System and method for processing and displaying information about real estate by developing and presenting a photogrammetric reality mesh - Patents.com

Abstract

The present invention discloses a system and method for multi-dimensional visualization of data and use of the visualization in any of a variety of applications, whereby a configuration of visualizations that can be used for a variety of purposes is generated. The system includes one or more processors, one or more databases, at least one Graphical User Interface (GUI), and control techniques for a user to control the display, where the display can typically be visualized via the GUI.

Description

[Technical field]

[0001] This application claims the benefit of priority to U.S. Provisional Application No. 63 / 199,458, filed December 30, 2020, the entire contents of which are incorporated herein by reference. [Background technology]

[0002] Currently, determining the attributes of a particular residential or commercial unit is laborious and unreliable. For example, users must painstakingly research the details of the unit they are interested in, and in order to determine the fair market value of that unit, users must properly compare dozens of attributes across many parameters to make that determination. For example: What is the general value of the neighborhood or submarket? How does the number of bedrooms and bathrooms affect the price? What amenities does the unit have and how does that affect the price? Are there any issues with the building itself, such as a rat problem? Is there any litigation pending regarding the building? What were previous sales prices? How much of the building consists of rental housing? How does the location of the apartment affect the price? Are the current owners behind on their taxes? If I make any changes to my unit, how will those changes affect the market value? Summary of the Invention [Problem to be solved by the invention]

[0003] Once all the data is available, algorithms can be used to determine pricing, however, the data changes frequently and it can be difficult or impossible to get fair pricing information in real time for a particular area or building.

[0004] To solve this problem, the present invention aims to simplify and automate the data collection and analysis process, allowing the user to select parameters and the invention uses an augmented reality approach to create and utilize visualizations of areas, buildings and even apartments. [Means for solving the problem]

[0005] The present invention relates to systems and methods for multi-dimensional visualization of data and use of the visualization in any of a variety of applications, which results in the creation of visualization configurations that can be utilized for a variety of purposes. The system includes one or more processors, one or more databases, at least one graphical user interface (GUI), and control techniques for a user to control the display, which may typically be visualized via a GUI. The processor of the present invention may include an engine for performing selected functions, such as processing specific data or types of data. Additionally or alternatively, the processor of the present invention may be pre-programmed to perform such functions. In one example of the present invention, data for a particular building at a particular location may be used to construct a two-dimensional, three-dimensional, or multi-dimensional visual arrangement of the building, including the building's amenities and appliances, which may (1) allow comparisons to be made on a unit-by-unit basis with other buildings (e.g., with neighboring buildings), (2) include additional dimensions of data, such as price, (3) allow a comparative fair value of the building or unit to be determined, or (4) achieve other desired results as identified below. In at least some embodiments, the value and / or other parameters may be determined automatically, and the user may be provided with the opportunity to select additional or alternative parameters.

[0006] That is, the present invention aggregates data from multiple public and private sources, does so on an ongoing basis, and uses that data to create multi-layered visualizations, typically augmented reality and / or 3D imagery. Using texture mapping (reality mesh), color coding, shading, fog, highlights, and user adjustable parameters that are automatically included by the system processor, convey detailed information about the area; the data is used to draw conclusions and form trends.

[0007] The primary goal of the inventive approach is to create a reality or augmented reality mesh visualization of a particular area to be used for a variety of purposes, some of which may be at least partially simultaneous, many of which are described herein. The area may be selected to be as large as an entire city or state, or as small as a portion of a single building. The selected area may be considered an augmented reality mesh. This reality mesh is an augmented reality type visual representation of a particular geographic area, region, or real estate submarket, and may be further augmented based on attributes of particular interest to a particular user. In the present invention, a graphical user interface (GUI) is used to display the reality mesh and allow the user or automatically to interact with and manipulate it.

[0008] The reality mesh of the present invention is a 3D computer file that is potentially actionable and contains metadata for applications such as control, and is created by photogrammetric processing of numerous aerial images of a geographical area that can be used to generate a virtual representation in 3D. The reality mesh file can be viewed in a geographic information system (GIS) such as a virtual globe software like Google Earth and can be placed in place to match a two-dimensional map. It augments the details of the area with vertical extrusions to help understand a city, submarket, neighborhood, block, or building. The problem faced by the real estate industry is that reality mesh files prior to the present invention do not contain metadata about legal properties and cannot provide additional insight beyond aesthetics. The present invention solves this gap by formulating a relational database data model that corresponds to the reality mesh, which can be expanded as data is collected and stored, including data about various attributes of each property, unit, or real estate market or submarket, and includes a user interface for displaying selected attributes of the properties, both in an augmented reality sense and in a selectable sense. The attributes may be user selectable, processor selectable, or both, and may include the ability for a user to expand or contract the field of view, including selecting an area of ​​interest across three dimensions. The database of the present invention is configured to be continually adjusted based on the introduction of new data or new data sources, and is further configured to allow for rapid delivery of selected content. Rapid delivery is important due to the large file sizes of high resolution reality mesh files. The database of the present invention is further created by normalizing the received data to allow for this rapid deployment.

[0009] In other examples, sales or rental data is used to highlight residential (or commercial) units that can be priced within a particular range of sales (such as using distinct colors for each range) or time frame or both to provide a fair market estimate for a particular property, and to allow comparisons to be made with other properties. In other words, a graphical user interface can be accessed and, if the need arises, one can see and determine how to price or improve a property, or understand market forces in general.

[0010] A reality mesh, also known more commonly as a photogrammetric model, is a texture-mapped model of a geographic area, with accurately scaled, high-resolution images. Typically, one or more images are taken by an aircraft or spacecraft and displayed on a Geographic Information System (GIS) or any mainstream web browser that supports common 3D graphic file display standards. The reality mesh models are then processed with advanced photogrammetry software that outputs homogenous polygonal models that can be viewed using a graphical user interface (GUI) using 3D model viewing software such as a web browser. Reality mesh models can be created in a variety of resolutions and fidelity. The objective in creating a reality mesh is to find a balance between image quality and file size to maximize system performance while considering the delivery method such as the web (low fidelity) or traditional local desktop GIS (high fidelity). As an example, our system modeling Manhattan uses a reality mesh with a resolution of 2 cm, which is highly detailed and performs well for web-delivered applications. This level of detail is advantageous over prior art in that it can be (1) displayed in an augmented reality manner (thus providing lifelike information to the viewer) and (2) overlaid with content such as informative coloring, highlighting, and / or text that is accurate to the actual submarket, property, building floor, or building architectural element embedded within the reality mesh.

[0011] It is a further goal of the present invention to provide the user with a visualization based on any combination of selectable parameters (including combinations of parameters) and to modify and display the relevant portions of the area, as well as to expose or highlight the area that best fits or fits the particular combination of parameters. Such visualization may include, but is not limited to, color, size, shading, fog, or labeling in relation to the targeted submarket, property, or neighborhood. In one such example, the data is periodically updated so that an augmented reality view of the area can show the current location of footholds and can be used to identify walking routes to the user based on current or forecasted weather. In another example, the augmented reality visualization can show changes in available spaces (such as apartments or stores) for rent, or available spaces that meet specific search criteria such as square footage, rent, building class, building operating costs, etc.

[0012] The system of the present invention includes a relational database and a GUI, and combines at least an x86 consumer-grade central processing unit (CPU) with a consumer-grade graphics processing unit (GPU) for handling the GUI display on an external monitor, command input from a touch screen or external mouse and keyboard, database queries, and display on an external display.

[0013] The technology of the present invention identifies, stores, and visualizes real estate information through curation and storage that includes the process of creating a coordinate bed (Figure 1) of coordinate options that a user may trap, manipulating the real estate footprint to match coordinates of parcels, buildings, floors, units, building elements, and infrastructure (cooling towers, water tanks, cellular towers, etc.) to identify and organize areas of the reality mesh.

[0014] Traditional Smart City GIS systems use shape files or unique 3D models of buildings with graduated levels of detail (LOD) aggregated into a virtual city model. The advantage of our solution is that it uses reality meshes to communicate unique building and floor data, which would otherwise be impossible without the coordinate-related knowledge of each property.

[0015] The visualization of the present invention is easy to understand and clickable so that the user can zoom in, zoom out, or obtain additional overlaid data. In other words, any spatial region, such as a real estate market, sub-market, building, floor, window, architectural element, etc., as an example, can be clicked by the user or highlighted by the system to reveal more information.

[0016] The present invention uses multiple data sources, such as, but not limited to, government records of properties and property listings, which may include both public and private sources. One such data source is actual imagery taken from overhead aerial or satellite equipment, which provides a structural starting point for the visualization. These initial images, which may vary in radius from being limited to a building to being extended to a city, are preferably high resolution images and can be used by the present invention to create the beginnings of an augmented reality approach to visualize selected areas. In the method of the present invention, the images are processed, at least in part, to more accurately identify edges and other attributes of the buildings. These edges are used in combination with other data obtained from additional sources to form an augmented reality visual display for each building, which is further enhanced based on factors such as, but not limited to, user and system selections.

[0017] Furthermore, the scale of the visualization can be adjusted based on user or system selection. The list of these data sources may include images of buildings and units, along with dimensional information, among other data. All data obtained by the system of the present invention from sources, which may include public and private sources, is preferably updated periodically, and a comparison is made periodically with previously received data (or normalized versions) to recognize which updates are relevant new information and which are e.g. temporary or erroneous. There are different kinds of error control checks in the system. As an example, floor-level error checks may be used, which include determining the correct spatial location of a floor or a unit on a floor in a building. This can be done for multiple units on one floor, but becomes more complicated when dealing with spatial representations of units in a 3D building model. The floor-level checks are performed as follows: A) Inspecting units by using available data on similar types of units on neighboring floors or similar buildings: in modern buildings, residential floors usually have the same column spacing and unit distribution between floors, and when new data is received that contradicts known column spacing or divided floor layouts, the inventive system is able to identify these anomalies by, but not limited to, visual comparison and when highlighted in the reality mesh. B) Floor level checking is also performed automatically by preventing the addition of unit coordinates that overlap the same area of ​​the floor. This is done by identifying the matching surface area from the newly added or updated unit and preventing the system from parsing the matching coordinates into the database.

[0018] The received images are preferably, but not necessarily, of high resolution. These images may include architectural variations such as ceiling height or low ceiling height, as well as appliances and fixtures. The system of the present invention considers these images along with available relevant data, such as dimensional information, that is stored in a normalized form in the database of the present invention or may be stored later, to create a multi-dimensional model and visualization of building architectural elements or features such as buildings, floors, units, land parcels, or rooftop infrastructure such as cooling towers, water towers, HVAC equipment, cellular transmitters, generators, or solar panels.

[0019] A graphical user interface associated with some of the systems of the present invention allows a user to interact with the visualization in any number of ways, including, but not limited to, rotating an image or replacing elements within the image.

[0020] The data sources used by the present invention are wide-ranging and include governmental and non-governmental sources. The system of the present invention includes a processor (which may in fact be multiple processors distributed such as a mesh network) that is programmed to periodically poll the data sources and update previously obtained data. List of Exemplary Data Sources is included as Appendix 1. As a result, the system of the present invention includes one or more databases, typically relational in nature, which can be reconfigured automatically or otherwise upon request. The processor of the present invention interfaces with the database and also with a dedicated graphical user interface, which allows the user to select any of a number of parameters to be displayed in the developed image for the desired building, unit or area. The image may be manipulable, such as rotatable by the system and / or user, and / or changeable viewpoint by the user, so that three-dimensional attributes can be displayed and / or distinguished, often in high resolution. Such imaging can include specific equipment, etc.

[0021] That is, the system of the present invention periodically polls a data source to create and update one or more database fields with visualized entries and uses these entries in the visualizations created and / or recreated by the processor of the present invention.

[0022] The selection of attributes to display can be user controlled and / or system controlled and selectable, and the user can select using a GUI, by clicking on a map, an object, selecting from a menu, voice activation, or any combination.

[0023] Data sources may be used to overlay the image with one or more additional dimensions, such as operating costs or tax data, calculated data such as rents, shared ownership, foreign ownership, market value, etc. Other real estate examples include real estate transaction details such as seller name, buyer name, sale price, and percentage ownership of the transferred property. Building mechanical examples include elevator inspection dates, boiler manufacture date and inspection date, cooling tower manufacture date, capacity, serial number, etc. Health and environmental examples include the presence and / or duration and / or time frame of litter, rodents, birds, air particulate levels, biological growth, etc. in a building's water tank or air handling equipment. These overlays can take many forms, including but not limited to zooming in and out of geographic areas or real estate markets, submarkets, buildings or portions of one or many buildings, changing colors to highlight selected markets, submarkets, buildings or properties, and selecting buildings or units for direct comparison. Again, all of these displays are preferably created to be displayed in augmented reality, such that a user can visualize a particular building, unit, or area in a multi-layered manner with desired and / or relevant data, the visualization incorporating the data in some way. The actual visualization may be user adjustable, customized based on the expressed or inferred needs of a particular user, etc., and may be created / displayed at least in part using augmented reality.

[0024] Some of these data sources may be personal sources. For some users, the personal source data may be combined with public source data, which allows for user-specific visualizations. In other cases, the personal source data may be filtered through anonymization routines to obscure data that could be used to identify the source, or to retain private data specific to a particular lease, appraisal report, mechanics contract, etc.

[0025] Many cities provide property coordinate boundaries as part of their open data initiatives. These coordinates typically originate from legacy / historic GIS systems and provide an association between parcels and city unique identifier codes (IDs) used for taxation or planning purposes. However, we have recognized that such data may be subject to errors and have developed an algorithm-based approach to "clean" the data (e.g., determine which data are erroneous) before updating or reconstructing the system's database.

[0026] Furthermore, because data available from multiple sources is not necessarily structurally consistent with one another, the system of the present invention includes routine processes to calibrate (or normalize) the data so that it can be stored in a uniform structure. As can be imagined, calls to data may be frequent, and the scope of such data may require extensive processing, so uniformity of storage is essential to the user experience.

[0027] Because at least some data, such as geographic data, are stored in public systems with significant history, the data associated with an accurate mesh depiction of the real world is not necessarily accurate, and the calibration process must compensate for this lack of accuracy. For example, building coordinates may be slightly off in space depending on the data source, and it is important to correct for this. In the present invention, the correction is preferably based at least in part on the overhead (e.g., aerial) imagery described above, which is used to manually adjust the coordinates. Furthermore, since data is constantly being updated by cities, municipalities, etc., and such errors may reintroduce themselves into the system of the present invention, the present invention provides routines to "check" for reintroduction of errors and avoid them as part of the calibration process. This is important because some changes, such as new construction, may be accurate. The present invention includes processing capabilities to distinguish between reintroduction of errors and temporary changes (e.g., scaffolding) and proper changes.

[0028] The system of the present invention ("System") process uses visual reporting to visualize the status of the system's coordinate fixation process, for example in a virtual globe within a web app (or equivalent element or engine). The visual reporting allows one to load any set of coordinates and quickly toggle on and off the original and modified coordinates, visually highlighting the differences, for example using primary colors. For example, if the system's original (government source) coordinates are styled red and the modified coordinates yellow, areas of finishing work in the reality mesh may be displayed in orange, and areas requiring coordinate refinement work may be displayed in red. The visual reporting also allows the user to adjust the brightness, contrast, hue, saturation, and gamma of the image to better expose building coordinates that require calibration. By cycling through different combinations, one can better explore areas of the mesh with texture colors from different real-world images and visually identify alignment issues that require corrective work.

[0029] Original coordinates vs. changed coordinates Figures 2 and 3 show the original footprint coordinates in yellow, overlaid with the modified coordinates in red by the system during the manual portion of the process (although this may also be automated at the parcel level), allowing the system's algorithms to identify buildings with a base set of coordinates that need to be adjusted to fit the reality mesh.

[0030] From the distortions along the surface of the mesh geometry it becomes clear that the original coordinates (yellow) do not fit the Reality Mesh building and need to be adjusted. The adjusted coordinates are acceptable and can be used by the application. The reporting tool also displays both coordinates simultaneously to allow manual or automatic checking.

[0031] Once calibrated, the coordinates become "core" to the system and can be used to correctly highlight buildings and legal property boundaries within the reality mesh. Without such calibration, especially in dense urban areas like Lower Manhattan, the results can be unnecessary and confusing. This can result in confusing distortions and inaccurate highlight placement.

[0032] Overlay with financial and other data provides "what if" scenarios that can be accomplished through user-selectable filtering.

[0033] Due to the volume of data and the need to retrieve it in different ways for visualization, file compression techniques can be used to compress the data and streamline the delivery of results. In short, a variety of techniques can be used, but they must be consistent with the process of both storing and delivering the data. Currently, the system uses a private cloud-based file server that is capable of actively compressing the reality mesh files before sending them to the client graphical user interface, but other file servers such as Amazon AWS or Microsoft Azure virtual hosting services could also be used. Alternatively, equivalent alternative technologies such as Content Delivery Networks (CDNs) can be used to improve the performance of hosting and file servers.

[0034] Once the database is set up and the data is regularly updated, its use in combination with a user interface is widespread. Appendix 2 lists many examples of use of the core of the present invention, all of which are believed to be novel and distinguishable in many ways from previous approaches. Effect of the Invention

[0035] Advantages of the Invention - Improved speed of dissemination of real estate and market information to stakeholders Improved and accurate comparison between real-world data and government or private property databases, multiple analyses and improved use of results and decision making New insights by combining multiple datasets in a 3D spatial context, leveraging simulation features such as Reality Mesh transparency, fogging, time span capabilities, weather, smoke, fire, flood and precise contextual callouts - Real-world representation of new data in real time Democratization of data by reducing arbitrage of information on assets - Maximize real estate value by comparing the performance of competitors Improved pricing based on up-to-date data to help accelerate sales cycles Introducing transparency to the real estate market by seeing all available space options in the market, not just the space offered to consumers by brokers ·Real-time search for apartments for rent or sale. Search for offices for rent or sale · Search all sales transactions of any kind of property and examine the details of each transaction The graphical user interface can present all or part of the application's data visualizations as an automated slideshow playing with constantly updated data customized to the user's industry role, significantly reducing the time needed to communicate changes in the real estate market. [Brief description of the drawings]

[0036] [Figure 1] FIG. 13 illustrates a real estate footprint to accommodate buildings of various heights and create a coordinate bed. [Diagram 2] FIG. 13 illustrates the unadjusted property footprint coordinates projected onto a reality mesh in accordance with the present invention. [Diagram 3] FIG. 13 illustrates adjusted property footprint coordinates projected onto a reality mesh in accordance with the present invention. [Figure 4] FIG. 1 is an architectural perspective view of various components of the present invention. [Diagram 5] This is a visualization example of how market participants can research the ownership and history of a unit, building, or market before renting or purchasing in the present invention. The infobox in the GUI shows the floor plan of a selected unit. [Figure 6] This is a visualization example of how market participants can research the ownership and history of a unit, building, or market before renting or purchasing in the present invention. The infobox in the GUI shows the floor plan of a selected unit. [Figure 7]FIG. 2 illustrates an example of a visualization of available office space in accordance with the present invention. [Figure 8] FIG. 1 illustrates an example visualization of New York City property taxes in square feet according to the present invention. [Figure 9] FIG. 1 illustrates an example visualization of New York City property taxes in square feet for peer properties in accordance with the present invention. [Figure 10] FIG. 1 shows an example of a complaint visualization for use in the present invention. [Figure 11] FIG. 1 shows an example of a complaint visualization for use in the present invention. [Figure 12] FIG. 2 is a diagram of the data hierarchy within the system, illustrating the nature of the data involved. [Figure 13] FIG. 1 illustrates an example of an infrastructure of the present invention. [Figure 14] FIG. 13 illustrates an example of a manipulation process for inspecting, modifying, or creating mesh coordinates that match a building envelope. [Figure 15] FIG. 13 shows an example of a fog effect where buildings containing query results are displayed without the fog effect as can be used in the present invention. [Figure 16] FIG. 13 shows an example of integration of core coordinates by coordinate floor selection. [Figure 17] FIG. 1 illustrates an example of a system that uses coordinate data to highlight buildings based on color-coded areas of urban zoning divisions in accordance with the present invention. [Figure 18] FIG. 1 is a diagram showing how existing potential floor-to-area ratio (FAR) differences can be represented in the present invention. [Figure 19] FIG. 2 shows details of the voice commands that are correctly accepted by the present invention, processed by a speech-to-text processor, and converted into text commands in the present invention. [Figure 20] FIG. 1 is a diagram showing an example of a mark-to-market analysis in the present invention. [Figure 21] FIG. 13 illustrates further attributes available for analysis in the present invention. [Figure 22] FIG. 13 illustrates the updated elevation of a polygonal control surface of the present invention. [Figure 23] FIG. 2 illustrates an example workflow of data during a typical user session. [Figure 24] FIG. 1 illustrates customization of the PropSee application infrastructure of the present invention. [Diagram 25] A diagram showing PropSee Application Infrastructure for Web. [Figure 26] FIG. 1 illustrates the PropSee application infrastructure iPad® Standalone. [Figure 27] FIG. 13 illustrates selected submarkets in Lower Manhattan by applying a fog effect to non-selected areas. [Figure 28] FIG. 1 illustrates a solution for fog solution grouping for residential buildings. [Figure 29] FIG. 13 illustrates an example of inverse mesh clipping logic. [Diagram 30] FIG. 13 illustrates a GUI used to manually position a floorplan within a mesh. [Diagram 31] FIG. 13 is a diagram illustrating the logic used by the system to select a viewing direction from partial suite coordinates. [Diagram 32] FIG. 13 shows an example of a visualization of an elevator core in a building in a reality mesh using the system's building coordinate data. [Diagram 33] FIG. 13 illustrates the system's use of floating lines in a GUI. [Diagram 34] Figure 1 shows rooftop infrastructure highlighted in a reality mesh that is part of the system. [Diagram 35] FIG. 1 illustrates an example of building centroid lines connected to indicate shared infrastructure ownership. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0037] The present invention can be thought of as having two core parts. The first part is a system and method for collecting, organizing, relating and embedding geographic real estate information, related information and physical building representations, linking them together and forming a photogrammetric reality mesh using some or all of the information. The second part includes systems and methods for retrieving, analyzing (or teaching) and displaying (or presenting) the processed information for purposes including, but not limited to, market data, statistics, financial analysis, which may include conclusions based on "what if" scenarios. That is why.

[0038] The present invention requires a consumer grade x86 CPU and 3D graphics acceleration GPU capabilities to operate. The present invention requires a high enough resolution display (1080p) to properly fit GIS (geographic information systems) and reality meshes within the GUI while presenting infoboxes to the user in readable text. The present invention requires access to large data stores including relational databases connected locally or over the Internet of at least 1 gigabyte in size, and capable of operating with much more data than a city the size of New York City would have with a data store of 100 gigabytes.

[0039] The present invention is further directed to the processor-based use of a relational database in combination with a GUI to allow a user to select attributes for display or further study or display, where the GUI and its display are self-adjusting based on the selected parameters. The GUI selection can be based on clicks, voice input prompts, and / or menu selections, by way of example only. The display itself may also be selectable, such as clickable, without limitation.

[0040] The invention is further directed to one or more physical displays with visualizations (herein referred to as displays) that change based on system and / or user selections, and the displays can include gradients that embed appropriate information within the presentation, preferably an augmented reality based presentation, and relate to the relevant data retrieved. The color gradients are contemplated and defined to reveal data outliers, where knowledge of the operational domain (e.g. boiler capacity) or business rules (e.g. tax assessments and abatements), market rents, expenses, etc. of the data set being visualized is required. The invention is directed, at least in part, to providing visualizations of real estate data superimposed on portions of properties and / or buildings, where such visualizations convey information based at least on content, color, atomization, shading, and / or highlighting.

[0041] The present invention is further directed to a method for updating and / or reconstructing a database based on periodic retrieval of data from a wealth of sources, the retrieved data resulting in ongoing modification of selectable menu items within a user GUI.

[0042] The present invention is further directed to the implementation of machine learning techniques that can be used to perform any or all of the database repopulation and / or reconfiguration and GUI reconfiguration, where the GUI reconfiguration can be based on any combination of the user's role, selections, and attributed fields or entries in the database.

[0043] The present invention is also directed to a methodology for volumetric mesh highlighting, where the highlighted regions may be highlighted in any number of ways, including but not limited to color or opacity / transparency, and the highlighting is preferably based on selectability by either or both of the processor of the present invention and the user role. Examples of user roles include real estate broker, tenant, property owner, financier, portfolio analyst, real estate appraiser, architect, stockbroker, etc. These include, but are not limited to, engineers, technicians ...

[0044] In at least some circumstances, particularly potentially, the present invention employs data structures and / or forms for input as described herein.

[0045] The present invention includes a system as shown in the architecture of Figure 4. The system of the present invention preferably includes an online, Internet-based file storage system, a relational database backend, a front-end client application including a GUI, and a front-end client application including a dashboard. Although the system is shown and discussed herein as including a database, a processor, and an application, each of these may be deployed as multiple such elements or devices, and these elements or devices may be distributed across several locations and / or arranged in an array configuration. An element or device described in the singular herein should not necessarily be construed as a single element or device.

[0046] The system can be deployed "offline" without needing an Internet connection. Such a system deployment paradigm is illustrated in Figure 13. The specific need is to run without Internet access, and the system includes all the necessary basemaps (vector topographical maps), reality mesh models, single building models, real estate market datasets, building datasets, city, state and federal government datasets, real estate transaction datasets, user information, user security information, GUI framework, and web browser stored on local disk on a mobile or desktop computing device connected to an external display of sufficient resolution (1080p or higher) to faithfully manipulate the GIS and reality mesh and interact with the data selectivity by mesh or the data selectivity by infoboxes. This same device can also be connected to a head mounted display or optical augmented reality glasses device to project the GIS, reality mesh and infoboxes directly in front of or above the user's viewing surface or field of view in a manner that spatially matches the geographic context proximate to the user's current / current location or other located area. This embodiment of the invention allows a user to stand in front of a building, look up at the building, and see data attributes related to the building, such as real estate information, available office or residential space for sale or rent, past property details, city permits, complaints, tax information, utility consumption data, or other data variables related to the subject property or to the neighboring properties of the user querying the system. Similarly, this embodiment of the invention also allows the user to remotely inspect the data attributes of a building without the user being present, using stored images of the real estate submarket, building, or floor being inspected.

[0047] Volumetric mesh enhancement techniques are described herein for buildings, floors, architectural elements, mechanical elements, or operational elements. These elements can be visualized in texture and geometry of the mesh image, or at any coordinate inside, on the surface, or outside the mesh. The polygonal mesh model can be extended to reveal elements including, but not limited to, real estate market or submarket performance, changes in real estate market or submarket performance over time, building ceiling height, amenity spaces, health safety, health check-in stations, window locations, door locations, rooftop water tanks, rooftop HVAC infrastructure, cooling towers, signage, life safety, outdoor terraces, rooftop solar potential, facade inspection dates, and other elements.

[0048] For operation, the system creates a reality mesh in 3D space, preferably on a Cartesian plane. The mesh is aligned within the virtual globe (aligned with a mathematical representation of the geographical location on a real Earth-type globe), preferably using a Cartesian coordinate system and a digital elevation model. A typical presentation of the mesh is drawn or projected on top of a two-dimensional satellite imagery map of the exact location, but the system is not limited to this and can present other base maps, such as nighttime satellite imagery, flood maps, or base maps with special styling and / or information design features. Base maps or other vector-based mapping data can also be placed on top of the reality mesh to depict road names, flood and evacuation zones, etc.

[0049] Systems that operate with reality meshes can also find applications in video games and film production. The system can generate user interface elements for buildings, neighborhoods, and cities that can be consumed by video game and film editing software for use in these areas. For example, the system can present areas of the reality mesh in a fog that contextually matches the plot or premise of a video game or film production.

[0050] The location of the reality mesh and the coordinate data stored in the system from the coordinate trapping process are relative to each other. When updated reality mesh files are added to the system, they must be positioned with the same alignment coordinates as the original mesh used for coordinate trapping. The mesh only contains exterior images of the structure. There is no interior image in the reality mesh for building structures, the interiors are hollow. The assumption commonly used and made in this document is that the location of the reality mesh will always be faithful and accurate to the real-world locations of real-world cities and buildings.

[0051] The user makes selections in the GUI in several ways. Primarily, the GUI provides the user with the ability to select any area of ​​the reality mesh, such as real estate markets, submarkets, buildings, floors, suites, windows, etc. Selections can be indicated by styling with icons, haze, color, styling with lighting projected by the GUI on the reality mesh, styling with mesh opacity / transparency, styling by clipping or removing sections of the mesh to show only one or more parcels or properties, styling by applying an opaque fog to the mesh and "punching holes" in the mesh to show buildings except those contained within the opaque fog, etc. The user can also make selections using information boxes (InfoBoxes) drawn in the GUI. InfoBoxes contain lists or tables of information generated from user inquiries and standard reports of information stored in a database. The user can select a building's name, address, or unique identifier in the InfoBox to open more detailed information depicting metadata or other information related to the InfoBox report. The info box can include unit specific images, such as photos of the bedrooms and kitchen for residential units, or the layout of the office space for office units. The info box can also include charts showing data displayed in a variety of user selectable chart formats, such as historical bar graphs providing information on rent and tax changes. The info box may or may not be placed relative to the building in context with the reality mesh, with lines connecting the reality mesh and the info box, depending on the need of the GUI to associate the info box with a particular building or real estate submarket or neighborhood relevant to the user.

[0052] The system's reality mesh is presented to the user's GUI as a compressed stream of texture image files, usually in PNG format, combined with vector geometry data organized as a tile set for efficient rendering performance by the CPU and GPU. The data is transmitted to the client system via a 3D mesh. Various compression protocols can be used that provide different levels of data size reduction depending on the client system platform. Compression protocols for reality mesh files used on Android devices and other devices may require different compression protocols than those required for Apple iOS devices. These compression techniques are important because high-resolution reality mesh files are very large and taxing on traditional consumer computing hardware, especially on mobile devices. With faster wireless technologies like 5G, this problem is somewhat mitigated. Client devices require the 3D graphics processing power of popular consumer video games to run applications with fidelity. The CPU and GPU processing power in current 5G mobile phones is sufficient to run applications. System memory requirements are not high because the system can buffer the necessary image textures in and out of memory as the GUI scrolls the reality mesh and loads and discards image textures depending on the display content.

[0053] The virtual globe software allows for a multitude of stylization effects exclusive to GIS software. A list of common stylization effects includes, but is not limited to, drawing 2D or 3D shapes, highlighting the terrain, drawing topographical features including terrain setbacks, inserting other 3D models, inserting 2D images, applying other texture images to buildings or other shape files, coloring the display, realigning loaded elements, colorization, clipping (deleting or removing parts of the reality mesh in horizontal planes), drawing vector lines on mesh and screen elements, measuring, and placing and positioning labels or icons relative to the user view of the display. These virtual globe stylizations can also be used during the process of capturing coordinate data that the system generates, whether through a manual coordinate capture process or through interaction with any other GIS. Multiple effects can be employed to enhance the system's GUI and make it easier for users to understand information from the system's display. For example, color-coding all buildings in New York City with high rates of bedbugs and highlighting all units in an apartment market that are vacant is an efficient way to narrow down potential options.

[0054] Sample insights gained for comparing multiple data sets: When two or more government or private property data sets are combined into the same view, the system reveals key insights. This allows market participants to comprehensively research the ownership and history of residential or office units or buildings, neighborhoods or submarkets prior to renting or purchasing; and identifies mortgages on properties with high loan-to-value ratios that are more likely to default; and identifies properties priced above the average square footage that may have a greater shot at profitability. See Figures 5 and 6.

[0055] As shown in Example A (see Figures 5 and 6), the system can load ResiRental's visualized apartment inventory with the city's ACRIS (Automated City Register Information System) data.

[0056] Example B (Figure 7) shows examples of available office space downtown and indoor environmental complaints from the New York City Department of Health and Mental Hygiene (DOHMH), highlighting space options located within buildings with reports of problematic indoor air quality, indoor sewage issues, asbestos, or mold.

[0057] Visualization functionality: Once the system has loaded the data required to perform a visualization, it provides the functionality to apply functions to the acquired data stored in the database and to modify the loaded stylization with other parameters. Example A shown in Figures 8 and 9 , which shows New York City property taxes per square foot for a downtown building. When the user selects a building, the GUI presents an information box with a feature called "Tax Difference" and changes the stylization to show an adjusted visualization showing the tax per square foot associated with the selected building. The color of the selected building, e.g., red or blue highlighting of the tax per square foot associated with it, indicates more or less real estate tax on the indicated building. This can be done only for peers, such as office buildings of the same type. In another example, this visualization feature can be applied to office tenants as well. Office tenants are typically obligated to pay a pro rata share of any tax increases over the base year in which the lease was signed. A visualization in the user interface can show the change in the rate of real estate tax increases over time for a single or multiple building spaces, using color coding. This feature allows tenants to use the system to evaluate whether they will be paying more or less tax, which can be useful to tenants, for example, to evaluate whether the tax system is being applied fairly.

[0058] Example B, which includes Figures 10 and 11, shows New York City Department of Buildings (DOB) complaints in the TriBeCa submarket area, with the building 56 Leonard Street highlighted in white in the GUI. The information box shows a list of DOB complaints by category. You can see complaints about illegal hotel rooms in residential buildings. Each category is also a feature, and selecting any category will filter the search results by buildings that fit the complaint category. The system shows the example of illegal hotel rooms in residential buildings, a growing problem in the city with the advent of online room rental services like AirBnB and Vacation Rentals By Owner (VRBO).

[0059] The system further integrates building footprint data available from public sources and unique building identifiers to link all information in the system. The geographic hierarchy of the system is by country, state, county, city, market, submarket, neighborhood, building, floor, unit, and facility / infrastructure item. By leveraging the public and private ecosystem of unique property identifiers, the system is able to select and visualize all geographic entities at an individual level, treating each entity uniquely within the system interface to visualize any associated data. Examples of related data or metadata include, but are not limited to, ownership, transaction history, municipal zoning, property tax records, government regulated condominium plans, commercial and residential property rents and vacancy rates, properties in litigation, energy, water and gas utility consumption meter data, health and education department data, fire and life safety data, insurance flood risk, building maintenance and operations data, property financial data, investment market data, restaurant inspections, current and past hotel booking activity from travel agency affiliate data feeds, criminal activity, proximity to subway stations, frequency of clusters testing positive for pathogens such as Covid-19, and more. Each data set can be tracked, for example, over time and changes can be visualized in the system using unique property identifiers common to government and private real estate datasets. Unique identifiers from these external datasets are added or combined with system-defined unique identifiers by the order of matching through manual or automated combining processes used by the system when properties are added. An example of an automated combining process is combining building addresses that match building IDs. An example of a manual join process would be a user creating a join (stored in a database table) to construct an identifier that includes different legal addresses but the same physical structure.

[0060] The concept of highlighting regions of a reality mesh is central to this invention, but is not limited to it. The highlighting can be either visible or invisible, and regions of the reality mesh can be made transparent by using a high opacity. The reason for making a reality mesh transparent is One reason is to mimic the appearance of windows on buildings. A key part of the invention is how the system allows for the selection of real-world objects in the mesh, either through a GUI or within the inventive system itself. Without this process, the system would be unable to classify or identify legal parcels within the reality mesh beyond indicating a single point with latitude and longitude values. Parcel coordinates provided by governments are typically stored in a legacy system format called cadastral maps. Cadastral maps are records of property boundaries collected from decades of government land management processes, using manual line-of-sight techniques that are not accurate to real-world urban structures. Different levels and generations of government have different standards of accuracy in their legacy land management systems, resulting in more or less accurate property data provided to the public. Additionally, urban areas with a history of buildings being demolished (legally or functionally divided into different properties) are often not reflected in public data with consistent standards. Additionally, there are few private market sources of property data stored in building or floor-level formats with common city and country standards in the United States, Canada, and European countries. Therefore, the system of the present invention compensates for these unknowns by using only the government provided coordinate data as the starting point and source of unique identifiers and using reality mesh coordinate trapping techniques to generate the required property coordinates using the precision created by the system's human operator.

[0061] The present invention uses its reality mesh to collect data using selected data sources, place the data in the present invention's database, and use the processed results to formulate visualizations at any or all of these levels, at least as requested. Because Western governments tend to organize real estate data by structure and tax place, the present system employs an organization method based on a similar paradigm, allowing for quick and intuitive data querying. If a jurisdiction uses a different method, the system can adjust accordingly with "normalized" data. Using transparent highlighting techniques, the system can create a user interface that allows touching or clicking on buildings and interacting by floors, units, windows, and superstructures.

[0062] As an example here, New York City in the United States is the best choice since New York City has various local laws (see LL84) enacted that require disclosure of a great many city data indicators and has an extensive data tracking program to track various aspects of real estate. This is important because it enables business models that rely on data disclosed from the city to provide / promote products and services. Generally, these data sources are publicly available, but different data sources may need to be normalized to be most useful in the system of the present invention. In the method of the present invention, data from these data sources are directly or indirectly brought into the database and may be supplemented by other data, such as private data from these or other sources.

[0063] Figure 12 illustrates the data hierarchy within the system of the present invention and indicates the nature of the data involved. While this description uses New York City as an example, tax location information and / or corresponding specific tax information may be included. A city, county, or other government agency that provides public access to a location identifier and relevant or applicable city, county, or other government agency dataset may also be substituted. Applicable government datasets that are relevant only to the region and not to specific properties, such as climate change or flood data, may also be used by the system to stylize the reality mesh and present the information to the user.

[0064] In the example detailed in Figure 12, we use OSS such as NASA's WorldWind, CesiumJS, or three.js. Use an open source virtual globe application like the Open Source WebGL Viewer to view a 3D world with real-world scale and physics on a Cartesian plane. A traditional GIS application environment can be created or simulated to project the reality mesh. Because the systems and methods detailed herein are not limited to GIS applications or web browsers, all the functionality described herein can be reproduced in a 3D environment with sufficient functionality to load the reality mesh and perform stylization effects on the reality mesh. The reality mesh can be rendered at various resolutions, the higher the resolution, the more computing resources required. See the infrastructure diagram in Figure 13 below.

[0065] Once a reality mesh is loaded and applied, it can be manipulated using coordinate data stored in the system's database. This data is constructed as 2D or 3D polygons of various dimensions and sizes, otherwise represented as volumetric regions of the reality mesh. These regions act as GUI contacts on the building surface, allowing the user to accurately and intuitively select any part of the building as inferred from the reality mesh resolution. The example mesh uses a very high resolution image texture mesh of 2cm=1 pixel, so the detail is very high, allowing small features like the spray paint on a manhole cover, as well as the make and model of the air conditioner and cell phone antenna on the rooftop to be identified. This is critical to the system as the image quality in the GUI is crisp and allows the user to accurately evaluate all aspects of the building.

[0066] All components of the application are preferably deployed in a client-server relationship over the public Internet or in an offline, standalone design, where the system and application can run on mobile and / or desktop computers running a local web browser, a local web server, a local copy of the web application, and a local copy of the database. A secure HTTPS connection may be required to operate an online implementation of the system over the public Internet.

[0067] The system provides a user interface that can be deployed on a client device in either online or offline mode via a web browser, web server, web application, database, and can be controlled with a mouse, a touch screen enabled device, or voice input, see Figure 13. The GUI also allows the user to export as graphics or render into video formats, making the information available offline.

[0068] The operational process for inspecting, correcting, or creating matching mesh coordinates to match building envelopes in the reality mesh is detailed in Figure 14. This method involves the critical step of creating and adjusting polygons that define 2D coordinates such as building footprints. Government provided (legacy) coordinates are associated with unique building identifiers (system, building IDs) and the unique identifiers they contain (government IDs) are combined with new IDs in a relational database. The legacy coordinate strings are then compared to the building representation in the reality mesh at grade (ground level) so that they can be modified to exclude sidewalks and other infrastructure at grade that are not part of the normalized structure or legal (or other) real estate. Modifications to adjust the coordinate strings to match the building are performed by removing coordinates that do not match the building exterior and adding new coordinates that match the building representation in the reality mesh. Coordinates that match the building exterior are stored in the system's database. Triangulation errors can occur throughout this process, which are distortions that occur in the GIS drawing system when coordinates are stored in non-contiguous strings. For example, to draw a polygon from coordinates, use GIS requires that all coordinates be read as continuous, and when the GIS draws coordinates that deviate from a linear sequence, distortions that appear as acute or obtuse triangles are introduced. Such errors can occur during this process when altered coordinate strings are entered in a nonlinear manner, or when coordinates with non-contiguous values ​​are parsed into the database so that they are superimposed along an edge, or as a non-contiguous string. Triangulation is performed on each individual building, or on all buildings in a submarket, to eliminate misplaced coordinates. Once the triangulation errors have been removed, the coordinate string is stored against a building ID and used to highlight the building.

[0069] An elevation bed of coordinates must also be created. This system is necessary because we need a set of coordinates close to the floor or building of interest to make accurate selections for anchoring the building or creating partial coordinates for the individual dwelling units. Many buildings require adjustments to the base coordinates, and this process is used to fit the reality mesh, to define unit boundaries for demised floors, or to add additional features such as solar panels, rooftop balconies, etc. The method of the present invention includes this adjustment process, along with any necessary adjustments to identify the coordinates of the building's infrastructure, such as water towers, cooling towers, and / or other HVAC or mechanical infrastructure.

[0070] The process of the system presents an existing set of real-world coordinates in spatial context with the target property in the reality mesh. The system does this by presenting the known coordinates in a semi-transparent polygon that encompasses the entire floor of the building. This provides a limited set of real-world coordinates that give the system precise accuracy since the user does not need to convert between various global and traditional coordinate systems as they select division points along the floor of the building. To the best of our knowledge, the present invention is the first to apply such a process to create such precision and the first known visualization with precision for an augmented reality display. Essentially, the coordinate selection process is simplified by a localized subset and possible options that are fed by a known selection of normalized coordinates in the coordinate bed. This allows for precise placement of units on the floor relative to the real-world image of the reality mesh. Figure 1 details the example of the 8th floor of 270 Broadway in New York. The coordinate bed encompasses the entire floor and the units derived from the selection of coordinates in the elevated coordinate bed result in units defined at the northeast corner of the building.

[0071] Our method provides clean vector transformations at architectural junctions in building structures by combining the first and last coordinate pairs from a known selection set, resulting in clean enhancement of the reality mesh at the corners of building floors.

[0072] Each spatially defined floor or unit (e.g., residential or commercial unit), or portion of a floor representing a common area (e.g., elevator lobby, hallway, or bathroom), can be stored in the database of the present invention as an associated object that is linked to the building table and various other associated entities in the database. Obvious examples of such units include apartments for rent, residential condominiums, office space for rent, office condominiums, lobbies, hotel rooms, mechanical rooms, cleaning storage rooms, utility access rooms, computer server facilities, etc.

[0073] The utility of storing coordinates of units and suites within a building is timely with the advent of augmented reality applications, making it useful for virtual tours and virtual representations of spaces for marketing purposes. Additionally, the present invention includes methods for more advanced visualization, including 3D floor stacking. Stacking building floor plans allows for the visualization of floors and suites in a building. This is extremely important as it gives you the ability to apply tint ranges to units and highlights that depict information such as lease end dates, rents, transaction history, tenants, vacant space, etc.

[0074] The application of this tool to investigating the rental apartment market, condominiums for sale, and sales history of buildings is clearly novel.

[0075] The present invention further includes mesh clipping, which can be used to clip out (or remove from the GUI) a portion of a building in the reality mesh at a selected elevation so that a floor plan image or computer-aided design (CAD) file can be placed in place in the reality mesh so that the floor plan is placed in the correct location and elevation with the correct spatial orientation. The system's process for creating the clipping coordinates necessary to support this capability is as follows: 1. The user or our system finds the unique ID of the building to create the clipping for and loads it using the Edit Clipping form in the system's Admin tool, which positions the virtual globe camera to the correct building ID in the reality mesh. 2. The system checks if there are existing clipping coordinates in the system for the selected building and loads these coordinates. If not, the system uses default coordinates based on the building centroid (center point of slope) coordinates and the core building coordinates. 3. The selected building is highlighted in partially transparent red and four horizontal clipping planes are drawn perpendicular to each other at 90 degree angles centered on the building's centroid: yellow, red, green and blue. 4. By clicking on each plane, the user can increase or decrease the plane’s proximity to building facades, property boundaries or structural walls so that the clipping fits the building, as evident in the Reality Mesh. 5. The two-step adjustment control in the Clipping Edit form allows the user to increase or decrease the plane's proximity to the building facade by making larger or smaller steps. Each step control contains a positive or negative correction to move the plane closer to or further away from the building. 6. After setting the desired parameters for the horizontal plane, the system displays a slider on the GUI, allowing the user to automatically scroll from top to bottom through each floor of the building to verify that the cropped plane displays only the desired elements of the building. At the same time, the system displays the saved floor plan of the selected building at the same height as the displayed (cropped) floor of the reality mesh, allowing the user to verify and edit (scale and rotate) the floor plan orientation on the two-dimensional axis. 7. The user of the present invention system saves the new or updated clipping plane coordinates to the system.

[0076] The system of the present invention provides an interface that allows a traditional home or office floor plan image to be displayed at the correct floor height and location with all portions of the reality mesh overlying the floor plan cleared or clipped from the GUI, allowing a deeper understanding of the target space in the context of the building, neighborhood, and spatial domain, clarifying the direction of sight and the location of the unit within the building.

[0077] The target building may be displayed with accurate (real-world) colorization, but from an information presentation, design, and emphasis standpoint, all surrounding urban structures may be displayed in a "clouded" manner, such as an opaque white color that obscures background buildings and enhances the emphasis on the building or buildings being displayed by the GUI.

[0078] Buildings with floors of various sizes require a lot of effort to create associations between all images and floors. This allows a building such as One World Trade Center (example in Figure 16) to accommodate different image files and coordinate layouts for each floor. Since the floors of the example building vary in physical parameters at different elevation ranges, this is an important feature that allows the system to adapt to numerous architectural styles and massings.

[0079] The system of the present invention allows building equipment such as furniture, HVAC equipment, security systems, security cameras, elevators, boilers, fire suppression equipment, plumbing infrastructure, etc. to be positioned in a way that represents real-world accuracy. Metadata such as manufacturer, serial number, asset ID, utility consumption data, equipment model and specifications can be stored in a database and associated with the equipment's exact spatial location within the system's reality mesh. The inclusion of this data allows for faster equipment inspections and, if necessary, faster asset relocation. In the security camera example, it is important that law enforcement agencies can use the system to quickly identify cameras installed in locations that are helpful in investigating criminal activity, camera imaging systems that utilize facial recognition technology, or security camera equipment can be precisely located and highlighted so there are no gaps in security camera coverage. Additionally, such metadata associated with data feeds from city, county, state, country, other government agencies, or real estate image archives can be associated with mesh coordinate data and visualized. If a state level government regulates the installation of certain air conditioning models and filters in certain retail buildings (shopping malls), the system can visualize existing or planned air conditioning equipment and air conditioning upgrades in the retail building in a reality mesh. By matching the upgrades and energy consumption permits, it is possible to visualize energy efficient buildings in the modern system. This is a very valuable feature for state governments to monitor compliance with new regulations.

[0080] Applications of the present invention include real estate and rental analysis, including building operations, inventory management, floor plan analysis, emergency services management, and various other real estate applications, views, interior design, and machine learning implementations that allow the system to be used to recognize and match photos of buildings to aid law enforcement investigations. This is done in the present invention by comparing the geometry and texture of the mesh to images of the building, allowing the computer to perform the matching task. Additionally, Building Information Management (BIM) data, demographic data, and traffic count data can also be used as sources of data that can be visualized by the system.

[0081] Once a floor is selected by the user or the like and the floor plan is displayed, the user can interact and receive information about the building, floor, or unit by clicking on other floors that are highlighted (see red in Figure 16), which when clicked can change color, e.g., to yellow. In this example, a dotted polygon is displayed on the newly selected floor. As shown in Figure 16, the floor highlight, dotted line, and clipped floor coordinates are all valid and constrained by the integration of the core coordinates made by the coordinate bed selection. This process serves to further associate the floor plan image with the data created in the previous process. The GUI for this process is described in Figure 16, which is an example of the 73rd floor of One World Trade Center, showing the in situ floor plan on the plane of the clipped floor. An example of the furniture and floor plan layout is also shown in the figure with a "fog" stylization of the mesh. Floor plan layouts are important for commercial and residential tenants as they can better differentiate spaces that fit their specific requirements. The system's reality mesh clipping feature also allows partial or complete building structures to be removed from the reality mesh down to grade (ground level) or foundation level, exposing the building's substructure. This obviously means that basements, parking lots, underground infrastructure, and foundation supports can be easily removed from the building represented in the reality mesh. It is very effective at exposing the lowest levels of the terrain below.

[0082] This section of the document details the application and use of the methods of the present invention.

[0083] Single-click on-mesh record search technology The method uses a relational database to store system inputs from one or more government or private source application programming interfaces (APIs), which are automated processes for pulling data from web services. An example of a government API is the New York City Open Data System. An example of a private API is the hotel reservation system operated by Expedia, Inc. The system can also use exports from government open data services and import them as comma separated values ​​(CSV), XML, or JSON data objects and store them in pre-built tables organized by levels of government (city, state, federal) and departments. The same data entry process exists for private data services that can export data with unique identifiers as CSV, XML, or JSON data objects and store them in tables organized by company or industry role. Examples of such data include, but are not limited to, private data sets owned by landlords and other businesses that are typically used by real estate data aggregators and real estate information companies. Other examples include data from government sources such as NYC Department of Buildings (DOB) complaints, Department of Housing and Community Renewal (DHCR) registration forms, DOB violations, DOB permits, Department of Environmental Protection (DEP) asbestos data, Department of Finance (DOF) property taxes, DOB façade safety, DOF financial records such as deeds and mortgages stored in ACRIS, DOHMH restaurant inspections, Department of Health (DSNY) graffiti tracking, DOB eviction orders, residential condominium plans registered with the NYS Attorney's Office, DOB certificates of occupancy, and NYC Department of Housing Preservation and Development (HPD) pest complaints (bed bugs, fleas, flies, rats, cockroaches, termites, etc.).

[0084] Figure 17 details an example of a system that uses coordinate data to highlight buildings based on color-coded ranges of land use classifications in a city. The selected buildings are displayed in red and linked to an infobox that displays the results of an entity transformation that retrieves a selection of building data from a relational database based on the buildings selected in the reality mesh. The infobox has several tabs for different data sources, with an ACRIS tab for New York City shown as an example. ACRIS is the Automated City Register Information System, managed by the New York City Department of Finance, that records and manages real estate ownership, mortgages, taxes, and other transactional financial records. All real estate financial transaction documents are registered in the system and made publicly available with a unique identifier, either the Borough Block Lot (BBL) code, which is one of New York City's methods of building identification, or the Building Information Number (BIN), which is issued by the Department of Planning. Many cities and counties in the United States have a similar ecosystem of unique identifiers managed by city departments to manage real estate data. As a result of selecting a building from the system's reality mesh, the user may be immediately presented with all ACRIS documents generated by queries resulting from the building ID associated with the system input. One novel aspect of this method of searching ACRIS documents is that selecting an area of ​​the mesh once will produce all real estate transaction records within that area, making the process of searching for copies of tax documents, deeds, loan agreements, etc. The benefit of this is that it reduces the time required for the system to run queries in the database, since New York City's ACRIS system stores so many municipal tax and real estate records, as well as all other municipal databases that contain legal and transactional documents necessary for municipalities to catalog tax and real estate records.

[0085] Any visualization returned in the GUI from a database query can be refined by user-selected filters (keywords) and parameters (date ranges) without any subsequent queries to the system, improving performance.

[0086] As a first example, New York City publicly provides restaurant inspection data (DOHMH If a user submits a query that returns all of the restaurants in a given building (see Table 1) with their associated building ID, the system can highlight all buildings containing restaurants with data. The data can include metadata related to all restaurant inspections, such as restaurant name, outdoor dining option, cuisine type, inspection date, inspection grade, and violations. The system can use any metadata field as a filter to select or exclude matching results from the returned dataset. For example, the system can select Mexican or Chinese cuisine types, and the visualization is instantly updated with the new criteria. This visual search analysis method is highly efficient because an individual's perceptual abilities are typically faster at identifying real-world buildings when they see the actual building highlighted on a map than they are at identifying a property from a list of addresses.

[0087] In a second follow-up example, if a user submits a query to return all mortgage agreements registered between all parties, the lender type (e.g., Bank of America or Citibank) is included as metadata in the government data export that was added as a field in the database and is therefore available as a filter and can be used to highlight document results by debtor, mortgagor, assignor, grantor, or lender. Extending this example, lenders can quickly research selected issues across their entire loans by area.

[0088] In a third follow-up example, if a user requests a result set containing all available residential condominiums for sale or rent in a city or submarket, the result set will include all coordinates that define partial floor highlights for all units. The result set will also include all metadata related to the residential condo market, such as asking price per square foot (PSF), unit size, last sale price, last sale date, or average price per square foot in the market, submarket, building group, or specific building. Each of these values ​​can be used to create a spectrum of colored result highlights across the mesh. More specifically, units that are in close proximity to and within sight of natural areas such as waterways, gardens, forests, parks, and coastlines will apply upward pressure on market value.

[0089] This method can be extended to be useful when processing government data that includes references to specific physically located parts of buildings, such as facade inspection scaffolding, water towers, cooling towers, cell phone antennas, security cameras, etc., for at least two reasons. A) Using this method, the characteristics of every building can be accurately identified and associated with it and stored in a database as objects. B) User queries can take into account coordinate domains when sorting infrastructure assets. Now that the geospatial domains of the water tower, emergency generator, and water cooler / chiller are known, they can be represented in 3D space, highlighted with attributes and metadata, and communicated to the user on various types of displays, especially devices with native capabilities for displaying augmented reality such as the iPad. These various types of devices can display the building infrastructure on their displays using cameras that capture real-world positions. The coordinates defined in this way by manual or automatic selection in the reality mesh are needed for this kind of augmented reality experience.

[0090] In the user experience of the present invention, by selecting a specific attribute, it becomes easier to see. The GUI of the present invention may display an augmented reality version of the desired attribute.

[0091] Another application of coordinate data is shown in Figure 18, which shows how the difference between existing and potential Floor Area Ratio (FAR) can be represented. FAR is a measure of the maximum buildable area on a legal lot. There is often a difference between the existing floor area and the buildable floor area according to the municipality's FAR rules. The FAR rules apply to different building exterior size options based on the underlying zoning district where the FAR can be used in different ways. The FAR calculation must also take into account the city zoning code and must be adjusted with the height restrictions as per the relevant zoning rights for the selected property parcel. The system can reference the FAR and zoning allowances of the neighboring lands of the selected parcel and consider the possibility of "air rights" transactions from neighboring lands and properties and maximization of density. The present invention can visually project the buildable floor area onto a three-dimensional representation of the building in a reality mesh based in part on customized building coordinates, which is required to project the potential floor area upwards in the reality mesh in the exact allowable location. The system generates this visualization taking into account the municipal zoning of the selected property and its immediate neighboring parcels.

[0092] The present invention allows for the use of stylized return result techniques, such as a city accented with white "fog" or "fogginess" over streets and buildings, normalized so that some buildings and floors have no fog.

[0093] Figure 15 is an example of stylized fog applied to a city to highlight a single building, floor or unit that does not include stylized fog processing. This technique highlights the target building or floor returned in the result set. This aids the user or viewer of the application in understanding the space or property returned by the query.

[0094] Similarly, the results of a GUI visualization can be projected into a reality mesh such that only the buildings that have results returned by the query are displayed in normalized real-world colors, and all other areas of the urban area represented in 3D by the reality mesh are covered in a stylized white mist.

[0095] This technique is only possible through the manipulation of collected data. While it seems possible for anyone to apply colorization effects to a reality mesh, the ability to "drill holes" in the same real world locations as buildings and floors requires a sophisticated processing system, similar to the one in this invention, that contains all the coordinate data related to buildings in the reality mesh.

[0096] Voice Control Sending system commands using customized text strings is particularly useful. On both mobile and desktop platforms, the system can accept voice commands to operate applications, interact with the reality mesh, and query stored real estate and government data. Customized spoken macros are stored by the system to recognize application functions for controlling the GUI and camera on the user device. These macros include terms such as "Spin," "Show Office Market," "Hide Office Market," "Show Rental Housing Market," "Show Rents," "Hide Rents," "Show Vacant Apartments," "Hide Vacant Apartments," "Vacant Apartments," "Recent Sales Prices," "Recent Sales Prices per Average Square Foot in a Specific Submarket," as well as all relevant real estate database fields and their variables and control modifiers (show, hide, none) within the system. These customized voice commands can be combined into macro commands to build more complex queries.

[0097] Figure 19 details the voice commands that are correctly accepted by the application, processed by the speech-to-text processor, and converted into text commands. Of course, other programs are possible. The command may be a simple action request to the application, such as "spin the map," which causes the application to orbit the camera around the GUI's reality mesh, or it may be a more complex request, such as a command + database query that is parsed by the speech converter. An example of such a query would be "Show me the 20th floor of 61 Broadway," and the user interface would display a virtual Fly through the virtual globe to the saved camera position at 61 Broadway, and the mesh will have the required protection. The system uses existing data to display floor plans, cropped at the 20th floor. Furthermore, a user can say, "Deed to 61 Broadway," and the user interface will fly to that building in the reality mesh, display the deed in question, and pull it from the system's association with a unique building ID.

[0098] With the speech function, you can use the speech command "highlight 61 Broadway ( When you say "61 Broadway," a database call is made to get the building's base coordinates, and the volumetric region of the building's projection within the mesh is enveloped with a colored highlight.

[0099] For example, "Fly to 61 Broadway, One World Trade Center, 28 Liberty." For example, users can send extended voice commands as a string, and the processed text command will cause the application GUI to move the virtual glove's camera to a specified building, hovering at a specific angle for a while before moving on to the next building, giving the user the sensation of flying. By adding additional conditions, it is possible to extend the GUI so that it flies over each building, briefly displays the title deed at a given camera position, and continues the sequence.

[0100] All this can be done by the user speaking to the application after enabling the application listener with a single click.Facial recognition is also available.

[0101] Cost Comparison The system includes mechanisms for comparing various costs, such as operating expenses and property taxes, in the subject building, such as, but not limited to, 1) costs averaged by submarket and / or 2) costs averaged by similar classes of commercial buildings (Classes A, B, C). These comparisons can also be made for similar property types, such as offices, apartment complexes, residential condominiums, hotels, leasehold positions, etc.

[0102] Examples of property operating expenses include, but are not limited to, insurance premiums, repairs, cleaning, labor, security, heating fuel, electricity, water and sewer, management, and administrative expenses. Property taxes are also stored in the database, but are organized separately from operating expenses. Each of these data is stored in the system and associated with each building record and its associated coordinate data.

[0103] The system can further generate visual output of this analysis, such as a GUI with the building represented 3D within a reality mesh.

[0104] Mark vs. Marker Analysis The system includes a mechanism to compare the existing rent for a building with the current market rent, calculate the difference, and apply a cap rate to the difference. Through this analysis, the system can estimate the reasonable value of the property. Figure 20 shows an example of this analysis. The system can generate a visual output of this analysis for any building represented three-dimensionally within the reality mesh. Mark-to-marker analysis requires the user's assumptions of office rents and fluctuating cap rates to output a visualization of the value difference that can be achieved at a future point in time. The system can automate these calculations by describing the average cap rates being paid for comparable properties in the target property's submarket.

[0105] Change in value This method allows you to calculate the difference between the purchase price and the selling price of a building.

[0106] This calculation is done algorithmically using relevant data regarding the buying and selling transaction, including but not limited to: 1) Operating rate at time of purchase 2) Occupancy rate at time of sale 3) Net operating income (NOI) at the time of purchase used to calculate the CAP rate 4) NOI at time of sale (CAP rate at time of sale) 5) All available financing information regarding the loan taken out on the property at the time of purchase 6) All possible loan information for loans placed on the property at the time of sale (although this data point is more important for historical record keeping in a system used to analyze the real estate investment sales market in the future). 7) Any further financing imposed on the property during the ownership period, such as mezzanine loans or refinancing The algorithms are adjusted periodically based on regular data collection and may be tuned through machine learning.

[0107] Any of the seven categorised information types can be visualised alone or in groups and compared to peer buildings, for example using a colour scale range applied to a reality mesh facilitated by the stored coordinate data.

[0108] Determining equity in a real estate transaction may be done by calculating the purchase price minus any debt at the time of purchase. This equals the capital invested in the transaction. Upon sale, the gain or loss is equal to the sale price minus any outstanding debt that is being redeemed at the time of sale. Calculating the difference between the original invested capital and the net proceeds at the time of sale after repaying existing debt provides a calculation to determine the equity multiple and internal rate of return (IRR), which are important metrics for investors.

[0109] This change in value can be visualized by the system for an individual building or for multiple buildings represented in a reality mesh.

[0110] Real estate production Building physical and financial attribute data is crucial to real estate transactions. Building parameters such as ceiling height, floor-to-ceiling windows, column spacing, loss factor, curtain wall type, floor area, etc. all play a role in determining fair market value.

[0111] These attributes, all or part, can be compiled and staged for a particular visualization and output via overlay in the display to convey specific information to various consumers of the information. Outputs can be development, finance, housing, office space, etc. An example of a finance output could be a visualization of existing loan size, terms, origination date and maturity, type of lender (debt fund, sovereign wealth fund, regional bank, national bank). This could be used to visualize highly indebted buildings that have a higher probability of mortgage default. This allows the user to filter the market and see which properties are being leveraged beyond a certain point set by the user. A tool in the system of the present invention can be used to highlight a set of buildings that meet certain criteria. The user can use this tool to identify financially stressed buildings by loan-to-value ratio. Buildings are financially stressed if they have mortgages that are causing rental yields to exceed the ability to make monthly mortgage payments. The visualization utilizes the coordinates created to stylize the reality mesh and presents output values ​​for single or multiple buildings, using comparative color ranges as an example. The system also takes into account the passage of time and can change the visualization of the output based on a user-entered time range or future dates.

[0112] Similarly, based on the data in the database, the expected growth (or decrease) in value can also be determined and visualized as well.

[0113] Over time, this data will become more strongly correlated with physical attributes and asset performance data, making it a predictor of financial performance. Certain physical attributes, such as column spacing or curtain walling (floor to ceiling glass, masonry, punch-outs, etc.), can be represented across the entire building surface, visualized by the texture of the mesh, and highlighted with great accuracy directly on the building in the reality mesh. This allows a user to, for example, query the system to return a representation of all buildings in a submarket with highlighting applied to buildings with a particular range of column spacing, and watch as it returns those results with the exact columns at the exact spacing projected onto the actual buildings in the mesh that meet the search criteria. Additionally, as this data is returned and made available in the user interface, attribute metadata such as window glass type, insulation coefficient, installation cost, replacement cost, or combination queries with government data such as third inspection date (required by law in NYC), HVAC inspection date (bacteriological testing of water coolers / chillers is required by law in NYC), water tower inspection date as required in NYC, etc. The system can also filter and highlight buildings that contain mechanical equipment like HVAC systems, elevators, escalator systems, and rooftop chillers that have been installed or repaired since a user-defined date. This allows users to identify buildings that have end-of-life mechanical equipment, life safety equipment, air handling equipment, electrical equipment, internet connectivity equipment, and utility systems, whose condition impacts the property's valuation. Examples of users of this feature include building owners and vendors of mechanical, life safety, air handling systems, electrical, and utility systems. Visualizing the data in this way can provide instant evidence of compliance violations, inspection fraud, and safety risks.

[0114] Telecommunications or wireless signal strength (including television signals) surveys are possible and useful with the system of the present invention because the stored coordinate data, including information on building materials, allows for three-dimensional modeling of signal propagation throughout dense urban areas of concrete, steel and glass structures, illustrating areas of weak signal strength in dense urban areas or areas without line of sight to cellular transmission infrastructure.

[0115] Available building power (watts per square foot), backup generators, green roofs, rooftop solar panels, and rooftop solar potential are also examples of unique infrastructure within buildings that can be visualized with this system. City governments are simultaneously enacting legislation to reduce CO2 and CO2 equivalent emissions, minimize the impact of emissions, and maximize alternative energy generation. Determining the coordinates of these building features and storing the data in a correlated manner is facilitated through this process.

[0116] The system contains a reality mesh that captures the state of the building infrastructure at a given point in time, allowing you to replicate the reality mesh with more recent photogrammetry. Once updated, changes to the urban fabric can be detected. This type of detection is made possible, at least in part, by the stored coordinate data. New buildings and expansions will be obvious as they lie beyond the system's core coordinate domain, allowing for automatic identification and updating within the system.

[0117] Predictive analytics The method of the present invention stores transaction data for the purchase, sale, and rental of various types of real estate such as office buildings, condominiums, retail buildings, office and residential apartments, as well as corporate units, hotels, land lease buildings, parking lots, industrial buildings, etc. Each transaction record includes attributes such as, but not limited to, the buyer, seller, cap rate, submarket, broker of either party, sale price, sale date, building class, building area, property dimensions, year built, year renovated, tenant information, vacancy information, hotel occupancy information, operating revenue, operating costs, last sale price, tenure, floodplain risk, curtain wall type, existing debt, existing loan term, loan size, lender, and lender type.

[0118] The system can compile price per square foot for individual transactions, or average price per square foot for a particular type of property in the general market, a specific submarket, or a pool of competing properties.

[0119] By examining trade data and organizing it by attributes, the system of the present invention can identify patterns, correlations, and anomalies, but only if the data is first visualized using the reality mesh coordinate data that is at the core of the system. That is, the system of the present invention can analyze the visualization and draw conclusions. The method of the present invention includes an interface that allows the user to selectively select different trade attributes, cycle through various combinations, and generate visualizations that reveal key factors for determining market valuation. Here are some examples of such economic questions:

[0120] Example 1: Of three office buildings purchased in the same year and sold in 2011, why did building X increase in value by three times while property Y only increased in value by two times? The inventive system provides a user interface that allows attributes of these transactions to be highlighted independently. The system can also draw colored (or otherwise distinguishable) wireframe polygons around the three-dimensional representation of the buildings in the reality mesh to display a second infographic variable in context on the highlighted mesh. By iterating through the visualization of various transaction attributes, the user (or the inventive system) can algorithmically identify which attributes are responsible for the difference in valuation between buildings X and Y. Again, these algorithms can be auto-tuned based on ongoing data collection (e.g., implementing machine learning to modify one or more algorithms).

[0121] When discussing coloring, it is important to recognize that color selection is preferably based on a spectral range reflecting, for example, expected minimum and maximum values, or another stylistic treatment such as opacity or area coloring may be used, similar to the "fog" effect described above. Color selection by range may involve the use of multiple colors, for example, red in the "high" range, blue in the "low" range, and other colors in between.

[0122] Example 2: Identify the properties with the highest sales price per square foot from residential apartment transactions in the same year, and identify concentrations within a particular building, submarket, or city. The system uses the building ID and unit number of the condos traded, and color-codes (or otherwise differentiates) the sales prices with a gradient. The coordinates associated with the selected IDs are projected onto a mesh, and highlighted units reveal patterns. To do.

[0123] Generally, a color gradient is used as the primary indicator to reveal the information, but the buildings and units returned in the query results may be indicated using other screen drawing elements such as lines drawn on the exterior of the building, labels, or graphs such as meters and gauges. For example, the system may float a dollar sign in the space next to the unit and color the dollar sign as an indication of the change in value, coloring it green for an increase in value and red for a loss.

[0124] Example 3: By analyzing the operating costs of various buildings traded in the same year, the user or the system can break this cost down into specific items such as property taxes, property insurance, repairs and maintenance, cleaning and management, payroll and security, heating fuel, electricity, water and sewerage, administrative fees, management fees, etc. The values ​​of each of these variables can be graphed and displayed as a color spectrum and projected with coordinates across the reality mesh to reveal insights and act as an efficient business intelligence module within the system. The user can create subsets of buildings to compare their operating cost costs to determine average costs and identify where savings can be made or reasons for cost differences.

[0125] Example 4: Analyzing the attributes of property transactions with different types of curtain walls is an important comparative physical building feature that can be visualized in the context of the present invention. The present invention maintains a record of the type of non-structural exterior cladding, such as glass panels, metal panels, floor-to-ceiling windows, masonry, or punch-out walls. This is an important factor in the valuation of real estate. Since the database of the present system contains records of real estate transactions resulting from masonry curtain walls that traded at price X, the present system can be used to predict the change in the valuation of a property when the curtain walls are replaced with floor-to-ceiling windows by using the comparative value difference of transactions of similar buildings with floor-to-ceiling windows. By extension, similar valuation change analysis can be performed with other physical attributes such as HVAC equipment upgrades and additions and / or modifications to inherent infrastructure such as life safety, telecom, pools, health clubs, porte-cochere, parking, roof decks, or changes in elevator ratios per square foot of area. Virtually each property type has different attributes that correlate to value.

[0126] The predictive analytics of the present invention can leverage archives of systems' historical leases, rents, hotel room rates, building mechanical systems such as generators, HVAC equipment, boilers, elevator systems, shared conference facilities, etc. This includes office and retail leases, residential apartment rentals, and hotel room reservations. Hotel reservation data can be analyzed by date and presented with time-based animation to identify months with the highest hotel reservation rates per door or room. Hotel reservation data can also be aggregated with data from websites such as Expedia to show current request rates per room for a user-selected date or date range. The data may be presented in real time using a feed from a private hotel booking API. Residential apartment rental data and sales price per square foot for condominiums may be analyzed as well to find seasonal changes in rents. To do this, the system presents the user with a GUI in which hotel booking data in one or more submarkets of a city, for example, can be colored on a gradient scale for each specific hotel room with the exact room (or unit) coordinates used to highlight the reality mesh by the GUI. This interface allows the user to control time-based animations in which highlighting of buildings, floors, and units is turned on and off by playing and pausing the presentation over time based on any of the date values ​​in the database, such as commercial rents, leasing activity, tenant data, residential rents, hotel booking records, or any of the attributes described in the following paragraphs.

[0127] The system analyzes the historical rents achieved for every building and uncovers correlations by comparing the achieved rents with physical and operational attributes. The following list is a partial list of variables that affect a building's rent: views, floor-to-ceiling windows vs. punch-out windows, ceiling height, building amenities (shared meeting rooms, cafeteria, gym, bike storage, conference facilities, parking), building metrics, column spacing, proximity to public transportation, building age, date of last renovation, backup power, rental concession package, tenant improvement allowance, lease terms (lease term, expansion, contract, and termination options), contiguous large vs. small parcel leases, operating expenses, property taxes, general real estate market performance, property performance in a particular submarket, and more. Certain buildings outperform their market and competitors. Often, a combination of several of the above key variables allows the property to achieve very expensive rents. By tracking multiple physical and operational variables, the system identifies those that result in higher rents and allows for greater predictability.

[0128] The analysis module can also use the coordinate highlighting system to project future property valuations based on changes in cap rates. Capacitation rates are defined as the comparative valuation metric of income-generating properties, calculated as net operating income divided by the purchase price, essentially the return an investor receives for the purchase price. This system interface allows users to calibrate the property's capacitation rate at a point in the future and use the coordinate highlighting system as a visual indicator to project the valuation and change in valuation from the date of the last sale. Users can narrow the results by selecting property types by class in specific submarkets and use this analysis to provide an explanatory narrative to audiences exploring cap rate compression and expansion trends in the commercial real estate market.

[0129] The system can present all or a preselection of the application data visualizations, including all analytical visualizations, as a "canned" automated presentation based on the user's role, and play these slides along with current data to the user at regular intervals. For example, an office leasing representative, as a user of the system, can play slides related to changes in the office leasing market in a submarket or across a city to quickly gain knowledge of major changes in the market. Similarly, a residential rental agent can play slides related to current rental activity in a graphical user interface to see what data is affecting current rental rates, vacancy levels, and current market inventory. These visualizations, displayed in slide format, can be distributed by users of the system as unique URLs or sent via email or text message, allowing time-efficient communication of market conditions to clients and colleagues.

[0130] Data Source The system consumes data from three main sources (A, B, and C below), but preferably does not utilise traditional GIS data available in its raw form from government and private sources, such as shapefiles, cadastral maps, road maps etc. The invention synchronises and stores the data it receives. The only external data source used to identify properties are municipally provided property and building footprint files containing unique identifiers, however the process works independently of government or third party provided property boundary data.

[0131] The synchronization process begins by importing tables of spatial and physical property attributes from city, county, state, or federal sources, and using them to associate volumetric regions in the Reality Mesh with a government-universal identification number (ID). This ID is used to construct database queries against public record sets. For example, once a government-specific property identifier code is associated with a volumetric region in the Reality Mesh, that region can be stylized, manipulated, hidden, or highlighted to view data in context or to visualize empty spaces. They can be used to correlate interrelatedly or thematically as in charts and graphs. Additionally, text symbols can be drawn in close proximity to relevant areas of the mesh to form the impression of a static or animatable meter displaying data including energy or water consumption, time variables such as emissions and tailpipe emissions, or financial information such as mortgage details, debt, net operating income, tenancies, etc. See for example Figure 21. This technique quickly improves the information conveying capabilities of a system.

[0132] In FIG. 21, there is reference to the building centroid, which is defined as the coordinate point approximately at the center of a floor of the building. This coordinate point can be derived from the adjusted building footprint or a coordinate selection from the coordinate bed of FIG. 1. The meter shown in FIG. 21 is therefore depicted relative to the building centroid, allowing for the correct visual context of the meter revealing property information. The meter can represent a scale of data such as energy, water, CO2 equivalent emissions, debt levels, or any property variable that is best understood depicted with a range of the color spectrum.

[0133] A) Traditional GIS files, manually adjusted Every unique building footprint has its vertices (coordinate points) adjusted to fit the mesh. This is done by adjusting the source coordinates to match the visual representation of the property, including the land boundaries, in the reality mesh. This is accomplished by the updated elevation of the polygon control surface changing to capture the updated coordinates as the user changes the location parameters, as shown in Figure 22. It is important to note that government footprint GIS data is typically based on cadastral maps and is not an accurate representation of real world coordinates. The photogrammetric model of the system, when calibrated on a Cartesian map, becomes an accurate real world data set and accurate coordinates can be extracted in the real world. This difference requires that the vertices of every government GIS footprint be adjusted slightly to match the photogrammetric mesh. This process is performed manually using various software tools that allow the user to select a vertex or edge and adjust its latitude, longitude, elevation, or any combination of those parameters. This step is required at the floor level, making it possible to support buildings with different floor to ceiling heights across different floor ranges. This manual process is required for every floor of every building in the system and an example of the software tools used in this process is shown in Figure 22.

[0134] B) Manually selected mesh coordinates The system of the present invention consumes selected coordinates directly from the reality mesh to create property coordinates for new properties and new or renovated buildings that have been constructed but do not exist in government records. The system can capture any coordinate on the mesh from the chosen selection and associate the selection with a new system building ID. A minimum of three unique coordinate pairs are required to properly assemble a 2D polygon that matches the reality mesh property representation. This same technique is used to classify building mechanical equipment such as rooftop potable water tanks, cooling towers, HVAC and air handling equipment, cellular and wireless communication infrastructure, security cameras, solar panels and solar panel locations, ventilation and exhaust systems, and elevator systems. Architectural and landscape features such as building entrances, security zones, parking, rooftop gardens, facade safety, and site landscaping can also be identified by the selected mesh coordinates and associated with the property's unique identifier.

[0135] C) Physical Property Information The system of the present invention consumes various types of documents and associates them with a system-specific property identifier. Examples of document types: Structured Tables, Microsoft Excel Files, CSV Files, Arg us model, text document, PDF file, JPEG image, PNG image, floor plan layout outfiles, and other metadata necessary or relevant to real estate transactions and property management. Examples of document content include:

[0136] Commercial and residential leases and subleases, property budgets and operating expenses, legal structures of property ownership (e.g. condominiums, cooperatives, cooperatives, freehold), building covenants, condominium plans, rent rolls, mortgage documents, financial records and contracts, deeds, tax records, development rights or air rights, market property transactions, certificates of occupancy, insurance documents, municipal assessments, environmental assessment reports, flood and other natural risk data collected for insurance purposes, planning laws, planning policies, city laws, city policies, public health surveys, air quality surveys, and other publicly or privately available documents containing information related to the real estate properties in the system.

[0137] The system of the present invention can create related documents that can be displayed similarly. Each document is associated with a property using various software tools. These documents are assimilated into the system as tables of data or stored in a database as binary large objects and associated with a unique property identifier. Some documents are also stored in the file system or database of the present invention in folder names that include the unique property identifier from the system.

[0138] Also, to highlight the floors correctly, the system needs to know the elevation of the top habitable floor of the building. This is done using the system's Building Elevation form, which allows each floor to have a unique floor height so it can match the building properly. To do this, it can use a variety of sources, and can also combine several overhead and public and private sources, look at facades and window spacing.

[0139] This information is refined as the system receives slab height measurements from the owner.

[0140] Residential and commercial dwelling units can be created using the "Create Partials" creation process. These are created in the system's augmented reality approach from floor coordinates using the 3D mesh. These are forms that allow you to select external points on the façade where the units are divided along the floors. The forms are also used to map rasterized floor plans to full or partial floors, clipping an area of ​​the mesh above the target unit to reveal the floor plan, allowing the system to display the floor plan in situ.

[0141] These coordinates come from traditional cadastral maps that record the dimensions and locations of land parcels, so manual calibration may be required to fit the buildings into the reality mesh. However, these coordinates may not match reality exactly, requiring the user to adjust the vertices to perfectly match the photogrammetry reality mesh.

[0142] The system includes the logic necessary for users to select the coordinates of rooftop infrastructure (cooling towers, water tanks, cell phone equipment, solar panels, etc.) from the system's core footprint dataset and associate them with the system's database. The system already has a database relationship to the building ID and the city-issued unique property ID, and can use available city open data to stylize the highlighting applied to the infrastructure in question. For example, a water tank that failed a sanitary inspection would be highlighted in red.

[0143] There are several ways that a user can select a parameter (or parameters) to display. In addition to clickable inputs as part of the graphical user interface, there are also clickable menus in the relevant parts of the graphical user interface. may be displayed, and the menus may change based on previous selections or may be customized for the type of user (e.g., real estate professional vs. general consumer). Additionally, such selections may be made by voice to speed up the process. Affirmative responses to selections may be color coded or otherwise indicated based on attributes of the selection (such as price range or greenhouse gas emission range).

[0144] Additionally, the GUI's info display box (info box) presented to the user when clicking on a highlighted security camera or rooftop infrastructure is connected with a thin red line connecting the centroid (center spatial coordinate) of the selected infrastructure, drawn on the screen in the context of the application interface, to the corner of the GUI's info box. This line remains fixed to the on-screen info box while the camera of the virtual globe (which may be a traditional satellite map or a rasterized city or road map, also known as a base map, projected onto a spherical user interface) moves, maintaining a direct index between the info box and the location of the reality mesh containing the selected infrastructure. This is useful because it creates an immediately unambiguous description of equipment information, status, and geographic location using current government data.

[0145] Our coding approach: The system uses a traditional LAMP stack (Linux, Apache, MySQL, PHP) web services that uses open source code for the operating system (Linux, GPL), web server (Apache, Apache 2.0 license), database (MySQL, GPL) and virtual globe (CesiumJS, Apache 2.0). There are various open source virtual globes available, including NASA World Wind, osgEarth, osimPlanet, CesiumJS, gvSIG 3D and KDE Marble. The system currently uses the open source CesiumJS due to its versatility with different reality mesh tileset formats.

[0146] All of the middleware or application level code used is proprietary. Open source code is used for some of the graphical user interface components, and for the virtual globe, which is licensed under Apache 2.0. While there are no proprietary modifications to the hosting service, the file system, or the Linux OS, the system's database schema contains a lot of proprietary design and information.

[0147] Figure 23 is an example of the data workflow in a typical user session. On the front-end client side of the application, the user is presented with a graphical user interface that is a reality mesh of the city area. Typically the first filter of interest is to narrow down the area, so submarkets are presented to the user and the user selects. A query is sent to the database and the values ​​needed to position the camera in the GIS or virtual globe are selected. These values ​​include longitude, latitude, altitude, heading, tilt, pitch, and roll. The values ​​returned from the DB to the GUI move the camera so that it is positioned over the correct submarket; for example, the Financial District, and the camera is focused on the area northeast of Battery Park in Manhattan.

[0148] The next step in this illustration is the selection of A class buildings. This search is composed as a query and sent to the database to return the coordinates of all A class buildings. The returned data consists of a large string of coordinates representing the spatial boundaries of the property in the reality mesh. The user sees all A class buildings highlighted in the pre-selected color.

[0149] Another selection is made from the previous result set of A-class buildings to request spaces for rent or available for rent within the buildings. This query uses volumetric highlighting techniques. It is used to return floor-level coordinates that are stylized to accurately highlight complete or partial floors or units of a mesh.

[0150] FIG. 24 illustrates the customization of the application infrastructure of the present invention.

[0151] The diagram shows the level of customization required for each infrastructure component of the application. Complete indicates that the component has been fully created for the system, while various other patterns indicate that less customization effort is required to set up the system.

[0152] Basemaps (traditional satellite, city, and road maps overlaid on the GIS virtual sphere) include public and private sources of terrain imagery, excluding the mesh. These are not used by the application, but provide location information for the mesh from elevated locations. For example, the system's example reality mesh covers downtown Manhattan, from Canal Street to the Battery. The area north of Canal Street is represented by the basemap, within which the reality mesh is spatially located. Public imagery providers for basemaps include NASA and OpenStreetMap. Applications Commercial imagery providers that can be accessed via a programming interface (API) key include Esri, Bing, and Mapbox.

[0153] Figure 25 shows an application infrastructure for the web. It shows how the application is configured to run in a standard web hosting design on remote web servers located in a virtualized cloud environment (such as AWS) accessible via the public internet. The virtualized servers run on Linux and The system has a file system, a database, and a web server, each of which is a necessary component for the application to run. The file system contains reality mesh tileset files, virtual globe mapping software, and customized JavaScript ( It includes HTML5, WebGL, and php files. All supported web browsers support the common HTML5 WebGL standard. The virtualization server also includes the open source MySQL database and the open source web server software Apache.

[0154] Figure 26 shows the application infrastructure iPad Standalone. This shows an application configured to run on an iPad as a packaged application without an Internet connection. This is done using Apple's XCode programming language. Applications are used to create packages for the graphical user interface, reality mesh, and web-delivered versions of the data of a system. This diagram shows how packages are built in XCode, not as standalone applications. Here is how the app is structured. The package is described by the item in the largest box. Applications must be sideloaded (transferred directly from the PC to the tablet via cable, rather than distributed through a cloud-based application store) to the tablet due to Mesh's large file size, which runs into limitations created by online application distribution methods such as Google Play or Apple's Appstore.

[0155] The standalone application uses the same JavaScript / PHP front-end client, as well as the CesiumJS open source virtual globe and web server. However, instead of a local connection to a database, the application uses a client-side JSON data file that contains all the possible coordinate data needed to generate the visualization within the Reality Mesh. The tileset file for the Reality Mesh is also saved as part of the package, which results in faster load times for the mesh since it can't be bottlenecked by a slow internet connection.

[0156] Figure 27: Fog effect This illustration shows a selection of submarkets in Lower Manhattan, with a fog effect applied to the non-selected areas. This makes the highlighted building in the selected market (World Trade Center) more visible, helping to understand the visualization. Visually " The "fog" effect is applied to the boundaries of each submarket in the reality mesh. This is created by using the volume highlighting function.

[0157] Mist grouping distortion solution To solve the problem of visual distortion encountered by GIS software, which disrupts multiple hole punches in the fog highlight (used to highlight search results) and creates polygon distortions that are represented as stray vectors drawn on the horizon, a solution is included in the system. This problem occurs when two or more buildings are next to each other in real space and share boundary coordinates. The solution is to create a set of matrices that combine all combinations of building coordinates that are involved in the distortion. For example, if three buildings (A, B, C) overlap, a matrix is ​​created with their respective coordinate combinations (AB, BC, CA, ABC). These are only the four possible combinations of buildings that require holes in the fog highlight for the three buildings. Using this matrix, the system can combine the boundaries of building A and building B, which can be used to define the holes. Using this method, the GIS is able to eliminate distortions. This technique is used manually in the code when distortions are seen in the GIS. If the combination matrices AB and BA are the same, their results are the same even if the keys (AB, BA, AC, CA, BC, CB) are different. Therefore, these combinations are not used again. The system includes additional ordering logic to avoid duplication of combination matrices. For example, if buildings with unique identifiers 12, 13, and 14 are used for fog highlighting, the system always sorts the buildings with unique identifiers in ascending order and checks for available matrices. This duplicate matrix data stored in the database also has the buildings organized in this way. This logic makes matching easy.

[0158] This functionality is important because it allows the system to visualize data about multiple buildings while obscuring the rest of the reality mesh with fog. Figure 28 shows an example of how this feature is used by the user interface. The orange and yellow buildings, which represent condominiums or rental type buildings, are punched through the fog for emphasis.

[0159] Inverse Clipping Logic With the defined coordinate plane, the system flips the plane orientation and switches the viewpoint to the opposite plane to clip all parts of the mesh except the target building. This allows the system to isolate a single building or block from the entire city mesh. The GIS then changes the mesh position values, but this needs to be done manually for every building to achieve this visualization.

[0160] This feature is important because it allows building-specific 3D stacking plans to be generated without neighboring buildings blocking views of the target property. Figure 29 shows how the user interface uses this feature to display available rental units in a building in Lower Manhattan.

[0161] Setting parts according to the floor plan Manually by the system to highlight suite locations on buildings in reality meshes The partial suite coordinates created in must align with the dimensions of the floor plan image file (PDF or rasterized image) when the mesh is clipped to a particular floor and the image file is drawn within the boundaries of the partial suite coordinates. Drawing in this manner, using only an initial set of partial coordinates captured from the exterior (outside the building in the mesh), can cause the GIS to exhibit alignment issues with the rendered floor plan image file. For example, the image file can be drawn beyond the boundaries of the partial coordinates and extend into space from the building in the mesh. This solution requires the manual effort of selecting another set of mesh coordinates from inside the mesh at the location of the clipped floor to accurately match the floor plan. The system solves this problem by providing a tool where a user-selected (or semi-automated or fully-automated) building clipped to a selected floor allows the user to align the floor plan file to an "anchor" coordinate that runs along the perimeter of the building in the mesh, and then select new interior coordinates that match the dimensions of the floor plan. This is a laborious task (more so for partial floor plans than for full floor plans) that requires resizing and rotating the image files to match the building floor parameters and the parameters of neighboring floor plans next to the unit or on the other side of the same floor. Once these coordinates are captured and stored within the system, it provides the user interface with the data necessary to draw the floors in place on the cut (or sliced) floors, as well as floating lines and polygon fences (later in this document) in the context of the floors and suites within the building of the reality mesh.

[0162] Figure 30 shows the tools the system uses to place floor plans onto clipped floors within buildings in the reality mesh. The small yellow dots act as control points for adjusting the image.

[0163] Automatic Display Logic The system can generate views from a particular suite or unit by calculating the building centroid and the partial suite centroid. A direction vector is determined by the location of the partial suite centroid relative to the building centroid on an orthogonal plane.

[0164] Many times in this document there is mention of a camera, which refers to how the end user sees an object (building, reality mesh, sliced ​​floor, etc.) in terms of angle, height, distance and rotation. References to "camera direction" in the following paragraphs define the direction in which a person standing at point A looks at point B, and can be understood as a vector representation of a line from point A to point B.

[0165] Panoramic views are also generated by the system in a similar manner to suite or unit views. The system uses the building centroid and partial centroids to calculate the camera direction. To realize the panoramic camera movement, a path is required for the camera to move. The stored coordinates of the partial units are used to create this path, and using the GIS clock, the system defines the clock tick events that move the camera from point to point giving the panoramic view. To calculate the points, the distance between the building centroid and the partial suite centroid is defined as the threshold distance, and any other partial coordinates whose distance from the building centroid is greater than the threshold are utilized for the panoramic viewpoint. As explained in Figure 31, the black square outline represents the building footprint and the blue lines represent the partial unit coordinates. The green coordinates indicate the points whose distance is greater than the previously calculated threshold.

[0166] This feature is very important because it gives the user a visual experience of seeing the real world view from a floor or unit through the streetscape reality mesh. It also allows the user interface to quickly navigate to a view without the user having to navigate using GIS controls. The view from a unit in a building is a valuable indicator of the commercial value that the unit can fetch in the market. This is an important feature for understanding the economic viability of a commercial or residential space for sale or rental as it impacts rental prices.

[0167] Elevator Core Using the stored floor coordinates and building height, the system of the present invention has the necessary parameters to create a polygon shape that approximates the dimensions of the building's elevator core and elevator cars. This visualization can be presented in a user interface with an opacity value cast onto the reality mesh so that the elevator core is visible through the transparent building in the mesh. Figure 32 illustrates this concept with a white opaque polygon drawn within a building in Lower Manhattan.

[0168] City data can include maintenance, permitting, and safety information related to elevator equipment, so this visualization can be useful for identifying maintenance issues or browsing through large volumes of specific hardware types or elevator equipment manufacturers to find specific information.

[0169] Floating Line The system allows partial unit coordinates to be manually set, which can then be used to draw polygons in space above where floors have been cut (cleared) from the reality mesh. Displaying units in this way allows the user to get a sense of the unit information while looking at, for example, a floor plan of a selected unit in a residential or commercial building. The user can then select other units based on the shape or elevation of the other units' floor plans without having to redraw the mesh and potentially lose spatial context. This improves awareness of the types of units being returned or drawn in a search query, because the user can visually filter smaller or larger units that may or may not be of interest. Figure 33 shows how these "floating lines" are used by the user interface to present this information. The floating lines can be stylized using colored solid or transparent polygons, and include floor plan image files (PDF or rasterized images) within these boundaries.

[0170] A polygonal fence surrounding some of the residences Similar to floating line logic, a polygon fence can be drawn upward from the plane or surface of the clipped building floor in the mesh using fractional unit coordinates (stored in the system through a process that defines the fractional units of the floor.) This polygon fence shows floor divisions by unit and is displayed as a vertically filled or outlined wireframe colored by a number of user-selected variables, such as number of bedrooms, rent, time on sale, etc. Polygonal fences can also indicate the interior wall materials of adjacent suites, as well as exterior window types (punch-out, floor-to-ceiling).

[0171] Adding rooftop infrastructure The system features a manual coordinate selection tool that is used to identify, describe, and store types of rooftop building equipment. For example, a user can select the coordinate boundary of the perimeter of a cooling tower roof, add a description, and store that real-world infrastructure item in the system's database with a unique ID. That ID can be bound to a query between government permits and maintenance records for the selected cooling tower in the associated building, and the user interface can visualize the cooling tower in a reality mesh using any of the available parameters (i.e., registration date, capacity, make, model, intended use) as styling variables. Supported infrastructure types include air handling units, cellular antennas, chillers, cooling towers, green roofs, rooftop units, solar panels, water towers, cranes, HVAC equipment, and more. Styling options include highlight colorization or polygonal lines. Clicking on the The user interface then displays an infobox displaying the equipment's fields and descriptions, and draws a line connecting the exact location of the infrastructure using the centroid of the stored coordinates.

[0172] Figure 34 shows three rooftop infrastructures and their infoboxes detailing them (cell phone antennas, cooling towers, and water towers).

[0173] Automatic presentation output The system comes with preselected slides that provide a series of visualizations that provide overviews and analyses of the office rental and sales market, residential rental and sales market, development market, city government open data, tenant industry concentration, and more. These automated presentations can be selected from a menu of options in the user interface or sent (emailed) to the user as compartmentalized URLs that play immediately when opened in a web browser. The data in the presentation slides reflects current government, commercial, real estate, and user data stored in the system database. Each slide plays at user-set intervals and includes a timer that appears before the next slide is played. Each slide can be paused and interacted with if it contains nested features such as a building camera control, a line indicator (as seen in the Cellular Tower Maintenance example in Figure 35, which shows a recent cellular maintenance request from T-Mobile Northeast LLC with a line connecting all the buildings), or a building information box (a window in the user interface).

[0174] While the invention has been described in detail in connection with only a limited number of embodiments, it should be readily understood that the invention is not limited to the above description.

Claims

1. 1. A method for generating and displaying a three-dimensional interactively augmented reality mesh visualization of a particular user-controllable geographic area using a processor-driven system, the system including one or more processors, at least one database, a graphical user interface (GUI), and a control technology engine for controlling the GUI; accessing an augmented reality mesh that includes data representing a geographic region and buildings having known real-world coordinate locations, the augmented reality mesh including exterior images of structures in the geographic region and mesh coordinates located at the real-world coordinate locations of the buildings, the augmented reality mesh being usable to generate a three-dimensional virtual representation of the geographic region; receiving legacy coordinates of a particular building and data identifying the particular building in the geographic region along with a file further including a first attribute of the particular building as metadata; composing data from said at least one database and said file; comparing the legacy coordinates of the particular building exterior represented by the existing data in the augmented reality mesh to identify errors in the legacy coordinates; providing updated coordinates that match the appearance of the particular building in the augmented reality mesh and adjusting the legacy coordinates to correct the identified errors; inputting said at least one database with updated coordinates of said particular building; selecting the geographic region for display; identifying content of said at least one database representing said geographic area and its surroundings, buildings including said particular building, and related building elements in said geographic area including coordinates in said geographic area; placing the content identified in the at least one database corresponding to the geographic region into fields to generate a data file; inputting a selection list into the GUI with the fields of content identified for selection, including a particular field associated with the first attribute; displaying in the GUI a visualized augmented reality mesh of the selected geographic area with one or more overlays for selected fields including the particular field, color coded based on numerical values ​​in the fields in response to receiving a selection of one or more fields and the selected geographic area, the overlays for the selected fields being positioned according to updated coordinates; Providing a user under control of said GUI with the opportunity to adjust a visualized reality mesh for a region or field; The method includes:

2. The legacy coordinates include a set of legacy coordinates defining a boundary; The step of adjusting the legacy coordinates includes: identifying a plurality of coordinates defined by the set of legacy coordinates that do not match an appearance of the particular building in an augmented reality mesh; removing at least one coordinate and replacing it with new coordinates that match the exterior appearance of the building to create a set of updated legacy coordinates; and triangulating the updated legacy coordinates to remove non-contiguous coordinates.

3. The step of adjusting the legacy coordinates includes: displaying in the GUI a visualized augmented reality mesh of an area including the particular building and a legacy overlay of the particular building having a location defined by the legacy coordinates; 2. The method of claim 1, further comprising: presenting an interface in the GUI to enable a user to select vertices or edges of the overlay and adjust the position of the overlay in three-dimensional coordinates to match the appearance of the particular building in the augmented reality mesh.

4. Preserving the legacy coordinates; 2. The method of claim 1, further comprising providing an opportunity for a user under control of the GUI to adjust the visualized reality mesh to toggle the display of the overlay for a selected field between a location defined by updated coordinates and a location defined by the original legacy coordinates.

5. 2. The method of claim 1, wherein the reality mesh has a resolution of 2 cm or greater.

6. 2. The method of claim 1, wherein the color coding includes shading based on the value of data in at least one of the fields and highlighting one or more particular regions of interest.

7. 2. The method of claim 1, wherein the selection list is established based on selected fields or fields stored in the at least one database.

8. 2. The method of claim 1, wherein said at least one database is continuously populated with said data from a plurality of sources in communication with said processor.

9. 2. The method of claim 1, wherein the at least one database includes fields related to building structure, contents, price history, and interior amenities.

10. 1. A method for a processor to formulate executable data files for formulating a multi-dimensional visualization on a graphical user interface (GUI) that is adjustable in orientation and content based on user input using a processor-driven system, the system including at least one database, a GUI, and a control technology engine that controls the GUI, the method comprising: providing access to an augmented reality mesh that includes data representing a geographic region and buildings having known real-world coordinate locations, the augmented reality mesh including exterior images of structures in the geographic region and mesh coordinates located at the real-world coordinate locations of the buildings, the augmented reality mesh being usable to generate a three-dimensional virtual representation of the geographic region; providing access to a database populated with data identifying a particular building in the geographic region, legacy coordinates of the particular building, metadata associated with the legacy coordinates including metadata for a first attribute of the particular building of the building provided in the legacy coordinates, and updated coordinates for the particular building that are an adjusted version of the calibrated legacy coordinates that match an appearance of the particular building in an augmented reality mesh; Identifying in a database a selected geographic area including buildings, related building elements in the geographic area, real estate having coordinates in the geographic area, together with one or more units for display; selecting, in said at least one database, all fields related to said geographic region and its vicinity, including a particular field related to said first attribute, for delivery to an executable file; forming a structured data file based on the user's device to provide a display to the user; A method according to claim 1, wherein a user may select one or more fields in a GUI, and the file is configured to present a visualized augmented reality mesh in a GUI display of the selected area with one or more overlays for the selected fields, the overlays being color coded based on the numerical values ​​of the fields, the selected fields having a particular field, and the overlays for the selected fields being positioned according to updated coordinates.

11. The method of claim 10, further comprising providing a user under control of the GUI with the opportunity to adjust the visualized reality mesh to toggle the display of the overlay for a selected field between a position defined by updated coordinates and a position defined by the original legacy coordinates.

12. 11. The method of claim 10, wherein said at least one database is continuously populated with data from a plurality of sources in communication with said processor.

13. 11. The method of claim 10, wherein the processor polls the data source on a programmable schedule.

14. 11. The method of claim 10, wherein the at least one database includes one or more fields that detail facilities on a property-by-property basis.

15. 11. The method of claim 10, wherein the processor executes the data file each time a user selects a GUI.

16. The method of claim 10 , wherein the GUI includes options for zooming in and zooming out of a display area.

17. 1. A method for generating a color-coded visualization of geographic locations based on data collected continuously from at least public sources and stored in at least one database using a processor-driven system, the system including one or more processors, the at least one database, a graphical user interface (GUI), and a control technology engine for controlling the GUI, the method comprising: providing access to an augmented reality mesh that includes data representing a geographic region and buildings having known real-world coordinate locations, the augmented reality mesh including exterior images of structures in the geographic region and mesh coordinates located at the real-world coordinate locations of the buildings, the augmented reality mesh being usable to generate a three-dimensional virtual representation of the geographic region; providing access to a database populated with data identifying a particular building in the geographic region, legacy coordinates of the particular building, metadata associated with the legacy coordinates including metadata for a first attribute of the particular building of the building provided in the legacy coordinates, and updated coordinates for the particular building that are an adjusted version of the calibrated legacy coordinates that match an appearance of the particular building in an augmented reality mesh; Selecting a geographic region for display; identifying at least one of said database fields and contents, including a particular field related to said first attribute representing said geographic area and its surroundings together with buildings including said particular building and associated building elements in said geographic area including coordinates in said geographic area; forming a data file for inputting a GUI display including the identified content; inputting a selection list into the GUI with fields of content identified for selection, including the identified field; determining a data range for each field for color coding in response to selecting one or more fields; displaying in the GUI a visualized augmented reality mesh of the selected geographic area with one or more overlays for selected fields having certain fields color coded based on the numerical values ​​in the fields, the overlays for the selected fields being positioned according to updated coordinates; Providing a user under control of said GUI with the opportunity to adjust said visualized reality mesh for an area or field; The method includes:

18. The method of claim 17, further comprising providing a user under control of the GUI with the opportunity to adjust the visualized reality mesh to toggle the display of the overlay for a selected field between a position defined by updated coordinates and a position defined by the original legacy coordinates.

19. 20. The method of claim 17, wherein the color coding is configured to potentially include shade, translucency, and haze based on the identified property interest.

20. 20. The method of claim 17, wherein the reality mesh has a resolution of 2 cm or greater.

21. 20. The method of claim 17, wherein the color coding includes value-based shading, semi-transparency, haze, etc. to highlight one or more particular regions of interest.

22. 20. The method of claim 17, wherein the color coding is based on historical financial data.

23. 20. The method of claim 17, wherein the color coding is based on financial information projected onto a reality mesh.

Images