Computer system, computer-implemented process, and computer program product for animation, simulation, or generating instructions for constructing a building

Abstract

Animation, simulation, or generating instructions of actions related to constructing a building involves building a complex model. This model (128) is parameterized to enable animation or simulation of positions, orientations, and presence of components in a building and of tasks, sequences of tasks, dependency relationships, or other information related to the actions to be animated or simulated or for which instructions are generated. The model is input to a platform (150) which performs and visualizes complex simulations and animations of the model, and generates instructions (172) for actions related to constructing the building. By integrating a parameterized model of a building and related data, which incorporates data about the three-dimensional structure of components of a building with information about tasks and other information associated with those components, into a platform that implements complex multi- dimensional simulations and animation, a computer system supports performing many actions related to constructing a building.

Description

COMPUTER SYSTEM, COMPUTER-IMPLEMENTED PROCESS, AND COMPUTER PROGRAM PRODUCT FOR ANIMATION, SIMULATION, OR GENERATING INSTRUCTIONS FOR CONSTRUCTING A BUILDINGBACKGROUND

[0001] Current technology in the building industry primarily focuses on the structure, design, and performance of buildings. Computer-aided design (CAD) tools, Building Information Modeling (BIM) software, and other architectural design and engineering programs allow buildings to be conceptualized and evaluated. These tools allow architects and engineers to create detailed three-dimensional models of buildings, incorporating not only geometric information about structural components but also data on materials, systems, and fixtures. Conventional CAD / BIM files primarily encode a finished, designed state of a building rather than the construction process itself. Current tools generally lack support for process-based evaluation, planning, optimization, animation, or simulation of the process of constructing a building.SUMMARY

[0002] This Summary introduces selected concepts in a simplified form that are described below in the Detailed Description. This Summary neither identifies key or essential features nor limits the scope of the claimed subject matter.

[0003] While significant advancements have been made in the use of technology in the design phase for buildings, less attention has been given to the process of constructing buildings. The transition from design of a building to constructing the building often involves manual interpretation of plans, on-site decision-making, and ad hoc problem-solving. This gap between designing a building and constructing the building can lead to inefficiencies, errors, and increased costs. Additionally, current technologies typically do not provide comprehensive and integrated solutions for identifying viable or optimized material usage, workforce planning, or dynamic adaptation to on-site conditions while constructing a building.

[0004] A computer system addresses these problems by implementing operations for animating, simulating, evaluating, optimizing, or generating instructions for actions related to constructing a building. To support such operations, the computer system generates a complex model of objects representing components of the building. This complex model is parameterized and labeled to enable animation or simulation of components in a building and of tasks related to the components of the building. The parameterized and labeled model is input to a platform which performs and visualizes complex simulations and animations of the model. By integrating acomplex parameterized and labeled model of a building, which incorporates data about the three- dimensional structure of components of a building with information about tasks and other information associated with those components, into a platform that implements complex multidimensional simulations and animation, a computer system can provide the various features explained herein.

[0005] One of the technical challenges in creating such a parameterized and labeled model of a building is data integration and representation. Software tools that are designed for animation and simulation of three-dimensional models are typically different from the software tools used for architectural design and engineering support, and thus typically do not represent the same data or do not represent data in the same way. Further, additional data used to help plan actions related to constructing a building originates from numerous, diverse sources. Finally, information useful for planning actions related to construction of a building may be present in the building files or additional data, but often is not organized in a manner that supports computer-based inference.

[0006] Addressing these challenges includes mapping elements in a building file or additional data, based on a schema for data used in the building file or additional data, to elements in a schema for data used by the animation and simulation tool. Elements in the building file or additional data which do not map directly to the schema used by the animation and simulation tool are captured in additional data structures which are then associated with the three- dimensional model. This information can be further processed to generate a building-specific knowledge graph representation of the building, by capturing dependencies and other relationships among objects, to assist in inference about planning construction of the building. Additionally, expertise knowledge graphs can be trained based on external information about building construction to capture general expertise which is useful for inference. The resulting parameterized model thus includes a three-dimensional model of objects with some properties specified as variable parameters, associated data structures capturing additional data related to the objects, and an associated building knowledge graph. Expertise knowledge graphs are further created.

[0007] Accordingly, in general, one aspect may provide a model parameterization module that can generate parameterized models from building files and additional data. In this aspect, a computer system includes a processing system with at least one processing device and computer storage containing computer program instructions. The system includes a model parameterization module that accesses at least a building file describing a building and additional data, and generates a parameterized and labeled model based on at least the building file and the additional data. The parameterized and labeled model includes at least data representing three-dimensional objectsrepresenting components of a building, where at least some information associated with the three- dimensional objects includes data specified as a variable parameter that can be assigned in an animation or simulation. The model parameterization module further extracts and preserves geometric representations of the building components, including their physical shape, dimensions, and spatial positioning, so that the resulting parameterized and labeled model maintains both semantic attributes and geometric fidelity for integration with USD-based animation and simulation. The computer storage is configured to allow access to the parameterized and labeled model for use by an animation and simulation module for animating or simulating at least one action related to constructing the building.

[0008] In general, one aspect may provide a model parameterization module that can generate parameterized models in which building components are represented by three-dimensional objects associated with data representing component fabrication tasks. In one aspect, a computer system includes a processing system with at least one processing device and computer storage containing computer program instructions. The system includes a model parameterization module that receives design data from architectural software, including a three-dimensional model of a structure of a building, and translates the received design data into a parameterized and labeled model of the building, where building components are represented by three- dimensional objects associated with data representing component fabrication tasks.

[0009] In general, one aspect may provide a computer-implemented process that receives a three-dimensional building model, parameterizes it with tasks and constraints, and simulates construction in a virtual environment. In this aspect, a computer-implemented process includes receiving a three-dimensional building model from architectural software, parameterizing the building model by associating tasks, materials, dependencies, or installation constraints with components of the building, and simulating assembly of the building in a virtual environment. The simulation includes one or more of building component preparation, material transportation, on-site logistics, assembly sequence, assembly methods, compliance with engineering standards, building codes, or environmental sustainability requirements, generating a cut list or bill of materials for components of the building, providing machine-readable instructions to automated cutting devices, generating human-readable component preparation or assembly instructions for human workers, or updating those instructions based on on-site sensor data or simulation feedback.

[0010] In general, one aspect may provide a computer system that transforms a three- dimensional building model into a parameterized model, simulates construction, and generates instructions based on feedback. In this aspect, a computer system includes a processing system with at least one processing device and computer storage containing computer programinstructions that configure the processing system to transform architectural design data into a three-dimensional model, parameterized with tasks, material properties, and dependencies. The system simulates constructing the building using the three-dimensional model including optimizing sequences of tasks and representations of assembly. The system translates the simulated constructing of the building into human-readable or machine-readable instructions and updates the simulation and instructions based on feedback from sensors monitoring constructing of the building.

[0011] In general, one aspect may provide an animation and simulation module and an instruction generation module. In this aspect, a computer system includes a processing system with at least one processing device and computer storage containing computer program instructions. The system includes an animation and simulation module having inputs receiving a parameterized and labeled model, including data representing three-dimensional objects representing components of a building, where at least some information associated with the three-dimensional objects is specified as a variable parameter that can be assigned in an animation or simulation, and having outputs providing data representing results of an animation or simulation of the parameterized and labeled model. The system includes an instruction generation module that accesses at least a parameterized and labeled model including data representing three-dimensional objects representing components of a building, where at least some information associated with the three-dimensional objects is specified as a variable parameter that can be assigned in an animation or simulation.

[0012] In general, this aspect may provide a control interface that can generate scripts for animations or simulations of construction actions using a parameterized and labeled model. In one aspect, a computer system includes a processing system with at least one processing device and computer storage containing computer program instructions. The system includes a control interface that accesses data representing actions relating to constructing a building and generates scripts that define instructions for positioning, sequencing, dependencies, or actions relating to three-dimensional objects within an animation or simulation of a parameterized and labeled model over time. The computer storage allows access to the scripts for use by an animation and simulation module to execute an animation or simulation of the parameterized and labeled model for animating or simulating at least one action related to constructing a building.

[0013] In general, this aspect may provide an instruction generation module that can use a parameterized and labeled model to produce instructions for actions related to constructing the building. In one aspect, a computer system includes a processing system with at least one processing device and computer storage containing computer program instructions. The system includes an instruction generation module that accesses at least a parameterized and labeledmodel including data describing three-dimensional objects representing components of a building, where at least some information associated with the three-dimensional objects is specified as a variable parameter that can be assigned in an animation or simulation. The instruction generation module generates instructions for an action related to constructing the building. The computer storage allows access to the instructions for presenting on a presentation device.

[0014] In general, this aspect may provide a comparison module that can evaluate differences between simulated construction actions and sensor data from performed actions. In one aspect, a computer system includes a processing system with at least one processing device and computer storage containing computer program instructions. The system includes a comparison module that accesses input data representing an animation or simulation of an action related to constructing a building and inputs from on-site sensors capturing performance of the action related to constructing a building. The comparison module generates a feedback output comprising data representing a comparison of the animation or simulation to the performed action based on the input data.

[0015] In general, this aspect may provide an optimization module that can select construction parameters based on optimization goals and results from simulations. In one aspect, a computer system includes a processing system with at least one processing device and computer storage containing computer program instructions. The system includes an optimization module that accesses data representing one or more optimization goals for one or more actions related to constructing a building, receives data relating to an animation or simulation of a parameterized and labeled model for the action related to constructing the building, and selects and outputs parameters for the action related to constructing the building based on the optimization goals and the data relating to the animation or simulation.

[0016] In general, this aspect may provide a control interface and an animation and simulation module, where the control interface can specify variable values over time and the module can execute simulations based on those scripts. In one aspect, a computer system includes a processing system with at least one processing device and computer storage containing computer program instructions. The system includes a control interface that accesses data representing actions relating to constructing a building and generates scripts that specify values over time for one or more variable parameters of a parameterized and labeled model representing a building including data representing three-dimensional objects representing components of a building, where at least some information associated with the three-dimensional objects is specified as a variable parameter that can be assigned in an animation or simulation. The system includes an animation and simulation module having inputs receiving the parameterized and labeled modeland the scripts, and having outputs providing data representing results of an animation or simulation of the parameterized and labeled model based on the scripts.

[0017] In general, this aspect may provide an animation and simulation module paired with a comparison module, where the comparison module can evaluate simulated actions against field sensor inputs. In one aspect, a computer system includes a processing system with at least one processing device and computer storage containing computer program instructions. The system includes an animation and simulation module having inputs receiving a parameterized and labeled model, including data representing three-dimensional objects representing components of a building, where at least some information associated with the three-dimensional objects is specified as a variable parameter that can be assigned in an animation or simulation and having outputs providing data representing results of an animation or simulation of the parameterized and labeled model. The system includes a comparison module that receives data representing an animation or simulation of an action related to constructing a building output by the animation and simulation module, accesses inputs from on-site sensors capturing performance of the action related to constructing building, and generates a feedback output comprising data representing a comparison of the animation or simulation to the performed action based on the input data.

[0018] In general, this aspect may provide an animation and simulation module paired with an optimization module, where the optimization module can evaluate simulation results against optimization goals. In one aspect, a computer system includes a processing system with at least one processing device and computer storage containing computer program instructions. The system includes an animation and simulation module having inputs receiving a parameterized and labeled model, including data representing three-dimensional objects representing components of a building, where at least some information associated with the three-dimensional objects is specified as a variable parameter that can be assigned in an animation or simulation and having outputs providing data representing results of an animation or simulation of the parameterized and labeled model. The system includes an optimization module that accesses data representing one or more optimization goals for an action related to constructing a building, receives the data relating to an animation or simulation of a parameterized and labeled model for the action related to constructing the building output by the animation and simulation module, and selects and outputs parameters for the action related to constructing the building based on the optimization goals and the data relating to the animation or simulation.

[0019] In general, this aspect may provide a model parameterization module that can generate parameterized models from design files and additional data, along with a control interface that can produce scripts for simulation. In one aspect, a computer system includes a processing system with at least one processing device and computer storage containing computer programinstructions. The system includes a model parameterization module that accesses at least a building file describing a building and additional data, and generates a parameterized and labeled model based on at least the building file and the additional data, where the parameterized and labeled model includes data representing three-dimensional objects representing components of a building, where at least some information associated with the three-dimensional objects includes data specified as a variable parameter that can be assigned in an animation or simulation. The computer storage allows access to the parameterized and labeled model for use by an animation and simulation module for animating or simulating an action related to constructing the building. The system includes a control interface that accesses data representing actions relating to constructing a building and generates scripts that specify values for one or more variable parameters over time for animation or simulation of the parameterized and labeled model. The computer storage allows access to the scripts for use by an animation and simulation module for instructing the animation or simulation model to execute an animation or simulation of the parameterized and labeled model based on the scripts.

[0020] In general, this aspect may provide a control interface combined with an optimization module, where the control interface can define simulations over time and the optimization module can select parameters based on optimization goals. In one aspect, a computer system includes a processing system with at least one processing device and computer storage containing computer program instructions. The system includes a control interface that accesses data representing actions relating to constructing a building and generates scripts that define an animation or simulation of a parameterized and labeled model by specifying values for one or more variable parameters of the model over time. The computer storage allows access to the scripts for use by an animation and simulation module for instructing the animation or simulation model to execute an animation or simulation of the parameterized and labeled model based on the scripts. The system includes an optimization module that accesses data representing one or more optimization goals for an action related to constructing a building, receives data relating to an animation or simulation of a parameterized and labeled model for the action related to constructing the building, and selects and outputs parameters for the action related to constructing the building based on the optimization goals and the data relating to the animation or simulation.

[0021] In general, this aspect may provide a system that integrates a control interface, instruction generation module, comparison module, and optimization module, which together can generate scripts, instructions, comparisons, and optimized parameters. In one aspect, a computer system includes a processing system with at least one processing device and computer storage containing computer program instructions. The system includes a control interface that accessesdata representing one or more actions relating to constructing a building and generates scripts that define an animation or simulation of a parameterized and labeled model by specifying values for one or more variable parameters of the model over time. The computer storage allows access to the scripts for use by an animation and simulation module for instructing the animation or simulation model to execute an animation or simulation of the parameterized and labeled model based on the scripts. The system includes an instruction generation module that accesses the parameterized and labeled model and generates instructions for an action related to constructing the building based on the data output by the animation and simulation module. The system includes a comparison module that receives input data representing an animation or simulation of an action related to constructing a building output by the animation and simulation module, accesses inputs from on-site sensors capturing performance of the action related to constructing building, and generates a feedback output comprising data representing a comparison of the animation or simulation to the performed action based on the input data. The system includes an optimization module that accesses data representing one or more optimization goals for an action related to constructing a building, receives data relating to an animation or simulation of a parameterized and labeled model for the action related to constructing the building, and selects and outputs parameters for the action related to constructing the building based on the optimization goals and the data relating to the animation or simulation.

[0022] In general, this aspect may provide a model parameterization module, a simulation module, and an instruction generation module, which together can translate design data, simulate construction processes, and output assembly instructions. In one aspect, a computer system includes a processing system with at least one processing device and computer storage containing computer program instructions. The system includes a model parameterization module that receives design data from architectural software, including a three-dimensional model of a structure of a building, and translates the received design data into a parameterized and labeled model of the building, where building components are represented by three- dimensional objects associated with data representing assembly tasks. The system includes an animation or simulation module that generates an animation or simulation of a process for constructing the building, where the animation or simulation includes a representation of assembly of building components. The system includes an instruction generation module that generates human-readable or machine-readable instructions for assembly of the building components, including one or more of component assembly sequences or step-by-step instructions for methods of assembling components.

[0023] In general, this aspect may provide an optimization module that can generate multiple parameter sets, compare their simulated performance, and support evaluation of preferredconstruction designs. In one aspect, a computer system includes a processing system with at least one processing device and computer storage containing computer program instructions. The system includes an optimization module that accesses data representing one or more optimization goals for one or more actions related to constructing a building, receives data relating to an animation or simulation of a parameterized and labeled model for the action related to constructing the building, selects and outputs a first set of parameters for the action related to constructing the building based on the optimization goals and the data relating to the animation or simulation, selects and outputs a second set of parameters for the action related to constructing the building based on the optimization goals and the data relating to the animation or simulation, and through a presentation device, enables comparison of performance of the action based on the first and second sets of parameters to evaluate options and determine the preferred design for construction based on criteria.

[0024] Any of the foregoing can include one or more of the following features. The at least some information associated with one or more of the three-dimensional objects can include at least a portion of the additional data. The at least some information associated with the three- dimensional objects can further include data specified as an associated parameter. The associated parameter can include data representing one or more of assembly sequences, material properties, labor assignments, or real-time adaptability based on site conditions or sensor feedback. The associated parameter can include one or more building knowledge graphs. The parameterized and labeled model can further comprise additional data structures storing semantic information extracted from the building file.

[0025] Any of the foregoing can include one or more of the following features. The system can further comprise an animation and simulation module having one or more inputs receiving the parameterized and labeled model, and having one or more outputs providing data representing results of an animation or simulation of the parameterized and labeled model. The animation or simulation can include a representation of fabricating the components of the building. The animation or simulation module can further configure the processing system to optimize the sequence of tasks for constructing the building by simulating multiple assembly sequences to reduce one or more of costs, time, labor requirements, material usage, or environmental impacts. The animation or simulation module can dynamically adjust the animation or simulation based on real-time updates to construction progress, material or equipment availability, workforce availability, or site conditions. The animation or simulation can be defined by one or more of an animation graph or an action graph or other graph data structure. The system can further comprise one or more agents, comprising computer program instructions that configure theprocessing system to generate one or more of an animation graph or an action graph or other graph data structure.

[0026] Any of the foregoing can include one or more of the following features. The system can further comprise an instruction generation module comprising computer program instructions that configure the processing system to generate human-readable or machine-readable instructions for an action related to constructing the building. The instruction generation module can configure the processing system to generate the instructions for the action related to constructing the building based on data output by an animation and simulation module using the parameterized and labeled model. The action can comprise assembly of the components of the building. The action can comprise a cut list or bill of materials for fabricating the components of the building. The instructions can include step-by-step instructions for methods of fabricating the components of the building. The instruction generation module can further provide updates to the instructions based on feedback from one or more of on-site data sensors, or changes in design, material availability, or equipment availability. The instruction generation module can further configure the processing system to optimize material cutting and preparation, including one or more of generating cut lists that reduce material waste based on material properties or providing instructions to one or more machines for automated cutting or fabrication. The instruction generation module can further configure the processing system to assign tasks to workers or machines based on one or more of availability, skill level, or performance metrics. Instructions generated based on an animation or simulation can include human-readable instructions, machine-readable instructions, or a combination of human and machine-readable instructions. The instructions can include a cut list or bill of materials for preparing one or more components from one or more pieces of material. The instructions can include instructions for ordering, fabricating, packaging, staging, or transportation of materials. The instructions can include a sequence of tasks to be performed.

[0027] Any of the foregoing can include one or more of the following features. The simulation can adjust based on one or more of data inputs from on-site devices that track progress of assembly, labor or equipment availability, or supply chain data reflecting one or more of material availability, material origin, delivery schedules, cost fluctuations, or environmental impact assessments. The on-site devices can include one or more of a camera, a LiDAR scanner, a GPS system, an environmental sensor, or an on-site graphical user interface.

[0028] Any of the foregoing can include one or more of the following features. The data can include one or more of a parameterized and labeled model of a building, additional data related to constructing the building, optimization goals, feedback, material types and properties, shapes and dimensions of building components, shapes and dimensions of raw material inputs for use inpreparing building components, material preparation techniques, construction techniques, or building files. The comparison module can further comprise an output interface enabling transmission of the feedback output to one or more of an animation or simulation module, an instruction generation module, or a control interface. The comparison module can further analyze information based on the onsite data to ensure conformance of the process of constructing the building to plan. The system can further comprise an instruction generation module that accesses at least a parameterized and labeled model including data representing three-dimensional objects representing components of a building, where at least some information associated with the three-dimensional objects is specified as a variable parameter that can be assigned in an animation or simulation, and generates instructions for an action related to constructing the building based on the data output by the animation and simulation module. The criteria can include one or more of construction cost, energy efficiency and usage, operational cost, construction time, return on investment, environmental impact, achievability of certifications, construction sequence, worker skill requirements, recyclability or reuse potential of materials at the end of building life, risk assessment including technical, financial, insurance, or lending- related risk metrics, as well as financial value projections such as return on investment (ROI), net present value (NPV), internal rate of return (IRR), contingency reserve reduction, and improvements to asset valuation under alternative construction scenarios.

[0029] The comparative analysis can be conducted for one or more criteria at both the wholebuilding level or more granularly by focusing on specific components, such as mechanical, electrical, and plumbing (MEP) systems, structural elements, or wall panels, enabling targeted optimization.

[0030] Any of the foregoing can include one or more of the following features. An animation or simulation of an action related to constructing the building, or outputs from a comparison of such animation or simulation with data from the building as actually constructed, can be used to verify completion of construction tasks. The verification can generate payment facilitation outputs, including release instructions or authorization holds, to enable automated, objective, and auditable facilitation of disbursements to subcontractors or vendors. The payment facilitation outputs can be formatted for use in progress-based financial transactions by general contractors, owners, lenders, or other stakeholders.

[0031] Any of the foregoing can include one or more of the following features. The data in the parameterized and labeled model, which are associated with an object representing a component of a building and specifying a variable parameter, includes one or more of a task to be performed associated with the component; a dependency of a first task to be performed associated with the component upon a second task to be performed; a state of constructing thecomponent; a state of availability of the component; a material for the component; a fastener associated with the component; supply information for a material for the component; a method of preparation associated with the component; a method of assembly associated with the component; a financial cost associated with the component; a location associated with the component; a measure of time associated with the component; or a worker associated with the component; or any combination of these. A method of preparation includes one or more of a tool, an action, a technique, an actor, a condition, a constraint, or alternatives therefor. A method of assembly includes one or more of a tool, an action, a technique, an actor, a condition, a constraint, or alternatives therefor.

[0032] Any of the foregoing can include one or more of the following features. The data in the parameterized and labeled model, which are associated with an object representing a component of a building and specifying a variable parameter, includes data representing a task associated with the component, and wherein the data representing the task associated with the component includes data representing one or more of: a worker associated with the task; a skill associated with the task; a skill level associated with the task; a physical capability associated with the task; equipment associated with the task; a state associated with the task; a dependency associated with the task; a sequence associated with the task; a standard associated with the task; a technique associated with the task; a degree of variability associated with the task; a time allotment for the task; a location associated with the task; a risk associated with the task; or a cost associated with the task; or any combination of these.

[0033] Any of the foregoing can include one or more of the following features. An animation or simulation of an action related to constructing the building, or outputs of a parameterized and labeled model used for such animation or simulation, can be analyzed at early stages of design, such as conceptual or schematic design, to estimate costs, schedules, and constructability constraints. The system can operate as a design co-pilot, providing scope recommendations, constraint testing, and comparative simulations that enable owners, architects, and builders to evaluate options and narrow the design space before committing to detailed design.

[0034] Any of the foregoing can include one or more of the following features. An animation or simulation of an action related to constructing the building, or outputs of a parameterized and labeled model used for such animation or simulation, can be analyzed to generate risk assessment outputs. The risk assessment can, without limitation, include identification of bottlenecks or chokepoints in task sequences, sensitivity testing across variables such as material availability, workforce allocation, material or labor costs, or environmental conditions, and generation of metrics such as likelihood of delay, probability of cost overrun, exposure to disruptions, or resilience under alternative scenarios. The risk assessment outputs can be used inconstruction planning as well as by financial stakeholders, including insurers, lenders, and developers, to inform underwriting, lending, or investment decisions. In addition to risk assessment, the system can generate financial value outputs, including projections of return on ROI, NPV, and IRR, anticipated reductions in contingency reserves, and asset valuation improvements associated with optimized sequencing, sustainability outcomes, or resilience. These outputs enable developers, owners, and financiers to capture financial upside as well as mitigate downside risks.

[0035] Any of the foregoing can include one or more of the following features. An animation or simulation of an action related to constructing the building animates or simulates one or more of a sequence of tasks associated with components of the building; a sequence of steps associated with the fabrication of a component of the building from a less finished starting material; one or more methods of assembly of components of the building; one or more fasteners associated with assembly of components of the building; one or more steps of material preparation associated with assembly of components of the building; one or more pathways for components through other components of the building; ordering, packaging, or transportation of materials for components of the building; or resource management, such as materials, labor, or equipment, with constraints such as cost, environmental impact, or building codes; or any combination of these. The animation or simulation includes constraints based on one or more of engineering constraints, optimization goals, regulatory compliance, energy efficiency, sustainability goals, supply chain constraints, material properties, workforce availability, worker skill, safety constraints, site access constraints, cost, time, or laws of physics, or any combination of these.

[0036] Any of the foregoing can include one or more of the following features. An output from an animation or simulation of an action related to constructing the building includes data representing one or more of an animation or simulation of movement of components of the building representing performance of the assembly or fabrication tasks; an animation or simulation of movement of workers representing performance of a task; a sequence of steps related to constructing the building; a modeled estimated sequence of tasks; a method or technique related to constructing the building; an animation or simulation of movement or operation of equipment representing performance of a task; a change in the state or other variable parameters of a component of the building; an estimated cost for performance of one or more tasks; an estimated time for completing performance of one or more tasks; dependencies constituting preconditions or consequences; numbers and types of dependencies between tasks; identification of the relationships between tasks; an estimated level of skill for workers; an estimated amount of materials; an estimated measure related to an optimization goal; anestimated measure of risks; or proposed adjustments to real-world actions; or any combination of these.

[0037] Any of the foregoing can include one or more of the following features. The optimization module optimizes instructions for one or more of: minimizing waste; optimizing assembly sequences; minimizing costs; reducing construction time; maximizing resource utilization; enhancing safety; improving quality control; minimizing environmental impact; maximizing energy efficiency; optimizing material usage; reducing waste; improving labor productivity; enhancing stakeholder satisfaction; increasing sustainability; ensuring regulatory compliance; maximizing return on investment; optimizing logistics and supply chain management; improving site layout efficiency; streamlining routing paths for mechanical, electrical, and plumbing systems; minimizing downtime; enhancing communication and coordination among teams; optimizing scheduling and sequencing; enhancing flexibility and adaptability to changes; optimizing equipment usage; improving risk management; maximizing durability and longevity of the structure; enhancing accessibility and inclusivity; optimizing maintenance and lifecycle costs; maximizing overall project performance and value; level of worker skill required; or potential for material reuse or recyclability at the end of the building’s useful life; or any combination of these.

[0038] Any of the foregoing can include one or more of the following features. The instructions generated include one or more of: daily workforce planning including personnel assignments, personnel including one or more of skills or expertise needed, sequences of steps for construction, optionally at various levels of detail, summaries related to one or more of material, equipment, or personnel availability, usage, cost, efficiency, or time, or any combination of these.

[0039] Any of the foregoing can include one or more of the following features. An optimization goal includes one or more of: minimizing waste; optimizing assembly sequences; minimizing costs; reducing construction time; maximizing resource utilization; enhancing safety; improving quality control; minimizing environmental impact; maximizing energy efficiency; optimizing material usage; reducing waste; improving labor productivity; enhancing stakeholder satisfaction; increasing sustainability; ensuring regulatory compliance; maximizing return on investment; optimizing logistics and supply chain management; improving site layout efficiency; streamlining routing paths for mechanical, electrical, and plumbing systems; minimizing downtime; enhancing communication and coordination among teams; optimizing scheduling and sequencing; enhancing flexibility and adaptability to changes; optimizing equipment usage; improving risk management; maximizing durability and longevity of the structure; enhancing accessibility and inclusivity; optimizing maintenance and lifecycle costs; maximizing overallproject performance and value; level of worker skill required; or potential for material reuse or recyclability at the end of the building’s useful life; or any combination of these.

[0040] Any of the foregoing aspects may be embodied as a computer system, as any individual component of such a computer system, as a process performed by such a computer system or any individual component of such a computer system, or as an article of manufacture including computer storage in which computer program code is stored and which, when processed by the processing system(s) of one or more computers, configures the processing system(s) of the one or more computers to provide such a computer system or individual component of such a computer system.

[0041] The following Detailed Description references the accompanying drawings which form a part of this application, and which show, by way of illustration, specific example implementations. Other implementations may be made without departing from the scope of the disclosure.BRIEF DESCRIPTION OF THE DRAWING

[0042] Figure l is a data flow diagram for a computer system which implements operations for animating, simulating, or generating instructions for actions related to constructing a building.

[0043] Figure 2 is a data flow diagram of an example implementation of the model parameterization, instruction generation, and animation or simulation modules of Figure 1.

[0044] Figure 3 illustrates an example implementation of knowledge graph generation.

[0045] Figure 4 is an example schema generated from a building file.

[0046] Figure 5 is a schematic illustration of an example set of parameterization types enabling animation and simulation.

[0047] Figure 6 is an example instance of a parameterized model of a simple structure (a garden shed).

[0048] Figure 7 is an example instance of a building knowledge graph of the garden shed.

[0049] Figure 8 is a flow chart of an example implementation of processing of a building file to generate a parameterized model for animations, simulations, and instruction generation.

[0050] Figure 9A is an example of set of parameters for animating operation of a door.

[0051] Figure 9B is an example illustrating simulation of construction of a wall assembly.

[0052] Figure 10 is an example of operation of a domain-specific agent using an expertise knowledge graph.

[0053] Figure 11 is an example of operation of an instruction generation module using a parameterized model to generate instructions.

[0054] Figure 12 is an example of using a parameterized model with physics simulation and behavioral simulation.

[0055] Figure 13 is a block diagram of an example general purpose computer.DETAILED DESCRIPTION

[0056] The computer systems, processes, and techniques described herein relate primarily to actions related to constructing buildings which have already been designed. However, in some implementations the system can also operate during earlier stages of design, such as conceptual, schematic, design development, or other pre-construction design activities, to analyze designs and provide outputs prior to finalizing the design of the building, for example but without limitation related to construction feasibility, cost estimation, scheduling, risk, required contingency reserves, ROI, NPV, IRR, or estimated asset valuation.

[0057] The term "building" refers to any structure or combination of structures, whether enclosed, partially enclosed, or open, that provides space for occupancy, use, or shelter, along with any associated elements or systems integral to its function or operation. Such a structure is formed by assembling a plurality of components, which may include but are not limited to, foundation elements, such as slabs and footings; structural frameworks, such as beams, studs, columns, or trusses; roofing systems, such as sheathing, underlayment, shingles, or other roofing materials; exterior coverings, such as cladding, siding, brick, or other facades; openings, like doors, windows, and skylights; interior walls and finishes, such as drywall, plaster, paint, panels, tiles, or molding; flooring systems, such as sub flooring, joists, and various surface materials; mechanical, electrical, and plumbing (MEP) systems, such as wiring, conduits, pipes, ducts, and HVAC components; insulation and barriers for thermal, moisture, fire, or sound control; fastening and connecting systems, such as screws, nails, bolts, adhesives, or brackets; or other elements that provide functional or aesthetic value, such as cabinets, countertops, fixtures, railings, and built-in furniture. These lists are presented by way of example only and are intended to be neither exhaustive nor required. Any element that is used in the final constructed state of a building is referred to herein as a "component." "Constructing" a building encompasses the process of assembling such components, or any other activity that involves the fabrication, arrangement, installation, or modification of such components, whether for part of a building, an entire building, new construction, renovation, retrofit, or repair.

[0058] Implementations of the computer systems, computer-implemented processes, and techniques described herein assist end users in performing various actions related to constructing a building. For example, these computer systems, computer-implemented processes, and techniques can implement operations for animating, simulating, evaluating, optimizing, orgenerating instructions for actions related to constructing a building. For example, they can generate detailed, step-by-step plans for selection, ordering, transportation, fabrication, sequencing, installation, or assembly of components of a building; simulate the process of constructing a building in a virtual environment; develop estimates of costs to construct a building; and make modifications to such plans based on feedback received from a user or based on on-site data. In some implementations, information related to constructing a building, which is generated by these computer systems, computer-implemented processes, and techniques, can be fed back into the design process. In some implementations, the system further provides risk analysis outputs derived from simulation of parameterized and labeled models, including outputs such as identification of bottlenecks or chokepoints in task sequences, sensitivity testing across variables such as material availability, workforce allocation, or environmental conditions, and generation of risk metrics including likelihood of delay, probability of cost overrun, exposure to disruptions, or resilience under alternative scenarios. Such risk outputs may be used not only in construction planning but also by financial stakeholders, including insurers, lenders, and developers, to inform underwriting, lending, or investment decisions. In some further implementations, the system may generate financial value creation outputs. Such outputs can include calculations of project-level return on investment (ROI), net present value (NPV), internal rate of return (IRR), based on simulated cost and schedule data; quantification of contingency reserve reductions from improved predictability; identification of financing efficiencies such as reduced cost of capital or improved loan terms; and projections of long-term asset valuation improvements associated with sustainability, energy performance, or resilience. At a portfolio level, such outputs may be aggregated across multiple projects to provide developers, financiers, and insurers with dashboards of financial performance and value creation opportunities.

[0059] There are several technical obstacles to overcome to build a computer system or computer-implemented processes that can assist end users in performing various actions related to constructing a building. For example, there are many different data sources to combine into a data structure that is a workable representation of the building and how the building will be constructed. Further, these data sources include complex and diverse data sets, such as three- dimensional models, time-dependent data sets, location-dependent data sets, user-dependent data sets, structured and unstructured data of various kinds, and extensive metadata associated with building construction. Such metadata typically includes, but is not limited to, classification codes (e.g., Uniformat, Masterformat, or OmniClass codes), manufacturer specifications, performance ratings, system assignments, assembly relationships, dependency chains, 4D scheduling data, phase assignments, level of development (LOD) specifications, code compliance tags,sustainability metrics, cost data, and lifecycle information. Many of these data sources were not designed to be integrated into a single environment or platform.

[0060] The resulting combined data set has a large number of dimensions and is a complex mix of different kinds of data. Further, these data sources typically are not static, resulting in challenges for maintaining the combined data set consistent with these data sources and up to date. Further, multiple users typically would use the input data sources and the combined data set in parallel. Implementations of solutions for various planning, optimization, and simulation problems using such a data set are computationally complex and consume significant storage and computational resources. Further, models of complex multi-dimensional systems are not capable of being fully understood or evaluated by the human mind. The following explains some example solutions to these complex technical problems.

[0061] A computer system receives inputs describing the structure and design of a building together with additional data and generates a parameterized and labeled model as a data structure representing a collection of objects. As used herein, an “object” is a data structure within the computer system that represents a building-related element or factor, defined in at least three spatial dimensions. An object is associated with parameters that describe its intrinsic properties, such as dimensions, material composition, cost, performance ratings, or state, and its relationships to other objects, such as physical connections, task dependencies, adjacency, containment, or sequencing. Objects can include, without limitation, building components, such as studs, beams, doors, windows, conduits, ducts, and fixtures; collections of building components, such as wall panels, floor assemblies, or roof trusses; and contextual construction elements that interact with or influence the building process, such as HVAC equipment awaiting installation, fabrication machinery, delivery vehicles, or site machinery such as cranes or excavators. This definition encompasses both physical elements of the constructed building and temporary or external elements relevant to constructing the building, so long as they are modeled with properties and relationships that can be parameterized or simulated.

[0062] Non-limiting examples of objects, as defined above, include structural components such as studs, bottom and top plates, king and jack studs, headers, sheathing panels, doors, windows, conduits, ducts, fixtures, beams, and columns. Objects may also include associated parameters, such as dimensions, material grade, fire rating, intended location, or fastening schedules. Further examples of objects include external or contextual elements, such as HVAC equipment awaiting assignment, or construction resources like delivery vehicles, forklifts, or excavators, with associated parameters such as cost, availability, operator requirements, or logistical constraints.

[0063] The parameterization of the model includes identifying categories of object attributes that are variable to enable animation or simulation of actions related to construction of the building.Example categories of attributes include, but are not limited to, one or more of: geometric description and pose, such as position and orientation; state information, such as visibility, lifecycle, or installed-state progression; assembly logic, including preconditions, dependencies, and ordering; specification and compliance status; tolerances and spatial clearances; temporal factors, including task durations and scheduling windows; resource assignment and capacity, including labor, crews, and equipment; economic and logistics factors, including rates, procurement status, and delivery or staging considerations; safety and regulatory requirements; or environmental or site context; or any combination of these.

[0064] The labeling or annotating of the model includes tagging objects with metadata, including, without limitation, relationships with knowledge graphs, material and dimensional data, assembly references, dependency links, classification and identifier data, tolerance and inspection data, fasteners, and environmental factors like load-bearing requirements, and other project information.

[0065] After generating such a parameterized and labeled model, an animation and simulation system can process the parameterized and labeled model to produce an animation, a simulation, or both, of actions related to constructing the building.

[0066] As used herein, an “action related to constructing a building” is any operation, task, process, design modification, or decision that contributes to or affects the construction of a building, whether physical, computational, logistical, or supply-chain related. Actions can include, without limitation, fabrication, assembly, or installation of building components; planning, sequencing, or routing of building elements or systems; optimization of materials, designs, sequences, schedules, or processes; estimating or analyzing costs; scheduling or allocating workforce and equipment; managing procurement, acquisition, transport, or logistics of materials, and monitoring, evaluating, or providing feedback on construction performance.

[0067] Examples of actions related to constructing a building, as defined above, can involve any portion of the building or any aspect of the construction process. Illustrative actions include, without limitation: assessing material availability, acquisition, packing, or transport; fabricating individual components; planning building layouts and system routing, such as for mechanical, electrical, and plumbing (MEP) subsystems; optimizing designs, material selections, component configurations, and construction sequences with respect to cost, time, energy efficiency, or sustainability; defining and executing assembly sequences and methods; estimating construction costs; and managing workforce and equipment availability, capability, and utilization. These examples are presented by way of illustration only and are intended to be neither exhaustive nor required.

[0068] Performance of such actions can be subject to one or more constraints, which the computer system can impose on an animation or simulation, such as: compliance with engineering standards, building codes, or other standards, environmental sustainability parameters, laws of physics, time to completion, overall cost of completion, cost of labor, cost of materials, cost of equipment, level of skill required, energy efficiency, performance response to external conditions, lifecycle cost, maintenance cost, or repair / replacement cost. Performance of such actions can be subject to one or more metrics of cost or value, each with its own method of calculation, which the computer system can use to evaluate an animation or simulation, such as: time to completion, overall cost of completion, cost of labor, cost of materials, cost of equipment, level of skill required, energy efficiency, performance response to external conditions such as weather, environmental impact, lifecycle costs, maintenance costs, or repair / replacement costs. These lists are presented by way of example only and are intended to be neither exhaustive nor required.

[0069] The computer system creates a model that includes not only objects which represent components of the building but also data structures, herein called “tasks.” As used herein, a task is a data structure that represents one or more activities contributing to an action related to constructing a building, whether performed in the physical world by a human, by a machine, using a machine, or modeled or represented in a simulated or planning environment. A task is associated with one or more objects representing building components, resources, or contextual elements affected by the activity represented by the task, and is parameterizable so that its properties, relationships, and outcomes can be varied, simulated, or optimized.

[0070] A task, as defined above, or collection of tasks, can include one or more of the following characteristics: a sequence or order in which tasks are or may be performed, including any dependency or coordination between or among tasks or objects, and grouping of tasks; a status of a task; one or more tools or machines used to accomplish a task; a method or technique used to accomplish a task; the resources, such as materials, labor, equipment, or skill used to perform a task; other resources, such as time or money, which are consumed by or constrain performance of a task; constraints, such as budget, regulations, or site conditions, on performance of a task; when the task should be performed; and any dependencies among these. These characteristics are presented by way of example only and are intended to be neither exhaustive nor required. Any one or more of these characteristics can be specified as an attribute of a task data structure, and can be specified as a variable parameter in the data structure representing a task.

[0071] The animation or simulation of an action related to constructing a building can relate to animation or simulation of, among other things, one or more of planning, optimization,estimating cost, explanation, instruction, developing solutions for performance, automation, execution, monitoring, or evaluation of performance of the action.

[0072] As used herein, “animation” refers to visualization of changes in object or system properties over time. For example, by evaluating one or more keyframed or piecewise functions of a property over time, visualizations of such changes can be rendered. Animation may visualize movement of objects in three-dimensional space or other time-varying information. In some embodiments, animation is used to visualize the progression of simulated states. In some embodiments animation is used independently for demonstration or training.

[0073] As used herein, “simulation” refers to computation of object or system state, state transitions, properties, schedules, or other characteristics, based on inputs such as system parameters, constraints, task dependencies, resource and capacity limits, calendars, and, in some embodiments, physics or spatial analyses. Simulation outputs can include, without limitation, task ordering, start and finish times, resource utilization, cost estimating, conflict indicators, and derived instructions. In some implementations, a simulation can provide inputs to an animation to visualize the modeled behavior (e.g., simulating steps to assemble a wall to evaluate time, feasibility, cost, or material usage).

[0074] In many applications, such as planning and optimization problems, multiple simulations are run with multiple different sets of values for the various parameters of the model. These multiple simulations may be run iteratively or in parallel. In some use cases explained herein, a simulation can have parameters of an underlying model which are based on real-world data, which therefore can change. By modeling performance of a system, a simulation typically generates data that is used to perform or decide how to perform yet other actions.

[0075] Based on results from the animation or simulation, and user interaction, the computer system can deliver outputs, whether human or machine-readable, related to performance of the action. Such outputs can include, without limitation, cost estimates, identification of physical and temporal conflicts, and construction planning and schedule optimization according to specified parameters and constraints. Planning and, optionally, optimization, yields a dependency-checked sequence and, in some embodiments, an optimized schedule with start and finish times, resource assignments, and constraints satisfied. From that plan, the computer system then generates instructions specifying performance of the actions required to construct the building. For example, the instructions can include one or more sequences of steps for actions related to constructing the building; indicate a bill of materials and suppliers from which to purchase the materials; provide cut lists for fabricating materials into components of the building; set out ordered day plans that specify tasks to be performed, equipment to be used, and workforce assignments; and specify methods or materials to be used to perform the action, together withexplanations of how to perform the methods so instructed. The instructions may, without limitation, reference model element identifiers, applicable tolerances and fastening schedules, and safety or compliance criteria. An animation can be used to visualize simulated state changes and illustrate how the action is to be performed. User input may be applied at any stage to set objectives, adjust parameters, or approve proposed plans.

[0076] The animation or simulation can implement a digital twin of the building or an action related to constructing the building. When the computer system has access to real-world data related to the action, such as by using sensors, cameras, microphones, wearables, smartphone interfaces, or other devices, that real-world data can be compared to this digital twin. Such comparison can be used for many purposes, such as to update or modify the digital twin or to provide feedback about the real-world performance of the action. Feedback of real-world data into the system also can trigger the computer system to perform a variety of other operations, such as re-execution of an animated or simulated action, or regeneration of a sequence of steps for an action, which may result in a different set of steps or different sequence of steps for the action from an originally generated sequence of steps. In some embodiments, such real-world feedback also drives adjustments to task sequences, and changes to material selections or the building design itself, when captured data indicates a design problem or other reason for a design change.

[0077] For a computer system to implement these various operations, complex and diverse data sets are combined. The structure of a building typically is defined in a computer using a three- dimensional spatial model, such as created by computer-aided design (CAD) tools, computer- aided design / computer-aided manufacturing (CAD / CAM) tools, building information modeling (BIM) tools, animation modeling tools, and architectural visualization tools. Each component of the building typically is defined as a three-dimensional object, typically as a set of vertices and edges. Various information about the component typically is stored as attributes of the object representing the component. In some cases, additional data sets provide such information about the components. Examples of relevant information about a component for a building include, but is not limited to, data describing local building codes, data relating to the supply chain for materials, data describing the workforce, including availability, skills, labor cost rates, and pace of work, data relating to available equipment, or various information about materials. As explained in more detail below, to enable animation and simulation of actions relating to constructing a building, such information is combined into a complex model.

[0078] To enable animation or simulation using a complex model of a building, certain information within the complex model is identified as the information that can be varied. The identification of the information in the model which can be varied in animation or simulation isreferred to herein as a kind of parameterization. For example, the locations of components of a building, such as walls, floors, and ceilings, and in turn their components such as posts, studs, beams, and joists, are typically specified in the data files created in the architectural design phase by their final positions after construction. To animate or simulate assembly of these components, locations, or other indicia of presence of the components are transformed into variable parameters of the model. Thus, as an example, when running an animation or simulation using the model, the position and orientation of a component can be varied to simulate movement of the component from a preparation stage to its installed location. Or, the visibility of a component, and its impact on other components based on the application of laws of physics, can be turned off unless that component is modeled as having been installed.

[0079] Additionally, components can change geometry or other attributes during fabrication or installation, and such modifications can be likewise parameterized and tracked as state transitions or associated derived geometry. For example, a stock stud may be cut to a specified length prior to placement in wall framing, and a rafter may be mounted and then its rafter tail trimmed to achieve the intended overhang. These operations reflect modification from an original to a subsequent size or shape and are represented in the model so that animation or simulation reflects the changed component.

[0080] Also, to enable animation or simulation of actions related to constructing a building, objects within the complex model further are associated with tasks, dependencies, and other information related to the actions to be animated or simulated. The association of these tasks, dependencies, or other information related to actions to be animated or simulated with their corresponding objects is also referred to herein as a kind of parameterization. In some cases, data about the tasks or other information also can be identified as information which can be varied and thus parameterized. For example, the installation of a stud for a wall, encompassing positioning the stud in a specific location and making use of certain fasteners and certain techniques, may be a task associated with that stud. The dependency of that task on other tasks, and its assignment to a particular worker, can be information which are varied in the model and thus parameterized. Similarly, the availability of optional materials for the stud can be information which is varied in the model and thus parameterized.

[0081] By integrating a parameterized, complex model of a building and related data, that incorporates data about the three-dimensional structure of components of a building with information about tasks and other information associated with those components, into a platform that implements complex multi-dimensional simulations and animation, a computer system can provide the various features explained herein.

[0082] With the foregoing explanation of the context in which a computer system is used for animating, simulating, or generating instructions for constructing a building, an example implementation will now be described.

[0083] Figure 1 is a data flow diagram of an example implementation of a computer system which implements operations for animation, simulation, or generating instructions for actions related to constructing a building.

[0084] One or more architect tools 108 output one or more building file(s) 118. Such tools encompass a variety of software applications which generate data files representing various aspects of buildings. In some implementations, these architect tools enable architects to create detailed three-dimensional models of buildings that include both geometric information about the foundation, walls, floors, ceilings, windows, doors, and other structural components of a building, as well as other data about materials, systems, fixtures, fasteners, and other building components. Many software tools output data files in formats that are compatible with Building Information Modeling (BIM) processes and workflows. BIM is a digital representation standard and process that involves both software interoperability and collaborative workflows for managing building data throughout a project’s lifecycle, encompassing the generation, management, and use of digital representations of physical and functional characteristics of spaces.

[0085] Example software tools capable of authoring or exporting BIM-compatible data include, without limitation, Autodesk Revit software, Graphisoft Archicad software, Vectorworks software, Bentley OpenBuildings Designer and MicroStation software, Nemetschek Allplan software, Trimble Tekla Structures software, BricsCAD BIM software, McNeel Rhino with Grasshopper software, SketchUp software, and Autodesk 3DS Max software. Coordination and checking tools such as Autodesk Navisworks and Solibri may also be used. Professionals may also employ general-purpose computer-aided design (CAD), computer-aided manufacturing (CAM), three-dimensional modeling, drafting, and visual programming environments to author or transform building data.

[0086] The building files 118 resulting from such software tools can have a proprietary format or an open format. Example file formats include, but are not limited to: RVT and RFA files (produced by Autodesk's Revit software), PLN files (produced by Graphisoft' s ArchiCAD software), NDW files (produced by Nemetschek's AllPlan software), EDF files or EDL files (produced by Acca software), NWD or NWC files (Navisworks files produced by Autodesk's software), Industry Foundation Classes (IFC) compliant files, the Construction Operation Building Information Exchange (COBie) compliant files, drawing (DWG) and drawing exchange format (DXF) files (from Autodesk), design (DGN) files (used by Bentley System’s Microstationsoftware), BIM Collaboration Format (BCF) files, Green Building XML (gbXML) files, scenegraph and visualization file formats, such as USD, USDZ, and GL transmission format (glTF) files, and reality-capture formats such as ReCap project (RCP) and ReCap scan (RCS) files, ASTM E57 3D imaging (E57) files, LASer LiDAR point cloud data (LAS and LAZ) files. In some embodiments, data from these sources and formats is normalized into a canonical scene representation, such as USD, for downstream simulation and animation. Execution and playback may then occur, for example and without limitation, in any USD-compatible environment, such as NVIDIA Omniverse.

[0087] Various data sources 110 provide additional data 116 related to the construction process. Examples of such data include but are not limited to the following: Material inventory data, supply chain data, information about materials, building code information, performance standard data, workforce data, equipment data, or construction methods, or combinations of these. In some cases, the additional data is specific to a building or buildings or a geographic area. In some cases, the additional data is generically applicable to any building. In some implementations, the data is received as data files or data streams from other computer systems which output such data. In some implementations, the additional data can be provided by a user through a user interface.

[0088] Some examples of additional data 116 include but are not limited to the following.

[0089] Material inventory data can be provided as additional data. Examples of material inventory data include stock levels of raw materials (e.g., amounts of lumber, steel, concrete, or other materials) immediately available for use with a building. Such information can include batch numbers and expiration dates for perishable materials (e.g., chemicals, sealants). Such data typically is provided in data files or data streams in formats such as CSV, Excel or other spreadsheet, JSON, or XML formats.

[0090] Supply chain data can be provided as additional data. Examples of supply chain data include data describing supplier details, material origins, supplier lead times, shipping, or logistics data such as tracking numbers and delivery schedules, cost fluctuations, or availability. Such data typically is provided in data files or data streams in formats such as EDI (Electronic Data Interchange), API feeds (JSON / XML), Excel or other spreadsheet formats, or CSV formats.

[0091] Cost and estimating data can be provided as additional data. Examples include unit-cost and productivity references, city cost indices, crew rates, and assemblies from sources such as the RSMEANS service, as well as vendor quotations and historical job-cost data. Such data may be received via API feeds, JSON, XML or other structured files, spreadsheet formats, such as CSV files, or other machine-readable formats.

[0092] The additional data can include a variety of information about materials. Such data can include, for example, material properties, such as strength, weight, thermal conductivity, environmental impact data, or sustainability certifications. Such information may be provided in the form of technical data sheets (TDS) and material safety data sheets (MSDS / SDS). Such data typically is provided in data files or data streams in formats such as PDF, CSV, Excel or other spreadsheet formats, JSON, XML, or proprietary formats (e.g., software-specific formats).

[0093] Building code information can be part of the additional data, such as local, state, and national building codes, zoning regulations and compliance requirements, and updates or amendments to existing codes. Such information typically is available in data files or data streams in PDF, database export (CSV / SQL), JSON, or XML formats.

[0094] The additional data also can include performance standard data, such as energy efficiency standards (e.g., LEED, BREEAM), structural performance standards (e.g., seismic ratings, wind load capacities), fire resistance standards, or safety standards. Such data typically is provided in data files or data streams in PDF, Excel or other spreadsheet, JSON, or XML formats.

[0095] Workforce data can be included in additional data. Such data can include, for example, worker availability, scheduling data, skill levels, certifications, attendance records, and productivity metrics. Such data typically is provided in data files or data streams in CSV, Excel or other spreadsheet, or human resource management software exports (e.g., spreadsheet, JSON, or XML) formats.

[0096] Equipment data also can be included in the additional data. Equipment data can include, for example, equipment availability, location, usage logs, maintenance schedules, logs and records, and equipment specifications and operational limitations, as well as equipment costs, including hourly or daily rates and mobilization or demobilization charges. In some embodiments, equipment data further includes operator-related requirements such as skill level, training and certification, and insurance requirements. Transportation and permitting constraints may also be provided, including heavy-load or weather-contingent permits, road-type limitations such as paved versus unpaved access, and seasonal or weather restrictions (for example, limits on heavy equipment use on dirt roads during muddy periods). Such data typically is provided in data files or data streams in CSV, Excel or other spreadsheet, JSON, XML, and proprietary equipment management software formats. Such data can be ingested as data files or data streams in CSV, Excel or other spreadsheet, JSON, XML, and proprietary equipment-management formats, and in some embodiments can also be generated by the system during simulation or execution. Equipment data can further include, for example, 3D design and engineering data, such as models of machinery, equipment layouts, and mechanical components created using computer-aided design (CAD) software like AutoCAD software, SolidWorks software, orCATIA software. These 3D models may depict detailed structural and functional aspects of machines, including geometric configurations, assembly structures, material properties, kinematic simulations, and dynamic interactions of parts within an operational environment, facilitating virtual testing, visualization, and integration within larger systems.

[0097] The additional data also can include information describing a variety of construction methods. For example, such information can include step-by-step instructions or best practices for specific construction tasks, and historical data on construction methods used in similar projects. Such information can be used, for example, for integration with AR / VR systems for training or real-time guidance. Such data typically is provided in data files or data streams in PDF or other text document or rich media document, video (e.g., MP4, AVI), XML, or JSON formats. As described in more detail below, in some embodiments, the system transforms and stores such information into a knowledge graph, with nodes and relationships that capture applicability conditions, constraints, and provenance. These structured representations are then associated with model objects and task graph elements to allow inferences, such as producing feasible construction sequences and sets of instructions.

[0098] The additional data also can include sensor data from constructing a building or from an already-constructed building. Such data can include, for example, data from LiDAR, multi- spectral imaging, or other advanced sensors, real-time monitoring data from "internet of things" (loT) devices on-site, and environmental data (e.g., temperature, humidity, noise levels). Such data typically is provided in data files or data streams in structured data arrays, CSV, JSON, XML, or proprietary sensor formats.

[0099] The building file(s) 118 and additional data 116 as output by architecture tools 108 and other data sources 110 typically are not all combined into a single model, and typically are not parameterized for the purpose of creating a platform that integrates this information to enable animation, or simulation, or generating instructions for actions relating to constructing a building.

[0100] Accordingly, the building file 118 and additional data 116 are processed by model parameterization module 122, which generates a parameterized and labeled model 128, as explained in more detail below. The model parameterization module 122 generates, and the parameterized and labeled model 128 thus includes, data representing a three-dimensional spatial model including a plurality of three-dimensional objects representing building components described by the building file 118. The parameterization can also introduce time-based state variables and scheduling attributes to support simulation and animation. The three-dimensional objects are labeled and annotated based on information in the building file 118 and the additional data 116.

[0101] As an example, the parameterized and labeled model 128 can be implemented using a Universal Scene Description (USD) data file. Universal Scene Description (USD) is a file format developed by Pixar Animation Studios for efficient interchange of 3D computer graphics data. The format supports hierarchical scene description which allows for complex relationships between objects, materials, geometries, and animations. In the example implementations described below in connection with Figures 2 through 8, the model parameterization module 122 can be implemented using a processing pipeline that transforms the building file(s) 118 and selected portions of the additional data 116 from their native or interchange formats (for example, RVT / RFA, IFC, DWG / DXF, DGN, or NWD / NWC) into a canonical scene representation, such as USD, which is augmented with variable parameters, labels, annotations, data structures, and links to associated knowledge graphs. For example, the three-dimensional objects are labeled and annotated based on information in the building file 118 and the additional data 116. Such labels can include, without limitation, stable identifiers, classifications, constraints, and pointers to associated items in the additional data 116 and to nodes in the knowledge graphs, enabling retrieval and downstream generation of sequences and instructions. In other embodiments, another scene-graph representation compatible with the described operations can be used. In some implementations, the model parameterization can preserve stable identifiers and provenance to maintain traceability to the source files.

[0102] This model is parameterized by identifying the information in the model which can be varied in animation or simulation. For example, the locations, visibility, and application of physics and collision rules, for components of a building can be specified as variable parameters. Also, objects within the complex model further are associated with tasks and other information related to the actions to be animated or simulated. In some cases, data about the tasks or other information also can be identified as information which can be varied and thus parameterized. For example, installation of an object may be a task associated with that object, and the time required to perform that task can be specified as a variable parameter. Further, the hourly cost of labor by workers possessing the necessary expertise to perform a task can be specified as a variable parameter. As another example, dependency of a task on another task, or assignment of a task to a particular worker or piece of equipment, can be specified as a variable parameter. Similarly, the availability of optional materials for an object, or the cost of materials for an object, can be information which is specified as a variable parameter. State information is another example of a parameter. For example, a component of a building can have state data representing whether that component has been scheduled for installation, installed, or approved, or other state. Similarly, a task can have state data indicating whether the task is open, scheduled, or completed, or in another state.

[0103] In some implementations, the additional data 116 is not static and can be updated over time. In some cases, such data can be updated in real time or near-real time. For example, the cost of material inputs may fluctuate over time. As another example, the availability of workers or equipment may vary on a daily or even hourly basis. For such additional data 116, the association of such data with objects in the model 128 can be specified as variable parameters. In some implementations, the model 128 can be connected to data sources 110 in a manner that provides live updates of the additional data 116. As another example, the current state or progress in constructing the building can be a form of live data that can be an input to update the model.

[0104] In some implementations, the one or more building files 118 is not static and can be updated over time. In some cases, such data can be updated in real time or near-real time. For example, during construction various issues may arise, or change orders from the owner may occur, or materials availability may change, resulting in a redesign of the building, and consequent changes downstream to actions relating to constructing the building. In such cases, the model 128 can be connected to receive building files 118 in a manner that provides updates, and possibly live updates as the building files are updated by the architectural tools 108. In some implementations, a simulation, feedback from users, feedback from workers involved in constructing the building, real-world sensor data, or other input can result in suggested changes to the core building design. In some implementations, such suggested changes may be implemented through the architectural tool. In some implementations, such suggested changes can be implemented directly within the parameterized model.

[0105] In some implementations, the building files 118 or additional data 116 may be incomplete by not specifying or by incompletely specifying information required for accurate animation, simulation, or instruction generation. The incompleteness can be due to any of a variety of reasons, such as errors or omissions of the professional preparing the building file or additional data, or the building file or additional data being specified at a low level of detail or not including certain information by design.

[0106] There are many ways to determine whether information is incomplete. For example, there may be known cases, such as fasteners, where the parameterized model can be scanned for such information to determine if such information is sufficiently specified. As another example, a failure in simulation or visible artifact in animation or failure or incomplete specification of generated instructions may reveal that information is missing. In response to detecting information, the parameterized model can be updated. In some cases, the parameterized model can be modified to specify the missing information as a variable parameter. In some cases, the parameterized model can be updated after being supplied with the missing information. In somecases, such missing information can be automatically generated using a form of interpolation (such as for level of detail issues) or by using a library of information (such as for fasteners).

[0107] As an example, a building file often omits specifications about fasteners, such as the type, location, and quantity of fasteners, to be used in constructing the building. In some cases, a building file may omit entirely any specification for types of fasteners to be used. In some cases, a building file may only specify generally types of fasteners without identifying the location for each individual fastener. Thus, the parameterization of the model can include adding objects to the model to represent fasteners, where various information about the fasteners are specified as variable parameters.

[0108] In some implementations, the system can include a processing module that identifies and resolves these gaps. For example, such a processing module can implement a multi-stage validation and completion process that operates on the parameterized and labeled model 128 after initial generation by the model parameterization module 122.

[0109] As an example, in some implementations, the system can include a validation module that performs physics simulations and collision detection on the parameterized and labeled model 128 to identify structural inconsistencies, missing components, and constructability issues. The validation module executes simulations of construction sequences, structural loads, and component interactions to detect conditions such as: unsupported structural elements lacking specified connections, missing fasteners at connection points where forces would be transmitted, geometric conflicts indicating missing spacers or intermediate components, or MEP system penetrations lacking specified sleeves, fire stopping, or structural modifications. When the simulation identifies such conditions, the validation module records the specific gap with location data, affected components, and constraint violation details.

[0110] After detection of gaps through simulation or other analysis, the system can apply a rulebased completion engine that accesses construction standards, building codes, and industry best practices encoded within expertise knowledge graphs 232. Such a rule-based engine evaluates construction rules such as "structural header requires jack studs," "joist spans require hangers or bearing support," or "sheathing edges require landing and specified nail schedule." For each violated rule, the engine retrieves component templates from the expertise knowledge graphs and automatically generates missing elements with appropriate placement, quantities, materials, and installation parameters. Each synthesized component is tagged with provenance data indicating the rule that generated it, confidence scores based on rule certainty, and references to the expertise knowledge graph nodes that provided the template.

[0111] In some implementations, the system can employ knowledge graph pattern retrieval to identify missing components by comparing local construction contexts against canonicalassembly patterns stored in the expertise knowledge graphs 232. In such an implementation, the system generates a context signature for each region or element incorporating component types, structural spans, load conditions, materials, and interface requirements.

[0112] In some implementations, vector-based similarity matching can be used. In such an implementation, the system transforms construction contexts into high-dimensional numerical representations. Each context is encoded as an embedding vector where dimensions capture attributes such as: spatial relationships between components (e.g., distance metrics, angular orientations), material properties (e.g., strength values, thermal coefficients), structural characteristics (e.g., load magnitudes, span lengths), functional requirements (e.g., fire rating codes, acoustic performance values), and assembly constraints (e.g., tool accessibility scores, worker skill levels). These embedding vectors are generated using graph neural networks trained on the expertise knowledge graphs, where the network learns to map similar construction scenarios to nearby points in the vector space.

[0113] As an example, using cosine similarity or Euclidean distance metrics over these construction context embeddings, the system retrieves the k-nearest canonical patterns from the expertise knowledge graphs. The system then performs structural alignment between the retrieved patterns and the local context, identifying which components from the canonical pattern are absent from the current model. Missing components are synthesized using parameters derived from the local context, tagged with provenance references to the retrieved patterns, and assigned confidence scores based on the vector similarity measure (e.g., cosine similarity of 0.95 yields 95% confidence) and the structural alignment quality between the pattern and the local context.

[0114] In some implementations, such a validation and completion process can operate iteratively, with each cycle of gap detection and component synthesis followed by re-simulation to identify newly revealed issues. To prevent infinite loops, the system can implement convergence criteria such as maximum iteration limits, confidence score thresholds below which no further synthesis occurs, and stability detection when no new gaps are identified across successive iterations.

[0115] In some implementations, the additional data 116 can provide information about additional objects to be included as objects in the three-dimensional model. For example, one or more human or robot workers may be created as virtual agents within the three-dimensional model. As another example, objects can be defined in the three-dimensional model to represent a piece of equipment or a tool. Various state information about the additional object, such as its location in the model or visibility in the model or the applicability of physics and collision rulesto the object or its availability over time, can be specified as a variable parameter of the object. Such variable parameters also can be associated with other data within the additional data.

[0116] One of the technical challenges in creating such a parameterized and labeled model of a building is data integration and representation. The software tools that are designed for animation and simulation of three-dimensional models, such as the OMNIVERSE platform, are typically different from the software tools used for architectural design and engineering support, such as the AutoCAD software. Different software tools typically do not represent the same data or do not represent data in the same way. For example, the schema for USD files is significantly different from the schema of most building files 118, as explained in more detail below. Further, additional data 116 used to help plan actions related to constructing a building originates from numerous, diverse sources. Finally, information useful for planning actions related to construction of a building may be present in the building files 118 or additional data 116, but often is not organized in a manner that supports computer-based inference.

[0117] Addressing these challenges includes mapping elements in a building file or additional data, based on a schema for data used in the building file or additional data, to elements in a schema for data used by the animation and simulation tool. Elements in the building file or additional data which do not map directly to the schema used by the animation and simulation tool are captured in additional data structures which are then associated with the three- dimensional model. This information can be further processed to generate a building knowledge graph representing the building, by capturing dependencies and other relationships among objects, to assist in inference about planning construction of the building. Additionally, expertise knowledge graphs can be trained based on external information about building construction to capture general expertise which is useful for inference. The resulting parameterized model thus includes a three-dimensional model of objects with some properties specified as variable parameters, associated data structures capturing additional data related to the objects, an associated building knowledge graph. Expertise knowledge graphs are further created.

[0118] Referring now to Figures 2 through 8, further example implementations related to the parameterization of a model will now be explained.

[0119] Turning now to Figure 8, a summary of an example implementation of how such a parameterized model can be created and used will now be explained. Figure 8 is a flow chart summarizing an example implementation of processing of a building file to generate a parameterized model for animations, simulations, and instruction generation.

[0120] A first stage 800 of this processing is importing a building file. The import involves identifying (802) the type of building file or additional data, and parsing and extracting (804) data from the data file.

[0121] A second stage 810 involves converting the parsed and extracted data into a three- dimensional model, such as a USD file. This second stage includes applying (812) the schema for the three-dimensional model to the extracted data to generate three-dimensional object definitions with associated properties. For the input data that is not represented by the schema for the three-dimensional model, additional data structures are defined (814) to represent this additional information. Further data associating the additional data structures and the three- dimensional model are then created (816).

[0122] A third stage 820 involves generating the variable-type param eterizations for the model. This stage involves identifying (822) those properties in the model that should be variable for an animation or simulation. These properties are then set as parameters (824) and constraints on those parameters are defined (826).

[0123] A fourth stage 830 involves generating the association-type parameterizations for the model, in this case in the form of one or more building knowledge graphs. Based on the various input data, nodes of a building knowledge graph are extracted (832), and edges between them are constructed (834). The building knowledge graph can be augmented (836) with expertise knowledge graphs.

[0124] After completion of stages 810 through 830, the parameterized model is ready for use in a variety of applications. In the event that data of the model is changed, the parameterized model is in a form in which the data can be exported back out to building files. As indicated in stage 840, the parameterized model can be used in several applications for various purposes. Such applications include, but are not limited to, generating animations 842, generating simulations 844, generating instructions 846, querying the model 848. These and other operations in stage 840 are explained in more detail below. Example implementations of such processing of stages 800 through 830 will now be explained in more detail in connection with Figures 2 through 7.

[0125] In Figure 2, a processing pipeline 200 receives input data files 202, such as one or more building files, such as IFC data files, Revit data files, COBie data files, or AutoCAD data files, or any combination of these. Other types of building files or data files with additional data also can be inputs. The purpose of the processing pipeline 200 is to preserve as much semantic information as possible from the building files while converting the data in those files into a parameterized model for animation, simulation, and instruction generation.

[0126] The processing pipeline 200 includes a universal parser framework 210 which can be implemented, for example, using a plug-in style architecture such as found in the OMNIVERSE platform. With such an architecture, a parser 214 can be created for each type of input building file or file with additional data, and the parser can be implemented as a plug-in within theuniversal parser framework. The plug-in architecture can detect the input format, such as by relying on a filename extension or file metadata, to select the appropriate plug-in. The plug-in is configured to extract semantic data from the input file into the semantics of an internal data format.

[0127] As noted above, the kinds of semantic information available in building files and additional data varies depending on the type of building file or additional data. The tables provided in the Appendix represent the kinds of semantic information found in industry foundation classes (IFC) standard compliant data files (Table I), Autodesk REVIT files (Table II), Construction Operations Building Information Exchange (COBie) data files, which emphasize facility management data (Table III). Other kinds of data files may provide yet other semantic information. For example, the BIM Collaboration Format (BCF) data files provide information about coordination issues with viewpoints, discussion threads, visual annotations, and camera positions and availability. As another example, green building XML (gbXML) data files include information related to energy analysis, such as thermal zones and properties, as well as information related to HVAC systems, occupancy and equipment schedules, and weather and climate information. As another example, CityGML data files include information about urban context, such as surrounding buildings and infrastructure, terrain, such as topography and landscape, transportation, such as roads and transit systems, and utilities, such as core urban infrastructure networks and their access locations.

[0128] The appendix also provides (Table IV) a summarizes a difference between semantic information captured by the USD native schema and semantic information found in typical building files and additional data.

[0129] Because of these differences, a parser 214 the processing pipeline parses a building file or additional data to extract pertinent semantic information. Information that can be mapped into the schema used for the three-dimensional model for animation is mapped to that schema. Other information is mapped to one or more additional data structures which are in turn associated with the three-dimensional model. An example of such additional data structures is described in more detail in connection with Figure 4. A schema mapping module 216 can be used to map the information extracted from the input file into the three-dimensional model or the additional data structures, to output the three-dimensional (3D) model and additional (BIM) data 212.

[0130] A parameterization engine 218 then processes the resulting three-dimensional model and associated additional data structures to identify and parameterize properties for the various objects in the model which can be varied for animation or simulation. The set of properties to be parameterized may be a set of predetermined properties. Examples of variable parameters that can be identified are explained below in connection with Figure 5.

[0131] A knowledge graph generator 220 also processes the resulting three-dimensional model and associated additional data structures to generate a knowledge graph representing the building, which is referred to herein as a building knowledge graph 224. This building knowledge graph is a form of associated parameters for the parameterized model, examples of which are explained below in connection with Figure 5. For example, the knowledge graph generator 220 identifies components, such as a bottom plate, top plate, and studs of a wall. This data contains one or more of geometrical, material, or placement details, and may contain other details. The knowledge graph generator decomposes each structure, such as a wall, into its atomic components (e.g., studs, plates) and represents these elements as object nodes. These nodes are tagged with their properties (dimensions, material, position) and their relationships to other nodes are defined as edges. A more detailed explanation of an example implementation such a knowledge graph and a process for generating the knowledge graph is provided below in connection with Figure 3.

[0132] Another kind of knowledge graph can be generated by the system, which is referred to herein as an expertise knowledge graph. Input files 202 can include general construction expertise which can be used, by expertise knowledge graph generator 230, to train and output knowledge graphs 232 representing expertise with respect to certain subject matter, such as building codes, construction techniques, material handling, and so on. An artificial intelligence model trains a knowledge graph on a corpus specific to a domain, such as but not limited to scientific literature, engineering manuals, construction manuals, documentation and code base for the animation or simulation platform, or feedback from various sources. An example use of expertise knowledge graphs is explained below in connection with Figure 10.

[0133] Turning now to Figure 3, an example implementation for generating a building knowledge graph will now be explained in further detail. Figure 3 illustrates an example implementation of kinds of nodes in the building knowledge graph and their interrelationships.

[0134] A format node 300 represents the type of the source of data, such as a file type for a building file or additional data file from which information was extracted. A format node can have attributes such as a format type (“format”), an identifier for the source file (“id”), a version for the schema (“version”), and a type (“typeld”). Additional data extracted from this file may be stored as a parameter node 301 which has a “has_parameter” relationship with its corresponding format node.

[0135] A domain node 302 is a node representing semantic information that is extracted from the source represented by the format node 300. There can be many types of such nodes, representing spatial elements, building elements, task elements, resource elements, temporal elements, costelements, or yet other kinds of semantic information which may be extracted from building files or additional data. A domain node is associated with an object in the three-dimensional model by establishing a “maps to” edge with a USD node 304. Additional data that is extracted and stored that is related to this domain node can be stored as a set of attributes of a quantity node 303, which has a “has quantity” edge relationship with the corresponding domain node.

[0136] A USD node 304 is a node representing a specific object in the USD (or other three- dimensional model representation used), and includes a reference (“path”) to that specific object. This reference associates the knowledge graph with that specific object in the three-dimensional model. Other information about that object, such as its type (“primType”), its visibility (“visibility”), or material can be included as attributes of the USD node. Also, any additional metadata to be associated with that object can be stored as an attribute node 305 that has a “has metadata” relationship with the corresponding USD node. The system extracts geometric representations of building elements (for example, meshes, boundary representations, or parametric solids) from source building files and maintains them in the corresponding USD prims, so that the physical shape, dimensions, and spatial positioning of each element are preserved in the USD scene graph alongside their semantic attributes linked through the knowledge graph.

[0137] With such a knowledge graph, a complete semantic representation of a building can be captured, with corresponding associations between nodes in the knowledge graph and objects in the three-dimensional model of the building.

[0138] To generate a knowledge graph such as shown in Figure 3, a knowledge graph generator (e.g., 220 in Figure 2) implements a multi-stage pipeline to transform heterogeneous building data into structured knowledge graphs. The generation process begins with entity extraction, where the system identifies and classifies entities from the building files 118 and additional data 116 using a combination of schema-based parsing and natural language processing. For structured data sources like IFC or Revit files, the system applies deterministic extraction rules based on the source schema, mapping entities such as IfcWall, IfcDoor, or IfcBeam to corresponding graph nodes with preserved attributes. For unstructured or semi -structured data sources such as specification documents, construction manuals, or safety guidelines, the system employs named entity recognition (NER) models trained on construction-specific corpora to identify entities including materials, tools, techniques, and building codes. Each extracted entity is assigned a unique identifier and enriched with metadata including source provenance, extraction confidence scores, and temporal validity markers.

[0139] Following entity extraction, the system performs relationship discovery to establish edges between nodes in the knowledge graph. A variety of techniques can be used. For example, forexplicit relationships encoded in the source data, techniques such as spatial containment (wall contains window), structural support (beam supports slab), or system membership (duct belongs to HVAC system), can be used to directly mapped entities to typed edges in the graph. For implicit relationships, the system can employ one more techniques including, but not limited to: co-occurrence analysis within sliding context windows to identify frequently associated entities; dependency parsing of technical text to extract subject-predicate-object triples that indicate relationships; and spatial reasoning algorithms that infer relationships based on geometric proximity, intersection, or alignment of components in the three-dimensional model. The system also can use entity resolution and linking to merge duplicate entities across different data sources, using similarity metrics based on attribute matching, string similarity, and contextual embeddings. Relationships can be weighted based on confidence scores derived from extraction method reliability, source authority, and validation against construction rules encoded in the expertise knowledge graphs.

[0140] In some implementations, the knowledge graph construction process can employ an incremental approach that enables continuous updates as new data becomes available. Rather than rebuilding the entire graph when building files or additional data are modified, the system can maintain a change detection mechanism that identifies delta updates through document versioning, timestamp comparison, or content hashing. Modified entities and relationships are processed through a validation pipeline that checks consistency with existing graph structure, verifies compliance with construction ontologies, and identifies potential conflicts or contradictions. The system can implement a multi-level representation where base facts extracted directly from source data are distinguished from inferred knowledge generated through reasoning rules. Graph enrichment processes augment the extracted knowledge by computing transitive relationships (if A supports B and B supports C, then A indirectly supports C), aggregating properties up the spatial hierarchy (room properties aggregate to floor properties), and inferring missing relationships using graph completion algorithms trained on construction patterns. The resulting knowledge graphs are stored in a graph database with support for efficient traversal operations, complex pattern matching queries, and version control to maintain consistency between the knowledge graph and the evolving parameterized model.

[0141] Figure 4 is an example, reduced schema, for the purposes of illustration, for additional data structures that capture semantic information from building files and additional data which is not natively captured by the USD schema. In this example, the different types of data are subclasses of a root class named “BimBase Root” 400, which is defined as subclass of the enhanced USD foundation class. Each subclass has a name with a prefix “Bim” so that it is readily distinguished as data originating from the building files or additional data. In thisexample there is a subclass for different kinds of semantic information. In this example, a “BimSpatialElement” class 402 captures additional spatial elements such as a site (BimSite 410) or a story (BimStory 412); a BimElement class 404 captures additional elements such as MEP elements (BimMEPElement 406), such as ducts (BimDuct 408) and cables (BimCable 410); a BimSystem class 406 capture addition building system elements such as plumbing (BimPlumbing 420), HVAC (BimHVAC 422), and electrical (BimElectrical 424). A BimAsset class 408 captures information about assets within the building, such as furnishings (BimFumishing 426) and equipment (BimEquipment 428). A full implementation includes a subclass for each kind of semantic information to be captured from the building files and additional data which are not captured within the USD scene data. Table IV of the Appendix lists many of the kinds of data that can be captured with these additional data structures.

[0142] Turning now to Figure 5, examples of different kinds of parameterization will now be explained. In this example, there are two categories of parameterization: variable parameters 500 and association parameters 502.

[0143] Variable parameters are parameters of objects within a three-dimensional model that can be subject to animation or simulation. Example variable parameters include, but are not limited to, geometric parameters 504, motion parameters 506, state parameters 508, material parameters 510, environmental parameters 512, and physics parameters 514.

[0144] As further examples, geometric parameters can include, but are not limited to, the position, rotation, scale, dimensions, or other properties of an object that define that object in three-dimensional space. Motion parameters can include, but are not limited to, data that defines velocity, acceleration, a path or trajectory, angular velocity, and handling constraints. Such motion parameters can be associated with specific geometric parameters which they influence. State information can include, but are not limited to, any property that has multiple possible states for an object, such as on or off, open or closed, installed or not installed, a flow rate, a temperature. State information can be defined, for example, by a set of discrete values, a range, or a function. Material parameters can include, but are not limited to, data representing how an object is visualized in animation or simulation, or how it should appear in real life, and can include information such as transparency, color, texture, technical properties, finishing, or other visual parameters. Environmental parameters can include, but are not limited to, information about emissions, water consumption, energy consumption, or carbon dioxide footprint. Physics parameters can include, but are not limited to, information about weight, load limitations, processes, or other factors.

[0145] Association parameters 502 are parameters associated with a scene, or one or more objects in a scene, or one or more properties of objects in a scene, that associate information notcaptured in the schema for the three-dimensional model but are relevant either to simulation or for data consistency or completeness and, in some embodiments, for cost estimating, planning and schedule optimization, workforce direction and management, monitoring and evaluation of field performance, and linkage to real-world events or operational parameters (for example, linking global supply-chain data for sensitivity analysis). Example association parameters include, but are not limited to, task dependencies 516, system relationships 518, spatial relationships and constraints 520, operational rules 522, and physics constraints 524. Association parameters are typically represented in building knowledge graphs or expertise knowledge graphs indicating relationships between two or more objects.

[0146] As further examples, task dependencies include data representing dependencies between actions related to objects in the model. For example, a task related to an object may precede, require, or trigger a task related to that object or another object or objects, or a skillset needed for that task. System relationships include data representing functional, structural, or operational dependencies between objects in the model. For example one object may feed, control, or monitor another object or objects. Spatial constraints include data representing spatial or structural relationships among objects in the model, such as whether one object contains, supports, or is adjacent to, one or more other objects. Operational rules include data representing logic to be applied in animation, simulation, or modification of the model. Such rules can be defined in a variety of ways, such as by using if-then rules, applying thresholds to enforce limits on values or changes to values, applying schedules, and so on. Physics constraints include data defining physical constraints on one or more objects, such as joints or parts of a system.

[0147] An illustrative example of a parameterized model, using a simple structure, a garden shed, will now be described in connection with Figures 6 and 7.

[0148] In this example, as shown in Figure 6, the three-dimensional model of the garden shed building is defined in part by static data 600, including information 601 describing the building as a whole, its foundation 602, which supports four timber framed walls 603, which support a roof 604. One of the walls supports a door 606 and a window 608. Each of these objects can be defined by a three-dimensional object and associated attributes.

[0149] A parameterization layer 610 includes data representing construction parameters 612, operational parameters 614, material parameters 616, and skills parameters 618. For example, construction parameters may include information about visibility, completion percentage, or state of assembly. Operational parameters can include information such as a door opening angle, a window slide position, or a light power switch location. Material parameters can include information such as wood weathering, metal corrosion, or paint fading. Skills parameters caninclude information such as identifiers for contractors. Such parameters can be represented as variable parameters or associated parameters.

[0150] Connections 620 with supporting information are enabled between the parameterization layer elements and other information, which can be represented by knowledge graphs. Examples of such other information include, but are not limited to the following. A four-dimensional construction sequence 622 can include, for example, a day-by-day schedule of construction such as foundation on day 1, framing on day 2, cladding on day 3 and finishing on day 4. Operations information 624 can include, for example, indications of whether heavy equipment is required, skills required, and materials required for assembly. Assembly instruction 626 can include, for example, indications of tools required, a sequence of steps to perform, and safety checks. Information about environmental simulations 628 can also be provided. Each of these can be represented by knowledge graphs which are connected to elements in the parameterization layer.

[0151] A corresponding illustrative knowledge graph representing this garden shed is shown in Figure 7. The knowledge graph includes nodes that represent actions, such as pouring the foundation, erecting walls, installing walls, mounting the roof, and fitting the door, and edges to indicate which actions precede others. The knowledge graph includes nodes that represent elements of the building, such as the foundation, frame, walls, doors, and windows, and edges that represent structural relationships among them. Further edges connect the nodes representing building elements to nodes that represent the materials the building elements are made of, such as concrete, timber, metal, and glass. Thus, such a building knowledge graph represents the structural relationships among elements, the material relationships among elements, and the construction dependencies among such elements.

[0152] Having now explained example ways to generate the parameterized and labeled model 128, and returning back to Figure 1, the parameterized and labeled model 128, and optionally the additional data 116, can be used to perform such tasks as animation, simulation, or instruction generation or other tasks, when they are inputs to several modules of a larger system.

[0153] In Figure 1, the parameterized and labeled model 128, and optionally the additional data 116, are input to an animation and simulation module 142. The animation and simulation module 142 includes a computer program executing on a computer which has sufficient computer processing and computer storage to handle complex rendering operations of three-dimensional models and to perform complex multivariable simulations including realistic simulation of real- world physics. Preferably such a platform also has an application programming interface to allow users to add functionality or integrate with other software. The platform also preferably enables real-time, multi-user collaboration. Such a platform also preferably incorporates tools for AI- driven content creation, predictive modeling, and procedural generation. The platform preferablysupports cloud-based infrastructure to facilitate collaboration across distributed teams. For example, platforms that include these capabilities include the UNITY platform, the UNREAL ENGINE platform, and the OMNIVERSE platform available from NVIDIA. In typical implementations, the platform includes these aforementioned capabilities as well as support for a universal scene description format (USD).

[0154] An example implementation of the animation and simulation module 142 includes the OMNIVERSE platform available from NVIDIA. The OMNIVERSE platform also may be used to implement the model parameterization module 122 and be connected to receive the building files 118 and additional data 116, including live updates of those data. In some implementations, the computer system includes the NVIDIA OMNIVERSE platform and other related solutions from NVIDIA.

[0155] In some implementations using the OMNIVERSE platform, some modules that can be used include the following. The OMNIVERSE Kit is a toolkit for building custom Omniverse applications. This toolkit can be used to create specialized interfaces and workflows to implement the techniques described herein. OMNIVERSE Create is a module used for real-time 3D content creation and visualization which supports real-time ray tracing, physically accurate materials, and complex scene composition. This module can be used for high-fidelity simulations and animations for the techniques described herein. The OMNIVERSE Nucleus module is a collaboration and data management layer, which allows real-time collaboration among various components and users. The computer system can use this model to ensure that all components and users use the latest version of the parameterized model. This model also allows users to collaborate in real-time across different software environments. The OMNIVERSE Farm module handles distributed rendering and simulation tasks. This module provides the computer system with the ability to scale up simulations and renderings efficiently across multiple processors. The OMNIVERSE Al module provides integrated artificial intelligence (Al) tools that support generative design, optimization, and real-time decision-making processes during simulation. This module also includes many components, such as generative artificial intelligence and machine learning workflows.

[0156] Among the related solutions from NVIDIA are the following software modules. The NVIDIA Modulus includes a physics-machine learning model framework. This module can be used in the computer system described herein to perform physics simulations that are both data- driven and physically accurate. The NVIDIA Inference Microservice (NIM) module includes a suite of inference microservices that accelerate the deployment of artificial intelligence models. The NIM module can be used in the computer system described herein for optimizing inference tasks in simulations and animations. The NVIDIA RTX module provides real-time,photorealistic rendering. This module can be used to provide realistic visualizations, especially with respect to light-material interactions.

[0157] Using a platform like the OMNIVERSE platform has several benefits. The platform supports and is compatible with numerous industry-standard data file formats, which enables the computer system to integrate with many data sources, such as tools like Autodesk Maya, 3ds Max, Blender, and others. The platform supports complex simulations, including physics-based animations and simulations relevant to constructing a building. The platform also supports workflows that involve both building files (such as CAD or BIM models) and live data feeds of such files.

[0158] A control interface 160 provides scripts 140 specifying characteristics of an animation or simulation to be performed by the animation and simulation module 142 or of instructions 172 to be generated by an instruction generation module 170 (explained below). The control interface 160 also can initiate the operation of the animation and simulation module 142 or of the instruction generation module 170.

[0159] The characteristics of animations or simulations to be performed, or instructions to initiate operations, are identified in Figure 1 as scripts 140. A "script" is any type of computer program that instructs an animation or simulation module about an animation or simulation operation to be performed or that instructs an instruction generation module about instructions to be generated. Thus, a script includes, but is not limited to, a computer program written in a scripting language like Python or JavaScript, compiled programs, declarative formats (e.g., XML, JSON), node-based programming, state machine definitions, animation curves, configuration files, executable workflows, shader programs, behavior trees, or physics simulation descriptors. In some implementations, such instructions can include a specification of computational models to be used or trained, such as artificial intelligence or machine learning models, reinforcement learning policies, generative models, procedural content generation algorithms, or neural motion controllers.

[0160] In some implementations, the control interface 160 includes a graphical user interface used by an individual. With such a graphical user interface, the individual can specify characteristics of an animation or simulation 150 to be performed or of a set of instructions 172 to be generated. In some implementations, the control interface 160 includes a command line interface used by the individual to establish characteristics of an animation or simulation to be performed or of a set of instructions to be generated.

[0161] In some implementations, the control interface 160 comprises a generative artificial intelligence component configured to transform input data into characteristics of an animation or simulation to be executed or of a set of instructions to be generated. For example, the generativeartificial intelligence component can translate natural language inputs into scripts and parameters for an animation or a simulation. As another example, the generative artificial intelligence component can translate natural language inputs into scripts and parameters for generating step- by-step instructions for an action.

[0162] In some implementations, the control interface 160 further operates as a design co-pilot that, upon receiving early-stage or conceptual design alternatives, queries the parameterized and labeled model 128 and additional data to simulate feasible construction approaches, generate order-of-magnitude or detailed cost and schedule impacts, and recommend scope adjustments, in order to enable users such as owners, architects, and builders to evaluate options and narrow design space prior to committing to detailed design.

[0163] In some implementations, the control interface 160 further operates as a risk assessment component configured to analyze outputs of the parameterized and labeled model 128 and associated simulations in order to identify bottlenecks or chokepoints in task sequences, perform sensitivity testing across variables such as material availability, workforce allocation, material or labor costs, or environmental conditions, and generate risk metrics including likelihood of delay, probability of cost overrun, exposure to disruptions, or resilience under alternative scenarios. The system can consume such risk assessment outputs to adjust sequences, resource allocations, or instructions, and can further be formatted for consumption by external systems or stakeholders, including financial, insurance, or project-management platforms, to support evaluation and decision-making processes.

[0164] In some implementations, the control interface can be implemented using, or can include, an animation graph editing module, which allows users to construct animations using node-based representations. Animation graphs, like those utilized in platforms such as NVIDIA OMNIVERSE, include a set of interconnected nodes that define states, transitions, and behaviors within an animation. Each node represents a specific function, such as a transformation, physicsbased interaction, or artificial intelligence-driven decision point. Edges between nodes represent dependencies or transitions between these states.

[0165] There are several types of nodes within an animation graph. Input nodes are used where user-defined parameters, variables, or current real-world data are introduced into the graph. Processing nodes specify mathematical, logical, or procedural operations to be applied, such as calculating forces, applying shaders, or adjusting animation timing. Output nodes execute the final animations or generate instructions for physical systems or for further computer system operations. Control nodes manage the flow of execution, including triggering sequences and setting priorities. Event nodes respond to specific triggers or conditions, to initiate or modify animations or simulations. State nodes represent different conditions or modes and managetransitions between these states. Data nodes store, retrieve, and manage information used within the graph, facilitating efficient reuse of data. Constraint nodes enforce rules and limitations. Feedback nodes dynamically adjust inputs or processes based on real-time data. Animation graphs can be stored in any of a number of formats, including but not limited to configuration files for the universal scene description (USD) format. In some implementations, the animation graph can include selections of techniques and modules from pre-existing libraries, or users can define modules to integrate various tools or workflows.

[0166] In some implementations, the control interface can be implemented using, or can include an action graph editing module, which allows users to construct simulations using node-based representations. Action graphs, like those utilized in platforms such as NVIDIA OMNIVERSE, include a set of interconnected nodes that define states, transitions, and behaviors within a simulation. Within the OMNIVERSE platform, action graphs are event-based; they can be combined with another graph called a push graph which includes a continuously evaluated node. Each node represents a specific function, such as a transformation, physics-based interaction, or artificial intelligence-driven decision point. Edges between nodes represent dependencies or transitions between these states. Typically one node tracks the passage of periods of time, which trigger operations to be evaluated as functions of time.

[0167] Custom animation graphs can be built to define specific animation behaviors, utilizing user-defined functions or machine learning models to generate procedural content or simulate complex interactions. Inputs to these nodes may include variables, such as but not limited to position, rotation, and time. Using action graphs, users can create simulations that account for techniques and constraints, such as physical behaviors, environmental conditions, or performance goals. The flexibility of animation and actions graphs, where nodes and edges can be added, removed, or reconfigured to build animation or simulation pipelines, allows complex workflows to be specified.

[0168] The OMNIVERSE platform provides additional graph structures that can be used to enable animation or simulation of other aspects of the model. For example, material graphs can be used to define visual appearance of materials. Physics graphs can simulate real-world physics and object interactions. MDL graphs create physically based, realistic materials. Al-based graphs leverage Al for tasks such as asset management. Deformer graphs control mesh deformation for character animation.

[0169] In some implementations, computer programs, herein called "agents", can be implemented to perform all or part of many of the functions described above for scripts. In some implementations such agents can be embedded into and operate with the OMNIVERSE platform or similar platforms.

[0170] For example, one kind of agent includes agents that generate and manipulate graphs that define animations, simulations, or scripts that instruct the instruction generation module. As an example, to set up a simulation, the user or a project manager agent provides values for simulation parameters, such as the type of fastening or material properties. These inputs are fed into the system's action graph. The action graph with the set values is loaded into a simulation environment. For example, an OmniGraph file is executed in the simulation environment (e.g., NVIDIA OMNIVERSE), where the assembly process is visualized. An agent can be provided that adjusts the action graph based on real-time conditions, such as tool availability or material delays.

[0171] Another kind of agent is a domain-specific agent, each of which uses a specialized expertise knowledge graph. Such agents can be used, for example, to assist in generating action graphs. For example, each object representing an element of a building can be assigned to one or more domain-specific agents (e.g., structural agents, tool agents, construction method agents) that manages that element's assembly. For example, a tools agent determines that studs need to be fastened using nails and that the plates need to be aligned with a predefined tolerance. For each element, the system creates action nodes that represent the specific tasks to assemble the element. These tasks may include but are not limited to cutting, fastening, and aligning the elements. Agents define the actions used for each task. Agents then analyze dependencies between components (e.g., the bottom plate must be in place before the studs are attached) and automatically create edges between action nodes that define the order of operations. After nodes and edges of a graph are defined, an agent generates an output file representing the graph.

[0172] In some implementations using an OMNIVERSE platform, the output file from an agent can be an OmniGraph file. This file stores the action graph and provides instructions for the simulation of physical assembly of components. The nodes represent specific assembly actions, while the edges indicate task dependencies (e.g., “Attach stud 1 to bottom plate”). Thus, instead of manually building this graph via the OmniGraph visual editor, the agents automate the process by applying predefined rules or learned patterns to the components and their relationships.

[0173] In some implementations, these agents perform these tasks automatically in the following ways. Agents generate action nodes by referencing expertise knowledge graphs. For example, the tools and equipment agent pulls data on the necessary tools (e.g., hammer, nail gun) and creates action nodes for attaching the studs to the plates using nails. Agents responsible for construction sequencing evaluate the order in which tasks must occur. They use algorithms to calculate dependencies, creating edges between action nodes. For example, the construction agent might specify that the top plate cannot be attached until all studs are in place. Agents collaborate to create a connected graph of actions, storing it in an OmniGraph file format. Thisstep automates what would typically be done manually in an OmniGraph visual editor. The action graph includes nodes representing actions (e.g., "cut the stud to size") and edges representing the dependencies (e.g., "fasten the stud after cutting"). Before execution, agents validate the action graph to ensure it complies with construction standards and safety regulations. Once validated, the simulation can proceed, driven by the OmniGraph file.

[0174] Thus, the term "scripts" encompasses a wide range of possible types of computer instructions that specify animation or simulation operations or that instructs an instruction generation module about instructions to be generated. In some implementations, the control interface, or a portion thereof, can be implemented as part of the animation and simulation platform, such as the OMNIVERSE platform.

[0175] In some implementations, changes to the building files 118 or additional data 116 can be provided, or notifications about such changes can be sent, to the control interface 160, to allow animation and simulation scripts 140 to be regenerated or to instruct that an animation or simulation to be re-executed.

[0176] Given a script 140 for an animation or simulation of an action relating to constructing a building, the animation and simulation module 142 uses the parameterized and labeled model 128, and current data from the additional data 116 to perform the specified animation or simulation, which generates corresponding animation or simulation output 150. This animation or simulation output 150 can be in many forms. For example, a three-dimensional virtual assembly of building components that involves a detailed step-by-step simulation, such as the installation of structural elements to construct a wall, can demonstrate the entire process and can identify potential clashes or inefficiencies prior to actual construction.

[0177] Turning now to Figure 9A, an example of using a parameterized model to perform an animation will now be described, using an example of operation of a door. As an illustrative example, a door is represented in the three-dimensional model with a set of static properties 900 and variable properties 910. For example, the static properties might include its size 902, the type 904 of door, such as hinged, sliding, or other type, its material 906, such as wood, fiberglass or other material, and fire rating 908. For example, the variable properties might include its locked state 912 (locked or unlocked), hinge friction 914, an angle of opening 916, and rotation of its handle 918. Various constraints 920 can be defined for the model of the door. For example, as indicated at 922, if the door state is locked, then the opening angle is limited to zero (0). As another example, as indicated at 924, a minimum handle rotation can be imposed before the door can be opened. As another example, as indicated at 926, the opening angle can be limited by presence of another object, such as a wall. Also, as indicated at 928, the speed of the door in theanimation can be limited by the mass of the door, hinge friction, and assumed force applied to the door.

[0178] An animation curve 930 also is associated with the door object. An animation curve defines one or more piecewise linear functions of one or more properties of one or more objects over time. In this example, the animation curve determines the angle of the door and handle rotation over time, and is defined by four points in time: zero (0.0) seconds, 0.5 seconds, 2.5 seconds and 3.0 seconds, with set values for the opening angle and handle rotation at each of those points in time (e.g., 0,0; 0,45; 85,45; and 90,0 in degrees respectively for angle rotation, handle rotation). In application, the animation curve is evaluated to generate the animation, providing set values for each variable property for each point in time over the animation, enabling the model to be rendered to provide a respective image for each point in time, with the output of the animation curve limited by the constraints 920. For example, if the state of the door is locked, the animation would not result in rendering the door opening. As another example, if the door is heavy, has lots of friction, and the presumed force is low, the determined speed at which the door can open may be used to limit the output of the animation curve.

[0179] Using such animation techniques with the parameterized model, it is further possible not only to animate operation of a building element such as a door, but also to animate or simulate its construction. To animate construction, the presence or absence of an element can be a variable parameter of that element, as well as its position. Further, a sequence of animations of elements can be used to visualize the sequential installation of each of those elements.

[0180] As an example, turning now to Figure 9B, an illustration of using a parameterized model to specify an animation or a simulation of constructing a wall assembly will be described. In this example, a wall assembly is represented in the three-dimensional model with a set of static properties 950 and variable properties 960.

[0181] By way of example, static properties 950 can include wall length 951, wall height 952, framing member type or material 953, such as nominal 2^4 SPF or 2x6 LVL, sheathing type 954, nominal stud spacing 955 such as 16 inches on-center, and opening descriptors 956 that define the location and size of doors or windows.

[0182] Variable properties 960 can include, for each component such as a bottom plate, studs, king or jack studs, a header, a top plate, or a sheathing panel, an installed state 961, a placement pose defined by position and orientation 962, fastening parameters 963 such as fastener type and count that satisfy a fastening schedule, and in some embodiments a tilt or bracing state 964 to support flat assembly and tilt-up sequences.

[0183] A set of constraints 970 governs feasible states and ordering. Examples include layout and plumb tolerance 971 that aligns studs to layout marks within tolerance, fastening schedulethresholds 972 that prevent a component from transitioning to installed until required fasteners are applied, opening and header rules 973 that bound adjacent member placement, sheathing landing requirements 974 that require panels to land on studs and meet edge-nailing, ordering constraints 975 such as requiring a header and jack studs before advancing sheathing over a corresponding opening and requiring bracing before tilt-up, and in some embodiments clearance or safety constraints 976 that limit access, reach, or tool operation.

[0184] One or more animation curves 980, or another state-progression function, can be defined, and in turn evaluated, to specify and visualize changes of state over time. However, in contrast to animation of an installed component, such as a door, the animation curves 980 are specified and controlled based on the sequence of tasks used to construct the component.

[0185] To this end, the system derives, such as from the model 950, 960 and constraints 970, a task graph 981 of discrete construction steps. Example steps for a wall construction include placing the bottom plate, placing studs, installing a header, and installing sheathing. Each task corresponds to one or more state transitions. The sequence of tasks encodes precedence and can allow parallelism. Each task may be associated with a duration parameter 982 defined as a fixed value, a range, or a distribution, and with resource requirements 983 that specify labor roles and skill levels, crew size, and tools and equipment. A calendar and capacity module 984 applies working hours and holidays, crew and equipment availability, and spatial concurrency to produce a scheduled sequence 985 with start and finish times, critical path, and float.

[0186] Thus, for each task, the corresponding task-dependent animation curve(s) can include time-dependent functions of the presence, position, and orientation, or other property of each element related to the task. In some implementations, the various resources for the task, such as tools or people, also have corresponding objects in the animation which can be animated.

[0187] During visualization, the animation curves 980 are driven by the task graph 981 according to the scheduled sequence 985. In some implementations, various constraints imposed by the model, such as collisions and physics, can limit the operation of the animation curves. In some implementations, the task states can advance only when the corresponding constraints, resource requirements, and capacity conditions are satisfied.

[0188] In some implementations, a number of other analyses can be performed for the animated construction. For example, a cost accumulator can compute labor, equipment, and consumables by applying corresponding rates to scheduled times and can apply productivity modifiers such as height, access, or weather. As another example, a conflict detector can evaluate scheduled activities together with model geometry and constraints to flag temporal or spatial conflicts such as resource over-allocation or interference or clearance violations. Such conflicts can be used to trigger automated resequencing of the task graph 981 and schedule 985, subject to constraints970. From the scheduled task graph (985, 981), the system can generate instructions that reference model element identifiers, required tools and materials, fastening schedules, and tolerances. Instructions may be ordered by the scheduled sequence and may include step durations and crew roles. Properties and relationships described above may be represented as attributes on scene-graph elements in a USD representation of the building to enable execution and playback in Omniverse.

[0189] Note that this example is illustrative and non-limiting, and other properties, constraints, task decompositions, scheduling methods, and cost models may be used.

[0190] An example implementation of the use of a knowledge graph layer to implement domain specific agents using expertise knowledge graphs will now be explained in connection with Figure 10. Each knowledge graph in a knowledge graph layer 1000 includes a set of nodes 1002, edges 1004 and a relationship 1006 with the building file or three-dimensional model for which it was generated.

[0191] Domain specific agents 1010 access expertise knowledge graphs in the knowledge graph layer 1000 through a reasoning engine 1020. Example domain specific agents include, but are not limited to, a construction agent 1012 which performs such tasks as sequence planning and resource allocation, a safety agent 1014 which performs such tasks as hazard detection and compliance checking, and a maintenance agent 1016 which performs such tasks as lifecycle tracking and service scheduling. A variety of different kinds of agents can be created and the invention is not limited to these examples.

[0192] The reasoning engine 1020 can include a query processor 1022, such as SPARQL or Cypher query engine, to apply queries to the expertise knowledge graphs. The information from expertise knowledge graphs is used by the domain specific agents to generate initial inferences, which are inputs back to the reasoning engine 1020. A rule-based inference engine 1024 provides its outputs based on its programmed logic, which are then constrained, based on the constraints imposed for the model, by a parameter checker 1026.

[0193] The outputs 1030 of the reasoning engine 1020 depend on the domain specific agents. For example, a construction agent produces step-by-step instructions 1032, a safety agent produces warnings or other safety alerts 1034, and a maintenance agent produces schedules and task timeline 1036.

[0194] In some implementations, the knowledge graph layer further supports a financial agent configured to generate financial data outputs from the parameterized and labeled model and simulated construction sequences. The financial agent accesses activity definitions, resource allocations, fabrication steps, logistics plans, and alternative scenarios produced by the animationand simulation module, and links these simulation outputs to cost assemblies, productivity norms, vendor catalogs, financing terms, and market data represented as nodes and edges in the knowledge graphs. The reasoning engine applies rules and probabilistic analysis to the simulated construction sequences to calculate schedule-dependent costs and cashflow timing, quantify contingency reserve requirements, and determine progress-based valuation of work performed for use in payment certification or loan drawdowns. The same pipeline can also generate valuecreation outputs, including identification of margin gains from optimized sequencing, release of capital through reduced contingency requirements, acceleration of revenue by simulating earlier delivery of rentable or saleable space, and phasing strategies aligned with market absorption or financing milestones. In this way, the financial agent transforms the simulation and knowledge graph framework into data that captures both financial risks and value opportunities inherent in a construction project.

[0195] An animation or simulation can also be used to identify modifications to building components which can be more efficiently accomplished during initial material fabrication rather than after assembly, such as making cavities in building components to allow passage of MEP systems. To identify such modifications, MEP routes through components of the wall are simulated in the animation and simulation module 142, and translated into specific modifications to the building components, such as the creation of holes or cut-outs through studs, joists, or other individual components of walls, floors, or other components, to enable the MEP systems to pass through without obstruction and follow a defined route. Such modifications for MEP routing can be simulated as applied either within a model of the fully constructed building, or as applied to individual components in their not yet assembled state, such as wall studs. The specific locations, dimensions, and angles of such MEP routing cavities can be output via the instruction generation module 170 for use in fabrication of components, for example, through a cut list that specifies the fabrication of wall studs from input materials such as dimensional lumber.

[0196] As another example, the simulation may involve several options for staffing and resource allocation, simulating different staffing configurations, equipment usage, and resource distributions, thereby providing insights into associated costs, timelines, and potential bottlenecks. Further examples can include simulations of environmental impact, wherein different construction methods or materials or building system layouts are simulated to determine their effects on the building's environmental footprint, such as energy efficiency, energy consumption, carbon emissions, or resource usage, which may be particularly valuable for projects targeting sustainability certifications like LEED or BREEAM. Fabrication simulationsmay be employed in coordination with a building simulation to model the preparation of building components from raw materials, ensuring that components are fabricated in an efficient manner and in time to be transported to the final installation location for timely use during building assembly. Logistics and supply chain simulations may also be employed to model the timing and routing of material deliveries, ensuring that materials, whether in raw form or in processed form as fabricated building components, arrive on-site, and to either the fabrication location or the final installation location within the building, precisely when needed to minimize delays and storage costs. Optimization through artificial intelligence can also be incorporated, wherein artificial intelligence-driven algorithms optimize various aspects of the construction process, such as resource allocation, construction scheduling, and multi-objective optimization to balance trade-offs between cost, time, quality, and sustainability, thus enhancing overall project efficiency and responsiveness to unforeseen changes.

[0197] As another example of the use of an animation or simulation output 150, the results of an animation or simulation can be used to review and assess one or more proposed actions related to constructing the building, as represented by the optimization module 132. Examples of such a review include, but are not limited to, evaluating one or more proposed actions, performing additional analysis of a proposed action, optimizing the proposed action by iterating through many options to meet certain optimization goals, identifying errors, or to calibrate the model. In some implementations, the optimization module, or a portion thereof, can be implemented as part of the animation and simulation platform, such as the OMNIVERSE platform. Typically, implementations of the optimization module 132 include one or more presentation devices, such as a display or audio output, which receive the animation or simulation output 150 and allow one or more individuals to interactively view the animation or simulation while performing additional tasks.

[0198] For example, the optimization module 132 may be used by one or more individuals to view and assess the results generated by a particular animation or simulation. For example, a team of construction workers or managers can view a virtual rendering of the assembly of part of a building to improve understanding of tasks to be completed.

[0199] As another example, the optimization module 132 can receive as inputs one or more optimization goals 106. Such optimization goals 106 also may be input through the control interface 160 and incorporated into the animation or simulation script 140. The optimization module 132 may be used to present one or more animations or simulations in connection with the optimization goals, or to analyze the implications of such animations or simulations. For example, a construction manager can review several options for constructing a part of a building, such as construction sequences and workforce allocation options, and compare those options andtheir computed dependencies and implications to the optimization goals 106. In some implementations, the outcomes of the different options can be presented to a human operator for selection. In some implementations, the computer system can select the best option or options with respect to data representing optimization goals. In some cases, the optimization goals 106 may be revised and the animation or simulation may be re-executed through the control interface 160.

[0200] As another example, the optimization module 132 can include an interface through which the optimization module 132 can be instructed to process information provided in the animation or simulation output 150 to generate additional data. For example, the construction manager, after reviewing several options for constructing a part of a building, could request the system to generate a list of the personnel assigned to the tasks, or to compute values not provided explicitly in the simulation results, such as total time for an individual worker to be onsite. In some implementations, the optimization module 132 may generate feedback 130, such as proposed changes either to the model (processed through the model parameterization module 122) so that the model is updated, or to the animation and simulation desired (processed through the control interface 160) so that the animation or simulation may be re-executed.

[0201] As a further example, the optimization module 132 can include functionality for both identifying design optimizations, and conducting comparative analyses between an original building design delivered from a building file 118 and its optimized alternatives generated by this computer system. Through the control interface 160, a user such as a project manager can interact with the optimization module 132 to assess various optimization goals 106, such as energy efficiency, construction cost, and operational costs. For instance, the optimization module 132 can be directed to analyze a standard MEP system configuration and layout specified within an original architect-provided building file 118 and modeled in the parameterized and labeled model 128, and identify an optimized configuration and layout that improves energy efficiency which can be animated or simulated in the animation and simulation module 142. Such an optimized layout might, for instance, reroute ductwork, piping, and electrical conduits to achieve better energy efficiency, while considering issues such as preserving fire safety barriers, maintaining thermal insulation integrity, ensuring acoustic separation, and avoiding load-bearing elements to maintain structural safety. Additional data 116 relating to the site and environment around the building could further be incorporated into the animation and simulation module 142 and used to analyze site-specific factors such as solar orientation, wind patterns, and topography in order to optimize HVAC placement for passive solar gain and natural ventilation, and thereby minimize energy use. Based on the direction of a user such as a project manager to incorporate design optimizations into an instance of the animation and simulation module 142, theoptimization module 132 can generate feedback 130 which forms the basis for modifications to the design input to the model parameterization module 122 and represented within the animation and simulation module 142. Utilizing the comparative analysis capabilities of the computer system as represented within the optimization module 132, the project manager might request the system to simulate and compare the up-front investment and long-term operating costs for each option. Based on the results, the optimization module 132 could provide feedback 130, such as presenting comparative financial data or presenting additional design scenarios that balance initial investment against future savings. The control interface 160 could then be used to refine the inputs and re-execute the simulation, enabling the manager to explore additional scenarios and ultimately make data-driven decisions about which design to construct.

[0202] In some implementations, the optimization module 132 can be built using one or more components providing artificial intelligence or machine learning or predictive modeling. For example, a large language model can be used to generate the scripts using natural language descriptions of the optimization problem to be performed. In some implementations, feedback based on the animation or simulation, such as selection of options by contractors, or results of comparing real-world data to the generated instructions, can be used to train models. Information about best practices and historical data can be used to train models. Feedback or identified patterns also can be used to dynamically adjust simulations or animations. In some implementations, the artificial intelligence component can be configured for model parameterization and simulation optimization. The artificial intelligence component can automatically identify and parameterize relationships within a building model using machine learning algorithms to, for example, detect patterns in material usage and configuration, dependencies including structural and sequence-of-assembly dependencies, or optimal workforce utilization. This component handles parameterization tasks related to geometry, physics, or dependencies. Predictive modeling or optimization based on datasets can be managed by an artificial intelligence component through advanced algorithms. In some implementations, an artificial intelligence component can optimize aspects of simulations, such as resource allocation, timing, and sequencing, through real-time adjustments, can perform complex multi-variable optimizations using deep learning models, or can refine the simulation process according to statistical methods or other parameters.

[0203] In some implementations, the parameterized and labeled model 128 as animated or simulated by the animation and simulation module 142, implements a digital twin of the building or a part of the building or an action related to constructing the building. When the computer system has access to real-world data related to the action, that real-world data can be compared to this digital twin. Such comparison can be used for many purposes, such as to update or modifythe digital twin or to provide feedback about the real-world performance of the action. Feedback into the system of real-world data also can trigger the computer system to perform a variety of other operations, such as re-execution of an animated or simulated action, or regeneration of a sequence of steps for an action.

[0204] Accordingly, as indicated in Figure 1, on-site sensors 158 provide onsite data 152. Examples of onsite sensors include, but are not limited to, still image and video cameras (whether visible, infrared, or multispectral imaging), microphones, heat, temperature, humidity, wind, pressure and other environmental sensors, bar code readers, LIDAR, location sensors such as GPS, tablet or other computers used to track worker attendance, material inventory, or equipment usage, and sensors on equipment that monitor activity or usage. The onsite data 152 collected by onsite sensors 158 are input to a comparison module 154. The comparison module 154 compares the onsite data 152 with the animation or simulation output 150.

[0205] In some implementations, the comparison module 154, or a portion thereof, can be implemented as part of an animation and simulation platform, such as the OMNIVERSE platform. Typically, implementations of the comparison module 154 would include one or more presentation devices, such as a display or audio output, which receive the animation or simulation output 150 and allow one or more individuals to interactively view the animation or simulation while viewing related visualization of the onsite data 152. In some cases, when the onsite data is being viewed in real-time, the comparison module 154 may make comparisons between the onsite data 152 and the animation or simulation output 150 and provide real-time feedback 146. Such feedback 146 can be input to the animation or simulation module 142, instruction generation module 170, or optimization module 132.

[0206] As an example implementation of the comparison module, images, or video of a part of a building based on the animation or simulation can be presented on a display adjacent to images or video of that part of the building as constructed. In some implementations, either the animation, or simulation, or images or video of the building as constructed, can be overlaid over the other in a visual representation, for instance to facilitate comparison. In some implementations, an augmented reality graphics, image, or video based on the animation or simulation can be overlayed on a view of part of the building as constructed. In some implementations, the augmented reality graphics, image, or video can be overlayed on a view of part of a building to be constructed, to help workers conceptualize the next steps, or further steps, in constructing that part of the building. In some implementations, the comparison module 154 can compare a structure in the constructed part of a building to corresponding structures in the parameterized model, and provide real-time feedback 146 such as signaling an alert if there is a mismatch, such as might occur if a component is installed incorrectly. Similarly, the comparisonmodule 154 also can be used to monitor robots doing construction. In some implementations, the feedback can be directed to the instruction generation module 170 so that updated instructions 172 can be generated. The feedback also can indicate that a section of a building has been successfully completed and can indicate to the instruction generation module 170 to generate instructions for a next stage of construction.

[0207] In some implementations, such feedback indicating progress, completion, or discrepancy status for particular construction steps or objects can be further used by the system to generate payment facilitation outputs. The payment facilitation outputs can include release instructions or authorization holds, enabling automated, objective, and auditable progress-based financial transactions with subcontractors, vendors, or other parties.

[0208] An example implementation of how animation or simulation results can provide feedback to update the parameterized model will now be explained in connection with Figure 12. As shown in Figure 12, a parameterized USD model 1250 includes a USD scene 1252 with various parameters which can be animated or simulated, such as material parameters 1254, geometric parameters 1256, and behavioral parameters 1258. Examples of material parameters include but are not limited to density, friction, and elasticity. Examples of geometric parameters include but are not limited to mass, volume, and cost of goods. Examples of behavioral parameters include but are not limited to sensors, controllers, and rules.

[0209] The system can also include access to various data representing environmental factors 1260. Example environmental factors include weather conditions 1262, usage patterns 1264, and time factors 1266. Examples of weather conditions include but are not limited to precipitation, wind, humidity, and temperature information. Examples of usage patterns include but are not limited to occupancy and traffic information. Information about time factors include but are not limited to, time of day, seasons, or other information.

[0210] The parameterized model 1250 and the environmental factors 1260 are inputs to a physics simulation module 1270 and a behavioral simulation module 1280, which provide simulation results 1290.

[0211] The physics simulation module 1270 includes a physics engine 1272 which, using the model 1260 and rigid body dynamics simulations, upon which collision detection and contact forces can be evaluated, as indicated at 1274. This module can output data about failure results 1294, such as stress and wear information. The physics simulation module also can include a force dynamics module 1276, which uses information about contact forces, collision detection, and environmental or other external factors, to generate further performance data based on gravity, wind, and loads. This information can be used to provide various state information 1296 for elements in the parameterized model, such as position, time, and sequencing information.

[0212] The behavioral simulation module 1280 performs a simulation of the performance of the model based on simulated sensors, controls, and logic. Behavioral parameters 1258 and time factors 1266 can be processed by a sensor simulation module 1282, to provide various simulated sensor information such as proximity and temperature. The simulated sensor information is input to a control simulation module 1284, which in turn outputs various simulated control information such as the states of thermostats and switches. This state information, combined with usage patterns 1264, can be processed by the logic simulation module 1286, to provide performance results 1292 for the building related to its environment, such as energy usage and comfort.

[0213] The simulation results 1290 are input to a feedback loop 1298, which processes them into changes to the parameterized model 1250. For example, the state information 1296 can be processed to provide parameter updates, such as changes to constraints in the parameterized model. The failure results 1292 and performance results 1294 can be processed for validation or compliance issues, resulting in changes to the parameterized model that are directed to improving validation or compliance. The failure results 1292 and performance results 1294 also can be processed for performance optimization issues, resulting in changes to the parameterized model that are directed to improving or optimizing performance.

[0214] Another advantage of this computer system is that, given an animation or simulation output 150 for an action relating to constructing a building that sets some values among the variable parameters, instructions 172, which can be human-readable or machine-readable or both, can be generated based on those values.

[0215] The instruction generation module 170 generates instructions 172 based on one or more of the parameterized model 128, animation or simulation output 150, scripts 140, additional data 116, feedback 146, user input 173, or yet other information such as a template, or combinations of these. These instructions 172 are sent to various presentation devices for use during performance of the relevant action related to constructing the building. Such presentation devices also may include input devices for use by workers to provide user input 173 back to the instruction generation module 170, for interactive review and generation of the instructions 172.

[0216] The instructions can direct performance of the action, whether by specifying a sequence of steps to be performed or methods to be used by one or more individuals, or by one or more machines (including automation or robots), or by combinations of these.

[0217] The instruction generation module 170 can be implemented in many ways. For example, a template for a kind of instruction can be created with placeholders for data in the parameterized model 128. As another example, a large language model can be used with a prompt that specifies how the instructions 172 should be generated using data from the parameterized model 128. As another example, one or more scripts can be created to programmatically access and process datain the parameterized model 128, such as with tasks associated with objects and actions, and format that data into instructions.

[0218] In some implementations, the instruction generation module 170, or a portion thereof, can be implemented as part of the animation and simulation platform, such as the OMNIVERSE platform. Typically, implementations of the instruction generation module 170 would include one or more presentation devices, such as a display or audio output or a printer, which provide the generated instructions 172 to an individual in human readable form. To provide instructions 172 to a machine in machine-readable form typically would involve transferring data through a computer network, or by using a computer storage device, or by transferring data through an electromechanical interface or radio interface, such as a wireless network, Bluetooth, or other connection to equipment. In some implementations, instructions can be generated in an instruction format used by the equipment. In some implementations, instructions may be generated in one format and subsequently transformed into the instruction format used by the machine.

[0219] As an example, after simulating an assembly of a wall resulting in a sequence of tasks for assembling components of the wall, the sequence and required steps for performance of tasks can be output in human-readable form, optionally along with an animation of the sequence of assembly. As another example, instructions 172 can be output to a computer assisted manufacturing device, such as a CNC device. For example, Hundegger Speed Cut machines are commonly used for cutting components for buildings. The instructions 172 output by the instructions generation module 170 can be in the form of instructions to such a machine, or to other software that integrates with such a machine, such as the CADWORK software.

[0220] As another example, a simulation may identify one technique for affixing two building components, and the instruction generation module can output instructions explaining the technique and an animation visualizing performance of that technique.

[0221] In some implementations, the instruction generation module 170 can operate in an interactive manner in response to user inputs and responses. For example, after presenting instructions 172 to a user, the user can provide an input indicating a request to view an animation or a request for more details, such as more detailed explanation or a video explaining how a particular step is to be performed. As an example, in response to a user selecting a portion of a displayed model in an animation, a different animation, such as from a different perspective, level of detail, etc., can be generated and viewed. As another example, if the instruction generation module includes a generative artificial intelligence model implemented using a large language model, the generative Al model can be instructed to provide increasing levels of detail for each step in response to user inputs requesting further explanation of a step.

[0222] As another example, a simulation may identify a set of materials that are available for a construction and the components of the building where those materials can be used, with tasks associated with those components for fabricating materials into installation-ready building components. The instruction generation module 170 can generate technical instructions for a user or a machine to place an order for the materials. Such instructions 172 can include, for example, a bill of materials, instructions for packaging and transportation of the materials, and a cut list for the order. The instruction generation module 170, for example, can output as instructions 172 a cut list and optionally fabrication instructions for how to prepare the material for each component in which the material is to be used.

[0223] Other examples of instructions include daily workforce planning and personnel assignments, personnel including skills or expertise needed, sequences of steps for construction at various levels of detail, methods of fabrication and installation of components at various levels of detail, safety and risk management procedures, and summaries related to material, equipment, or personnel availability, usage, cost, or time.

[0224] An example implementation of an instruction generation module will now be explained in connection with Figure 11. As shown in Figure 11, the instruction generation module receives, as an input, a parameterized model 1100, which includes the three-dimensional model 1102 of the building, its variable parameters 1104, and its association parameters 1106, including connections to one or more building knowledge graphs, which define dependencies, sequences, and rules. To generate instructions, the instruction generation module 170 can implement an intelligent routing mechanism that selects the most appropriate generation approach based on multiple factors evaluated at runtime.

[0225] An analysis module 1110 processes the parameterized model 1100 to select a method for generating the instructions. For example, for simple, frequently-occurring components with well- established assembly patterns, the analysis module 1110 can determine that a template-based generator 1132 can be used directly to produce step-by-step instructions 1142 based on previously generated instructions. For such a purpose the analysis module 1110 can store data about previously processed component types 1114. For new or more complex components, additional tools can be provided. For example, for components requiring spatial understanding or involving complex three-dimensional relationships, the system can use a hybrid retrieval- augmented generation (RAG) architecture with visual -language model (VLM) integration 1134 to produce annotated visual instructions 1144, such as annotated three-dimensional views for one or more instructions.

[0226] To make such selections, the analysis module 1110 can assess the complexity of the component and its assembly requirements, the presence of specialized requirements (such asprecision tolerances or safety-critical connections), and the availability of historical instruction data for similar components. The selection mechanism implemented by the analysis module 1110 can consider available computational resources and response time requirements. In some implementations, the selection algorithm can attempt to apply multiple techniques for generating instructions. For example, the selection algorithm can use a weighted scoring system that balances generation quality, computational efficiency, and user requirements. The selection mechanism can default to simpler approaches if more complex methods fail or timeout, and conversely can escalate to more advanced generation approaches when simpler methods are determined to be insufficient to meet accuracy, compliance, or user-defined requirements. In some implementations, for example, the analysis module 1110 can implement a form of graph traversal algorithm to systematically process the parameterized model 1100 to identify actionable items, their dependencies, and any constraints, as indicated at 1112, and determine the component's complexity. This information 1112 is output to the selected instruction generator, as dependencies among the actionable items also determine, in part, the sequence in which the actionable items can be constructed.

[0227] In some implementations, the instruction generation module 170 can output instructions in multiple formats, including human-readable text, annotated diagrams, interactive three- dimensional visualizations, or machine-readable code for robotic or automated equipment, as shown at outputs 1142 and 1144. The module can further incorporate feedback from on-site sensors, simulation results, or user input 173 to update or regenerate instructions dynamically. Predefined instruction templates can be stored and indexed by component type for use with the template-based generator 1132. Retrieval-augmented generation in the hybrid RAG / VLM model 1134 can access the knowledge base 1120, which includes corpora such as prior task graphs, technical manuals, or historical instruction data, to enrich instruction detail.

[0228] A knowledge base 1120 also includes one or more expertise knowledge graphs relevant to construction activities, such as tool mapping 1122, material mapping 1124, and skills mapping 1126. The expertise knowledge graphs provide context to the generative models (1132, 1134, 1136, 1138).

[0229] For example, the hybrid RAG / VLM model 1134 accesses the knowledge base 1120, which provides context, such as tools, materials, and required skills, to a visual -language model to produce the visual instructions 1144.

[0230] Domain-specific agents 1136 use the generated instructions (1142, 1144) and the expertise knowledge graphs in knowledge base 1120 to generate skill-based task assignments 1146. Specifically, based on the actionable items and their sequences of actions, the tools, skills, and materials for each item and task can be determined. For example, based on its knowledgegraph, a tool mapping module 1122 identifies required, available, optional, or alternative equipment to be used to construct an item. Based on its knowledge graph, a material mapping module 1124 identifies required, available, optional, or alternative materials to be used to construct an item. Similarly, based on its knowledge graph, a skills mapping module 1126 identifies required, available, optional, or alternative expertise, whether human or machine or both, to be used to construct an item.

[0231] Finally, a validation generator 1138 generates information 1148 about warnings, tips, safety requirements, or other information based on the instructions (1142, 1144), and constraints identified for the actionable items as indicated by 1112, by accessing data in the knowledge base 1120. In some implementations, the instruction generation module 170 can employs graph traversal algorithms operating on the building knowledge graph 224 and expertise knowledge graphs 232 to systematically generate construction instructions. In such implementations, the instruction generation module 170 implements a multi-stage process beginning with identification of a target component within the parameterized and labeled model 128. Starting from the target component's corresponding node in the building knowledge graph, the system performs a reverse traversal through dependency edges to identify all prerequisite components and tasks that must be completed before the target component can be installed. This traversal employs depth-first search with cycle detection to prevent infinite loops in cases where circular dependencies exist. The identified dependencies are then processed through a topological sorting algorithm to establish a valid construction sequence that respects all dependency constraints. For each node in the topologically sorted sequence, the instruction generation module 170 extracts relevant attributes including material specifications, dimensional requirements, fastening methods, and spatial relationships from both the parameterized model 128 and the associated knowledge graphs. The system queries the expertise knowledge graphs 232 using these extracted attributes to retrieve applicable assembly patterns, best practices, and construction methods. These retrieved patterns, combined with the component-specific data, form a comprehensive context that is processed by a large language model (LLM) configured with construction-specific prompt templates. The LLM generates human-readable instructions for each construction step, maintaining coherence across the instruction sequence by preserving a context window of the previous three to five steps. To optimize performance, the system implements a caching mechanism that stores generated instructions for frequently occurring component types. For example, a semantic similarity matching with a threshold of 0.95 cosine similarity can be used to determine when cached instructions can be reused.

[0232] In some implementations, the instruction generation module 170 can employ a hybrid retrieval-augmented generation (RAG) architecture that combines visual understandingcapabilities with knowledge graph queries. In such implementations, the instruction generation module extracts visual context from the three-dimensional model by capturing viewport renderings of the components to be assembled and their surrounding context. These visual representations are processed alongside component metadata to generate high-dimensional embedding vectors using either general-purpose embedding models or custom models fine-tuned on construction documentation. The embedding vectors enable similarity searches within a vector database containing historical construction patterns, assembly sequences, and validated instruction sets. The retrieval process identifies the k-nearest neighbor construction scenarios based on cosine similarity or Euclidean distance metrics in the embedding space. Retrieved examples provide contextual templates and proven instruction patterns that are combined with the current construction context. A visual -language model (VLM) processes this combined information to generate instructions that include both textual directions and visual annotations. The VLM generates bounding boxes, arrows, and highlighting overlays on three-dimensional renderings to indicate specific components, connection points, or assembly directions. This visual augmentation enhances instruction clarity, particularly for complex spatial relationships or when multiple similar components are present in the work area.

[0233] In some implementations, the instruction generation module 170 can implement a template-based generation system that combines predefined instruction templates with dynamic data retrieved from knowledge graphs. In such implementations, the instruction generation module maintains a library of instruction templates categorized by component type, assembly method, and construction phase. When generating instructions for a specific component, the module first identifies the most appropriate template based on component classification within the building knowledge graph. The selected template contains placeholders for variable information such as specific dimensions, material grades, fastener specifications, and tool requirements. To populate these placeholders, the module can execute structured queries against the knowledge graphs using query languages such as SPARQL or Cypher. For example, a query might retrieve all tools required for a particular component by traversing REQUTRES TOOL edges, identify material specifications through MADE FROM relationships, and gather safety requirements via HAS SAFETY REQ connections. The query results are validated against current availability data from the additional data 116 before being inserted into the template. After placeholder substitution, a language model performs naturalization to ensure grammatical correctness, add appropriate transitional phrases between steps, and adjust the technical complexity based on the specified skill level of the intended workforce.

[0234] In some implementations, the instruction generation module 170 can model the construction process as a finite state machine where each state represents a constructionmilestone and transitions represent tasks with associated preconditions and postconditions. The state machine maintains persistent state information including completed tasks, available resources, and current construction phase. For each state transition, the system evaluates preconditions by querying the building knowledge graph to verify that prerequisite components are installed, required tools are available, and necessary inspections are completed. After validating preconditions, the module generates a comprehensive context object containing the current state, available resources, environmental conditions, and any active constraints. This context object is processed by instruction generation algorithms that produce state-specific instructions optimized for the current construction phase. The generated instructions undergo validation against safety requirements, building codes, and technical specifications before being released. The state machine architecture enables rollback capabilities, allowing the system to regenerate instructions if a task fails or if changes to the construction plan necessitate returning to a previous state. Progressive disclosure mechanisms within the state machine reveal instructions incrementally based on worker skill levels and task complexity, preventing information overload while ensuring all necessary details are available when needed.

[0235] In some implementations, the instruction generation module 170 can implement a multiagent system where specialized agents handle different aspects of construction instruction generation. In such implementations, each agent maintains expertise in a specific domain, such as structural framing, MEP systems, safety compliance, or finishing work. When the system receives a request to generate instructions for a component, a routing mechanism analyzes the component type and construction context to determine which agents should participate in instruction generation. Each selected agent queries its domain-specific portion of the expertise knowledge graphs 232 to retrieve relevant standards, methods, and constraints. For example, a structural agent might retrieve load-bearing requirements and connection specifications, while an MEP agent retrieves routing constraints and system integration requirements. The agents operate semi-autonomously, generating preliminary instructions within their domains before engaging in a collaborative resolution phase. During this phase, agents exchange messages to identify and resolve conflicts, such as competing space requirements or incompatible installation sequences. The message-passing protocol includes priority rankings, constraint specifications, and proposed compromises. Once consensus is achieved, the agents' outputs are merged and harmonized by a coordination agent that ensures consistency in terminology, sequencing, and formatting across the unified instruction set.

[0236] In some implementations, the instruction generation module 170 can implement a tokenization system that represents construction operations as discrete tokens within a learned vocabulary of assembly actions. Similar to how natural language models tokenize text or howprotein folding models tokenize amino acid sequences, such a tokenization system encodes construction tasks into a specialized token vocabulary. Each token represents an atomic construction operation such as "FASTEN STUD TO PLATE", "APPLY ADHESIVE", "INSERT ANCHOR", or "ALIGN PERPENDICULAR". Complex construction sequences are thus represented as token chains, where the order and combination of tokens encode valid assembly procedures. The system learns these token sequences from historical construction data, building codes, and validated assembly methods stored in the expertise knowledge graphs 232.

[0237] The tokenization approach enables the system to predict valid construction sequences using transformer-based architectures trained on tokenized construction patterns. Given a partial sequence of construction tokens and the current building context, the model predicts the next most likely tokens, effectively generating assembly instructions as a sequence generation task. For example, after tokens ["LAY FOUNDATION", "INSTALL SILL PLATE"], the model might predict ["ANCHOR SILL", "CHECK LEVEL", "INSTALL RIM JOIST"] as the highest probability continuation. The model learns constraints implicitly from training data. For example, it would not predict "INSTALL ROOF SHEATHING" immediately after foundation work because such sequences do not appear in valid construction patterns. This tokenized representation also enables efficient encoding of alternative assembly methods as different token sequences that achieve the same end state, allowing the system to generate multiple valid instruction sets and select among them based on available resources, worker skills, or optimization criteria.

[0238] In some implementations, the tokenization system can implement hierarchical token structures where high-level tokens can be decomposed into sequences of lower-level tokens, similar to how complex proteins have hierarchical structures from primary to quaternary. A high- level token like "FRAME WALL" expands into a sequence of subtokens: ["POSITION BOTTOM PLATE", "MARK STUD LOCATIONS","CUT STUDS TO LENGTH", "FASTEN STUDS", "ATTACH TOP PLATE", "SQUARE FRAME", "ADD BLOCKING"]. This hierarchical approach enables the system to generate instructions at varying levels of detail based on worker expertise — experienced framers might receive high-level tokens while apprentices receive fully expanded sequences. The token prediction model learns these compositional patterns, understanding that certain token subsequences frequently occur together and can be abstracted into higher-level operations. The system also implements token embeddings that capture semantic similarities between construction operations, allowing it to identify functionally equivalent token sequences even if they use different specific methods or tools. This tokenized approach integrates with the knowledge graphs by using graph structure to inform token transition probabilities — tokens aremore likely to follow each other if their corresponding entities are connected in the building knowledge graph 224 through dependency or sequence relationships.

[0239] Having now described several example implementations and used cases, an additional use case of such a system, which provides additional advantages, is to combine multiple different building projects into one model. For example, a general contractor may be constructing multiple buildings across different project sites at one time; or may be constructing a complex project such as a multi-family apartment building that internally contains multiple sub-project worksites. The general contractor may be able to optimize material, equipment, and personnel usage, and improve costs, time of completion, or other factors, over several projects or within one complex project, simultaneously. Such an implementation is achieved by including the building files for multiple building projects and the additional data for the multiple building projects into one parameterized and labeled model for which animation and simulations can be performed. As another example, multiple parts of a single building, such as a large multi-family apartment building, can be treated as multiple distinct yet interdependent projects, and the computer system can optimize resource allocation among them. For example, the computer system can optimize how best to allocate where and when different types of tradespeople work on different apartments within the same building.

[0240] A further use case of such a system, which provides additional advantages, is to simulate and compare an original building design and optimized designs subsequently produced by this computer system, in order to evaluate options and determine the preferred design for construction based on criteria such as construction cost, energy efficiency and usage, operational cost, construction time, return on investment, environmental impact, achievability of certifications, construction sequence, worker skill requirements, and recyclability or reuse potential of materials at the end of building life. This comparative analysis can be conducted for one or more criteria at both the whole-building level or more granularly by focusing on specific components, such as mechanical, electrical, and plumbing (MEP) systems, structural elements, or wall panels, enabling targeted optimization. It also allows stakeholders to assess whether the proposed optimizations justify their implementation by comparing changes in upfront investment with the modeled impact on long-term costs and revenues. By providing a clear analysis of tradeoffs, the system aids in deciding whether to adopt the optimized design in full, selectively integrate certain elements, or retain the original design. Additionally, it offers the flexibility to test various scenarios and sensitivity analyses, refining the decision-making process by highlighting the most impactful factors under different conditions. For example, an original architect-provided building design may feature industry-standard routing for MEP systems. Asubsequent design optimized by this system for energy efficiency could reroute MEP systems by strategically positioning HVAC units, reconfiguring ductwork and electrical wiring, and implementing home run manifold-based plumbing layouts. While such an optimized design may incur a higher initial installation cost compared to standard layouts, the improved energy efficiency could result in significantly lower operating costs over time. By simulating these scenarios, the system can provide a clear comparison of the payback period and potential longterm savings, helping stakeholders make informed choices of whether to construct the original or the optimized design.

[0241] Having now described an example implementation, some information about the implementation on a computer system will now be provided. One or more computers can be used to implement such a system, using one or more general -purpose computers, such as client devices including mobile devices and client computers, one or more server computers, or one or more database computers, or combinations of any two or more of these, which can be programmed to implement the functionality such as described in the example implementations.

[0242] Figure 13 is a block diagram of a general-purpose computer which processes computer programs using a processing system. Computer programs on a general-purpose computer generally include an operating system and applications. The operating system is a computer program running on the computer that manages access to resources of the computer by the applications and the operating system. The resources generally include memory, storage, communication interfaces, input devices, and output devices.

[0243] Examples of such general-purpose computers include, but are not limited to, larger computer systems such as server computers, database computers, desktop computers, laptop and notebook computers, as well as mobile or handheld computing devices, such as a tablet computer, handheld computer, smart phone, media player, personal data assistant, audio and / or video recorder, or wearable computing device.

[0244] With reference to Figure 13, an example computer 1200 comprises a processing system including at least one processing unit 1202 and a memory 1204. The computer can have multiple processing units 1202 and multiple devices implementing the memory 1204. A processing unit 1202 can include one or more processing cores (not shown) that operate independently of each other. Additional co-processing units, such as graphics processing unit 1220, also can be present in the computer. The memory 1204 may include volatile devices (such as dynamic randomaccess memory (DRAM) or other random-access memory device), and non-volatile devices (such as a read-only memory, flash memory, and the like) or some combination of the two, and optionally including any memory available in a processing device. Other memory such asdedicated memory or registers also can reside in a processing unit. Such a memory configuration is delineated by the dashed line 1204 in Figure 13. The computer 1200 may include additional storage (removable and / or non-removable) including, but not limited to, solid state devices, or magnetically recorded or optically recorded disks or tape. Such additional storage is illustrated in Figure 13 by removable storage 1208 and non-removable storage 1210. The various components in Figure 13 are generally interconnected by an interconnection mechanism, such as one or more buses 1230.

[0245] A computer storage medium is any medium in which data can be stored in and retrieved from addressable physical storage locations by the computer. Computer storage media includes volatile and nonvolatile memory devices, and removable and non-removable storage devices. Memory 1204, removable storage 1208 and non-removable storage 1210 are all examples of computer storage media. Some examples of computer storage media are RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optically or magneto-optically recorded storage device, magnetic cassettes, magnetic tape, magnetic disk storage, or other magnetic storage devices. Computer storage media and communication media are mutually exclusive categories of media.

[0246] The computer 1200 may also include communications connection(s) 1212 that allow the computer to communicate with other devices over a communication medium. Communication media typically transmit computer program code, data structures, program modules, or other data over a wired or wireless substance by propagating a modulated data signal such as a carrier wave or other transport mechanism over the substance. The term "modulated data signal" means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal, thereby changing the configuration or state of the receiving device of the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media include any non-wired communication media that allows propagation of signals, such as acoustic, electromagnetic, electrical, optical, infrared, radio frequency, and other signals. Communications connections 1212 are devices, such as a network interface or radio transmitter, that interface with the communication media to transmit data over and receive data from signals propagated through communication media.

[0247] The communications connections can include one or more radio transmitters for telephonic communications over cellular telephone networks, and / or a wireless communication interface for wireless connection to a computer network. For example, a cellular connection, a Wi-Fi connection, a Bluetooth connection, and other connections may be present in thecomputer. Such connections support communication with other devices, such as to support voice or data communications.

[0248] The computer 1200 may have various input device(s) 1214 such as a various pointer devices, whether single pointer or multi-pointer devices, such as a mouse, tablet and pen, touchpad and other touch-based input devices, stylus, image input devices, such as still and motion cameras, audio input devices, such as a microphone. The computer may have various output device(s) 1216 such as a display, speakers, printers, and so on, also may be included. These devices are well known in the art and need not be discussed at length here.

[0249] The various storage 1210, communication connections 1212, output devices 1216 and input devices 1214 can be integrated within a housing of the computer, or can be connected through various input / output interface devices on the computer, in which case the reference numbers 1210, 1212, 1214 and 1216 can indicate either the interface for connection to a device or the device itself as the case may be.

[0250] An operating system of the computer typically includes computer programs, commonly called drivers, which manage access to the various storage 1210, communication connections 1212, output devices 1216 and input devices 1214. Such access generally includes managing inputs from and outputs to these devices. In the case of communication connections, the operating system also may include one or more computer programs for implementing communication protocols used to communicate information between computers and devices through the communication connections 1212.

[0251] Any of the foregoing aspects may be embodied as a computer system, as any individual component of such a computer system, as a process performed by such a computer system or any individual component of such a computer system, or as an article of manufacture including computer storage in which computer program code is stored and which, when processed by the processing system(s) of one or more computers, configures the processing system(s) of the one or more computers to provide such a computer system or individual component of such a computer system.

[0252] Each component (which also may be called a “module” or “engine” or “computational model” or the like), of a computer system such as described herein, and which operates on one or more computers, can be implemented as computer program code processed by the processing system(s) of one or more computers. Computer program code includes computer-executable instructions and / or computer-interpreted instructions, such as program modules, which instructions are processed by a processing system of a computer. Generally, such instructions define routines, programs, objects, components, data structures, and so on, that, when processed by a processing system, instruct the processing system to perform operations on data orconfigure the processor or computer to implement various components or data structures in computer storage. A data structure is defined in a computer program and specifies how data is organized in computer storage, such as in a memory device or a storage device, so that the data can be accessed, manipulated, and stored by a processing system of a computer.

[0253] The implementations described above are presented by way of example only and can be modified in a variety of ways. Accordingly, other implementations are within the scope of the following claims and the subject matter defined in the following claims is not limited to these example implementations. What is claimed is:APPENDIXSemantic information found in industry foundation classes (IFC) standard compliant data files:Geometric Representation: 3D shapes, curves, surfaces, and solid modelsSpatial Structure: Hierarchical organization (Site —> Building —> Storey —> Space)Building Elements: Physical components (walls, doors, windows, slabs, columns, beams)Property Sets (Psets): Standardized property collections a. Pset W allCommon (FireRating, AcousticRating, ThermalTransmittance, IsExternal, LoadBearing) b. Pset DoorCommon (FireRating, Security Rating, Handicap Accessible, SelfClosing) c. Pset WindowCommon (ThermalTransmittance, GlazingAreaFraction, IsExternal) d. Pset SpaceCommon (Reference, IsExternal, GrossPlannedArea, NetPlannedArea)Quantity Sets (Qto): Measurable quantities e. Qto WallBaseQuantities (Length, Height, Width, GrossArea, NetArea, Volume) f. Qto DoorBaseQuantities (Height, Width, Area, Perimeter)Relationships: Complex interconnections g. IfcRelAggregates (parent-child spatial relationships) h. If cRelContainedlnSpatial Structure (element-space relationships) i. IfcRelConnects (physical connections between elements) j . IfcRel Assigns (grouping and classification)Materials: Layered material definitions with propertiesClassifications: Reference to external classification systemsActors & Organizations: Project participants and rolesScheduling (4D): Time-based construction sequencesCost Data (5D): Budget and cost informationFacility Management: Operational and maintenance dataTable ISemantic information found in Autodesk REVIT files:Families & Types: Parametric component definitions Parameters: System, shared, and project parameters k. Instance parameters (unique to each element) l. Type parameters (shared across instances)Views & Sheets: 2D drawings and 3D viewsSchedules: Tabular data extractionPhases: Construction phases and demolitionDesign Options: Alternative design scenariosWorksets: Collaboration divisionsMaterials: Appearance, physical, and thermal properties MEP Systems: Mechanical, electrical, plumbing networks Structural Connections: Analytical models and connections Rooms & Spaces: Spatial analysis dataTable II77Semantic information found in Construction Operations Building Information Exchange (COBie) data files:Components: Maintainable building elements Types: Equipment and product types Systems: Building systems and assemblies Zones: Spatial groupings for maintenance Attributes: Custom properties for FM Documents: Operation and maintenance manuals Jobs: Maintenance tasks and procedures Resources: Materials and tools required Spare Parts: Inventory management data Warranties: Warranty tracking informationTable III78Table IV82

Claims

CLAIMS1. A computer system, comprising: a processing system including at least one processing device and computer storage, wherein computer program instructions in the computer storage are processed by the at least one processing device to configure the processing system; a model parameterization module comprising computer program instructions in the computer storage to be processed by the processing system to configure the processing system to: access at least a building file describing a building and additional data, and generate a parameterized and labeled model based on at least the building file and the additional data, wherein the parameterized and labeled model includes at least data representing three-dimensional objects representing components of a building, wherein at least some information associated with the three-dimensional objects includes data specified as a variable parameter that can be assigned in an animation or simulation; and wherein the computer storage is configured to allow access to the parameterized and labeled model for use by an animation and simulation module for animating or simulating at least one action related to constructing the building.

2. The computer system of claim 1, wherein the at least some information associated with one or more of the three-dimensional objects includes at least a portion of the additional data.

3. The computer system of claim 1, wherein the at least some information associated with the three-dimensional objects further includes data specified as an associated parameter.

4. The computer system of claim 3, wherein the associated parameter includes data representing one or more of assembly sequences, material properties, labor assignments, or real-time adaptability based on site conditions or sensor feedback.

5. The computer system of claim 3, wherein the associated parameter includes one or more building knowledge graphs.

6. The computer system of claim 1, wherein the parameterized and labeled model further comprises additional data structures storing semantic information extracted from the building file.

697. The computer system of claim 1, further comprising an animation and simulation module having one or more inputs receiving the parameterized and labeled model, and having one or more outputs providing data representing results of an animation or simulation of the parameterized and labeled model.

8. The computer system of claim 7, wherein the animation or simulation includes a representation of fabricating the components of the building.

9. The computer system of claim 8, wherein the animation or simulation module further configures the processing system to optimize the sequence of tasks for constructing the building by simulating multiple assembly sequences to reduce one or more of costs, time, labor requirements, material usage, or environmental impacts.

10. The computer system of claim 9, wherein the animation or simulation module further generates cost estimating outputs derived from the parameterized and labeled model and simulated construction sequences, the outputs including one or more of: projected total project cost, cost by building component or subsystem, labor cost, material cost, equipment cost, or schedule-related cost impacts.

11. The computer system of claim 9, wherein the animation or simulation module dynamically adjusts the animation or simulation based on real-time updates to construction progress, material or equipment availability, workforce availability, or site conditions.

12. The computer system of claim 11, wherein the animation or simulation is defined by one or more of an animation graph or an action graph or other graph data structure.

13. The computer system of claim 12, further comprising one or more agents, comprising computer program instructions that configure the processing system to generate one or more of an animation graph or an action graph or other graph data structure.

14. The computer system of claim 1, further comprising an instruction generation module comprising computer program instructions that configure the processing system to generate human-readable or machine-readable instructions for an action related to constructing the building.7015. The computer system of claim 14, wherein the instruction generation module configures the processing system to generate the instructions for the action related to constructing the building based on data output by an animation and simulation module using the parameterized and labeled model.

16. The computer system of claim 14, wherein the instruction generation module further provides updates to the instructions based on feedback from one or more of on-site data sensors, or changes in design, material availability, or equipment availability.

17. The computer system of claim 14, wherein the instruction generation module further configures the processing system to optimize material cutting and preparation, including one or more of: generating cut lists that reduce material waste based on material properties; or providing instructions to one or more machines for automated cutting or fabrication.

18. The computer system of claim 17, wherein the instruction generation module further configures the processing system to assign tasks to workers or machines based on one or more of availability, skill level, or performance metrics.

19. The computer system of claim 14, wherein the instructions include a cut list for preparing one or more components from one or more pieces of material.

20. The computer system of claim 14, wherein the instructions include instructions for ordering, fabricating, packaging, staging, or transportation of materials.

21. The computer system of claim 14, wherein the instructions include a sequence of tasks to be performed to assemble the components of the building.

22. The computer system of claim 14, wherein the instructions include step-by-step instructions for methods of fabricating the components of the building.

23. A computer system, comprising: a processing system including at least one processing device and computer storage, wherein computer program instructions in the computer storage are processed by the at least one processing device to configure the processing system;71a model parameterization module comprising computer program instructions that configure the processing system to: receive design data from architectural software, including a three-dimensional model of a structure of a building; translate the received design data into a parameterized and labeled model of the building, comprising a plurality of three-dimensional objects representing building components and a knowledge graph including nodes, representing building components and associated with their respective three-dimensional objects, and edges representing relationships among the building components.

24. A computer-implemented process, comprising: receiving a three-dimensional building model from architectural software; parameterizing the building model by associating tasks, materials, dependencies, or installation constraints with components of the building; simulating assembly of the building in a virtual environment, wherein the simulation includes one or more of building component preparation, material transportation, on-site logistics, assembly sequence, assembly methods, compliance with engineering standards, building codes, or environmental sustainability requirements, generating a cut list for components of the building, providing machine-readable instructions to automated cutting devices, generating human-readable component preparation or assembly instructions for human workers, or updating those instructions based on on-site sensor data or simulation feedback.

25. The computer-implemented process of claim 24, wherein the simulation adjusts based on one or more of data inputs from on-site devices that track progress of assembly, labor or equipment availability, or supply chain data reflecting one or more of material availability, material origin, delivery schedules, cost fluctuations, or environmental impact assessments.

26. The computer-implemented process of claim 24, wherein the on-site devices include one or more of a camera, a LiDAR scanner, a GPS system, an environmental sensor, or an on-site graphical user interface.

27. The computer-implemented process of claim 24, wherein simulating assembly of the building further comprises generating financial data outputs derived from the parameterized and labeled model and simulated construction sequences, the financial data outputs including one or more of: schedule-dependent costs and cashflow timing; contingency reserve requirements determined by72stochastic variation in labor, equipment, or material availability; progress-based valuation of work performed for payment certification or loan drawdowns; identification of cost savings or margin improvements from optimized sequencing; release of capital through reduced contingency requirements; acceleration of revenue resulting from simulated earlier delivery of rentable or saleable space; or phasing strategies aligned with market absorption or financing milestones.

28. The computer-implemented process of claim 24, wherein simulating assembly of the building further comprises generating cost estimating outputs derived from the parameterized and labeled model and simulated construction sequences, including one or more of: projected total project cost, cost by building component or subsystem, labor cost, material cost, equipment cost, or schedule-related cost impacts.

29. A computer system comprising: a processing system including at least one processing device and computer storage, wherein computer program instructions in the computer storage are processed by the at least one processing device to configure the processing system to: transform architectural design data into a three-dimensional model, parameterized with tasks, material properties, and dependencies; simulate constructing the building using the three-dimensional model including optimizing sequences of tasks and representations of assembly; translate the simulated construction of the building into human-readable or machine- readable instructions; and update the simulation and instructions based on feedback from sensors monitoring construction of the building.

30. A computer system, comprising: a processing system including at least one processing device and computer storage, wherein computer program instructions in the computer storage are processed by the at least one processing device to configure the processing system; an animation and simulation module having one or more inputs receiving a parameterized and labeled model, including data representing three-dimensional objects representing components of a building, wherein at least some information associated with the three- dimensional objects is specified as a variable parameter that can be assigned in an animation or73simulation, and having one or more outputs providing data representing results of an animation or simulation of the parameterized and labeled model; and an instruction generation module comprising computer program instructions in the computer storage to be processed by the processing system to configure the processing system to: access at least a parameterized and labeled model including data representing three- dimensional objects representing components of a building, wherein at least some information associated with the three-dimensional objects is specified as a variable parameter that can be assigned in an animation or simulation.

31. The computer system of claim 9, wherein the animation or simulation module further generates financial data outputs derived from the parameterized and labeled model and simulated construction sequences, wherein the financial data outputs include one or more of: scheduledependent costs and cashflow timing; contingency reserve requirements determined by stochastic variation in labor, equipment, or material availability; progress-based valuation of work performed for payment certification or loan drawdowns; identification of cost savings or margin improvements from optimized sequencing; release of capital through reduced contingency requirements; acceleration of revenue resulting from simulated earlier delivery of rentable or saleable space; or phasing strategies aligned with market absorption or financing milestones.74

Images