Metaverse model for building sustainability
Patent Information
- Application Number
- PCT/US2023/024255
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-06-02
- Publication Date
- 2024-12-05
Smart Images

Figure US2023024255_05122024_PF_FP_ABST
Abstract
Description
METAVERSE MODEL FOR BUILDING SUSTAINABILITYTECHNICAL FIELD
[0001] This application relates to digital twins. In particular this application relates to combining multiple models to create a comprehensive digital twin of a system.BACKGROUND
[0002] Planning, constructing, managing and re-adapting buildings involve the intervention of several individuals, covering many different roles. For example, architects, engineers, owners, and city stakeholders will interact in the planning phase of the building. In construction phase, contractors and engineers might have to make decisions that deviate from the original plan and evaluate the consequences afterwards. During operations, the building manager, the owner and the tenants may interact to implement decisions that impact energy consumption, spread of airborne disease, and comfort of the building users to name a few examples.
[0003] Currently, all such interactions take place either through separate conversations or paper communications. These interactions do not necessarily maintain historical information, or require use of several digital tools, each targeting a specific aspect or phase of the building life cycle. Models that emulate many aspects of the system across different life cycle phases are desired.SUMMARY
[0004] A comprehensive digital twin or metaverse model of a system includes a plurality of models that each represent a particular aspect of the system. A three- dimensional (3D) engine renders a scene of the model based on two or more of themodels. The scene is embodied as a 3D visualization of some or all of the system and is presented to user or stakeholder via a user interface. The user interface enables a stakeholder to provide information about the system that is incorporated into the digital twin of the system. Additional information is received and included in the appropriate model representing the aspect of the system relating to the new information. In addition to stakeholders, additional information relating to the system may be generated by outside sources or applications. A set of importers and adapters receive the information and process the information for use in the digital twin model.
[0005] A method for creating a comprehensive digital twin model includes receiving information relating to a system and using the received information to update a plurality of system models each representing an aspect of the overall system. A virtual twin of the system is created by a combination of two or more of the models. A 3D visualization of some or all of the system is rendered in a 3D modeling engine. During the system life cycle, updated or new information relating to the system may be received. The new or updated information is incorporated into the plurality of models and used to update the 3D visualization displayed to a user. Information may be provided to the digital twin by a user via a user interface, or information may be received from outside sources or applications through a set of importers or adapters that enable the receipt of information and ingestion of that information to the digital twin model.BRIEF DESCRIPTION OF THE DRAWINGS
[0006] The foregoing and other aspects of the present invention are best understood from the following detailed description when read in connection with the accompanying drawings. For the purpose of illustrating the invention, there is shown in the drawingsembodiments that are presently preferred, it being understood, however, that the invention is not limited to the specific instrumentalities disclosed. Included in the drawings are the following Figures:
[0007] FIG. 1 is a diagram of an embodiment of a comprehensive digital twin according to aspects of this disclosure.
[0008] FIG. 2 is an illustration of a comprehensive digital twin according to aspects of embodiments of this disclosure.
[0009] FIG. 3 is a block diagram of a comprehensive digital twin according to aspects of embodiments of this disclosure.
[0010] FIG. 4 is a process flow diagram for creation of a comprehensive digital twin of a system according to aspects of embodiments of this disclosure.
[0011] FIG. 5 illustrates an exemplary computing environment 500 within which embodiments of the invention may be implemented.DETAILED DESCRIPTION
[0012] Currently there is no systematic approach to maintain a connection between multiple stakeholders and multiple system life cycle phases. Furthermore, such connections happen on an ad-hoc basis. A system will be described that comprises a building system, but the methods and systems described may be used to model and develop any complex system. Several tools are available to translate Building Information Modelling (BIM) to 3D geometry. There is also availability of tools that simulate energy demand for given buildings. However there exists no solutions to provide a complete pipeline from BIM to a 3D geometrical model and further to 3D energy or comfortsimulation models that provide a 3D visualization of outcome. Such a solutions would require integration of building data, calibration of models, performance optimization, predictive maintenance, upgrade planning, building re-purposing, and interactive exploration of options. An embodied solution provided in this disclosure may be referred to as “building metaverse” providing a comprehensive digital twin of a building’s systems and includes a 3D scene Tenderer that supports user interactions via a graphical interface.
[0013] A system for generating a comprehensive digital twin (or metaverse) for a complex system provides a graphical interaction with the building model to change component configuration, assign materials or other conditions (e.g. thermal load and air flow) to various components, another graphical interaction with the building allows the addition of assets, including physics properties and behavior, to evaluate what-if scenarios. Examples may include solar panels, additional insulation layers, roof top garden, or other features relating to a building system.
[0014] Embodiments of a metaverse model may provide graphical representations to display on-screen information and key performance indicators selected by a user. These representations may include on-screen textual information, graphs, or 2D and 3D maps. Some embodiments may generate documentation based on the digital twin model, which may be exported using an exporter for stakeholder to implement.
[0015] A metaverse model according to embodiments herein include a set of importers for receiving available information including engineering information such as ventilation conduits, electrical connections, energy demand of computing clusters or other energy payloads, data from building sensors, material profiles for selected building components (e.g. wall and insulation, shades or naked windows), environmental conditions, such asclimate and weather forecast (cloud coverage, temperature, wind) , and building occupancy information.
[0016] The metaverse modelling system further contains a set of editors for allowing a user to specify or update a 3D geometrical model of the building (interior and exterior) and the components inside it (e.g. furniture, wall dividers, vents); environmental conditions, such as climate and weather forecast (cloud coverage, temperature, wind), building occupancy, and building operating conditions (e.g. cooling air flow, set point temperature).
[0017] Some available information may come from external tools or applications. To ingest this information, a set of connectors allow the model to realize a connection. For example, a connector to connect with a cost calculator to evaluate design and operating cost of a given change in building design I operation mode, or connection with life cycle assessment tool to evaluate environmental impact (e.g. carbon emission), or connection with energy system-level simulation, optimization, and / or Al models. Further information may be received by connectors may provide connection with 3D simulation, optimization and / or Al models for user comfort prediction (ventilation, temperature, illumination), risk of airborne infection (ventilation) or connection with evacuation planning simulation model, or connection with predictive maintenance model to predict building component failure, or connection with building physical controller to adjust operation mode.
[0018] A data store may store capability to track the building entities in time, to store past and current information of the building. Accordingly, such a system may be utilized from the conceptual design phase to store progressive information and models pertinent to subsequent phases of the building life cycle or may be created at intermediate stagesof the building life cycle (e.g., during operation and prior to realizing extensions or reproposing of the building). A preferred embodiment which uses a building and its many systems as a complex system. It should be noted that the methods and systems for creating a comprehensive digital twin model may be extended to any complex system.
[0019] FIG. 1 is a diagram of an embodiment of a comprehensive digital twin according to aspects of this disclosure. For illustration, a real-world system is embodied as a building 101. Various aspects of the building 101 are processed by the comprehensive digital twin. Information may be provided by one of more stakeholders across any phase of the building 101. Using information from multiple sources, the comprehensive digital twin may be used to simulate an aspect of the overall system. With reference to FIG. 1 , a specific room within building 101 may be displayed to a user as a virtual environment 103. By observing the virtual environment 103, a stakeholder may provide updated information to the comprehensive digital twin. The additional information may be combined with other updates from other stakeholders, or alternatively additional information may be obtained from other data sources. For example, sensors installed throughout the building 101 may provide information relating to the state of the building 101. In other embodiments, additional information may be obtained from a predictive weather monitoring application. Information relating to the weather, such as temperature or cloud cover may be provided to the comprehensive digital twin. The updated information is processed in the virtual environment 103. A user may observe the virtual environment 103 via a user interface to view the effects of updated information pertaining to the system.
[0020] FIG. 2 is an illustration of a comprehensive digital twin according to aspects of embodiments of this disclosure. Continuing from the building example of FIG. 1 , acomprehensive digital twin of the building is created in a digital version which may be housed in a computing platform including a computer processor and memory. The computing platform may be remotely implemented in a cloud network 201 . The digital twin may exchange information with a plurality of sources. For example, stakeholders 219 may interact with the digital twin via a user interface between the stakeholder 219 and the digital twin. Stakeholders may include engineers and designers among others who interact with the digital twin during various life cycle stages of the building system. During construction for example, an engineer may provide construction details relating to the building 101. This information may include information stored as Building Information Modeling (BIM) 203 data and may further specify specific materials 205 to be used to construct components of the building 101 .
[0021] Other information may be utilized during other lifecycle phases of the building. For example, climate or weather information 207 may be used during the ongoing use of the building to evaluate the performance or implementation of renewable energy resources. Other criteria, including battery capacity and usage 209, photo-voltaic cells in solar panels 213, thermal and ventilation models 215 may be included in the comprehensive digital twin to provide a full and accurate visualization of the state of the system. Further data may be provided to the comprehensive digital twin to calibrate the digital twin and / or simulate the real-world system and enhance the reliability of virtual representation of the system more comprehensively. In the building example, this additional information may include occupancy profiles of the building 211 or energy simulations 217.
[0022] The comprehensive digital twin 201 receives multiple data sources and aggregates the various sources of information to create a comprehensive view of the system throughout the system’s life cycle. This view may be rendered as a 3D visual representation that may interact with one or more stakeholders 219 of the system. Stakeholders 219 may provide updated information relating to the system and the comprehensive digital twin 201 will use the information to update the visual representation of the system while taking into account all other data inputs relating to the system.
[0023] FIG. 3 is a block diagram of a comprehensive digital twin 301. The comprehensive digital twin 301 includes a plurality of models 303, which are representative of aspects of the overall system. A data store 305 is provided to store data relating to the models 303 and other data relating to the system. Data store 303 may further store historical information including past states of the system. Using data from data store 303 and the simulation results from one or more of the models 303, a post processing 3D engine 307 may generate a 3D virtual representation of the system based on multiple models 303. The 3D engine 307 may provide the 3D virtual representation to a user interface 311 to provide a visualization to one or more stakeholders 219.
[0024] The comprehensive digital twin 301 includes importers 307 and adapters 309 for receiving information from the system and its operating environment. Information regarding weather 313, information generated by sensors within the system, in this case a building 101 , as well as system changes provided by stakeholders 219 are presented to an importer 307 which may provide the information to one or more models 303 or to data store 303.
[0025] Other outside processes, such as occupancy models 321 , comfort 323 and environmental 329 models, energy simulations 325 or cost calculators 327 provide their information to adapters 309, which process the information output from these processes and convert the information into a format ingestible by models 303 and data store 305. Exporters 308 may provide information representative of the digital twin and provide information to shareholders 219 by means including but not limited to documentation 330.
[0026] FIG. 4 is a process flow diagram for creation of a comprehensive digital twin of a system according to aspects of embodiments of this disclosure. Information is collected from stakeholders or may be provided from outside sources or applications 401. Stakeholders may provide information regarding the system during any phase of the system’s life cycle. Outside sources or applications may include energy simulations, occupancy and comfort simulations, weather reports and similar data relating to the system. Information from these applications may be provided to adapters which format the data for ingestion and use by the comprehensive digital twin for simulating and visualization of the system. A plurality of models within the comprehensive digital twin are updated based on the new information 403. The models may each represent an aspect of the overall system and some or all of these models may be combined to construct a comprehensive model of the system 405. A 3D engine may be used to render a 3D visualization of the comprehensive digital twin 407. The visualization may provide a 3D perspective of the system, or of sub-systems within the overall system. The visualization may be updated to display the effect of additional information that is received at the digital twin.
[0027] The 3D visualization may be provided to one or more stakeholders 409 via a user interface. The stakeholders may view the state of the system through the visualization representing the comprehensive digital twin and query values of interest (temperature, humidity in the building space) at specific state(s). As the state of the system changes, the digital twin will update the visualization to show any changes to the stakeholders. Additionally, stakeholders may provide data relating to a proposed change to the system and observe the effects of the proposed change through the digital twin. Stakeholders can also visualize historical states of the system, or past design alternatives evaluated to help them inform future decisions. Based on the observations, additional information 411 may be provided and received by the comprehensive digital twin via the user interface 401 . The information is used to update the models in the digital twin which are used to create the updated visualization of the system.
[0028] Users of the building metaverse would be able to utilize it for several scenarios, including the following:
[0029] While planning for a building construction project, evaluate layout, size, material choices for their impact in terms of monetary and environmental cost, both considering upfront cost and cost during building operation (e.g., choosing material 1 for insulation might have less upfront cost but would require an increased energy cost during operation as it provides less insulation). Impact of the design decisions conforming to building standards and codes may also be visualized to inform users or utilized as constraints in design optimization studies.
[0030] When modifying an existing building to achieve net-zero energy profile, the user could evaluate addition of renewable energy sources, such as photovoltaic panels, orhardware changes (e.g., insulation, battery storage), or different usage profiles (e.g., set points for AC), taking into account building specific information. For example, utilize historic data on occupancy as well as simulated occupancy for a future period, to estimate hourly energy demand, compute estimated solar energy given the location-specific weather conditions.
[0031] When looking for tenants for a building, the building metaverse could be used to provide a photorealistic visualization of the building environment, including e.g. aesthetic of a given design choice, lighting conditions throughout the day in the different areas, simulated flow of occupants through the building based on different usage scenarios, and accounting for specialized settings like hospital buildings, and at the same time displaying expected energy requirements, overlaying thermal maps for the different zones, and potentially additional key performance indicators
[0032] When re-purposing a building, due to building ownership transfer or new tenants with different business models (including new layout and occupancy patterns), evaluate before such transition what is the impact of the change, from the economic and environmental point of view.
[0033] FIG. 5 illustrates an exemplary computing environment 500 within which embodiments of the invention may be implemented. Computers and computing environments, such as computer system 510 and computing environment 500, are known to those of skill in the art and thus are described briefly here.
[0034] As shown in FIG. 5, the computer system 510 may include a communication mechanism such as a system bus 521 or other communication mechanism for communicating information within the computer system 510. The computer system 510further includes one or more processors 520 coupled with the system bus 521 for processing the information.
[0035] The processors 520 may include one or more central processing units (CPUs), graphical processing units (GPUs), or any other processor known in the art. More generally, a processor as used herein is a device for executing machine-readable instructions stored on a computer readable medium, for performing tasks and may comprise any one or combination of, hardware and firmware. A processor may also comprise memory storing machine-readable instructions executable for performing tasks. A processor acts upon information by manipulating, analyzing, modifying, converting or transmitting information for use by an executable procedure or an information device, and / or by routing the information to an output device. A processor may use or comprise the capabilities of a computer, controller or microprocessor, for example, and be conditioned using executable instructions to perform special purpose functions not performed by a general-purpose computer. A processor may be coupled (electrically and / or as comprising executable components) with any other processor enabling interaction and / or communication there-between. A user interface processor or generator is a known element comprising electronic circuitry or software or a combination of both for generating display images or portions thereof. A user interface comprises one or more display images enabling user interaction with a processor or other device.
[0036] Continuing with reference to FIG. 5, the computer system 510 also includes a system memory 530 coupled to the system bus 521 for storing information and instructions to be executed by processors 520. The system memory 530 may include computer readable storage media in the form of volatile and / or nonvolatile memory, suchas read only memory (ROM) 531 and / or random-access memory (RAM) 532. The RAM 532 may include other dynamic storage device(s) (e.g., dynamic RAM, static RAM, and synchronous DRAM). The ROM 531 may include other static storage device(s) (e.g., programmable ROM, erasable PROM, and electrically erasable PROM). In addition, the system memory 530 may be used for storing temporary variables or other intermediate information during the execution of instructions by the processors 520. A basic input / output system 533 (BIOS) containing the basic routines that help to transfer information between elements within computer system 510, such as during start-up, may be stored in the ROM 531 . RAM 532 may contain data and / or program modules that are immediately accessible to and / or presently being operated on by the processors 520. System memory 530 may additionally include, for example, operating system 534, application programs 535, other program modules 536 and program data 537.
[0037] The computer system 510 also includes a disk controller 540 coupled to the system bus 521 to control one or more storage devices for storing information and instructions, such as a magnetic hard disk 541 and a removable media drive 542 (e.g., floppy disk drive, compact disc drive, tape drive, and / or solid-state drive). Storage devices may be added to the computer system 510 using an appropriate device interface (e.g., a small computer system interface (SCSI), integrated device electronics (IDE), Universal Serial Bus (USB), or FireWire).
[0038] The computer system 510 may also include a display controller 565 coupled to the system bus 521 to control a display or monitor 566, such as a cathode ray tube (CRT) or liquid crystal display (LCD), for displaying information to a computer user. The computer system includes an input interface 560 and one or more input devices, such asa keyboard 562 and a pointing device 561 , for interacting with a computer user and providing information to the processors 520. The pointing device 561 , for example, may be a mouse, a light pen, a trackball, or a pointing stick for communicating direction information and command selections to the processors 520 and for controlling cursor movement on the display 566. The display 566 may provide a touch screen interface which allows input to supplement or replace the communication of direction information and command selections by the pointing device 561. In some embodiments, an augmented reality device 567 that is wearable by a user, may provide input / output functionality allowing a user to interact with both a physical and virtual world. The augmented reality device 567 is in communication with the display controller 565 and the user input interface 560 allowing a user to interact with virtual items generated in the augmented reality device 567 by the display controller 565. The user may also provide gestures that are detected by the augmented reality device 567 and transmitted to the user input interface 560 as input signals.
[0039] The computer system 510 may perform a portion or all of the processing steps of embodiments of the invention in response to the processors 520 executing one or more sequences of one or more instructions contained in a memory, such as the system memory 530. Such instructions may be read into the system memory 530 from another computer readable medium, such as a magnetic hard disk 541 or a removable media drive 542. The magnetic hard disk 541 may contain one or more datastores and data files used by embodiments of the present invention. Datastore contents and data files may be encrypted to improve security. The processors 520 may also be employed in a multiprocessing arrangement to execute the one or more sequences of instructions containedin system memory 530. In alternative embodiments, hard-wired circuitry may be used in place of or in combination with software instructions. Thus, embodiments are not limited to any specific combination of hardware circuitry and software.
[0040] As stated above, the computer system 510 may include at least one computer readable medium or memory for holding instructions programmed according to embodiments of the invention and for containing data structures, tables, records, or other data described herein. The term “computer readable medium” as used herein refers to any medium that participates in providing instructions to the processors 520 for execution. A computer readable medium may take many forms including, but not limited to, non- transitory, non-volatile media, volatile media, and transmission media. Non-limiting examples of non-volatile media include optical disks, solid state drives, magnetic disks, and magneto-optical disks, such as magnetic hard disk 541 or removable media drive 542. Non-limiting examples of volatile media include dynamic memory, such as system memory 530. Non-limiting examples of transmission media include coaxial cables, copper wire, and fiber optics, including the wires that make up the system bus 521 . Transmission media may also take the form of acoustic or light waves, such as those generated during radio wave and infrared data communications.
[0041] The computing environment 500 may further include the computer system 510 operating in a networked environment using logical connections to one or more remote computers, such as remote computing device 580. Remote computing device 580 may be a personal computer (laptop or desktop), a mobile device, a server, a router, a network PC, a peer device or other common network node, and typically includes many or all of the elements described above relative to computer system 510. When used in anetworking environment, computer system 510 may include modem 572 for establishing communications over a network 571 , such as the Internet. Modem 572 may be connected to system bus 521 via user network interface 570, or via another appropriate mechanism.
[0042] Network 571 may be any network or system generally known in the art, including the Internet, an intranet, a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a direct connection or series of connections, a cellular telephone network, or any other network or medium capable of facilitating communication between computer system 510 and other computers (e.g., remote computing device 580). The network 571 may be wired, wireless or a combination thereof. Wired connections may be implemented using Ethernet, Universal Serial Bus (USB), RJ-6, or any other wired connection generally known in the art. Wireless connections may be implemented using Wi-Fi, WiMAX, and Bluetooth, infrared, cellular networks, satellite or any other wireless connection methodology generally known in the art. Additionally, several networks may work alone or in communication with each other to facilitate communication in the network 571.
[0043] An executable application, as used herein, comprises code or machine- readable instructions for conditioning the processor to implement predetermined functions, such as those of an operating system, a context data acquisition system or other information processing system, for example, in response to user command or input. An executable procedure is a segment of code or machine-readable instruction, subroutine, or other distinct section of code or portion of an executable application for performing one or more particular processes. These processes may include receiving input data and / or parameters, performing operations on received input data and / orperforming functions in response to received input parameters, and providing resulting output data and / or parameters.
[0044] A graphical user interface (GUI), as used herein, comprises one or more display images, generated by a display processor and enabling user interaction with a processor or other device and associated data acquisition and processing functions. The GUI also includes an executable procedure or executable application. The executable procedure or executable application conditions the display processor to generate signals representing the GUI display images. These signals are supplied to a display device which displays the image for viewing by the user. The processor, under control of an executable procedure or executable application, manipulates the GUI display images in response to signals received from the input devices. In this way, the user may interact with the display image using the input devices, enabling user interaction with the processor or other device.
[0045] The functions and process steps herein may be performed automatically or wholly or partially in response to user command. An activity (including a step) performed automatically is performed in response to one or more executable instructions or device operation without user direct initiation of the activity.
[0046] The system and processes of the figures are not exclusive. Other systems, processes and menus may be derived in accordance with the principles of the invention to accomplish the same objectives. Although this invention has been described with reference to particular embodiments, it is to be understood that the embodiments and variations shown and described herein are for illustration purposes only. Modifications to the current design may be implemented by those skilled in the art, without departing fromthe scope of the invention. As described herein, the various systems, subsystems, agents, managers and processes can be implemented using hardware components, software components, and / or combinations thereof.
Claims
CLAIMSWhat is claimed is:1 . A system for a comprehensive digital twin of a system comprising: a plurality of models, each model representing an aspect of the system; and a three-dimensional (3D) Tenderer for generating a 3D scene based on two or more of the models.
2. The system of Claim 1 further comprising: a user interface for displaying the 3D scene to one or more stakeholders of the system.
3. The system of Claim 2, further comprising: a set of importers in communication with the user interface for receiving information from one or more of the stakeholders of the system and updating at least one of the models based on the received information.
4. The system of Claim 1 , further comprising: a set of connectors for exchanging information between the comprehensive digital twin and a lifecycle tool of the system.
5. The system of Claim 3, wherein the lifecycle tool is a cost calculator for calculating a cost involved with the system.
6. The system of Claim 3, wherein the lifecycle tool is an environmental impact assessment tool.
7. The system of Claim 3, wherein the lifecycle tool is a predictive maintenance model to predict failure of a component of the system.
8. The system of Claim 3 further comprising a physical controller for adjusting operation of the system based on information from one or more of the plurality of models.
9. The system of Claim 3, wherein at least one of the one or more stakeholders is a designer during construction of the system.
10. The system of Claim 3, wherein at least one of the one or more stakeholders is an engineer during redesign of the system.11 . The system of Claim 1 , wherein the system comprises a building.
12. A method for creating a comprehensive digital twin of a system, comprising: receiving information relating to the system; updating a plurality of models, each model representative of an aspect of the system; constructing a virtual twin of the system based on one or more of the models; and rendering a three-dimensional visualization of the system depicting the constructed virtual twin.
13. The method of Claim 12, further comprising: receiving updated information related to the system; and updating the plurality of models with the updated information.
14. The method of Claim 12, wherein the information is received from a stakeholder of the system.
15. The method of Claim 12, wherein the information is received from an external application.
16. The method of Claim 12, further comprising: constructing or reconstructing the virtual twin throughout the lifecycle phase of the system.
17. The method of Claim 12, further comprising: receiving a portion of the information relating to the system from a first stakeholder of the system; receiving additional information relating to the system from a second stakeholder; and constructing the virtual twin of the system based on the information received from the first stakeholder and the second stakeholder.
18. The method of Claim 12, further comprising: displaying the three-dimensional visualization to one or more users via a user interface.
19. The method of Claim 18, further comprising: receiving the information relating to the system from a user via the user interface.
20. The method of Claim 12, wherein the system is a building and the information relating to the system is weather information for the proximity of the building.
Citation Information
Patent Citations
Multifactor analysis of building microenvironments
US20210173969A1
Building data platform with digital twin based predictive recommendation visualization
US20230169223A1
