Digital twin-based architecture for predictive maintenance with criticality analysis

Abstract

Various examples are provided related to Digital Twin-based predictive maintenance system augmented with criticality analysis framework. In one example, a virtual environment is provided including virtualized digital sensing device models corresponding to physical sensing devices and digital building component models corresponding to physical building components; a building information model-based digital twin is generated that provides a virtual representation of a physical building based at least in part on the virtualized digital sensing device models and the digital building component models; sensor data from at least one of the physical sensing devices and / or physical building components is received; a fault is identified using the sensor data; a criticality analysis is performed to identify a criticality score for the fault; and a user interface including a virtual representation of the physical building is generated and showing at least a subset of the fault based at least in part on the criticality score.

Description

DIGITAL TWIN-BASED ARCHITECTURE FOR PREDICTIVE MAINTENANCE WITH CRITICALITY ANALYSISCROSS REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to, and the benefit of, U.S. provisional application entitled “Digital Twin Based Architecture for Predictive Maintenance with Criticality Analysis” having serial no. 63 / 637,447, filed April 23, 2024, which is hereby incorporated by reference in its entirety.BACKGROUND

[0002] Facilities Management (FM) is a post-construction discipline that ensures that facilities operate safely and efficiently. The role of FM is to aid organizational performance and end-user experience. FM over the years has evolved from a purely technical function of caretaking, cleaning, repairs, and maintenance to include advanced functions like real estate management, financial management, change management, human resource management, building and engineering services maintenance, and utilities supply. This evolution of FM has instigated a sophisticated approach towards operating, maintaining, improving, and adapting facilities to better support the business objectives of organizations. As such, the current generation of FM is expected to contribute to the extension of the lifespan and performance improvement of assets.

[0003] Maintenance and repair functions of FM can facilitate the goal of extending the lifespan and performance improvement of assets. The maintenance and repair practice in the United States has historically been noted for relying on reactive maintenance strategies. In addition, there is substantial dependence on time-Zuse-based maintenance approaches (preventive maintenance) that result in the inefficient utilization of financial resources, excessive maintenance backlogs, and sudden asset failures.SUMMARY

[0004] Aspects of the present disclosure are related to Digital Twin- (DT) based predictive maintenance system augmented with a criticality analysis framework. In one aspect, among others, a system comprises at least one computing device; and at least one memory comprising instructions that when executed cause the at least one computing device to at least: provide a virtual environment that includes a plurality of virtualized digital sensing device models corresponding to a plurality of physical sensing devices, and a plurality of digital building component models corresponding to a plurality of physical building components; generate a Building Information Model (BIM)-based digital twin that provides a virtualrepresentation of a physical building based at least in part on the plurality of virtualized digital sensing device models and the plurality of digital building component models; receive sensor data generated by at least one of the plurality of physical sensing devices, at least one of the plurality of physical building components, or any combination thereof; identify at least one fault based at least in part on the sensor data; perform a criticality analysis to identify at least one criticality score for the at least one fault; and generate a user interface comprising the virtual representation of the physical building and showing at least a subset of the at least one fault based at least in part on the at least one criticality scoree. In one or more aspects, at least one fault can comprise a detected fault that is detected based at least in part on the sensor data and a threshold value. At least one fault can comprise a diagnosed fault that is diagnosed based at least in part on the detected fault and a fault graph model that relates the detected fault to the diagnosed fault. The fault graph model can indicate at least one probability in association with at least one edge connecting the diagnosed fault to the detected fault, and the diagnosed fault can be a highest-probability fault among a plurality of faults related to the detected fault through the fault graph model. The instructions, when executed, can further cause the at least one computing device to at least: generate, based at least in part on a machine learning model, a predicted fault based at least in part on a detected fault, a diagnosed fault, or any combination thereof, wherein the at least one fault comprises the detected fault, the diagnosed fault, and the predicted fault. The instructions, when executed, can further cause the at least one computing device to at least initiate a response to the at least one fault based at least in part upon the at least one criticality score. The response can comprise initiating repair or maintenance of a building component.

[0005] In another aspect, a method comprises providing, by at least one computing device comprising processing circuitry, a virtual environment that includes a plurality of virtualized digital sensing device models corresponding to a plurality of physical sensing devices, and a plurality of digital building component models corresponding to a plurality of physical building components; generating, by the at least one computing device, a Building Information Model (BIM)-based digital twin that provides a virtual representation of a physical building based at least in part on the plurality of virtualized digital sensing device models and the plurality of digital building component models; receiving, by the at least one computing device, sensor data generated by at least one of the plurality of physical sensing devices, at least one of the plurality of physical building components, or any combination thereof; identifying, by the at least one computing device, at least one fault based at least in part on the sensor data; performing, by the at least one computing device, a criticality analysis to identify at least one criticality score for the at least one fault; and generating, by the at least one computing device, a user interface comprising the virtual representation of the physical building and showing at least a subset of the at least one fault based at least in part on the atleast one criticality score. In another aspect, at least one fault can comprise a detected fault that is detected based at least in part on the sensor data and a threshold value. At least one fault can comprise a diagnosed fault that is diagnosed based at least in part on the detected fault and a fault graph model that relates the detected fault to the diagnosed fault. The fault graph model can indicate at least one probability in association with at least one edge connecting the diagnosed fault to the detected fault, and the diagnosed fault can be a highest- probability fault among a plurality of faults related to the detected fault through the fault graph model. The method can comprise generating, by the at least one computing device, based at least in part on a machine learning model, a predicted fault based at least in part on a detected fault, a diagnosed fault, or any combination thereof, wherein the at least one fault comprises the detected fault, the diagnosed fault, and the predicted fault. The method can comprise initiating, by the at least one computing device, a response to the at least one fault based at least in part upon the at least one criticality score. The response can comprise initiating repair or maintenance of a building component.

[0006] Other systems, methods, features, and advantages of the present disclosure will be or become apparent to one with skill in the art upon examination of the following drawings and detailed description. It is intended that all such additional systems, methods, features, and advantages be included within this description, be within the scope of the present disclosure, and be protected by the accompanying claims. In addition, all optional and preferred features and modifications of the described embodiments are usable in all aspects of the disclosure taught herein. Furthermore, the individual features of the dependent claims, as well as all optional and preferred features and modifications of the described embodiments are combinable and interchangeable with one another.BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Many aspects of the present disclosure can be better understood with reference to the following drawings. The components in the drawings are not necessarily to scale, emphasis instead being placed upon clearly illustrating the principles of the present disclosure. Moreover, in the drawings, reference numerals designate corresponding parts throughout the several views.

[0008] FIG. 1 shows an example of a Digital Twin-based predictive maintenance system augmented with a criticality analysis framework, in accordance with various embodiments of the present disclosure.

[0009] FIG. 2 shows an example of a fault model for a Digital Twin-based predictive maintenance system with a criticality analysis framework, in accordance with various embodiments of the present disclosure.

[0010] FIG. 3 shows an example flowchart describing the functionalities of the Digital Twin-based predictive maintenance system with the criticality analysis framework, in accordance with various embodiments of the present disclosure.

[0011] FIG. 4 shows an example of a user interface of a platform generated using the Digital Twin-based predictive maintenance system with a criticality analysis framework, in accordance with various embodiments of the present disclosure.

[0012] FIG. 5 shows an example of a solution architecture for the Digital Twin based predictive maintenance system prototype in Autodesk Tandem, in accordance with various embodiments of the present disclosure.

[0013] FIG. 6 shows an overview illustrating an example of alignment of Tandem functionalities and built-in tools to proposed RA, in accordance with various embodiments of the present disclosure.

[0014] FIG. 7 shows an example of a visualization of actual real-time data from selected extracted datapoints, in accordance with various embodiments of the present disclosure.

[0015] FIG. 8 shows an example of a system solution architecture for Unreal Engine- Azure Digital Twin (UE-ADT) BIM-based DT system for PdM, in accordance with various embodiments of the present disclosure.

[0016] FIG. 9 shows an overview illustrating an example of tools used in the loT-Game engine prototype and their alignment to the layers of the proposed reference architecture (*not formally executed), in accordance with various embodiments of the present disclosure.

[0017] FIG. 10 shows an example of alignment of DT and SkySpark taxonomies in Revit, in accordance with various embodiments of the present disclosure.

[0018] FIG. 1 1 shows an example of preserved Revit parameters as metadata in UE (Generated during import of BIM model into Unreal Engine), in accordance with various embodiments of the present disclosure.

[0019] FIG. 12 shows an example of a model graph created for the system solution architecture prototype, in accordance with various embodiments of the present disclosure.

[0020] FIG. 13 shows an example of a graph-based virtual representation of physical entities and sensor data updates in an ADT instance, in accordance with various embodiments of the present disclosure.

[0021] FIGS. 14A and 14B show an example of a user interface visualization of telemetry and dynamic condition monitoring of sensors and a table of rules for dynamic visualization, in accordance with various embodiments of the present disclosure.

[0022] FIG. 15 shows an example of a computing system for execution of a Digital Twinbased predictive maintenance system augmented with a criticality analysis framework, in accordance with various embodiments of the present disclosure.DETAILED DESCRIPTION

[0023] The present disclosure describes various examples of systems and related methods for a Digital Twin (DT)-based predictive maintenance system augmented with a criticality analysis framework. The system can perform fault detection and analysis that includes a criticality analysis for the predictive maintenance items that are identified. This can include the use of Building Information Modeling (BIM) as a component of the DT. The system can provide a real-time view of managed assets by providing critical real-time data, contextual insights, and / or analytics that help identify potential issues. The use of DTs in facilities management (FM) of buildings can overcome the limitations of existing systems such as building automation systems (BAS), computerized maintenance management systems (CMMS), etc., in supporting advanced predictive maintenance management. Such limitations include an inability to perform predictive maintenance and analytics-based criticality analyses that support maintenance decision-making as provided using the mechanisms described herein. Real-time condition monitoring, made possible by the proliferation of sensing devices and the aspects of the present disclosure can augment predictive maintenance which can provide about 8%-40% cost savings in maintenance expenditure compared to reactive and preventive maintenance.

[0024] The mechanisms described can perform predictive maintenance and analyticsbased criticality analyses in view of the following principles and as described through the figures. For example, development of the architecture for predictive maintenance with criticality analysis can include creating a pattern to operationalize the data fabric architecture presented earlier. As DTs are systems by design, a principal element of their development is the design of a system architecture that responds to the required functionality of the DT. The reference architecture for the DT-based PdM solutions described can include features that correspond to one or more of the following concepts as integrated with criticality analysis frameworks.

[0025] Moving to FIG. 1 , shown is a reference architecture for a DT-based predictive maintenance system 100 with a criticality analysis module. The predictive maintenance system 100 can include a DT and / or BIM-based system in a networked environment of components that can communicate over a wired or wireless network. The predictive maintenance system 100 can be referred to as a criticality analysis predictive maintenance system 100 since it includes modules or components that work in concert to perform criticality analysis. The predictive maintenance system 100 can include an information layer 103, an application layer 106, and a service layer 109. The information layer 103 can include a physical environment layer 112 and a virtual environment layer 115. The physical environmentlayer can include physical building components 118 and sensing devices 121 , which can sense aspects of a physical environment, including any measurable physical parameter that can be sensed using a sensor device 121. This can include sensing parameters of the physical building components 118, as well as aspects of physical structures in the physical environment. The physical building components 118 and sensing devices 121 can transmit data to the components of the virtual environment layer 115 as shown.

[0026] The virtual environment layer 1 15 can include digital building component models 124, digital sensing device models 127, a BIM-based digital model 130, a rules database 133, and a conditions database 136. The information layer can also include a maintenance management system or other operational technology systems 139, corresponding to one or more Computerized Maintenance Management Systems (CMMS) or Integrated Workplace Management Systems (IWMS).

[0027] Application layer 106 can receive or retrieve data from information layer 103. The application layer 106 can include a condition monitoring and fault detection module 142 or modules, a fault diagnosis module 145, a condition prediction module 148, a criticality analysis module 151 , and an alarm module 154.

[0028] The systems can include a data-driven PdM execution framework for building systems such as mechanical, electrical, and plumbing (MEP) systems based on BIM and loT technologies. The framework can include (1) an information layer, (2) an application layer, and (3) a user interface or service layer. In the information layer, the Construction Operations Building information exchange (COBie) specification can be used to extract information for predictive maintenance from the pre-stored as-built models to populate the FM system, which in their framework can include a Computerized Maintenance Management System (CMMS) or Integrated Workplace Management Systems (IWMS). A CMMS database can be used to manage maintenance records, work orders, and work requests. Sensing devices 121 can be installed on the MEP or other physical building components 1 18 to measure the real-time condition of the physical building components 1 18 as well as other physical parameters or a building. The real-time sensor data can be stored in .csv files or an SQL database, or another data structure.

[0029] The physical environment layer 1 12 can include the actual equipment or machines being managed as well as the physical structures of buildings. In some examples, sensing devices can be separated into a standalone layer known as the sensing layer. The sensing layer can be responsible for real-time updates and bidirectional coordination of the physical building components 1 18 with their virtual representations through condition monitoring, location detection, and control. However, in the example shown, the physical environment layer 1 12 includes both physical building components 118 and sensing devices 121.

[0030] In some examples, a five-dimension DT model for management of complex equipment. A mathematical expression to depict the model (MDT) can include,MDT= (PE, VE,Ss, DD, CN) (1) where PE refers to a physical entity in the physical environment layer 112, VE stands for the virtual equipment in the virtual environment layer 115, Ss stands for services for the PE and VE, DD is the DT data, and CN is the connection among PE, VE, Ss, and DD.

[0031] PE can refer to the functional system or subsystems that perform a defined task whereas the sensors collect the states of the system or subsystems and the relevant working conditions. The VE can be a high-fidelity digital model of the PE that represents its geometry, physical properties, behaviors, and rules in the virtual world. The VE can be modeled as follows,VE = (Gv, Pv, Bv,Rv) (2) where Gv is a 3D solid model developed in CAD modeling software; Pv simulates the physical properties of the PE; Bv describes the behavior of the PE governed by its driving factors; and Rv comprises the rules of constraints, associations, and deductions. The rules serve as the brain that makes the VE judge, evaluate, optimize and / or predict.

[0032] The Service model (Ss) incorporates services that optimize the operations of the PE and ensures the fidelity of the VE. Thus, the Ss can describe the function, input, output, quality, and state of the PE and calibrates the VE to be in sync with the PE. An example of Ss can be the temperature output monitoring for a chiller. For example:Ss = (Function, Input, Output, Quality , State) (3)

[0033] The DT Data model (DD), expressed in equation (4), can include the data from the PE (Dp), data from the VE (Dv), data from the Ss (Ds), domain knowledge (Dk), and fused data (Df). Df is a fusion of Dp, Dv, Ds, and Dk.DD = (Dp, Dv, Ds,Dk, Df) (4)

[0034] The bidirectional delivery of data is governed by the Connection Model (CN). For a given connection, CNxX, the delivered data can be modeled as follows,CNxX = (Datasource, Unit, Value, Scope, Sampling interval) (5)

[0035] For example, temperature data from a chiller measured every 15 minutes can be expressed as,CN _PV -temperature = (Temp sensor, fahrenheit, 45, 32 — 212,15m) (6)

[0036] A workflow for the DT can be provided according to equations (1) to (6). This framework can provide a comprehensive approach to developing a DT that leverages both physics and data models but is equipment-centric. Physics-based components can involvethe processing of large amounts of data. The framework can provide a DT for predictive management in buildings.

[0037] The overall framework can integrate the data from a 3D geometric model and maintenance management systems, essentially making such systems part of the DT architecture provided according to the BIM-based digital model 130. Integration of maintenance management systems 139 into the information layer 103 can provide a rich source of maintenance history and a highly efficient data source for work order and maintenance resource management. Further modification details of the information layer 103 are discussed below. As a general model, the information layer 103 can incorporate a PE and VE, and is expressed as follows,IL = PE, VE) (7)

[0038] For clarity, the information layer is detailed to incorporate the expression of the PE and VE as described above. This can establish the relationship between the monitored systems, physical building components 118, and sensing devices 121. The sensing devices 121 described here can include representations of the same in the virtual environment layer 115, for example, as digital sensing device models 127. Sensing device 121 functionalities can be monitored to ensure the DT is working properly as defined using the BIM-based digital model 130 and other components of the virtual environment layer 115, in view of the predetermined rules stored and defined in the rules database 133.

[0039] The system architecture for a DT can identify sensing devices 121 as physical entities that have geometry, function, input, output, quality, and state. Consequently, adequate virtual representations of sensor devices 121 are provided as digital sensing device models 127 in the virtual environment layer 115.

[0040] The rules database 133 can include operational setpoints, facts and rules for management, and other information that describes which data is to be provided to the application layer 106 and the service layer 109 for the user interface 157. The rules database 133 can include user designer specified that specify criticality information such as a hierarchy of enterprise importance. Cost, time, energy consumption, location, and other factors can be indicated using a numerical or symbolic measure of criticality. Each of the physical building components 118, and sensing devices 121 , as well as corresponding digital building component models 124 and digital sensing device models 127, can be associated with data that indicates location, and expected and actual energy consumption, as well as cost data including running costs, costs of downtime, costs for certain errors or maintenance events, and so on. As a result, each of these entities can specify static and dynamic parameters that can be mapped to criticality decisions according to rules specified in the rules database 133.

[0041] The physical environment layer 1 12 can include physical entities as described using building components 1 18 and physical sensing devices 121. These can provide data including real time or near real time sensor data to the conditions database 136. In this context, real time and near real time can refer to times with negligible or minimal delay in processing and display / use of the value in the system. For the purposes of the document, real time can refer to time under a centisecond, and near real time can refer to time under one second, or under a predetermined number of seconds such as under 5 seconds, under ten seconds, or under a minute.

[0042] The virtual environment layer 1 15 can include a BIM-based digital model 130 that relates the physical building components 118, physical sensing devices 121 , and other components according to the rules database 133 and the conditions database 136, to provide a digital twin of the physical environment layer 1 12. The virtual environment layer 115 can describe the geometry (Gv) of the physical entities to an appropriate level of development (LoD), identifies the physical properties (Pv) of the physical entities such as location, entity ID, etc., and behavioral parameters (Bv) such as set-points, flow rates, voltage, etc. The data in the VE must be mapped to the maintenance management system to ensure that each physical entity is in the asset inventory.

[0043] Furthermore, the conditions database 136 can receive and store real-time sensor readings from the physical sensing devices 121. The conditions database 136 can be linked to the maintenance management system 139 and a rule database 133 to develop a knowledge base for fault detection and fault diagnosis, which can be carried out in the application layer 106 using data received from the information layer 103. The application layer 106 can include five modules or more that are designed to logically handle real-time input from the information layer 103.

[0044] The condition monitoring and fault detection module 142 can perform condition monitoring using an event stream approach and draw on the rules database to detect faults. Once faults are detected, the fault diagnosis module 145 can follow a hierarchical rule-based or artificial intelligence-driven approach to identify an underlying condition that caused the fault. This underlying condition can be utilized for the condition prediction module 148 to predict future faults and provide empirical insights on the predicted future faults. The criticality analysis module 151 carries the analysis further to assess the criticality of the current and future faults output, prior to the triggering of an alarm in the alarm module 154. The functionalities can be arranged in this logical sequence to prevent the event of ‘alarm showers’. The output from the application layer 106 feeds the service or visualization layer 109 which the end user interacts with as an intelligent reality.

[0045] Criticality analysis in the context of the predictive maintenance system 100 can refer to the process of prioritizing faults that have been detected and diagnosed by the fault detection module 142 and the fault diagnosis module 145, and / or predicted using the condition prediction module 148. The criticality analysis module 151 can support decision-making on the allocation of resources for maintenance actions. For a single cycle of operation, the process can begin with collection of the sensor data at a set interval e.g., every five minutes, from the information layer 103. For example, from the application or device responsible for condition monitoring e.g., from the building automation systems (BAS) or building management system (BMS) that implements the information layer 103. The sensor data can be transmitted and received via an appropriate telemetry transfer protocol. The time interval for sensor data collection can vary for different equipment or systems based on their usage pattern and / or process parameters being monitored.

[0046] Following this, fault detection module 142 can run a fault detection based on a particular fault analysis technique e.g., traditional single variable technique. This technique involves checking the reported sensor reading for the specific process parameter e.g., pressure, temperature, flow rate, vibration, etc., for deviations (high or low threshold values) relative to a set acceptable limit, for example, as defined in the rules database 133. Sensor data, maintenance history, diagnosed faults, and other information can be received. The sensor data can include a building ID, a device ID, a timestamp, one or more values, and can append other types of criticality data such as a location, documents, and so on. Locations can refer to a predefined area (e.g., room, floor, building, set of rooms, set of floors, buildings, etc.) that is associated with a criticality value. If no fault is detected, the sensor data collection and monitoring cycle continues. When a fault is detected by the fault detection module 142, the fault diagnosis model 145 can perform a fault diagnosis using a fault tree model.

[0047] For reference, FIG. 2 shows one example fault graph model 200 with a set of fault nodes corresponding to problems or fault types. The fault types can include items that are detected using sensed setpoints as well as those that do not have sensor data (data from sensor devices 121) or building component data (data from building components 1 18) reported directly for them. In one example, sensor data can indicate that the chiller temperature is lower than a threshold (e.g., a threshold difference from a setpoint). This can be caused by a number of other problems or fault types, indicated by edges between the nodes, which indicate a direction such as an arrowhead. Upstream items at the base of an arrow can be causes while downstream items at the arrow or tip can be effects. Causes for the low chiller temperature can include improper or fluctuating line voltage, among other causes indicated at the bases of the various arrows connected to the low chiller temperature fault node. Some fault nodes can be checked (verified or refuted) by sensor data and component data. Refutedfault nodes can be discarded, while verified fault nodes can be provided as potential causes. Unrefuted (and unverified) fault nodes can also be provided as potential causes. Potential causes can be referred to as “diagnosed faults” in some examples.

[0048] As a result, the fault model 200 can refer to a failure mode according to binary or directed graph (cause-and-effect) models curated on specific failure modes that can be encoded into the rules database. Accordingly, fault isolation can also be performed, which involves deduction of a probable fault and effect. For example, if a temperature sensor attached to a chiller reports a low or high value, the algorithm runs through the possible diagnoses (e.g., unrefuted), which are coded based on historical failure modes or assigned probabilities for each causal edge between fault nodes using expert knowledge, to isolate one or more diagnosed fault, which can then be sorted and provided according to resulting probability, with the most probably diagnosed faults provided higher in the list. In some examples, a single fault is provided. In some cases, an unknown cause may be identified, as indicated by the arrows with no starting point.

[0049] Moving back to FIG. 1 , the condition prediction module 148 can refer to prediction of future faults based on the “detected faults” from sensor and component data setpoints analyzed using the fault detection module 142 and diagnosed faults resulting from the fault diagnosis module 145. The condition prediction module 148 can include rules-based algorithms as well as machine learning algorithms that are trained using training data that includes verified mappings between detected faults, diagnosed faults, and predicted faults. The machine learning algorithms can include Neural network algorithms such as Long Short- Term Memory (LSTM). Deep Learning such as variational autoencoders and generative artificial intelligence can also be trained and utilized. The service layer 109 can provide feedback data for further training of the machine learning models. For example, the user interface 157 can enable a user to confirm detected faults, diagnosed faults, and predicted faults. Moreover, predicted faults that are not corrected can be verified for training purposes if the predicted fault shows up as a detected fault within a predetermined time period from detection, where the predetermined time period can be fault-type specific based on expected timeline until a predicted fault is predicted to occur.

[0050] The detected faults, diagnosed faults, and predicted faults can be dispatched to the criticality analysis module 151 , which until this point remains in an idle state. For a single fault, there will be only one fault in a fault queue in the criticality analysis module 151. For multiple or simultaneous faults, a queue can be established based on the order in which faults were dispatched to the condition database. The presence of a posted fault triggers the running of the criticality analysis algorithm which calculates the cost of the fault and likely downtime based on data from the maintenance management database. A priority rating is assigneddepending on the outcome of the calculation and an advisory is generated. An alternative approach will be to use a predetermined criticality ranking to sort and order the detected faults in an ordinal sequence from highest priority to lowest priority. The outcome suggests the level of urgency maintenance staff can attach to planning and scheduling interventions.

[0051] FIG. 3 illustrates a flowchart 300 of one illustrative cycle of the predictive maintenance system 100. The flowchart can be viewed as a method performed by one or more computing devices. For illustrative purposes, the various actions can be separated into ‘fault analysis’, ‘criticality analysis’ and ‘alarm and visualization’ categories. However, the underlying instructions can be executed as a single application or multiple applications that implement the predictive maintenance system 100.

[0052] In the fault analysis section, the predictive maintenance system 100 can collect sensor data and physical component data. This data can be retrieved from the conditions database 136, and can include data generated using the sensing devices 121 and electronic portions of various physical building components 118. In some examples, the term “sensor data” can be used to refer to any of this data, since the electronic portions of various physical building components 118 can include sensor devices 121. The sensor device 121 can also include sensors that are separate from the physical building component 118 but can nevertheless indicate data in association with a physical building component 118. Physical building components 118 can include physical building locations, structures, devices, machines, and other components.

[0053] The predictive maintenance system 100 can run a detection algorithm or process on the sensor data. This can include comparison of sensor data to predetermined threshold values according to the rules database 133. The predictive maintenance system 100 can identify a detected fault based at least in part on the sensor data indicating a value for a particular parameter, which crosses a threshold value. If no fault is detected, then collecting and monitoring sensor data can continue.

[0054] If a fault is detected, the predictive maintenance system 100 can run a fault diagnosis algorithm or process. The predictive maintenance system 100 can identify a diagnosed fault using a hierarchical rule-based approach to identify an underlying condition that caused the fault. For example, the diagnosed fault can be identified using a fault graph model 200. The predictive maintenance system 100 can also use the fault graph model 200 to isolate a fault. The fault graph model 200, the detected faults, and the diagnosed faults can also be provided as inputs to a machine learning algorithm or rules based algorithm that generates predicted faults. The detected faults, the diagnosed faults, and the predicted faults can be labeled using labels that indicate detected, diagnosed, and predicted. This data can be used for a criticality analysis, in view of other data including an associated ID of the sensordevice 121 and / or building component 1 18, locations thereof, cost data, time data, and other parameters as discussed.

[0055] In the criticality analysis, the predictive maintenance system 100 can maintain a fault queue. The detected faults, the diagnosed faults, and the predicted faults can be placed in the fault queue. For a single fault, there will be only one fault in the fault queue (e.g., in the condition database 136 or a separate fault queue database). For multiple or simultaneous faults, a queue can be established based on the order in which faults were dispatched to the fault queue. The presence of a posted fault triggers the running of the criticality analysis algorithm which calculates the cost of the fault and likely downtime based on data from the maintenance management database. A priority rating is assigned depending on the outcome of the calculation and an advisory is generated. An alternative approach can be to use a predetermined criticality ranking to sort and order the detected faults in an ordinal sequence from highest priority to lowest priority. The outcome suggests the level of urgency maintenance staff can attach to planning and scheduling interventions.

[0056] The predictive maintenance system 100 can trigger a notification to a client device such as a mobile phone, tablet, laptop or other mobile device, or any computer system. In some examples, Multimedia Messaging Service, Short Messaging Service, or another messaging protocol can be used. Email and other electronic messaging services can also be used. The notification can include a link to a website that includes a user interface 157 of the service layer 109, or can otherwise trigger the user interface 157 to be displayed on a client device.

[0057] FIG. 4 shows a user interface 157 generated using the predictive maintenance system 100. The user interface 157 can include a virtualized three-dimensional representation of a building according to the DT generated using the virtual environment layer 115. The user interface 157 can include the ability to navigate a building in three dimensions. The user interface 157 can be generated in response to a notification received from an alarm, by navigating to a website provided by the service layer 109, or by selecting a link such as a uniform resource locator or other network address link provided in the notification.

[0058] The user interface 157 can generated by automatically placing a viewpoint so that one or more fault icons or fault elements are in view. The fault elements can include visual representations that indicate an identified fault such as one or more of a detected fault, a diagnosed fault, a predicted fault, or any combination thereof. A fault element can include a widget or other user interface element. Each fault element can be color coded indicating a priority level or criticality of the fault. A list of the faults can be generated in response to the selection of another user interface element from the user interface 157. A selection of a particular fault element can provide a fault name indicating the fault type, a potential fixincluding an action to perform and a textual or other description of how to perform the fix or correction, a location of the fault (e.g., of the sensor or component associated with the fault) and / or a location where the fault correction or fix is to be applied. A list of probable causes and corrections for each can be provided as well, along with criticality information such as financial and energy costs, and so on.

[0059] End user service and visualizations represent the front-end of the proposed predictive maintenance system 100. To achieve this, a heads up display (HUD) and user widgets can be provided together with a custom time series analytics platform or another Internet of Things (loT) time-series data analysis tool such as Azure Time Series Insights (TSI) or Azure Data Explorer. Telemetry flow to the loT Hub devices (e.g., sensor devices 121) can be routed to the time series analytics platform for storage and processing. Telemetry processing produced historical charts that can be used to track the condition of the real-world entity including physical building components 1 18 and sensor devices 121. A Web app can be deployed to act as a middleware between the user experience application 157 and time series analytics platform to enable live queries of the data on demand.

[0060] For example, a sensor user interface element or widget can be provided. Once clicked in the Digital Twin end user application, a Hyper Text Transfer Protocol (HTTP) request can be made to the Web app or service layer 109 to query the required data in the various databases discussed. The raw telemetry payload can be stored in JavaScript Object Notation (JSON) or another data structure format and historical charts can be then furnished to update the user interface 157 for visualization. The sensor device 121 can be individually represented using individual widgets or elements shown as circular items in the figure.

[0061] The elements can include a dynamic radial virtual material that changes color based on reported values such as criticality, location or other values. In one nonlimiting example, green signifies that the current sensor value is within acceptable range; amber means approaching an undesirable limit; red means the value is at an undesirable limit; no ring color means the sensor is not receiving data. The bottom of the screen or another user interface area can provide a visualization of telemetry inside of the user interface 157 after establishing a live connection. The HUD design enables user interaction with telemetry such as text input in the form of markers and downloading of historical data. Individual sensor widgets can be interacted with. At the equipment level, the application can track the time left for replacement of the equipment using the metadata from the associated file.Prototypes development

[0062] Overview of real-world testbed. DT system prototypes were developed in the context of an existing building on the campus of a higher education institution in the Southeast U.S.A. The selected building was a three-story 57,000 square foot facility that containedclassrooms and offices spread over three levels. Specifically, the study was limited to the attic space of the building, which served as a mechanical room. The location of the mechanical room in the attic space imposes challenges such as reduced visibility of managed assets and accessibility, making the case for the need for a system that increases asset visibility for maintenance purposes. Moreover, considering the humid subtropical location of the facility and its potential impact on the operational efficiency of equipment like air handling units (AHUs) located in an attic, the study focused on monitoring a selection of process parameters operation of an AHU. Additionally, the AHU was one of the few equipment categories that had complete connectivity within the network of maintenance operational technology (OT). In consultation with the managers of the facility, the research team limited the research to monitoring selected temperature and pressure sensors installed on the AHU. The technical configuration of the institution's OT informed the choice.

[0063] In terms of OT, the facility monitored equipment condition with a combination of Metasys BAS and a maintenance intelligence platform known as SkySpark. Metasys interacts directly with sensors installed on the equipment and communicates the recorded sensor data to SkySpark using the BACnet protocol at 15-min intervals. SkySpark's role in the maintenance of the facility was limited to trending analytics mostly in the form of charts that provided no visualizations of the asset and imposed a high cognitive load on users to interpret. Additionally, end users had to be skilled at writing specialized code with the Axon programming language to create desired analytics. This limited the use of SkySpark to only a few technical experts. Since Metasys is a proprietary vendor-specific BAS that was managed by a third party, the project team was not provided access for integration into the DT system. As such, the stakeholders of the project, including the research team owner's Information Technology department, Facilities Services, and Business Affairs and Technical Services decided that the DT system should be able to:• Interact with SkySpark in real-time to retrieve sensor data obtained from the BAS.• Provide real-time analytics on process parameters of interest i.e., selected temperature and pressure readings.• Visualize managed assets and individual sensors and their real-time condition.• Execute a rule-based PdM program based on defined thresholds.

[0064] Platform selection. Two approaches were identified for platform development i.e., build or buy. In terms of DT design and implementation, the build approach involves assembling resources to create a custom platform whereas the buy approach involves purchasing a pre-built platform that offers preprogrammed and all-in-one offering for developing a DT. Essentially, the choice of platform is chiefly determined by the availability or lack of personnel who are proficient with software engineering tools and coding skills to createa DT twin system from scratch. In addition, the scale of implementation and the nature of the organization could also determine the type of platform that is chosen. Regardless of the approach, it is worth noting that the DT is a system of systems. Therefore, even custom- developed DT solutions are an assembly of multiple software such as game engines such as Unreal Engine or Unity, public cloud resources such as Microsoft Azure or Amazon Web Services, and loT hardware.

[0065] Since the project was connected to a real-world implementation program, two prototypes were developed: one developed with a plug and play tool (buy) and one using either an loT or Gaming Engine (build) to build a DT-based PdM system. The criteria for selecting the tools to be used were as follows.• Evidence of the tool being used in research or practice for DT development.• Support for data and system integration with SkySpark.• Support for BIM-based visualization.• Support for real-time analytics i.e. , time series graphs.• Support for PdM threshold setup.

[0066] The study settled on Autodesk Tandem for the buy approach. Tandem was selected because there was evidence of its use in practice by the University of Birmingham's facilities management unit; it supported direct integration with SkySpark; relied on BIM f or asset visualization; and provided real-time analytics. A hybrid of Unreal Engine (UE) and Microsoft Azure (AZ) for the build approach. The ensuing solutions are hereafter referred to as Prototype 1 (Buy)- using Autodesk Tandem and Prototype 2 (Build)- using UE-AZ.

[0067] Reference architecture for DT-based PdM systems. This research utilizes the reference architecture (RA) of the DT-based PdM system shown in FIG. 1 to create the unique solution architectures for two different prototypes. The proposed RA comprises three layers i.e., information layer, application layer, and service and visualization layer, representing the core features of a DT. The application layer encompasses any component, physical or virtual, that produces, observes, or contains data or information related to the managed asset. The contents of the information layer serve as the input for the application layer, which plays the role of the brain of the DT. The processing of the data from the information layer in the form of fault detection and diagnosis (FDD), predictions, criticality analysis, and alarm generation happens in the application layer. The output of the application layer is visualized in the third layer i.e., the service and visualization layer. The three-layer RA provides a pattern that ensures consistency of implementation with respect to technology assembly and deployment for DT-based PdM. Furthermore, the RA serves as a tool for validating the conformity of deployed DTs to the needs of a DT-driven PdM practice.

[0068] Prototype 1. Prototype 1 relied on the buy approach and was developed within Autodesk Tandem, which is a commercial tool that enables a no or low code ‘plug and play’ approach to developing DTs of constructed facilities. Tandem's BIM-based design offers a quick time to deployment with BIM-based visualizations. Moreover, Tandem's quick process for developing and deploying BIM-based DTs, made it a useful test-bed for testing DT development by FM professionals with limited BIM and computing skills. To apply the proposed reference architecture to a predictive DT implementation in Autodesk Tandem, a technical review of the software was undertaken to understand its features. This was beneficial to developing a suitable solution architecture that follows the pattern of the proposed reference architecture. The main findings at the time of creating the prototype were.• Tandem provides a seamless and low intensity import of BIM models from Revit, which is a widely used BIM authoring tool in the United States.• Updates to the models in Revit are captured in real-time when the Revit file is hosted on the Autodesk Construction Cloud. This functionality is valuable for continuous updates to the as-built model in the event of facility upgrades, replacements, or renovation.• Tandem only supports numeric data formats. This means that data in the other formats such as Booleans (yes / no; on / off) have to be codified into numeric values.• Facility template definition is the critical step in aligning DT data to the data points from the BAS or business informatics / intelligence tool.• Tandem provides telemetry visualization but does not offer anomaly detection functionality.• The Tandem Software Development Kit (SDK) is needed to integrate FDD and criticality analysis.FIG. 5 illustrates an example of a solution architecture created to govern the implementation.

[0069] To practically implement the solution architecture, four sensors hosted by the AHU, tagged as AHU-01 -RM-401 , were chosen to be tracked. These were the chilled water temperature sensor (CHW Temp), chilled water valve position (CHW Vlv %), cooling coil discharge air temperature (Cool Coil DAT), and cooling coil discharge air temperature setpoint (Cool Coil DAT SP). These four were selected due to the availability of data and for ease of managing the live data connections for prototyping. In addition, the sensors chosen provided a comparative set of data for a major HVAC system failure that had occurred on the campus previously. The extent to which Tandem met the requirements of the RA is shown in FIG. 6, which shows an overview of alignment of T andem functionalities and built-in tools to proposed RA. Therefore, to support DT-based PdM, the software development kit (SDK) will need to integrate the developed PdM engine into Tandem as a plugin. The SDK was not publiclyavailable at the time of conducting this study, so this functionality was not executed. The steps involved in setting up the Tandem prototype are explained in detail in the following subsections.

[0070] BIM-based representation of physical objects'. The as-built Revit file, managed within the Autodesk Revit 2023 version, was purged of irrelevant elements such as sheets, detail drawings, and unused elements. This was done to reduce the size of the file and retain only those model elements that are relevant to the DT. The Revit file was then uploaded to the Autodesk Construction Cloud (ACC) to enable real-time cloud collaboration between stakeholders and the research team. A new facility was created in Tandem to receive the Revit file as the base data source for 3D visuals and non-graphical data on model elements. A prerequisite in Tandem involves the selection of an appropriate classification system.

[0071] There are four built-in classification systems, i.e., Masterformat, UniClass, Uniformat, and Simple Categories. Tandem enables customization of each of these built-in classification systems to suit the user's needs. The purpose of selecting a classification system is to have a standardized method for tagging assets. For this prototype, ‘Simple categories’ was selected as it broke down the building assembly into product types, e.g., air terminals, mechanical equipment, etc. The classification was then enriched with custom parameters that represented the sensors as mentioned earlier. The Revit file was subsequently uploaded into Tandem. Once uploaded to Tandem, all model elements assumed the status of assets. An asset in Tandem is a digital replica of a real-world entity. Consequently, the asset, i.e., AHU- 01 -RM 401 , had four parameters that represented the sensors attached to the asset in the real world.

[0072] Integration of sensor data'. The integration of the sensor data into the visualized 3D model was achieved through Tandem streams. The streams functionality in Tandem enables the creation of uniform resource locators (URLs) that provide an application programming interface (API) connection to integrate sensor data from any source. To integrate the data from SkySpark, a stream was set up and its URL was shared with the managers of the SkySpark system to make a POST request to the URL. Each post request, at 15-min intervals, published a JSON payload with the data reported by the four sensors to SkySpark via Metasys. To ensure accurate data mapping, the key for the first payload had to be mapped to the corresponding parameter. After this configuration, the system automatically registers data to the appropriate parameter through a POST request via the Tandem API.

[0073] Analytics and visualization'. Tandem provides a charts functionality that enables plotting of graphs from the sensor data. From a predictive analytics perspective, the telemetry captured in the charts provide insights into real-time and historical conditions through thegraph of FIG. 7, which illustrates a visualization of actual real-time data from selected datapoints extracted from the Johnson Controls NAE via SkySpark. However, a trained eye is needed to examine the charts to determine anomalies. This may place a higher cognitive load on users in the event of having several sensors.

[0074] Prototype 2. Prototype 2 was developed within a hybrid loT-Game engine platform. Microsoft Azure resources were used to establish the loT infrastructure whereas Unreal Engine 5 (UE5) was selected as the game engine. Due to security protocols, this prototype could not be integrated with sensor data from SkySpark. In lieu of the real-world data, a mock loT device simulator was deployed with parameters that enabled creation of synthetic data similar to that reported in the real world. The prerequisites provided in the creation of this prototype are as follows:• Azure resources- Azure Digital Twins (ADT), Azure Function App, Azure Event Grid, Azure Event Hubs, Azure loT Hub, Azure.• SignaIR Service, Azure Time Series Insights.• Unreal Engine (UE): UE 4.27, UE 5.0, ADTLink Plugin for UE 4.27 and 5.0.• Mock loT device simulator: Built from https: / / github.com / codetunez / mock-devices.

[0075] An example of the system solution architecture for UE-ADT BIM-based DT system for PdM developed in line with the reference architecture is shown in FIG. 8. FIG. 9 shows the overview of an example of tools used in the loT-Game engine prototype for this implementation against the components (layers) of the proposed reference architecture. The ADTLink Plugin served as a middle-ware between UE and ADT, helping to automate the creation of twins in the ADT from the BIM model. This saved a lot of effort in terms of writing the Digital Twin Definition Language (DTDL) code to create the DT models in ADT.

[0076] BIM-based representation of physical objects'. In this prototype, the BIM model had to be exported to a file format that is compatible with UE5. First, the model element had to be aligned with the data in SkySpark. This involved creating digital replicas of the sensors and aligning their names to those found in SkySpark. To speed up the creation of twins in the loT platform using the ADTLink plugin, two new parameters were added to the BIM model. The two parameters were “ADT_Modelld” and “ADT_Twinld.” “ADT_Modelld” aligned each model entity to a particular model interface while “ADT_Twinld” specified the name of the twin. The twin names were aligned with the data points in the BAS and SkySpark. The challenge faced here was that the datapoints in SkySpark were generic for each AHU. This meant that it would be impossible to create unique twin IDs for each sensor if the DT were scaled to include other AHUs. As such, AHU01 was appended to the data point names as shown in FIG. 10, which illustrates an example of alignment of DT and SkySpark taxonomies in Revit.

[0077] Following the alignment of model elements with SkySpark data points, the Revit model was exported to a Udatasmith’ file using the Datasmith plugin. The Datasmith file format was identified as the appropriate file format for this exchange as it enables the preservation of metadata and physics-based rendering (PBR) associated with each model element. This is important as it helps match the DT closely to its physical counterpart. For example, Datasmith enabled preservation of the parameters created in the BIM such as installation date, which was used in this prototype to estimate RUL. Prior to the export, the model was purged to remove unused model elements to reduce the size of the Revit file and also reduce the number of static meshes to be processed by the Unreal Editor.

[0078] The import process into the UE editor followed the visual dataprep process. The UE visual dataprep allows merging different geometry, which is great for federating different disciplinary models, as performed in this project. Also, the materials and component mesh in the Revit file were swapped for more realistic ones for enhanced visualization using the visual dataprep process. The preservation of metadata from Revit is illustrated in FIG. 1 1 , which illustrates an example of preserved Revit parameters as metadata in UE.

[0079] Configuring DT data models in UE and integrating with APT: The DTDL was used to create model interfaces in line with the RealEstateCore (REC) ontology as a way of establishing a standardized data model for virtualizing real-world entities. The models were created using the UE Blueprints visual programming and facilitated by the ADTLink plugin for UE. Ten DTDL models based on REC were created as shown in the model graph of FIG. 12 and synchronized with the ADT cloud through the ADTLink plugin. The model interfaces were used to create twins of the building, level, room, equipment, and sensors. For consistency, the same AHU was maintained in this prototype. However, additional sensors were added to the four used in the previous prototype. A portion of the virtualized physical world in the ADT environment is illustrated in FIG. 13, which shows a graph-based virtual representation of physical entities and sensor data updates in the ADT instance for AHU01 DAP.

[0080] Sensor data simulation and integration: Since live access to data from SkySpark or Metasys was not obtained, the mock-device app was utilized to simulate sensor data. First, a sample of historical data from SkySpark was inspected to determine lower and upper boundaries. Subsequently, a typescript file was programmed to generate the data on demand. The typescript code has not been shared as it contains secure connection credentials to Azure resources. The simulated sensor app was configured to send telemetry to the Azure loT Hub. Once data was registered with the loT Hub, a function written to ingest the telemetry into the ADT instances is triggered. An example of this is shown in FIG. 13, where the value for AHU01 DAP sent from the simulated device to the loT Hub is registered in ADT.

[0081] Analytics and visualization-. The UE heads up display (HUD) and user widgets were used together with Azure TSI to produce analytics and visualizations from telemetry. Telemetry flow to the loT Hub devices was routed to TSI for storage and processing into trends using aggregated averages over a 2-hour period at all times. As a result, anytime the user(s) of the system request analytics, the aggregated average over the last 2 h are visualized. Telemetry was further processed into interactive charts that can be used to track the historical condition of the real-world entity. An Azure Web app was deployed to act as a middleware between the UE application and TSI to enable live queries of the data on demand.

[0082] For example, when a sensor widget is clicked in the UE application, an ‘http’ request is made to the Web app to query the required data. The raw telemetry payload in JSON format and the historical charts are then furnished to the UE HUD for visualization. The sensors were individually represented using UE user widgets that were fitted with a dynamic radial material that changes color based on the reported value. FIG. 14A illustrates an example of the visualization of telemetry inside of UE after establishing a live connection through SignaIR and the Azure Web app. The HUD design enabled user interaction with telemetry such as text input in the form of markers and downloading of historical data. Individual sensor widgets could be interacted with as shown in the bottom right of FIG. 14A. At the equipment level, the application was designed to be able to track the time left for replacement of the equipment using the metadata from the Revit file (top right frame of FIG. 14A). Table 1 of FIG. 14B summarizes the rules that were encoded in UE editor to visualize anomalies and to track the Remaining Useful Life (RUL) of the AHU.Evaluation of prototypes

[0083] Subsequent to the deployment of the prototypes, an evaluation was conducted among a broader pool of potential end users and academic researchers. The evaluation focused on two dimensions: (1) the ease of development and usability of the DT system and (2) suitability of the prototype for PdM. The evaluation adopted a human risk and effectiveness strategy within the Framework for Evaluation in Design Science (FEDS). The strategy and theoretical framework adopted was aimed at rigorously evaluating the two prototypes to provide insights into the organizational and technical implications associated with the build or buy approach in relation to the design and implementation of DT-based PdM systems. Three groups of participants were involved in the evaluation. These were academic researchers, industry professionals, and industry-based researchers in industry. Overall, thirty-three individuals participated in the evaluation. The rationale for selecting the groups of participants included.• Academic researchers in the field of digital technologies in the AECO industry were involved because of their familiarity with the concepts of digital innovation and emerging technologies. Furthermore, they are key stakeholders in the development of knowledge related to the application of digital innovation in the industry. As such, their input provides a good measure of the validity of the research outputs. Researchers formed the bulk of the participants as they were more familiar with the concepts and technicalities involved in the research outputs and could provide critical and constructive peer feedback.• Industry professionals, mostly in the FM, BIM, and DT space, were selected because of their knowledge of maintenance management and the current digital support for the practice. Their domain expertise and experience provide valuable inputs into refining the outputs to meet industry needs. Industry professionals were selected from different states and organizations across the US. They included Facility Managers, Facilities Technical Services Executives, Digital Twin Software Company Executives, and Owners' agents.• Industry-based researchers, i.e. , persons working in non-academic institutions but carrying out research on DTs, provide a unique blend of the two perspectives held by the aforementioned groups, helping to bridge gaps between the opinions of the two previous groups.

[0084] To ensure that the prototypes were fit for purpose and could be scaled up within the case study organization and others of similar structure, an evaluation was conducted. The evaluation process involved a detailed demonstration of the two prototypes, the development approach, and the system functionalities. Following the demonstrations, participants were asked to answer questions on the: (1) ease of development and usability of the DT system and (2) suitability of the DT system for PdM. These measures were to help ensure that the prototypes were sufficiently practical to implement, use, and effective for the intended use case. Specific rationales for the two measures are as follows.• Ease of development and usability: the rationale was to examine how user-friendly the prototype was and to also determine how straightforward its implementation is. By examining the prototypes with this measure, the extent to which the process and tools can be adopted without excessive complexity or steep learning curves could be estimated. This is vital for increasing the adoption and implementation of DTs.• Suitability of the DT system for PdM evaluating the prototypes suitability for PdM was aimed to ensure that the system can adequately fulfill the requirements of PdM 1.0 as planned.

[0085] Evaluation results. Of the thirty-three (33) participants, twenty-two (22) were academic researchers, five were industry professionals, and six (6) were industry-based researchers. The researchers had between 1 and 5 years of experience in the area of BIM, construction information modeling, robotics, and DTs. The five industry professionals included three (3) facility management professionals and two (2) DT software experts who had experience working with BAS systems and maintenance management systems with experience ranging from 5 to 30 years. The six industry-based researchers included two (2) DT researchers working in construction companies with over five years’ experience; two (2) BIM / DT program managers for a consulting firm with over 15 years combined experience; one (1) director of campus operations at a state college with a research program on BIM and DTs; and one (1) design professional working on DT research and development for a consulting company.

[0086] Ease of development and usability of system. This dimension of the evaluation related to: (1) the learning curve and extent of technical expertise associated with set up the DT system for the specific use case i.e. , PdM and (2) the learning curve associated with the end user's utilization of the system and its functionalities for PdM. Participants reported their opinions on a five-point Likert scale. The scale spanned from 1 to 5, where 1 indicated "strongly disagree," and five denoted "strongly agree." The majority of respondents (55%) reported they somewhat agreed that the learning curve for setting up and using Prototype 1 would be easy to navigate while about 27% reported strong agreement. Some participants (12%) reported that the system development and utilization will not be easy. To provide some rigor to this evaluation, a counterbalancing measure was included where participants were asked to opine on the statement “I would need a technical person to be able to set up and use this DT system.” Approximately 48% reported that they somewhat agreed that they would need technical support, while 36% strongly agreed. For Prototype 2, about 59% of participants reported that they somewhat agreed they would be able to easily learn how to set up and use the system. Only 19 % indicated a strong agreement with being able to easily learn how to set up and use such a system. About 16% of the participants reported moderate disagreement, suggesting that Prototype 2 would be somewhat difficult to set up and use. Regarding the need for technical assistance, 53% reported moderate agreement while 31% reported strong agreement.

[0087] Suitability of prototype for PdM. In this evaluation, participants were asked to provide opinions on whether or not they found the analytics provided by the system to be capable of facilitating PdM. Over 63% reported strong agreement in favor of Prototype 1 , with about 30% reporting moderate agreement, while about 6% indicated moderate disagreement on the suitability of the analytics in Prototype 1 for PdM. Similarly, 50% strongly agreed thatthe analytics provided by Prototype 2 can facilitate PdM. Moderate agreement on the suitability of the analytics of Prototype 2 for PdM was reported by about 47%.

[0088] Participants’ preferred prototype. To obtain an understanding of the comparative strengths of the two prototypes and by extension the build or buy approach for DT-based PdM system implementation, participants were invited to select between Prototype 1 and Prototype 2, guided by the comprehensive demonstrations. Among the participants, a total of twenty-two individuals, representing 73% of the sample, opted for Prototype 2 as the preferred system for real-world PdM application, while eight individuals (27%) favored Prototype 1. Further analysis revealed that academic researchers and industry-based researchers were more inclined to Prototype 2 while industry professionals preferred Prototype 1. Analysis of individual responses revealed that industry professionals found Prototype 1 to easier to operate compared to Prototype 2. On the other hand, academic researchers and industry-based researchers preferred the flexibility of customization that Prototype 2 afforded.

[0089] The results of the evaluation revealed that there is an overall positive sentiment regarding both prototypes’ potential suitability for PdM, relative ease of development, and usability. However, the majority of participants preferred Prototype 2 over Prototype 1 , with the distinguishing factor being the intuitive nature of the user interface of Prototype 2 and its ability to perform and visualize anomalies in line with PdM 1.0. This implies that DT-based PdM systems must adopt intuitive user interfaces and be fitted with functionalities that reduce the cognitive load of users with respect to anomaly detection. This gives credence to the build approach as it enables the end users to customize functionalities such as type of user interface for specific use cases unlike the buy approach where functionalities are preprogrammed.

[0090] Furthermore, the results highlight the need for the development of technical guidance to aid in the development of custom-built DT systems as well as the setup of prebuilt DT platforms within the AECO industry. While participants reported a moderate to high level of agreement with the two approaches being easy to learn in terms of setup and utilization, there was a markedly high indication of the need for technical assistance in both approaches. As such, it can be inferred that there is a satisfactory level of foundational skill within the AECO industry that can be enriched through technical training to facilitate the development of DTs in either the build or buy approach. However, the choice of preferred prototype suggests that people with research backgrounds and expertise may be more adept at implementing DT solutions in the build approach. The results of the evaluations revealed the multifaceted considerations that must be made when selecting an approach and platform(s) for developing a DT-based PdM system. These considerations include:• Ease of system development or setup, organizations seeking to implement a DT- based PdM system must assess their technical capacity to determine their ability to successfully design and implement a system using either a build or buy approach. For organizations with technical capacity to develop custom software programs, the build approach is more suitable as it affords the creation of context-driven solutions. Such organizations are likely to succeed with an in-house research and development team or external consultants. On the other hand, organizations with limited resources for custom build approaches may adopt the buy approach. The results of the evaluation indicated that the commercial pre-built platform provided a similar level of suitability for PdM compared to custom-built platform.• Ease of system use: organizations must assess the degree of difficulty associated with the end user utilization of systems regardless of the approach or platform chosen. The results showed that approximately 84 % of respondents believed they needed technical assistance to learn how to use both Prototype 1 and Prototype 2. This suggests that there is a significant learning curve for all kinds of users of DT- based PdM systems. However, there is a marginal advantage for systems that utilize intuitive user interaction such as that found in Prototype 2 to be less complex to use.Additional factors for consideration include pricing structure of platform, data and system integration, DT implementation success, visualization capability and quality, time to deployment, security, and scalability and flexibility.

[0091] DTs have the potential to transform the AECO industry. More specifically, DTs can positively impact the management of constructed facilities through enhanced data and system integration. The AECO industry is witnessing an advancement of analytics facilitated by artificial intelligence (Al), the growing utilization of BIM in facilities management (FM), and loT technologies. This has led to a concomitant interest in DTs, especially for operational purposes in FM. Such operational reasons include the need to be proactive and precautionary. DTs a provide real-time view of managed assets by providing critical real-time data, contextual insights, and / or analytics that help identify potential issues, providing a valuable platform for PdM. Developing DT-based PdM systems, however, has not taken off on a large scale in the AECO industry. A notable barrier to adoption and implementation is the low level of knowledge on design and implementation of DT-based PdM systems related to the selection of DT development platform and comparative solutions to use as benchmarks. In view of this, this study has developed two prototypes using two different approaches, i.e., build (custom platform development) or buy (utilizing a pre-built platform) in a real-world context.

[0092] With reference to FIG. 15, shown is a schematic block diagram of the computing environment in accordance with various embodiments of the present disclosure. The computing environment includes one or more computing devices 1500. Each computing device 1500 includes at least one processor circuit, for example, having a processor 1503 and a memory 1506, both of which are coupled to a local interface 1509. To this end, each computing device 1500 may comprise, for example, at least one server computer or like device. The local interface 1509 may comprise, for example, a data bus with an accompanying address / control bus or other bus structure as can be appreciated.

[0093] Stored in the memory 1506 are both data and several components that are executable by the processor 1503. In particular, stored in the memory 1506 and executable by the processor 1503 is a DT-based PdM System 1515, Operating System 1518, and potentially other applications. Also stored in the memory 1506 may be a data store 1512 and other data. In addition, an operating system may be stored in the memory 1506 and executable by the processor 1503.

[0094] It is understood that there may be other applications that are stored in the memory 1506 and are executable by the processor 1503 as can be appreciated. Where any component discussed herein is implemented in the form of software, any one of a number of programming languages may be employed such as, for example, C, C++, C#, Objective C, Java®, JavaScript®, Perl, PHP, Visual Basic®, Python®, Ruby, Flash®, or other programming languages. Additionally, it is understood that terms such as “application,” “service,” “system,” “engine,” “module,” and so on may be interchangeable and are not intended to be limiting.

[0095] A number of software components are stored in the memory 1506 and are executable by the processor 1503. In this respect, the term “executable” means a program file that is in a form that can ultimately be run by the processor 1503. Examples of executable programs may be, for example, a compiled program that can be translated into machine code in a format that can be loaded into a random access portion of the memory 1506 and run by the processor 1503, source code that may be expressed in proper format such as object code that is capable of being loaded into a random access portion of the memory 1506 and executed by the processor 1503, or source code that may be interpreted by another executable program to generate instructions in a random access portion of the memory 1506 to be executed by the processor 1503, etc. An executable program may be stored in any portion or component of the memory 1506 including, for example, random access memory (RAM), read-only memory (ROM), hard drive, solid-state drive, USB flash drive, memory card, optical disc such as compact disc (CD) or digital versatile disc (DVD), floppy disk, magnetic tape, or other memory components.

[0096] The memory 1506 is defined herein as including both volatile and nonvolatile memory and data storage components. Volatile components are those that do not retain data values upon loss of power. Nonvolatile components are those that retain data upon a loss of power. Thus, the memory 1506 may comprise, for example, random access memory (RAM), read-only memory (ROM), hard disk drives, solid-state drives, USB flash drives, memory cards accessed via a memory card reader, floppy disks accessed via an associated floppy disk drive, optical discs accessed via an optical disc drive, magnetic tapes accessed via an appropriate tape drive, and / or other memory components, or a combination of any two or more of these memory components. In addition, the RAM may comprise, for example, static random access memory (SRAM), dynamic random access memory (DRAM), or magnetic random access memory (MRAM) and other such devices. The ROM may comprise, for example, a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable readonly memory (EEPROM), or other like memory device.

[0097] Also, the processor 1503 may represent multiple processors 1503 and / or multiple processor cores and the memory 1506 may represent multiple memories 1506 that operate in parallel processing circuits, respectively. In such a case, the local interface 1509 may be an appropriate network that facilitates communication between any two of the multiple processors 1503, between any processor 1503 and any of the memories 1506, or between any two of the memories 1506, etc. The local interface 1509 may comprise additional systems designed to coordinate this communication, including, for example, performing load balancing. The processor 1503 may be of electrical or of some other available construction.

[0098] Although the DT-based PdM System 1515, the Operating System 1518, and other various systems described herein may be embodied in software or code executed by general purpose hardware as discussed above, as an alternative the same may also be embodied in dedicated hardware or a combination of software / general purpose hardware and dedicated hardware. If embodied in dedicated hardware, each can be implemented as a circuit or state machine that employs any one of or a combination of a number of technologies. These technologies may include, but are not limited to, discrete logic circuits having logic gates for implementing various logic functions upon an application of one or more data signals, application specific integrated circuits (ASICs) having appropriate logic gates, field-programmable gate arrays (FPGAs), or other components, etc. Such technologies are generally well known by those skilled in the art and, consequently, are not described in detail herein.

[0099] The flow diagrams of FIGS. 1-3, 5 and 8 show examples of the functionality and operation of implementations of components described herein. The components describedherein can be embodied in hardware, software, or a combination of hardware and software. If embodied in software, each element can represent a module of code or a portion of code that includes program instructions to implement the specified logical function(s). The program instructions can be embodied in the form of, for example, source code that includes human-readable statements written in a programming language or machine code that includes machine instructions recognizable by a suitable execution system, such as a processor in a computer system or other system. If embodied in hardware, each element can represent a circuit or a number of interconnected circuits that implement the specified logical function(s).

[0100] Although the flowcharts and sequence diagram show a specific order of execution, it is understood that the order of execution can differ from that which is shown. For example, the order of execution of two or more elements can be switched relative to the order shown. Also, two or more elements shown in succession can be executed concurrently or with partial concurrence. Further, in some examples, one or more of the elements shown in the flowcharts can be skipped or omitted.

[0101] Also, one or more or more of the components described herein that include software or program instructions can be embodied in any non-transitory computer-readable medium for use by or in connection with an instruction execution system such as, a processor in a computer system or other system. The computer-readable medium can contain, store, and / or maintain the software or program instructions for use by or in connection with the instruction execution system.

[0102] A computer-readable medium can include a physical media, such as, magnetic, optical, semiconductor, and / or other suitable media. Examples of a suitable computer- readable media include, but are not limited to, solid-state drives, magnetic drives, or flash memory. Further, any logic or component described herein can be implemented and structured in a variety of ways. For example, one or more components described can be implemented as modules or components of a single application. Further, one or more components described herein can be executed in one computing device or by using multiple computing devices.

[0103] It should be emphasized that the above-described embodiments are merely possible examples of implementations, merely set forth for a clear understanding of the principles of the present disclosure. Many variations and modifications may be made to the above-described embodiment(s) without departing substantially from the principles of the present disclosure. All such modifications and variations are intended to be included herein within the scope of this disclosure.

[0104] Where a range of values is provided, it is understood that each intervening value, to the tenth of the unit of the lower limit unless the context clearly dictates otherwise, between the upper and lower limit of that range and any other stated or intervening value in that stated range, is encompassed within the disclosure. The upper and lower limits of these smaller ranges may independently be included in the smaller ranges and are also encompassed within the disclosure, subject to any specifically excluded limit in the stated range. Where the stated range includes one or both of the limits, ranges excluding either or both of those included limits are also included in the disclosure.

[0105] It is also understood that this disclosure is not limited to the specific devices, methods and conditions or parameters described and / or shown herein, and that the terminology used herein is for the purpose of describing particular embodiments by way of example only and is not intended to be limiting of the claimed disclosure. Also, as used in the specification including the appended claims, the singular forms “a,” “an,” and “the” include the plural, and reference to a particular numerical value indicates at least that particular value, unless the context clearly dictates otherwise. Ranges may be expressed herein as from “about” or “approximately” one particular value and / or to “about” or “approximately” another particular value. When such a range is expressed, another embodiment includes one particular value and / or to the other particular value. Similarly, when values are expressed as approximations, by use of the antecedent “about,” it will be understood that the particular value forms another embodiment.

[0106] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs.

Claims

CLAIMSTherefore, at least the following is claimed:

1. A system, comprising: at least one computing device; and at least one memory comprising instructions that when executed cause the at least one computing device to at least: provide a virtual environment that includes a plurality of virtualized digital sensing device models corresponding to a plurality of physical sensing devices, and a plurality of digital building component models corresponding to a plurality of physical building components; generate a Building Information Model (BIM)-based digital twin that provides a virtual representation of a physical building based at least in part on the plurality of virtualized digital sensing device models and the plurality of digital building component models; receive sensor data generated by at least one of the plurality of physical sensing devices, at least one of the plurality of physical building components, or any combination thereof; identify at least one fault based at least in part on the sensor data; perform a criticality analysis to identify at least one criticality score for the at least one fault; and generate a user interface comprising the virtual representation of the physical building and showing at least a subset of the at least one fault based at least in part on the at least one criticality score.

2. The system of claim 1 , wherein at least one fault comprises a detected fault that is detected based at least in part on the sensor data and a threshold value.

3. The system of any of claims 1 or 2, wherein at least one fault comprises a diagnosed fault that is diagnosed based at least in part on the detected fault and a fault graph model that relates the detected fault to the diagnosed fault.

4. The system of claim 3, wherein the fault graph model indicates at least one probability in association with at least one edge connecting the diagnosed fault to the detected fault, and the diagnosed fault is a highest-probability fault among a plurality of faults related to the detected fault through the fault graph model.

5. The system of any of claims 1-4, wherein the instructions, when executed, further cause the at least one computing device to at least: generate, based at least in part on a machine learning model, a predicted fault based at least in part on a detected fault, a diagnosed fault, or any combination thereof, wherein the at least one fault comprises the detected fault, the diagnosed fault, and the predicted fault.

6. The system of any of claims 1-5, wherein the instructions, when executed, further cause the at least one computing device to at least initiate a response to the at least one fault based at least in part upon the at least one criticality score.

7. The system of claim 6, wherein the response comprises initiating repair or maintenance of a building component.

8. A method, comprising: providing, by at least one computing device comprising processing circuitry, a virtual environment that includes a plurality of virtualized digital sensing device models corresponding to a plurality of physical sensing devices, and a plurality of digital building component models corresponding to a plurality of physical building components; generating, by the at least one computing device, a Building Information Model (BIM)-based digital twin that provides a virtual representation of a physical building based at least in part on the plurality of virtualized digital sensing device models and the plurality of digital building component models; receiving, by the at least one computing device, sensor data generated by at least one of the plurality of physical sensing devices, at least one of the plurality of physical building components, or any combination thereof; identifying, by the at least one computing device, at least one fault based at least in part on the sensor data; performing, by the at least one computing device, a criticality analysis to identify at least one criticality score for the at least one fault; and generating, by the at least one computing device, a user interface comprising the virtual representation of the physical building and showing at least a subset of the at least one fault based at least in part on the at least one criticality score.

9. The method of claim 8, wherein at least one fault comprises a detected fault that is detected based at least in part on the sensor data and a threshold value.

10. The method of any of claims 8 or 9, wherein at least one fault comprises a diagnosed fault that is diagnosed based at least in part on the detected fault and a fault graph model that relates the detected fault to the diagnosed fault.11 . The method of claim 10, wherein the fault graph model indicates at least one probability in association with at least one edge connecting the diagnosed fault to the detected fault, and the diagnosed fault is a highest-probability fault among a plurality of faults related to the detected fault through the fault graph model.

12. The method of any of claims 8-11 , comprising generating, by the at least one computing device, based at least in part on a machine learning model, a predicted fault based at least in part on a detected fault, a diagnosed fault, or any combination thereof, wherein the at least one fault comprises the detected fault, the diagnosed fault, and the predicted fault.

13. The method of any of claims 8-12, comprising initiating, by the at least one computing device, a response to the at least one fault based at least in part upon the at least one criticality score.

14. The method of claim 13, wherein the response comprises initiating repair or maintenance of a building component.

Images