Server, system and method for industrial meta universe

Abstract

In some embodiments, a system includes a virtual reality environment that includes a digital twin of at least a portion of a physical industrial environment. In some embodiments, the system is configured to change the virtual reality environment based on a change in the physical industrial environment. In some embodiments, as a non-limiting example, the system receives the change through a sensor, such as a temperature and / or pressure sensor. In some embodiments, changes in the physical environment are detected by analysis of the images. In some embodiments, the system is configured to detect the change using artificial intelligence. In some embodiments, the system is configured to predict an effect of a change in the virtual and / or physical environment on other system components through a variable change in the virtual model. In some embodiments, the system is configured to display the effect in a virtual environment prior to achieving the effect in a physical environment.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Provisional Application No. 63 / 343,884, filed on May 19, 2022, which is incorporated herein by reference in its entirety for all purposes. Background Art

[0003] Industrial organizations often have many systems that are sometimes located far apart. Currently, problems with a specific sensor or device can often only be viewed and analyzed by an operator or technician at a specific location. While three-dimensional (3D) models can provide additional insight, they still often fail to provide many of the insights that can be obtained through physical or appropriate remote analysis.

[0004] Customers need asynchronous access to multiple systems and software products and services to access their digital representations of their physical assets. Additionally, the collaboration environment is often limited to VoIP technology and screen sharing. Collaboration is sequential, error-prone, and has limited scope for accessing large amounts of data across many assets.

[0005] Thus, there is a need in the art for an industrial environment with at least some immersive capabilities (hereinafter referred to as an "industrial metaverse") that enables at least one person to be fully immersed in a realistic industrial environment and provides collaboration capabilities in a fully immersive, partially immersive, or non-immersive manner. Summary of the invention

[0006] The systems and methods described herein are directed to an industrial metaverse (hereinafter referred to as the "system") comprising one or more virtual environments. In some embodiments, the one or more virtual environments comprise a virtual representation of an industrial environment. In some embodiments, the industrial environment comprises one or more components. In some embodiments, the one or more components comprise one or more software and / or hardware components.

[0007] In some embodiments, the software component includes programming instructions for performing one or more computer-implemented steps. In some embodiments, the hardware component includes one or more computers including one or more non-transitory computer-readable media for storing the programming instructions, and one or more processors for executing the programming instructions and implementing the computer code steps.

[0008] In some embodiments, the one or more computers include a virtual display. In some embodiments, the virtual display is configured to be coupled to a user's head, wherein at least a portion of the virtual display is configured to move with the user and / or remain in the user's field of view when the user moves his or her head. In some embodiments, the virtual display is configured to enable a perspective view of a virtual environment to change while the virtual display remains in the user's field of view.

[0009] In some embodiments, the virtual display is configured to prevent external light from entering the user's field of view. In some embodiments, the virtual display is configured to enable some or all light to enter the user's field of view. In some embodiments, the visual display is configured to display at least a portion of the adjacent environment (e.g., the physical room in which the user is located) in the user's field of view. In some embodiments, the visual display is configured to enable the user to see the adjacent environment through at least a portion of the visual display.

[0010] In some embodiments, the first visual display is configured to enable the user to see at least a portion of the content being displayed and / or displayed in the second visual display. In some embodiments, the virtual display includes a virtual reality (VR) headset (e.g., Meta 2 or other commercially available head-mounted displays). In some embodiments, as non-limiting examples, the virtual display includes a commercially available augmented reality head-mounted display and / or augmented reality glasses. In some embodiments, the virtual display includes one or more of a screen (e.g., a computer screen), a projector, a cell phone, a tablet, an LED display, an OLED display, a joystick, a controller, a power source, and / or any conventional display and / or associated control device configured to enable a user to view and / or manipulate digital images.

[0011] In some embodiments, the software includes one or more rendering software that generates a virtual (e.g., 3D) environment when executed. In some embodiments, the software includes one or more analysis software configured to provide an analysis display of raw data (e.g., time series) data. In some embodiments, the analysis display includes analysis items such as one or more charts, graphs, images, videos, statistics, and / or any conventional analysis used in an industrial environment. In some embodiments, the software includes programming instructions configured to display the analysis display in a virtual environment. In some embodiments, the programming instructions are configured to enable a user to perform manipulation of one or more analysis displays in a virtual environment.

[0012] In some embodiments, the analysis display includes a virtual screen. In some embodiments, the virtual screen is configured to display one or more analyses and / or analysis results. In some embodiments, the manipulation includes moving the virtual screen from one location within the 3D environment to another location. In some embodiments, the manipulation includes selection of one or more analysis items within the analysis display.

[0013] In some embodiments, the hardware includes one or more physical components. In some embodiments, non-limiting examples of physical components include one or more pumps, sensors, reactors, pipes, valves, heaters, stairs, tanks, vehicles, drones, lights, scaffolding, floors, ceilings, the earth, water, the sky, the sun, the moon, the stars, and / or any conventional physical structure found in a real (also referred to as "real life") environment. In some embodiments, the software includes programming instructions configured to generate one or more virtual components. In some embodiments, the one or more virtual components include a virtual rendering of one or more physical components.

[0014] In some embodiments, the present disclosure relates to a system for generating a virtual environment. In some embodiments, the system includes one or more computers, the computer including one or more processors and one or more non-transitory computer-readable media. In some embodiments, the one or more non-transitory computer-readable media include program instructions stored thereon, and when the program instructions are executed, the one or more computers implement one or more computer steps described herein.

[0015] In some embodiments, the steps include generating, by the one or more processors, a virtual environment comprising a three-dimensional (3D) representation of at least a portion of a physical industrial environment. In some embodiments, the steps include generating, by the one or more processors, an asset link between one or more virtual assets in the virtual environment and one or more physical assets in the physical industrial environment. In some embodiments, the steps include generating, by the one or more processors, a data link between the virtual environment and a historical database, the historical database containing asset data about the one or more physical assets. In some embodiments, the steps include generating, by the one or more processors, a virtual graphical user interface (GUI) in the virtual environment, the virtual GUI being configured to enable a user to access and display the asset data in the virtual environment. In some embodiments, the steps include generating, by the one or more processors, one or more user controls in the virtual environment, the one or more user controls being configured to navigate the virtual environment, select the one or more virtual assets, and / or control the virtual GUI.

[0016] In some embodiments, the display of the virtual environment in the user's field of view is configured to change based on a change in the position of the user's head. In some embodiments, the asset data includes at least one analysis result of raw data associated with the one or more virtual assets. In some embodiments, selecting the one or more virtual assets includes manipulation of a virtual asset actuator. In some embodiments, manipulation of the virtual asset actuator results in remote manipulation of a corresponding physical asset actuator. In some embodiments, the physical asset actuator includes one or more of a joystick, a valve, a switch, and a computer setting.

[0017] In some embodiments, the one or more non-transitory computer-readable media further include program instructions stored thereon, which, when executed, cause the one or more computers to generate, via the one or more processors, one or more virtual models of the one or more physical assets. In some embodiments, each of the one or more virtual models includes one or more mathematical equations configured to represent the physical behavior of the one or more physical assets. In some embodiments, selecting the one or more virtual assets includes virtually manipulating the virtual asset controller.

[0018] In some embodiments, the virtual manipulation causes a change in one or more variables of the one or more virtual models. In some embodiments, the effect of the change in the one or more variables is displayed in the virtual environment. In some embodiments, the effect is a virtual effect. In some embodiments, the virtual effect includes a theoretical result from a calculation performed by the one or more virtual models. In some embodiments, the virtual effect does not include a change to the one or more physical assets. In some embodiments, the one or more non-transitory computer-readable media also include program instructions stored thereon, which, when executed, cause the one or more computers to execute a user's command through the one or more processors for physically manipulating one or more physical assets corresponding to the virtual manipulation after the virtual effect occurs in the virtual environment.

[0019] In some embodiments, the one or more non-transitory computer-readable media further include program instructions stored thereon, which, when executed, cause the one or more computers to receive, via the one or more processors, one or more images of the one or more physical assets. In some embodiments, the steps include performing, via the one or more processors, a comparison of the one or more images with one or more previous images of the one or more physical assets. In some embodiments, the steps include performing, via the one or more processors, an identification of visual changes in the one or more images based on the comparison. In some embodiments, the steps include altering, via the one or more processors, the one or more virtual assets in the virtual environment to reflect the visual changes in the one or more images. In some embodiments, the comparison includes the execution of artificial intelligence. In some embodiments, at least a portion of the one or more previous images is used to train the artificial intelligence. In some embodiments, the one or more non-transitory computer-readable media further include program instructions stored thereon, which, when executed, cause the one or more computers to predict, via the one or more processors, a change in a variable in one or more virtual models based on the visual changes.

[0020] In some embodiments, the selection of the one or more virtual assets in the virtual environment is configured to cause a display of a time series graph to be generated in the virtual environment. In some embodiments, the one or more virtual assets are connected to the one or more physical assets via a software as a service (SaaS) subscription program. In some embodiments, the physical industrial environment includes a supervisory control and data acquisition (SCADA) platform configured to monitor and / or control the one or more physical assets. In some embodiments, at least a portion of the SCADA can be accessed via the virtual environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 A high-level schematic diagram of a system showing a portion of several features and techniques in accordance with some embodiments is shown.

[0022] Figure 2 Illustrated are three non-limiting examples of platforms that may be integrated into a system in accordance with some embodiments.

[0023] Figure 3 Illustrate integration of one or more system modules for process simulation, AVEVA PI and AVEVA Data Hub according to some embodiments.

[0024] Figure 4The diagram illustrates one or more system module integrations for AVEVA Insight, third party cameras, AVEVA VISION AI, and a Human Machine Interface Supervisory Control and Data Acquisition (HMI SCADA) system in accordance with some embodiments.

[0025] Figure 5 Depicts the integration of one or more system modules from the AVEVA XR Toolkit in accordance with some embodiments.

[0026] Figure 6 An initial startup of a system in accordance with some embodiments is shown.In some embodiments, the visual display is configured to enable a user to see at least a portion of their immediate environment on the display.

[0027] Figure 7 A user is shown crossing a border in accordance with some embodiments.

[0028] Figure 8 Illustrated is a transition view in which at least a portion of both a proximal environment and a virtual environment are displayed simultaneously, in accordance with some embodiments.

[0029] Fig. 9 A user is shown fully immersed in a virtual portal environment in accordance with some embodiments.

[0030] Fig.10 Shown is a teleportation feature enabled by the system in accordance with some embodiments.

[0031] Fig.11 The diagram illustrates a view in a virtual environment where a user is in close proximity to an industrial environment overview.

[0032] Fig.12 A user is shown passing through a portal and the user's virtual controllers, which are virtual representations of two physical controllers in the real world, in accordance with some embodiments.

[0033] Fig.13 Depicted is a user in a first virtual industrial environment within a virtual environment after passing through a first portal, according to some embodiments.

[0034] Fig.14 The diagram illustrates a user selecting a first component (Pump MT01A) according to some embodiments.

[0035] Fig.15 Shown is a full highlight of a selected component and a partial highlight of a second identified component in accordance with some embodiments.

[0036] Fig.16Depicted is an analysis display in a first virtual industrial environment in accordance with some embodiments.

[0037] Fig.17 A user is shown teleporting to a second virtual industrial environment portal in accordance with some embodiments.

[0038] Fig.18 Depicted is an analysis window in the form of an industrial environment summary adjacent to a second portal in accordance with some embodiments.

[0039] Fig.19 Illustrate a second virtual industrial environment generated by the system after a user passes through a second portal, according to some embodiments.

[0040] Fig. 20 The position of the sun at a certain moment in time is shown in accordance with some embodiments.

[0041] Fig.21 A time series graph of selected thermocouples in a second virtual industrial environment is shown in accordance with some embodiments.

[0042] Fig. 22 Depicts a system-generated controller menu overlaid on a virtual controller in accordance with some embodiments.

[0043] Fig.23 A user is shown selecting a controller menu using another virtual controller in accordance with some embodiments.

[0044] Fig.24 Illustrate a menu display generated by the system in response to selecting a controller menu in accordance with some embodiments.

[0045] Fig.25 Illustrate an analysis display including a portion of a data center including AVEVA Connect as a non-limiting example of accessing a public cloud platform in accordance with some embodiments.

[0046] Fig.26 Filtering by "bad" components in a virtual window (eg, an AssetExplorer window) in an analysis display in accordance with some embodiments.

[0047] Fig. 27 Illustrating the ability to select components from a second virtual industrial environment while in a first industrial environment, in accordance with some embodiments.

[0048] Fig.28 Depicts how the system may be configured to enable a user to select a state analysis window within a virtual window, according to some embodiments.

[0049] Fig.29Shows how the system may be configured to enable a user to select a time range within a virtual window, according to some embodiments.

[0050] Fig.30 Depicting a user selecting a high temperature portion of a graph in accordance with some embodiments.

[0051] Fig.31 A user is shown selecting a thermocouple assembly in a virtual industrial environment in accordance with some embodiments.

[0052] Fig.32 Illustrate a launch data center window in accordance with some embodiments.

[0053] Fig.33 A non-limiting example of a time series graph of thermocouples 3 displayed in the same portion of a previously used data center is shown in accordance with some embodiments.

[0054] Fig.34 A user is shown leaving a virtual industrial environment through a portal in accordance with some embodiments.

[0055] Fig.35 The diagram illustrates a user in front of a third portal leading to a third virtual environment in accordance with some embodiments.

[0056] Fig.36 Illustrated is a virtual CAD environment in the form of a ship's engine room, in accordance with some embodiments.

[0057] Fig.37 Illustrate a user launching an analysis window in accordance with some embodiments.

[0058] Fig.38 The user is shown selecting a component in front of him in the analysis display.

[0059] Fig.39 The assembly is shown in a vertically elevated position from below a base in accordance with some embodiments.

[0060] Fig.40 The assembly is shown being lowered to a base by a user in accordance with some embodiments.

[0061] Fig.41 Illustration of selection of a component and an analysis window displaying a 2D schematic diagram of the component in accordance with some embodiments.

[0062] Fig.42 Depicted is a user navigating out of a ship's engine room toward a CAD overview perspective using fly mode, in accordance with some embodiments.

[0063] Fig.43 A user is shown selecting a "Jump To" option on a controller menu in accordance with some embodiments.

[0064] Fig.44 Illustrating user selection of a jump location in accordance with some embodiments.

[0065] Fig.45 Depicts a user teleporting to a jump location by jumping in accordance with some embodiments.

[0066] Fig.46 An overview of the concept of an XR module is shown in accordance with some embodiments.

[0067] Fig.47 The diagram illustrates system modules and connections between the modules that form at least a portion of a system according to some embodiments.

[0068] Fig.48 According to some embodiments Fig.47 First zoomed in view of .

[0069] Fig.49 According to some embodiments Fig.47 A second enlarged view of .

[0070] Fig.50 is a third enlarged view according to some embodiments.

[0071] Fig.51 According to some embodiments Fig.47 Fourth enlarged view of .

[0072] Fig.52 Illustrate additional system modules connected to a manager module in accordance with some embodiments.

[0073] Fig.53 is a diagram illustrating the concept of glue codes according to some embodiments.

[0074] Fig.54 Included is example glue code according to some embodiments.

[0075] Fig.55 The diagram illustrates a computer system 1010 capable of implementing or including the described systems and methods, according to some embodiments of the system. DETAILED DESCRIPTION

[0076] The following non-limiting examples are intended to help those of ordinary skill understand how to make and use various combinations of the systems and methods presented herein in accordance with some embodiments. It should be understood that each example is part of an overall system, and that some embodiments presented herein can be easily combined with some other embodiments. Thus, any portion and / or combination of portions of the description of the system in accordance with some embodiments presented in this disclosure can be used to define the boundaries for which protection is sought.

[0077] Figure 1 A high-level schematic diagram of a system that represents a portion of multiple features and techniques of the system in accordance with some embodiments is shown. In some embodiments, the system includes one or more virtual displays, which in some embodiments include one or more VR head displays, tablets, laptops, and / or personal computers. In some embodiments, the virtual display is configured to interface with users and / or data. In some embodiments, the system is configured to generate a multi-user environment for collaboration. In some embodiments, the system is configured to generate a first perspective for a first user in a multi-user environment and a second perspective for a second user in the same multi-user environment. In some embodiments, the system includes an XR (meta) module, a common visualization platform (CVP), a knowledge database, engineering information, operational information, and one or more of 1D, 2D, 3D, structured, and streaming data.

[0078] In some embodiments, the system includes multiple features that, when combined, provide a virtual environment for multiple users to collaborate on engineering and operational information that is independent of their interface hardware. In some embodiments, a knowledge graph system is used to combine information from simulations, 3D computer-aided drafting (CAD), engineering data, and operational sensor data and generate a visualization technology layer that presents this information in a virtual (e.g., 3D) environment for real-time interaction.

[0079] In some embodiments, the system is configured to generate a virtual environment to safely interact with colleagues and partners (i.e., users). In some embodiments, the system is configured to enable users to access link information connected to objects in the virtual environment. In some embodiments, the system is configured to display real-time operation information (e.g., component information) in a 3D (virtual) or other environment. In some embodiments, the system is configured to enable multiple device types to be connected, thereby allowing a variety of rich interactions, including viewing different perspectives and / or viewing the same perspective, and the manipulation of items in the virtual environment. In some embodiments, the system includes real-time collaboration, multi-hardware device support, or semantic organization of information in a knowledge graph.

[0080] In some embodiments, the system includes a miniature and independent module component that can be configured and can create a customized application. In some embodiments, the module component includes an XR metamodule. In some embodiments, the XR metamodule includes a "meta" script (i.e., glue code). In some embodiments, the XR metamodule can be used as a single independent and customizable component. In some embodiments, one or more XR metamodules include pre-configured variables. In some embodiments, each variable is calibrated for one or more conditions. In some embodiments, one or more hardware components (e.g., desktop, touch, VR, HoloLens, mobile, etc.) are tested and calibrated to provide the best user experience.

[0081] In some embodiments, the XR meta-modules are application independent and can be easily maintained and customized for the end user through a low-code or no-code approach while creating the desired data exchange interactions. In some embodiments, these modules are ready-to-use with minimal configuration effort, but many customizations are possible. In this way, according to some embodiments, the system creates a common standard for all AVEVA products that easily meets the requirements of most or all users. Figure 2 Illustrated are three non-limiting examples of platforms that may be integrated into a system in accordance with some embodiments.

[0082] In some embodiments, the system is configured to generate and / or display a process simulation. In some embodiments, the process simulation includes a first principles based model. In some embodiments, as a non-limiting example, the first principles based model is integrated with analysis software such as AVEVA PI software and / or AVEVA Data Center. Figure 3 Illustrate integration of one or more system modules for process simulation, AVEVA PI and AVEVA Data Center according to some embodiments.

[0083] In some embodiments, the system includes, as a non-limiting example, artificial intelligence (AI) software, such as AVEVA Vision AI. In some embodiments, the system includes one or more AI algorithms. In some embodiments, the system is configured to integrate with a secure, managed Software as a Service (SaaS) solution for collecting, storing, and visualizing industrial data to enable faster, smarter business decisions. Non-limiting examples of SaaS include AVEVA Insights, which integrates disparate data to provide a comprehensive understanding of how components are performing and enables users across the organization to access the data from anywhere. Figure 4 The diagram illustrates one or more system module integrations for AVEVA Insight, third party cameras, AVEVA Vision AI, and a Human Machine Interface Supervisory Control and Data Acquisition (HMI SCADA) system in accordance with some embodiments.

[0084] In some embodiments, the system is integrated with one or more extended reality systems and is configured to enable augmented reality (AR), virtual reality (VR), and mixed reality (MR) environments. In some embodiments, non-limiting examples of extended reality systems include AVEVA XR Toolbox. Figure 5 Depicts the integration of one or more system modules from the AVEVA XR Toolbox in accordance with some embodiments.

[0085] Figure 6 An initial startup of a system according to some embodiments is shown. In some embodiments, the visual display is configured to enable a user to see at least a portion of their immediate environment on the display. In some embodiments, the system is configured to enable a user to define the boundaries of a virtual environment within the displayed immediate environment. Such boundaries can help ensure user safety.

[0086] Figure 7 A user is shown crossing a border in accordance with some embodiments. Figure 8 Illustrated is a transition view in which at least a portion of both a proximal environment and a virtual environment are displayed simultaneously, in accordance with some embodiments.

[0087] In some embodiments, the virtual environment includes a virtual portal environment. Fig. 9 A user in accordance with some embodiments is shown fully immersed in a virtual portal environment. In some embodiments, the virtual portal environment includes one or more portals configured to transport the user to a virtual industrial environment. In some embodiments, the one or more portals are arranged according to their relative positions on a real-world map. In some embodiments, the system is configured to enable a user to customize the arrangement of one or more portals within the virtual portal environment. In some embodiments, the system is configured to display one or more industrial environment summaries. In some embodiments, the system is configured to display one or more analytical displays that display one or more industrial environment summaries adjacent to one or more portals. In some embodiments, the system is configured to enable a user to customize one or more industrial environment summaries. In some embodiments, the one or more industrial environment summaries include one or more analyses using one or more integrated systems described herein.

[0088] Fig.10A teleportation feature enabled by the system in accordance with some embodiments is shown. In some embodiments, the system is configured to enable a user to traverse a virtual environment by physically traversing an adjacent environment. In some embodiments, the system is configured to enable a user to traverse a virtual environment by selecting a remote location, wherein upon confirmation, the user is teleported (defined as being automatically or otherwise transported across space and / or distance) to the remote location in the virtual environment. In some embodiments, the remote location includes a portal. In some embodiments, the system is configured to enable a user to teleport to a remote location by selecting a location on a menu.

[0089] Fig.11 The diagram illustrates a view in a virtual environment where a user is in close proximity to an industrial environment summary. In some embodiments, the system is configured to zoom one or more industrial environment summaries as the user approaches and / or moves away. In some embodiments, the system is configured to change the perspective of one or more industrial environment summaries to correspond to a real-world view and / or a change in the real-world perspective.

[0090] Fig.12 A user is shown walking through a portal, according to some embodiments, and the user's virtual controllers are also shown, which are virtual representations of two physical controllers in the real world. Fig.13 Depicts a user in a first virtual industrial environment within a virtual environment after passing through a first portal, in accordance with some embodiments. In some embodiments, the virtual industrial environment includes a digital twin of one or more components in a corresponding physical industrial environment. In some embodiments, the system is configured to associate data collected from the one or more components by the one or more integrated systems and / or analyzed by the one or more integrated systems with each of the one or more industrial components, and display the associated and / or analyzed data to the user via an analytical display. In this non-limiting example, in accordance with some embodiments, the one or more components include two pumps.

[0091] Fig.14 The diagram illustrates a user selecting a first component (Pump MT01A) in accordance with some embodiments. In some embodiments, in accordance with some embodiments, the system is configured to enable a user to select one or more components using one or more virtual controllers. In some embodiments, the system is configured to highlight one or more selected components. Fig.15 A fully highlighted display of a selected component and a partially highlighted display of a second identified component in accordance with some embodiments are shown. In some embodiments, the system is configured to enable a user to generate one or more analysis displays within the virtual environment. In some embodiments, the system is configured to enable a user to place an analysis display anywhere in the virtual environment. In some embodiments, the system is configured to enable a user to generate and / or remove an analysis display based on selection of an identified component and / or a selected component. Fig.16Depicted is an analysis display in a first virtual industrial environment in accordance with some embodiments.

[0092] Fig.17 A user is shown teleporting to a second virtual industrial environment portal in accordance with some embodiments. In some embodiments, the system is configured to alert the user when at least a portion of the virtual display is outside the boundary. In some embodiments, in accordance with some embodiments, the alert includes generation of a boundary grid and / or a transition to a neighboring environment view. Fig.18 Depicted is an analysis window in the form of an industrial environment summary adjacent to a second portal in accordance with some embodiments.

[0093] Fig.19 The diagram illustrates a second virtual industrial environment generated by the system after a user passes through a second portal in accordance with some embodiments. In some embodiments, the system includes a camera configured to capture one or more images from the physical industrial environment. In some embodiments, the system is configured to determine one or more component dimensions from the one or more captured images. In some embodiments, the system is configured to generate the virtual industrial environment from the one or more images. In some embodiments, the camera includes a scanner. In accordance with some embodiments, Fig.19 The second virtual industrial environment shown in the diagram is generated via images captured by a laser scanner.

[0094] In some embodiments, the system is configured to generate real-world environmental conditions in the virtual environment. According to some embodiments, some non-limiting examples of real-world conditions include rain, clouds, snow, and sun position. In some embodiments, the system is configured to display environmental conditions on one or more analysis displays.

[0095] In some embodiments, the system is configured to continuously and / or intermittently record one or more aspects of the physical industrial environment. In some embodiments, according to some embodiments, the system is configured to store the records in one or more databases such as a historical database. In some embodiments, the system is configured to store only changes in the physical industrial environment. In some embodiments, the system is configured to use AI (e.g., AVEVA Vision AI) to determine changes in the physical industrial environment. In some embodiments, by storing only changes in the physical industrial environment, valuable computer storage resources are saved.

[0096] In some embodiments, the system is configured to integrate changes in the physical environment into the virtual environment as components, images, and / or videos. In some embodiments, the system is configured to combine changes in the environment with static components that do not change in response to a user's query and / or request. In some embodiments, the query is for a specific time range. In some embodiments, the time range is in the past. Advantageously, in some embodiments, by recreating the physical industrial environment during a time period of interest in the virtual environment, the system is configured to enable the user to "travel through time" and witness past environmental factors integrated into the virtual environment by one or more systems described herein from a current perspective. Fig. 20 The position of the sun at a certain moment in time is shown in accordance with some embodiments.

[0097] As a non-limiting example, in some embodiments, changes in the environment may include a person manipulating a component. In some embodiments, the person may not have reported the manipulation, but the system enables the user to witness their past presence from a current perspective. In some embodiments, the system is configured to enable the person to fast forward, rewind, play, and pause virtual time while in the virtual environment. In some embodiments, this enables the user to witness multiple physical industrial environment changes in a short real time. In some embodiments, the system is configured to associate a time series graph in the analysis display with the changes generated in the virtual environment. Fig.21 A time series graph of selected thermocouples in a second virtual industrial environment is shown in accordance with some embodiments.

[0098] Fig. 22 Depicts a system-generated controller menu overlaid on a virtual controller in accordance with some embodiments. Fig.23 Shown according to some embodiments, the user selects the controller menu using another virtual controller. In some embodiments, by overlaying the controller menu on the virtual controller, the user can access the menu simply by looking at the virtual controller.

[0099] Fig.24 Illustrate a menu display generated by a system in response to selecting a controller menu in accordance with some embodiments.In some embodiments, the controller menu includes one or more navigation options and / or one or more analysis options. Fig.25 The diagram illustrates an analysis display according to some embodiments, which includes a portion of an AVEVA data center, the AVEVA data center including AVEVA Connect as a non-limiting example of accessing a public cloud platform that provides a central location for secure access to various integrated systems in a virtual environment. Fig.26 Filtering by "bad" components in a virtual window (eg, an asset browser window) in an analysis display in accordance with some embodiments is shown.

[0100] In some embodiments, the system is configured to enable a user to access components from a second virtual industrial environment while in a first industrial environment. In some embodiments, this saves time and computer resources because the user does not have to exit the first virtual environment to access items in the second virtual environment. Fig. 27 The diagram illustrates the ability to select a component from a second virtual industrial environment while in a first industrial environment in accordance with some embodiments. In some embodiments, the system is configured to generate multiple analysis displays. In some embodiments, the system is configured to enable a user to display different component analyses from different virtual industrial environments while in any virtual environment in accordance with some embodiments.

[0101] Fig.28 Depicts how the system may be configured to enable a user to select a state analysis window within a virtual window, according to some embodiments. Fig.29 Shows how, according to some embodiments, the system is configured to enable a user to select a time range within a virtual window. In some embodiments, according to some embodiments, the system is configured to enable a user to select one or more analysis elements within a virtual window. Fig.30 Depicting a user selecting a high temperature portion of a graph in accordance with some embodiments. Fig.31 A user is shown selecting a thermocouple assembly in a virtual industrial environment in accordance with some embodiments. In some embodiments, the system is configured to display data from the newly selected assembly in the same analysis display that displayed the analysis of the previously selected assembly. In some embodiments, the system is configured to display data from the newly selected assembly in a new and / or different analysis display.

[0102] In some embodiments, the system is configured to enable a user to automatically display an integrated system in an existing or previous analysis display in a new analysis display. Fig.32 Shown is a launch AVEVA Data Center window in accordance with some embodiments. Fig.33 A non-limiting example of a time series graph of thermocouples 3 displayed in the same section of a previously used AVEVA data center is shown in accordance with some embodiments. Fig.34 A user is shown leaving a virtual industrial environment through a portal in accordance with some embodiments.

[0103] Fig.35 The diagram illustrates a user in front of a third portal leading to a third virtual environment in accordance with some embodiments. In some embodiments, the system is configured to enable the user to enter a virtual computer-aided drafting ("CAD") environment. In some embodiments, the virtual CAD environment includes a virtual representation (1D, 2D, 3D) of a computer-aided drafting system display. In some embodiments, the virtual CAD environment includes components and / or structures that do not yet exist in the physical world. Fig.36Illustrated is a virtual CAD environment in the form of a ship's engine room, in accordance with some embodiments.

[0104] In some embodiments, the system is configured to record changes in the virtual environment. In some embodiments, the system is configured to record changes in the virtual CAD environment. In some embodiments, according to some embodiments, the system is configured to store the records in one or more databases such as a historical database. In some embodiments, the system is configured to store only changes in the virtual environment. In some embodiments, by storing only changes in the virtual environment, valuable computer storage resources are saved. In some embodiments, the system is configured to enable the user to fast forward, rewind, play and pause virtual time while in the virtual environment to witness sequential changes in the virtual environment. In some embodiments, this enables the user to witness multiple structural and / or component changes made during the life cycle of the project in a very short real time.

[0105] Fig.37 The diagram illustrates a user-activated analysis window according to some embodiments. In some embodiments, the system is configured to enable a user to manipulate components in a virtual environment. In some embodiments, non-limiting examples of manipulating components include moving, rotating, activating, fixing, adding and / or deleting components and / or structures. In some embodiments, the system is configured to enable a user to manipulate components via an analysis display. Fig.38 The user is shown selecting a component in front of him in the analysis display. Fig.39 The assembly is shown in a vertically elevated position from below a base in accordance with some embodiments. Fig.40 The assembly is shown being lowered to a base by a user in accordance with some embodiments. Fig.41 Illustration of selection of a component and an analysis window displaying a 2D schematic diagram of the component in accordance with some embodiments.

[0106] Fig.42 Depicted is a user navigating out of a ship's engine room toward a CAD overview perspective using fly mode, in accordance with some embodiments. Fig.43 A user is shown selecting a "Jump To" option on a controller menu in accordance with some embodiments. Fig.44 Illustrating user selection of a jump location in accordance with some embodiments. Fig.45 Depicting a user in accordance with some embodiments teleporting to a jump location by jumping. In some embodiments, the jump location comprises a portal entrance. In some embodiments, the system is configured to enable the user to place one or more jump locations anywhere within the virtual environment.

[0107] Fig.46An overview of the concepts of an XR module according to some embodiments is shown. In some embodiments, the XR module supports one or more integrated systems. In some embodiments, the XR module supports XR applications. In some embodiments, the XR application can interact with one or more of a desktop mouse, keyboard, touch, a VR device such as HoloLens 2, and a mobile phone or other device. In some embodiments, the integrated system includes the system described herein, as well as AVEVA products such as CVP SDK and XR Studio.

[0108] In some embodiments, (XR) modules are miniature and / or independent components that enable users to configure the system into customized applications. In some embodiments, modules are independent of templates. In some embodiments, modules can be easily customized and / or maintained.

[0109] Fig.47 The diagram illustrates system modules and connections between the modules that form at least a portion of a system according to some embodiments. Fig.48 According to some embodiments Fig.47 First magnified view of . Fig.49 According to some embodiments Fig.47 A second enlarged view of . Fig.50 is a third enlarged view according to some embodiments. Fig.51 According to some embodiments Fig.47 Fourth enlarged view of . Fig.52 Illustrate additional system modules connected to a manager module in accordance with some embodiments.

[0110] Fig.53 1 is a diagram illustrating the concept of glue code according to some embodiments. In some embodiments, the glue code includes scripts that help facilitate communication between modules so that end users can customize templates as needed. In some embodiments, the system includes code samples within the template to help users customize. Fig.54 Included is example glue code according to some embodiments. Fig.55 The diagram illustrates a computer system 1010 capable of implementing or including systems and methods according to some embodiments of the system. In some embodiments, the computer system 1010 can operate and / or process computer executable code of one or more software modules of the above-described systems and methods. In addition, in some embodiments, the computer system 1010 can operate and / or display information within one or more graphical user interfaces (e.g., HMIs) integrated or coupled with the system.

[0111] In some embodiments, the computer system 1010 may include one or more processors 1032. In some embodiments, the one or more processors 1032 may reside in or be coupled to one or more conventional server platforms (not shown). In some embodiments, the computer system 1010 may include a network interface 1035a and an application interface 1035b coupled to at least one processor 1032 capable of processing at least one operating system 1034. In addition, in some embodiments, the interfaces 1035a, 1035b coupled to at least one processor 1032 may be configured to process one or more of the software modules (e.g., such as enterprise applications 1038). In some embodiments, the software application module 1038 may include server-based software and may be operable to host at least one user account and / or at least one customer account and to transfer data between one or more of these accounts using at least one processor 1032.

[0112] In view of the above embodiments, it should be understood that the system can employ various computer-implemented operations involving data stored in a computer system. In addition, according to various embodiments, the above-mentioned database and the models described throughout the present disclosure can store analytical models and other data on a computer-readable storage medium within the computer system 1010 and on one or more non-transitory computer-readable storage media coupled to the computer system 1010. In addition, in some embodiments, the above-mentioned applications of the system can be stored on a computer-readable storage medium within the computer system 1010 and on a computer-readable storage medium coupled to the computer system 1010. In some embodiments, these operations are those that require physical processing of physical quantities. Typically, although not necessarily, in some embodiments, these physical quantities take the form of one or more of electrical signals, electromagnetic signals, magnetic signals, optical signals, or magneto-optical signals that can be stored, transmitted, combined, compared, and otherwise processed. In some embodiments, the computer system 1010 may include at least one computer-readable medium 1036 coupled to at least one of at least one data source 1037a, at least one data storage device 1037b, and / or at least one input / output device 1037c. In some embodiments, the computer system 1010 may be embodied as computer readable code on a computer readable medium 1036. In some embodiments, the computer readable medium 1036 may be any data storage device capable of storing data that can then be read by a computer (such as computer 1040). In some embodiments, one or more non-transitory computer readable media 1036 may be any physical or material medium that can be used to tangibly store desired information or data or instructions and can be accessed by the computer 1040 or processor 1032. In some embodiments, the computer readable medium 1036 may include a hard drive, a network attached storage device (NAS), a read-only memory, a random access memory, a flash-based memory, a CD-ROM, a CD-R, a CD-RW, a DVD, a tape, other optical and non-optical data storage devices. In some embodiments, various other forms of computer readable media 1036 may send or convey instructions to a remote computer 1040 and / or at least one user 1031, including routers, private or public networks or other wired and wireless transmission devices or channels. In some embodiments, the software application module 1038 may be configured to send data and receive data from a database (e.g., from a computer-readable medium 1036 comprising a data source 1037a and a data storage device 1037b that may include a database), and data may be received by the software application module 1038 from at least one other source.In some embodiments, at least one of the software application modules 1038 may be configured within the computer system 1010 to output data to at least one user 1031 via at least one graphical user interface presented on at least one digital display.

[0113] In some embodiments, one or more non-transitory computer-readable media 1036 can be distributed over a conventional computer network via a network interface 1035a, where a system embodied by computer-readable code can be stored and executed in a distributed manner. For example, in some embodiments, one or more components of the computer system 1010 can be coupled to send and / or receive data via a local area network ("LAN") 1039a and / or an Internet-coupled network 1039b (e.g., such as wireless Internet). In some embodiments, the network 1039a, 1039b can include a wide area network ("WAN"), a direct connection (e.g., via a universal serial bus port), or other forms of computer-readable media 1036, or any combination thereof.

[0114] In some embodiments, the components of the networks 1039a, 1039b may include any number of personal computers 1040, including, for example, desktop computers and / or laptop computers, or any fixed, typically non-mobile Internet devices coupled via the LAN 1039a. For example, some embodiments include one or more of the personal computers 1040, databases 1041, and / or servers 1042 coupled via the LAN 1039a that may be configured for any type of user, including an administrator. Some embodiments may include one or more personal computers 1040 coupled via the network 1039b. In some embodiments, one or more components of the computer system 1010 may be coupled via the Internet (e.g., such as the network 1039b) to send or receive data. For example, some embodiments include at least one user 1031a, 1031b that is wirelessly coupled and accesses one or more software modules of a system including at least one enterprise application 1038 via an input and output ("I / O") device 1037c. In some embodiments, the computer system 1010 may enable at least one user 1031a, 1031b to be coupled via a LAN 1039a to access enterprise applications 1038 via I / O devices 1037c. In some embodiments, the user 1031 may include a user 1031a coupled to the computer system 1010 using a desktop computer, and / or laptop computer coupled via the Internet 1039b, or any fixed, typically non-mobile Internet device. In some embodiments, the user may include a mobile user 1031b coupled to the computer system 1010. In some embodiments, the user 1031b may connect using any mobile computing device 1031c to wirelessly couple to the computer system 1010, including but not limited to one or more personal digital assistants, at least one cellular telephone, at least one mobile phone, at least one smart phone, at least one pager, at least one digital tablet, and / or at least one fixed or mobile Internet device.

[0115] The subject matter described herein is intended to make technical improvements in the field of industrial analysis by using innovative hardware and software. The present disclosure describes the details of how a machine including one or more computers implements the system and its improvements over the prior art, the computer including one or more processors and one or more non-transitory computer-readable media. The instructions executed by the machine cannot be performed in the human brain or derived by humans using pen and paper, but require the machine to convert process input data into useful output data. In addition, the claims presented herein do not attempt to link the judicial exception to conventional steps known to be implemented by general-purpose computers; nor do they attempt to bundle the judicial exception by simply linking it to the technical field. In fact, the systems and methods described herein were unknown and / or did not exist in the public domain at the time of filing the application, and they provide technical improvement advantages that were not known in the prior art. In addition, the system includes unconventional steps that limit the claims to useful applications.

[0116] It should be understood that the system is not limited in its application to the details of the construction and arrangement of the components stated in the preceding description or shown in the accompanying drawings. The systems and methods disclosed herein fall within the scope of numerous embodiments. The foregoing discussion is presented to enable those skilled in the art to implement and use embodiments of the system. Any portion of the structure and / or principle included in some embodiments may be applied to any and / or all embodiments: it should be understood that the features of some embodiments shown herein may be combined with features according to some other embodiments. Thus, some embodiments of the system are not intended to be limited to what is shown, but rather should be given the broadest scope consistent with all principles and features disclosed herein.

[0117] Some embodiments of the system are described using specific values ​​and / or set points. These values ​​and set points are not limiting, but are merely examples of higher configurations compared to lower configurations, and are intended to help a person of ordinary skill in implementing and using the system.

[0118] Any text in the drawings is part of the disclosure of the system and should be understood to be readily incorporated into any description of the boundaries of the system. Any functional language in the drawings is a reference to a system configured to perform the described function, and the structures shown or described in the drawings should be considered to include the system as described therein. Any drawings depicting a graphical user interface are a disclosure of a system configured to display the contents of the graphical user interface. It should be understood that the use of the description of the images in the drawings to define the boundaries of the system does not require a corresponding textual description in the written specification to fall within the scope of the present disclosure.

[0119] Furthermore, as Applicant's own lexicographer, Applicant assigns express meanings to the following terms and / or disclaims claim scope:

[0120] Applicants define any use of "and / or", such as "A and / or B", or "at least one of A and / or B" to mean only element A, only element B, or elements A and B together. In addition, the statement "at least one of A, B, and C", the statement "at least one of A, B, or C", or the statement "at least one of A, B, or C, or any combination thereof" are defined to mean only element A, only element B, only element C, or any combination of elements A, B, and C, such as AB, AC, BC, or ABC.

[0121] When used in connection with values, "substantially" and "approximately" encompass differences of less than 5% of the same unit and / or scale being measured.

[0122] As used herein, "simultaneously" includes the lag and / or delay times associated with conventional and / or proprietary computers, such as the processors and / or networks described herein, attempting to process multiple types of data simultaneously. "Simultaneously" also includes the time it takes for a digital signal to travel from one physical location to another, whether over a wireless and / or wired network, or within a processor circuit.

[0123] As used herein, "can" or "may" or derivatives thereof (e.g., the system display can display X) are used for descriptive purposes only and should be understood to be synonymous and / or interchangeable with "configured to" (e.g., the computer is configured to execute instruction X) when defining the boundaries of the system. In some embodiments, the phrase "configured to" also refers to the step of configuring a structure or computer to perform a function.

[0124] Additionally, the phrase "configured to" means that the limitations listed in the specification and / or claims must be arranged in a manner to perform the listed functions: "configured to" excludes structures in the art that "can" be modified to perform the listed functions, but the disclosure associated with the art does not explicitly teach to do so. For example, a statement "configured to receive fluid from structure X at an upper portion and deliver fluid to a container of structure Y from a lower portion" is limited to systems in which structure X, structure Y, and the container are all disclosed as performing the listed functions. The statement "configured to" excludes systems that may "can" perform the listed functions simply by virtue of their construction, but the associated disclosure (or lack thereof) does not provide teaching to make such modifications to satisfy the functional limitations among all of the listed structures. Another example is "a computer system configured or programmed to execute a series of instructions X, Y, and Z." In this example, the instructions must be present on a non-transitory computer-readable medium such that the computer system is "configured to" and / or "programmed to" execute the enumerated instructions: "configured to" and / or "programmed to" excludes art teaching computer systems having a non-transitory computer-readable medium that is merely "capable of" having the enumerated instructions stored thereon, without the teachings of instructions X, Y, and Z programmed and stored thereon. The statement "configured to" may also be interpreted as being synonymous with being operably connected when used in conjunction with a physical structure.

[0125] It should be understood that the phrases and terms used herein are for descriptive purposes and should not be considered limiting. The use of "including," "comprising," or "having" and variations thereof herein is meant to include the items listed thereafter and their equivalents as well as additional items. Unless otherwise specified or limited, the terms "mounted," "connected," "supported," and "coupled," and variations thereof, are used in a broad sense to include direct and indirect mounting, connection, support, and coupling. Furthermore, "connected" and "coupled" are not limited to physical or mechanical connections or couplings.

[0126] The foregoing detailed description will be read with reference to the drawings, wherein like elements in different drawings have like reference numerals. The drawings, which are not necessarily drawn to scale, depict some embodiments and are not intended to limit the scope of embodiments of the system.

[0127] Any of the operations described herein that form part of the present invention are useful machine operations. The present invention also relates to equipment or devices for performing these operations. The device can be specially constructed for the desired purpose, such as a special-purpose computer. When defined as a special-purpose computer, the computer can also perform other processing, program execution or routines that are not part of the special-purpose, while still being able to operate for the special-purpose. Alternatively, these operations can be processed by a general-purpose computer that is selectively activated or configured by one or more computer programs stored in a computer memory, a cache, or obtained through a network. When data is obtained through a network, the data can be processed by other computers on the network (e.g., a cloud of computing resources).

[0128] Embodiments of the present invention can also be defined as machines that transform data from one state to another state. Data can represent articles, which can be represented as electronic signals and manipulate data electronically. In some cases, the transformed data can be visually depicted on a display, thereby representing the physical object obtained from the transformation of the data. The transformed data can be saved to a storage device generally or in a specific format that enables the construction or depiction of physical and tangible objects. In some embodiments, the manipulation can be performed by a processor. In such an example, the processor thus transforms data from one thing to other things. Further, some embodiments include methods that can be processed by one or more machines or processors that can be connected through a network. Each machine can transform data from one state or thing to another state or thing, and can also process data, save data to a storage device, transmit data through a network, display results, or pass results to another machine. Computer-readable storage media used herein refers to physical or tangible storage devices (contrary to signals), and includes but is not limited to volatile and non-volatile, removable and non-removable storage media implemented by any method or technology for tangible storage of information (such as computer-readable instructions, data structures, program modules or other data).

[0129] Although method operations are presented in a particular order according to some embodiments, the execution of these steps does not necessarily occur in the order listed unless explicitly specified. In addition, other housekeeping operations may be performed between operations, operations may be adjusted so that they occur at slightly different times, and / or operations may be distributed in a system that allows processing operations to occur at various intervals associated with the processing, as long as the processing of the superimposed operations proceeds in the desired manner and results in the desired system output.

[0130] It will be appreciated by those skilled in the art that, although the invention has been described above in conjunction with specific embodiments and examples, the invention is not necessarily limited thereto, and the appended claims are intended to include numerous other embodiments, examples, uses, modifications and deviations to these embodiments, examples and uses. The entire disclosure of each patent and publication cited herein is incorporated by reference, just as each such patent or publication is incorporated herein by reference individually. Various features and advantages of the invention are set forth in the following claims.

Claims

1. A system for generating an industrial metaverse, comprising: One or more computers, the one or more computers comprising one or more processors and one or more non-transitory computer-readable media, the one or more non-transitory computer-readable media comprising program instructions stored thereon, the program instructions, when executed, causing the one or more computers to: generating, by the one or more processors, a virtual environment comprising a three-dimensional (3D) representation of at least a portion of a physical industrial environment; generating, by the one or more processors, asset links between one or more virtual assets in the virtual environment and one or more physical assets in the physical industrial environment; generating, by the one or more processors, a data link between the virtual environment and a historical database including asset data regarding the one or more physical assets; generating, by the one or more processors, a virtual graphical user interface GUI in the virtual environment, the virtual GUI being configured to enable a user to access and display the asset data in the virtual environment; and generating, by the one or more processors, one or more user controls in the virtual environment, the one or more user controls configured to navigate the virtual environment, select the one or more virtual assets, and control at least a portion of the virtual GUI; The display of the virtual environment in the user's field of view is configured to change based on a change in the position of the user's head.

2. The system according to claim 1, The asset data includes at least one analysis result of raw data associated with the one or more virtual assets.

3. The system according to claim 1, Wherein selecting the one or more virtual assets comprises manipulation of an actuator of the virtual asset.

4. The system according to claim 3, Wherein manipulation of the virtual asset actuator results in remote manipulation of the corresponding physical asset actuator.

5. The system according to claim 4, Wherein the physical asset actuator comprises one or more of a joystick, a valve, a switch, and a computer setting.

6. The system according to claim 1, The one or more non-transitory computer-readable media also include program instructions stored thereon that, when executed, cause the one or more computers to: One or more virtual models of the one or more physical assets are generated, by the one or more processors, the one or more virtual models each comprising one or more mathematical equations configured to represent physical behavior of the one or more physical assets.

7. The system according to claim 6, wherein selecting the one or more virtual assets comprises virtual manipulation of a virtual asset controller; wherein the virtual manipulation results in a change in one or more variables of the one or more virtual models; and The effects of changes in the one or more variables are displayed in the virtual environment.

8. The system according to claim 7, wherein the effect is a virtual effect; wherein the virtual effects include theoretical results from calculations performed by the one or more virtual models; and Wherein the virtual effect does not include changes to the one or more physical assets.

9. The system according to claim 8, The one or more non-transitory computer-readable media also include program instructions stored thereon that, when executed, cause the one or more computers to: The one or more processors execute a user's command for physically manipulating the one or more physical assets corresponding to the virtual manipulation after the virtual effect occurs in the virtual environment.

10. The system according to claim 1, The one or more non-transitory computer-readable media also include program instructions stored thereon that, when executed, cause the one or more computers to: receiving, by the one or more processors, one or more images of the one or more physical assets; performing, by the one or more processors, a comparison of the one or more images with one or more previous images of the one or more physical assets; performing, by the one or more processors, identifying visual changes in the one or more images based on the comparing; and The one or more virtual assets in the virtual environment are altered, via the one or more processors, to reflect the visual changes in the one or more images.

11. The system according to claim 10, wherein the comparison includes the performance of artificial intelligence; and Wherein at least a portion of the one or more previous images is used to train the artificial intelligence.

12. The system according to claim 10, The one or more non-transitory computer-readable media also include program instructions stored thereon that, when executed, cause the one or more computers to: A change in a variable in one or more virtual models is predicted based on the visual change, by the one or more processors.

13. The system according to claim 1, Wherein selection of the one or more virtual assets in the virtual environment is configured to result in generation of a display of a time series graph in the virtual environment.

14. The system according to claim 1, The one or more virtual assets are connected to the one or more physical assets via a Software as a Service (SaaS) subscription program.

15. The system according to claim 1, wherein the physical industrial environment comprises a Supervisory Control and Data Acquisition (SCADA) platform configured to monitor and / or control the one or more physical assets; and Wherein at least a portion of the SCADA is accessible via the virtual environment.

Images