Automated industrial process recipe generation

Abstract

A technique is provided for automatically generating process definitions for industrial processes used to create products in an industrial plant, including: capturing sensor data associated with an individual performing a set of process operations on a set of process materials to manufacture a product; analyzing the sensor data associated with an individual performing a set of process operations on a set of process materials to manufacture a product; and identifying the process definition based on the analysis of the sensor data associated with an individual performing a set of process operations on a set of process materials to manufacture a product, the process definition including a set of process materials, equipment used to manufacture the product, a set of process operations applied to the materials to manufacture the product, a sequence of process operations, timing of process operations, and / or the quantity of materials used in the process.

Description

Technical Field

[0001] This patent application relates generally to processes in industrial plants, and more specifically, to process definitions for automatically generating industrial processes for creating products in industrial plants. Background Technology

[0002] Industrial or manufacturing processes (such as those used in chemical, petroleum, pharmaceutical, paper, or other industrial process plants to produce physical products from source materials) typically include one or more process controllers coupled to one or more field devices via analog, digital, or combined analog / digital buses or via wireless communication links or networks. These field devices (which may be, for example, valves, valve positioners, switches, and transmitters (e.g., temperature, pressure, level, and flow rate sensors)) are located within the process environment and typically perform physical or process control functions such as opening or closing valves, measuring process parameters, etc., to control one or more processes executing within the process plant or system. Intelligent field devices, such as those conforming to well-known FOUNDATION... ® Field devices using the Fieldbus protocol can also perform control calculations, alarm functions, and other control functions typically implemented within a controller. These process controllers (which can be located centrally or distributed within the plant environment) receive signals indicative of process measurements performed by the field devices and / or other information related to the field devices, and execute a controller application that runs, for example, different control modules. These different control modules make process control decisions based on the received information, generate control signals, and communicate with the field devices (such as HART). ® WirelessHART ® and FOUNDATION ® Control modules or blocks are coordinated within Fieldbus field devices. Control modules in the controller send control signals to field devices via communication lines or links, thereby controlling the operation of at least a portion of the process plant or system.

[0003] Information from field devices and controllers is typically provided from the controllers via high-speed data paths to one or more other hardware devices, such as operator workstations, personal computers or computing devices, data history repositories, report generators, centralized databases, or other centralized management computing devices typically located in control rooms or other locations away from harsh plant environments. Each of these hardware devices is typically centralized within the process plant or a portion thereof. These hardware devices execute applications that, for example, enable engineers to configure parts of the process or enable operators to perform functions related to controlling the process and / or operating the process plant, such as changing the settings of process control routines, modifying the operation of control modules within controllers or field devices, viewing the current status of the process, viewing alarms generated by field devices and controllers, simulating process operation for training purposes or testing process control software, maintaining and updating configuration databases, etc. The high-speed data paths utilized by the hardware devices, controllers, and field devices may include wired communication paths, wireless communication paths, or a combination of wired and wireless communication paths.

[0004] For example, DeltaV sold by Emerson Inc. ™The control system comprises multiple applications stored in and executed by various devices located at different locations within the process plant. Configuration applications (residing in one or more workstations or computing devices) enable users to create or modify process control modules and download these modules to dedicated distributed controllers via high-speed data paths. Typically, these control modules consist of interconnected functional blocks, which are objects in an object-oriented programming protocol. These objects perform functions within the control scheme based on inputs and provide outputs to other functional blocks within the control scheme. Configuration applications also allow configuration designers to create or modify operator interfaces used by viewing applications to display data to operators and enable operators to change settings within process control routines, such as setpoints. Each dedicated controller, and in some cases, one or more field devices, stores and executes a corresponding controller application that runs the control modules allocated and downloaded to it to implement the actual process control functions. Viewing applications (which may run on one or more operator workstations (or on one or more remote computing devices communicatively connected to the operator workstations and the data highway)) receive data from the controller application via the data highway and display that data to the process control system designer, operator, or user using a user interface, providing any of several different views, such as an operator's view, an engineer's view, a technician's view, etc. Data history applications are typically stored in and executed by a data history device that collects and stores some or all of the data provided via the data highway. Meanwhile, configuration database applications may run on another computer attached to the data highway to store the current process control routine configuration and associated data. Alternatively, the configuration database may reside on the same workstation as the configuration application.

[0005] Historically, when developing industrial or manufacturing processes, the steps of these processes were manually recorded. The initial process developers provided key process, material, equipment, and procedural knowledge and information to those implementing the process, such as through paper lab notebooks, PDFs, and spreadsheets. Consequently, the accuracy and completeness of process knowledge and information are easily compromised, which not only increases the total time required to complete the process but also introduces risks into the products produced by that process. Unfortunately, such risks can lead to safety issues for both plant workers and end-product users, and in some cases, such as when the risks are introduced into processes used to produce pharmaceuticals, chemicals, and other potentially hazardous and / or fatal products, they can result in injury or death. Exacerbating these problems, manual technology transfer makes it difficult for product manufacturers to record and accurately report the data necessary to identify the causes and contributing factors to product safety and quality, as well as to comply with regulatory requirements. Furthermore, this manual technology transfer increases the time required for process testing and deployment.

[0006] Several digital tools have attempted to address these issues; however, these attempts have shortcomings and limitations. For example, product (relative to process) lifecycle management tools (“PLM”) address the lifecycle stages of the physical end product and are not easily adapted to and / or optimized for aspects specific to the lifecycle stages of the industrial and manufacturing processes that produce the physical end product. Furthermore, PLM is not easily customized, extended to, and / or integrated with site-specific execution systems (if any).

[0007] Recently, some process knowledge management systems or tools (“PKA”) have attempted to allow engineers and / or other process personnel to define processes a priori in a manner similar to those used in object-oriented technologies (e.g., objects, classes, modules, instances, etc.). Such PKA tools provide a user interface and database that enable users to define and store a set of related process definitions (from generalized expressions down to specific concrete implementation values, such as experimental process definitions, general process definitions, site process definitions, control process definitions) and site-specific parameter values ​​that can be passed from the tool to a specific site for implementation at that site. However, these tools still require users to manually define process objects and parameters in advance at the tool level for various object levels (e.g., objects, classes, modules, instances, etc.), which is both cumbersome and time-consuming.

[0008] Furthermore, manually recording and defining process steps in this way can lead to many of the same problems as manually recording and defining process steps in a lab notebook. Specifically, manual recording of a process typically does not, or cannot, occur in real time as the scientist or engineer performs the process, but is usually completed long after the actual process is finished. Therefore, whether using a lab notebook or PKA tools, manually recording and defining process steps can result in process steps, measurements, etc., being omitted or otherwise misrepresented because the recording occurs after a given step is completed, or even after the entire process is finished. This affects the accuracy of the recorded process. When any element of the process—such as the amount of time taken for a given step (e.g., how quickly or slowly one material is poured into another), the amount of time between steps, the order of steps, specific measurements of materials, specific types of materials, specific methods of mixing or combining materials, specific properties and quantities of results, etc.—is incorrect, especially when multiple of these elements are incorrect, the process can become more difficult to accurately reproduce in future batches. Reproducing an inaccurate process can lead to incorrect results, or in some cases, unsafe or dangerous results. Summary of the Invention

[0009] In one aspect, a computer-implemented method is provided for automatically generating a process definition for an industrial process used to create a product in an industrial plant. The method may include capturing sensor data associated with an individual performing a set of process operations on a set of process materials to manufacture a product; analyzing the sensor data associated with the individual performing the set of process operations on the set of process materials to manufacture the product; and identifying the process definition based on the analysis of the sensor data associated with the individual performing the set of process operations on the set of process materials to manufacture the product. The process definition includes one or more of the following: the set of process materials used to manufacture the product, one or more pieces of equipment used to manufacture the product, the set of process operations applied to the materials to manufacture the product, the sequence of the process operations, the timing of the process operations, or information regarding the quantity of materials used in the process. The method may include additional, fewer, or alternative operations, including those discussed elsewhere herein.

[0010] On the other hand, a computer system is provided for automatically generating process definitions for industrial processes used to create products in an industrial plant. The computer system may include: one or more sensors configured to capture sensor data associated with an individual performing a set of process operations on a set of process materials to manufacture a product; one or more processors; and a memory storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to: analyze the sensor data associated with the individual performing the set of process operations on the set of process materials to manufacture the product; and identify a process definition based on the analysis of the sensor data associated with the individual performing the set of process operations on the set of process materials to manufacture the product, the process definition including one or more of the following: the set of process materials used to manufacture the product, one or more pieces of equipment used to manufacture the product, the set of process operations applied to the materials to manufacture the product, the sequence of the process operations, the timing of the process operations, or information regarding the quantity of materials used in the process. The system may include additional, fewer, or alternative functionalities, including those discussed elsewhere herein.

[0011] In another aspect, a non-transitory computer-readable storage medium is provided for storing computer-readable instructions for automatically generating process definitions for an industrial process used to create a product in an industrial plant. When executed by one or more processors, the computer-readable instructions cause the processors to: analyze sensor data associated with an individual performing the set of process operations on a set of process materials to manufacture the product; and identify the process definition based on the analysis of the sensor data associated with an individual performing the set of process operations on the set of process materials to manufacture the product, the process definition including one or more of the following: the set of process materials used to manufacture the product, one or more pieces of equipment used to manufacture the product, the set of process operations applied to the materials to manufacture the product, the sequence of the process operations, the timing of the process operations, or information regarding the quantity of materials used in the process. The instructions may direct additional, fewer, or alternative functionalities, including those discussed elsewhere herein.

[0012] In one aspect, a computer-implemented method is provided for automatically generating a configuration hierarchy for industrial processes used to create products in an industrial plant. The method may include analyzing sensor data associated with an individual performing a set of process operations on a set of process materials to manufacture a product; identifying, based on the analysis of the sensor data associated with the individual performing the set of process operations on the set of process materials to manufacture the product, the set of process operations applied to the materials to manufacture the product and one or more pieces of equipment used in each of the process operations; and determining, based on the set of process operations applied to the materials to manufacture the product and the one or more pieces of equipment used in each of the process operations, a hierarchy of the industrial plant associated with each of the process operations. The method may include additional, fewer, or alternative operations, including those discussed elsewhere herein.

[0013] On the other hand, a computer system is provided for automatically generating a configuration hierarchy for industrial processes used to create products in an industrial plant. The computer system may include one or more processors and a memory storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to: analyze sensor data associated with an individual performing a set of process operations on a set of process materials to manufacture a product; identify, based on the analysis of the sensor data associated with the individual performing the set of process operations on the set of process materials to manufacture the product, the set of process operations applied to the materials to manufacture the product and one or more pieces of equipment used in each of the process operations; and determine, based on the set of process operations applied to the materials to manufacture the product and the one or more pieces of equipment used in each of the process operations, a hierarchy of the industrial plant associated with each of the process operations. The system may include additional, fewer, or alternative functionalities, including those discussed elsewhere herein.

[0014] In another aspect, a non-transitory computer-readable storage medium is provided for storing computer-readable instructions for automatically generating a configuration hierarchy for an industrial process used to create a product in an industrial plant. When executed by one or more processors, the computer-readable instructions cause the processors to: analyze sensor data associated with an individual performing a set of process operations on a set of process materials to manufacture a product; and, based on the analysis of the sensor data associated with the individual performing the set of process operations on the set of process materials to manufacture the product, identify the set of process operations applied to the materials to manufacture the product and one or more pieces of equipment used in each of the process operations; and, based on the set of process operations applied to the materials to manufacture the product and the one or more pieces of equipment used in each of the process operations, determine a hierarchy of the industrial plant associated with each of the process operations. The instructions may direct additional, fewer, or alternative functionality, including functionality discussed elsewhere herein.

[0015] In one aspect, a computer-implemented method is provided for visualizing a process definition of an industrial process for creating a product in an industrial plant. The method may include analyzing sensor data associated with an individual performing a set of process operations on a set of process materials to manufacture a product; generating a visualization of performing the set of process operations on the set of process materials to manufacture the product based on the captured sensor data associated with the individual performing the set of process operations on the set of process materials to manufacture the product, wherein the visualization exemplifies one or more of the following: the set of process materials used to manufacture the product, one or more pieces of equipment used to manufacture the product, the set of process operations applied to the materials to manufacture the product, the sequence of the process operations, the timing of the process operations, or information regarding the quantity of materials used in the process; and providing the visualization of the individual performing the set of process operations on the set of process materials to manufacture the product via a user interface. The method may include additional, fewer, or alternative operations, including those discussed elsewhere herein.

[0016] On the other hand, a computer system is provided for visualizing process definitions of industrial processes used to create products in an industrial plant. The computer system may include one or more processors and a memory storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to: generate a visualization of performing a set of process operations on a set of process materials to manufacture the product, based on captured sensor data associated with an individual performing a set of process operations on a set of process materials to manufacture the product, wherein the visualization exemplifies one or more of the following: the set of process materials used to manufacture the product, one or more pieces of equipment used to manufacture the product, the set of process operations applied to the materials to manufacture the product, the sequence of the process operations, the timing of the process operations, or information regarding the quantity of materials used in the process; and provide the visualization of the individual performing the set of process operations on the set of process materials to manufacture the product via a user interface. The system may include additional, fewer, or alternative functionalities, including those discussed elsewhere herein.

[0017] In another aspect, a non-transitory computer-readable storage medium is provided for storing computer-readable instructions for visualizing a process definition of an industrial process for creating a product in an industrial plant. When executed by one or more processors, the computer-readable instructions cause the processors to: generate a visualization of performing a set of process operations on a set of process materials to manufacture the product, based on captured sensor data associated with an individual performing a set of process operations on a set of process materials to manufacture the product, wherein the visualization exemplifies one or more of the following: the set of process materials used to manufacture the product, one or more pieces of equipment used to manufacture the product, the set of process operations applied to the materials to manufacture the product, the sequence of the process operations, the timing of the process operations, or information regarding the quantity of materials used in the process; and provide the visualization of the individual performing the set of process operations on the set of process materials to manufacture the product via a user interface. The instructions may direct additional, fewer, or alternative functionality, including functionality discussed elsewhere herein.

[0018] The advantages will become more apparent to those skilled in the art from the following description of the preferred embodiments shown and described by way of illustration. As will be appreciated, embodiments of this disclosure may have other and different implementations, and modifications may be made to their details in various aspects. Therefore, the drawings and description are to be regarded in an illustrative rather than restrictive manner. Attached Figure Description

[0019] The accompanying drawings described below depict various aspects of the systems and methods disclosed herein. It should be understood that each drawing depicts an embodiment of a specific aspect of the disclosed systems and methods, and each drawing is intended to correspond to one possible embodiment thereof.

[0020] The arrangement currently under discussion is shown in the figures; however, it should be understood that this embodiment is not limited to the precise arrangement and tools shown in the figures:

[0021] Figure 1 An exemplary computer system according to one embodiment is described, the exemplary computer system being used to automatically generate process definitions for industrial processes for creating products in an industrial plant, automatically generate configuration hierarchical structures for process definitions for industrial processes for creating products in an industrial plant, and / or visualize process definitions for industrial processes for creating products in an industrial plant;

[0022] Figure 2 An exemplary individual according to one embodiment performs a set of process operations on a set of process materials to manufacture a product, and an exemplary sensor is configured to capture sensor data associated with the individual performing the set of process operations on the set of process materials to manufacture the product;

[0023] Figure 3 An exemplary plant hierarchy structure in an industrial plant according to one implementation scheme is depicted;

[0024] Figure 4 A flowchart is depicted of an exemplary computer-implemented method according to one embodiment for automatically generating process definitions for industrial processes used to create products in an industrial plant;

[0025] Figure 5 A flowchart is depicted illustrating an exemplary computer-implemented method according to one embodiment, which is used to automatically generate a configuration hierarchy structure for an industrial process definition used to create a product in an industrial plant; and

[0026] Figure 6 A flowchart is depicted of an exemplary computer-implemented method according to one embodiment, which is used to visualize the process definition of an industrial process for creating a product in an industrial plant.

[0027] While the systems and methods disclosed herein may be embodied in many different forms, specific exemplary embodiments thereof are shown in the accompanying drawings and will be described in detail herein. It should be understood that this disclosure is intended to be seen as an illustration of the principles of the systems and methods disclosed herein, and not as an attempt to limit the scope of protection of the systems and methods disclosed herein to the illustrated specific embodiments. In this regard, before explaining in detail at least one embodiment consistent with the systems and methods disclosed herein, it should be understood that the systems and methods disclosed herein are not limited to the construction details and component arrangements described above and below, in the accompanying drawings, or in the examples.

[0028] Methods and apparatus consistent with the systems and methods disclosed herein can have other embodiments and can be implemented and performed in various ways. Furthermore, it should be understood that the wording and terminology used herein, as well as the abstract contained below, are for descriptive purposes and should not be considered limiting. Detailed Implementation

[0029] The novel systems, methods, and techniques provided herein allow and enable the accurate generation of process “recipes” in real time as a process is performed by an individual (e.g., a scientist, engineer, operator, worker, etc.). In one embodiment, sensor data can be captured in real time as the individual performs the process. Sensor data may include images and / or videos of the individual, equipment, materials, etc., captured as the individual performs steps of the process, as well as audio data captured as the individual performs steps of the process. For example, in some examples, the audio data may include data associated with words or phrases spoken by the individual as they perform steps of the process. In some examples, the sensor data may include data from sensors associated with equipment involved in steps of the process. For example, the sensor data may include data from motion sensors or weight sensors indicating the time and / or amount of time the equipment is used, or the weight / size of materials used with the equipment. Furthermore, in some examples, the sensor data may include location data associated with equipment or materials used and moved as the individual completes steps of the process.

[0030] Machine learning models can be trained to associate sensor data with specific process steps. Sensor data can be analyzed (e.g., using machine learning models) to identify a process definition of a “recipe” that includes a set of process materials used to manufacture the product, one or more pieces of equipment used to manufacture the product, a set of process operations applied to the materials to manufacture the product, a sequence of process operations, timing of the process operations, information about the quantity of materials used in the process, etc. Furthermore, sensor data can be analyzed to identify the hierarchical structure of the plant associated with each step / element of the process or with the process as a whole. Advantageously, the accuracy of the recorded process is greatly improved because the data associated with the steps of the process is captured in real time as the individual performs the process and does not rely on estimates and / or approximations of the process steps after they have been performed.

[0031] Process definitions determined based on sensor data can be provided to users via a user interface. In some examples, the process definition can be provided via the user interface as a written or otherwise graphical depiction of a series of steps, measurements, times, etc. Users can interact with the process definition provided via the user interface to adjust, clarify, or correct the process determined based on sensor data, as well as with annotations associated with the individual steps of the process.

[0032] Furthermore, in some examples, a visual depiction of the process can be provided via a user interface. For example, in some examples, an augmented reality (AR) depiction of a process performed by an individual can be provided as an overlay on the actual factory where the individual performs the process, or an overlay on an image or video of the factory where the individual performs the process. For example, this allows a new user who wishes to reproduce the steps of the process to follow the steps and timings actually performed by the individual and shown in the AR overlay. Additionally, in some examples, when multiple processes (or the same process) performed by multiple users are captured, an AR depiction of the process performed by each individual can be overlaid on the actual factory where the individual performs the process, or overlaid on an image or video of the factory where the individual performs the process, making it possible to compare / contrast the performance of the process by multiple individuals. For example, in some examples, the performance of the process by multiple individuals can be compared for training purposes. As another example, the performance of the process by multiple individuals and the corresponding results of each performance of the process can be compared to identify how any differences between the performances of the process might affect the final result of the process.

[0033] Now refer to the attached diagram, Figure 1An exemplary computer system 100 according to one embodiment is described. This exemplary computer system is used to automatically generate process definitions for industrial processes used to create products in an industrial plant, automatically generate configuration hierarchical structures for the process definitions for creating products in an industrial plant, and / or visualize the process definitions for creating products in an industrial plant. As described below, Figure 1 The advanced architecture shown may include hardware and software applications, as well as various data communication channels for transmitting data between various hardware and software components.

[0034] System 100 may be implemented within or in conjunction with an industrial environment 102 (e.g., an industrial plant) including industrial equipment 103 (which may include field equipment in an industrial plant), and may include one or more sensors 104 located within the industrial environment 102. In some examples, one or more sensors 104 may be attached to or integrated within multiple pieces of industrial equipment 103 in the industrial environment 102. Generally, sensors 104 may be configured to capture data associated with an individual (e.g., an operator, scientist, engineer, or other individual in the industrial environment 102) performing a set of process operations on a set of process materials to produce a product. For example, Figure 2 An exemplary individual is depicted performing a set of process operations on a set of process materials in an industrial environment 102 to manufacture a product, and an exemplary sensor 104 is configured to capture sensor data associated with the individual performing the set of process operations on the set of process materials to manufacture the product.

[0035] Sensor 104 can be configured to capture various types of data associated with an individual performing a set of process operations on a set of process materials to manufacture the product. Sensor data may include image data, video data, and / or audio data associated with the individual when using equipment 103 to perform process operations on the set of process materials to manufacture the product. Furthermore, sensor 104 can be configured to capture data associated with equipment 103 and / or the materials involved in the set of process operations. Additionally, sensor 104 can be configured to capture location data associated with the individual, materials, equipment 103, etc., when an individual uses equipment 103 to perform process operations on the set of process materials to manufacture the product.

[0036] System 100 may also include a computing device 106, and in some cases, one or more user interface devices 108 (which may include, for example, smartphones, smartwatches or fitness trackers, tablets, laptops, virtual reality headsets, smart or augmented reality glasses, wearable devices, etc.). Sensor 104, computing device 106, and user interface device 108 may be configured to communicate with each other via a wired or wireless computer network 110. To facilitate such communication, device 103, sensor 104, computing device 106, and / or user interface 108 may each include a wireless transceiver for receiving and transmitting wireless communications.

[0037] Despite Figure 1 The diagram shows a device 103, five sensors 104, a computing device 106, a user interface 108, and a network 110, but in various embodiments, any number of such devices 103, sensors 104, computing devices 106, user interfaces 108, and / or networks 110 may be included. Furthermore, although in Figure 1 Only equipment 103 and sensor 104 are shown within industrial environment 102, but in some examples, computing device 106, user interface device 108, and / or network 110 may also be located within industrial environment 102. Furthermore, although computing device 106 and user interface device 108 are... Figure 1 While shown as separate devices, in some examples these devices may be integrated into a single device, such as a fixed or mobile computing device 106 with user interface components. Similarly, although sensor 104 is shown as separate from computing device 106 and / or user interface device 108, in some examples sensor 104 may be attached to or integrated with computing device 106 and / or user interface device 108.

[0038] In some implementations, computing device 106 may include one or more servers, which may include multiple redundant or replicated servers as part of a server cluster. In a further aspect, such servers may be implemented as cloud-based servers, such as cloud-based computing platforms. For example, such servers may be any one or more cloud-based platforms, such as Microsoft Azure, Amazon AWS, etc. Such servers may include one or more processors 110 (e.g., CPUs) and one or more computer memories 112.

[0039] Memory 112 may include one or more forms of volatile and / or non-volatile, fixed and / or removable memory, such as read-only memory (ROM), electronically programmable read-only memory (EPROM), random access memory (RAM), erasable electronically programmable read-only memory (EEPROM), and / or other hard disk drives, flash memory, microSD cards, etc. Memory 122 may store an operating system (OS) (e.g., Microsoft Windows, Linux, UNIX, etc.) that enables the functions, applications, methods, or other software discussed herein. Memory 112 may also store an automation process definition generator application 114, a process visualization application 115, a machine learning model 116, and / or a machine learning model training application 117.

[0040] Additionally or alternatively, memory 112 may store data from various sources, including industrial process terminology data and historical sensor data associated with historical industrial processes. Industrial process terminology data may also be stored in an industrial process terminology database 118, which may be accessible or otherwise communicatively coupled to computing device 106. Furthermore, historical sensor data associated with historical industrial processes may be stored in historical sensor data associated with historical industrial process database 119. In some embodiments, industrial process terminology data, historical sensor data associated with historical industrial processes, and / or other data from various sources may be stored on one or more blockchains or distributed ledgers.

[0041] The automated process definition generator application 114 may include enabling sensors 104 to capture sensor data associated with an individual when an individual uses equipment 103 to perform a process in an industrial environment 102. For example, the automated process definition generator application 114 may analyze the sensor data captured by sensors 104 to identify a process definition for the process. The process definition may include, for example, a set of process materials used to manufacture a product, equipment 103 used to manufacture the product, a set of process operations applied to the materials to manufacture the product, a sequence of process operations, timing of the process operations, information about the quantity of materials used in the process, etc.

[0042] For example, in some examples, the automated process definition generator application 114 can analyze images or videos of an individual performing a process captured by sensor 104 to identify various aspects of the process. For instance, as shown in the images or videos of the process, the proximity of an individual to a specific piece of equipment 103 can indicate that the individual is performing a step associated with that equipment 103. Furthermore, as shown in the images or videos of the process, the individual's movement can indicate what type of step the individual is performing, such as a step of moving materials between locations, a step of heating materials, a step of mixing or stirring multiple materials together, etc. Additionally, the automated process definition generator application 114 can analyze images or videos of an individual performing a process to identify specific materials involved in the process, for example, based on shape, size, color, or other visual characteristics associated with the material or its container, or based on barcodes, symbols, and / or optical character identifiers of words or serial numbers printed on the material itself or its container. Similarly, the automation process definition generator application 114 can analyze images or videos of individual execution processes to identify a specific piece of equipment 103 involved in the process in the industrial environment 102, for example, based on shape, size, color or other visual characteristics associated with equipment 103, or based on barcodes, symbols and / or optical character identifications of words or serial numbers printed on equipment 103 in the industrial environment 102, or based on the location of equipment 103 in the industrial environment 102.

[0043] Furthermore, in some examples, the automated process definition generator application 114 can analyze audio data captured by sensor 104 and associated with an individual execution process to identify aspects of the process. For example, the audio data can be analyzed to identify materials, equipment 103, and / or process steps based on sounds associated with specific materials and / or equipment 103 and / or sounds associated with specific process steps. For example, specific equipment 103 may emit specific sounds during a mixing step, and the automated process definition generator application 114 can analyze audio data captured by sensor 104 and associated with an individual execution process to identify equipment 103 and / or identify that a mixing step is being performed by equipment 103.

[0044] Additionally, audio data can be analyzed to identify words or phrases spoken by the individual during the process. For example, when performing a process, the individual may verbally describe the steps of the process (e.g., such as...). Figure 2Examples include phrases like "stir the first material with the second material together for 30 minutes" and "heat one pound of material for 20 minutes using the equipment name." For instance, natural language processing and / or a version of natural language processing notified by the Industrial Process Terminology Database 118 can be used to analyze an individual's verbal description of a process to identify process steps, the order of process steps, the amount of time associated with each step of the process, the equipment used during each step of the process, the materials used during each step of the process, and their corresponding quantities, etc.

[0045] Furthermore, in some examples, the automation process definition generator application 114 can analyze data captured by sensors 104 associated with or integrated with equipment 103 as an individual performs a process. For example, equipment 103 may include sensors 104 (e.g., electronic scales) configured to measure the weight of objects placed on or within equipment 103 and to transmit the measured weight of the objects to computing device 106. The automation process definition generator application 114 can associate this weight measurement with materials involved in the process. As another example, equipment 103 may include sensors 104, such as motion sensors or proximity sensors, which can capture data as equipment 103 performs steps of the process (e.g., when equipment 103 moves to perform process steps, and / or when materials are placed into equipment 103). The automation process definition generator application 114 can use this motion or proximity data associated with equipment 103 to determine, for example, the timing of specific steps involving equipment 103 and their relationship to other steps of the process, and / or the duration of the individual steps involving equipment 103 during the process.

[0046] Additionally, a piece of equipment 103 may include a sensor 104, such as a position sensor, which can capture data as the equipment 103 moves within an industrial environment. In some examples, a material or a container of material may also include a corresponding position sensor 104. Furthermore, in some examples, an individual may have a position sensor 104 attached to their body (e.g., on their clothing, on a lanyard, etc.). The automation process definition generator application 114 can use this position associated with the equipment 103, material, and / or individual to determine, for example, the timing of a specific step involving the equipment 103 and its relationship to other steps in the process, and / or the duration of each step involving the equipment 103 during the process. That is, the automation process definition generator application 114 can determine that a process step involving the individual, material, and / or equipment is in progress at a specific time or duration based on the individual, material, and / or equipment 103 being in the same location as each other at a specific time and / or duration.

[0047] In some examples, the automated process definition generator application 114 may determine, based on data captured by sensor 104 when an individual performs a set of process operations on a set of process materials to manufacture a product, the set of process materials used to manufacture the product, the equipment 103 used to manufacture the product, the set of process operations applied to the materials to manufacture the product, the sequence of process operations, the timing of the process operations, information about the quantity of materials used in the process, etc., based on data captured by sensor 104 when an individual performs a set of process operations on a set of process materials to manufacture the product, as discussed in more detail below.

[0048] In some examples, the automated process definition generator application 114 may further determine hierarchical levels associated with one or more steps of the process, for example, such as Figure 3 The hierarchical structure is illustrated in the diagram. That is, in one example, the hierarchical structure of industrial environment 102 may include a region level 302, a unit module level 304, an equipment module level 306, and a control module level 308. A specific piece of equipment 103 may be associated with a specific level of the hierarchical structure of industrial environment 102. For example, based on determining that a specific piece of equipment 103 involves a specific step of the process, automation process definition generator application 114 may determine the hierarchical level associated with each step of the process, or the hierarchical level associated with the process as a whole.

[0049] The process visualization application 115 may include visualizations of generated processes (e.g., as identified by the automation process definition generator application 114). For example, in some examples, the process visualization application 115 may generate an interactive user interface display that lists the steps of the process, i.e., in text or in the form of diagrams or graphs such as flowcharts. Furthermore, in some examples, the process visualization application 115 may generate visualizations of the steps of an individual process execution. For example, the process visualization application 115 may generate augmented reality (AR) visualizations of the steps of an individual process execution, which may be overlaid on an image or video of the industrial environment 102, or may be overlaid on the actual industrial environment 102 (e.g., using a projector). The visualization of the steps of an individual process execution may be transmitted to a user interface device 108, where it may be made available to a user. In some examples, a user may interact with the visualization of the steps of the individual process execution via the user interface device 108, as discussed in more detail below, for example, to isolate specific steps of the process, modify or correct specific steps of the process, etc.

[0050] In some examples, the machine learning model 116 may be executed on the computing device 106, while in other examples, the machine learning model 116 may be executed on a separate computing system independent of the computing device 106. For example, the computing device 106 may transmit data captured by the sensor 104 when an individual performs a set of process operations on a set of process materials to manufacture a product to another computing system, wherein the trained machine learning model 116 is applied to the data captured by the sensor 104 when the individual performs the process, and the other computing system may, based on the application of the trained machine learning model 116 to the data captured by the sensor 104 when the individual performs the process, transmit to the computing device 106 predictions or identifiers of the set of process materials used to manufacture the product, the equipment 103 used to manufacture the product, the set of process operations applied to the materials to manufacture the product, the sequence of process operations, the timing of the process operations, information about the quantity of materials used in the process, etc. Furthermore, in some examples, the machine learning model 116 may be trained by a machine learning model training application 117 that runs on the computing device 106, while in other examples, the machine learning model 116 may be trained by a machine learning model training application that runs on a separate computing system from the computing device 106.

[0051] Regardless of whether the machine learning model 116 is trained on computing device 106 or elsewhere, it can be trained by machine learning model training application 117 using training data corresponding to historical sensor data captured as an individual performs historical processes, and from each of the corresponding historical processes, a set of process materials for manufacturing the product, equipment 103 for manufacturing the product, a set of process operations applied to the materials to manufacture the product, a sequence of process operations, timing of process operations, information about the quantity of materials used in the process, etc., such as those available from database 119. The trained machine learning model 116 can then be applied to new sensor data captured when an individual performs a new process (e.g., new sensor data captured by sensor 104) to determine various elements of the new process: such as a set of process materials for manufacturing the product, equipment 103 for manufacturing the product, a set of process operations applied to the materials to manufacture the product, a sequence of process operations, timing of process operations, information about the quantity of materials used in the process, etc.

[0052] In various aspects, the machine learning model 116 may include machine learning procedures or algorithms that can be trained by and / or employ neural networks, which may be deep learning neural networks or combined learning modules or procedures that learn from one or more features or feature datasets in a specific region of interest. The machine learning procedures or algorithms may also include natural language processing, semantic analysis, automated reasoning, regression analysis, support vector machine (SVM) analysis, decision tree analysis, random forest analysis, K-nearest neighbor analysis, Naive Bayes analysis, clustering analysis, reinforcement learning, and / or other machine learning algorithms and / or techniques.

[0053] In some implementations, the artificial intelligence and / or machine learning-based algorithms used to train the machine learning model 116 may be included in the computing device 106 (or Figure 1 Libraries or packages that execute on other computing devices (not shown). For example, such libraries may include TENSORFLOW-based libraries, PYTORCH libraries, and / or SCIKIT-LEARN Python libraries.

[0054] Machine learning may involve identifying and recognizing patterns in existing data (such as training a model based on historical sensor data captured as an individual performs a historical process and a set of process materials used to manufacture the product, equipment 103 used to manufacture the product, a set of process operations applied to the materials to manufacture the product, a sequence of process operations, timing of process operations, information about the quantity of materials used in the process, etc.) to facilitate prediction or identification of subsequent data (such as using machine learning model 116 to determine predictions for possible sets of process materials used to manufacture the product, possible equipment 103 used to manufacture the product, possible sets of process operations applied to the materials to manufacture the product, possible sequences of process operations, possible timing of process operations, information about the possible quantity of materials used in the process, etc.) based on new sensor data associated with an individual's execution of a new process order.

[0055] Machine learning models can be created and trained based on example data (e.g., “training data”) or data (which may be referred to as “features” and “labels”) to make effective and reliable predictions on new inputs, such as test-level or production-level data or inputs. In supervised machine learning, example inputs (e.g., “features”) and their associated or observed outputs (e.g., “labels”) can be provided to a machine learning program running on a server, computing device, or other processor so that the machine learning program or algorithm can determine or discover rules, relationships, patterns, or other machine learning “models” that map such inputs (e.g., “features”) to outputs (e.g., labels), for example, by determining and / or assigning weights or other metrics to various feature categories of the model. Subsequent inputs can then be provided to such rules, relationships, or other models so that the model executing on the server, computing device, or other processor can predict the expected outputs based on the discovered rules, relationships, or models.

[0056] In unsupervised machine learning, servers, computing devices, or other processors may need to find their own structure within unlabeled example inputs. This may involve multiple training iterations performed by the server, computing device, or other processor to train multiple generations of models until a satisfactory model is generated—for example, a model that provides sufficient predictive accuracy when given test-level or production-level data or inputs. The disclosure in this paper may utilize one or both of such supervised or unsupervised machine learning techniques.

[0057] In addition, memory 112 may also store additional machine-readable instructions, including any one of one or more application programs, one or more software components, and / or one or more application programming interfaces (APIs), which can be implemented to facilitate or perform the features, functions, or other disclosures described herein, such as any methods, processes, elements, or limitations illustrated, depicted, or described in the various flowcharts, diagrams, charts, figures, and / or other disclosures relating to this document. For example, in some examples, the computer-readable instructions stored on memory 112 may include instructions for performing methods 400, 500, or 600 (which are described below respectively in relation to) via algorithms executed on processor 110. Figure 4 , Figure 5 and Figure 6 (Described in more detail) Instructions for any step of any of the processes described herein. It should be understood that one or more other applications may be envisioned and executed by processor 110. It should be understood that, given the current state of development of mobile computing devices, all process functions and steps described herein may coexist on mobile computing devices such as user interface device 108.

[0058] Generally, the user interface device 108 may include a user interface display 120, or be configured to communicate with such a user interface display, which may receive input from a user and provide audible or visible output to the user. In some examples, the user interface display 120 may be configured to provide augmented reality (AR) output to the user, such as AR output overlaid on, or overlaid on, images or videos of, the industrial environment 102 and / or equipment 103.

[0059] Additionally, the user interface device 108 may include one or more processors 122 and one or more computer memories 124. The memory 124 may include one or more forms of volatile and / or non-volatile, fixed and / or removable memory, such as read-only memory (ROM), electronically programmable read-only memory (EPROM), random access memory (RAM), erasable electronically programmable read-only memory (EEPROM), and / or other hard disk drives, flash memory, microSD cards, etc. The memory 124 may store an operating system (OS) (e.g., iOS, Microsoft Windows, Linux, UNIX, etc.) that enables the functions, applications, methods, or other software discussed herein.

[0060] The memory 124 may also store instructions that, when executed by one or more processors 122, cause one or more processors 122 to receive graphics, AR renderings, and / or other visualizations generated by the computing device 106 and associated with an individual execution process, and to display such graphics, AR renderings, and / or other visualizations to a user of the user interface device 108. Furthermore, when executed by one or more processors 122, the instructions may cause one or more processors 122 to receive input from a user of the user interface device 108, and to modify the graphics, AR renderings, and / or other visualizations based on the input from the user of the user interface device 108. For example, user input may include a request to isolate graphics, AR renderings, and / or other visualizations associated with a specific step or portion of the individual execution process. As another example, user input may include a request to speed up or slow down graphics, AR renderings, and / or other visualizations associated with a specific step or portion of the individual execution process. Furthermore, as another example, user input may include a request to modify graphics, AR renderings, and / or other visualizations associated with a specific step or portion of the individual execution process.

[0061] Figure 4A flowchart is depicted of an exemplary computer-implemented method 400 according to one embodiment, which is used to automatically generate a process definition for an industrial process for creating a product in an industrial plant. One or more steps of method 400 may be implemented as a set of instructions stored on a computer-readable storage medium (e.g., memory 112 and / or memory 124) and executable on one or more processors (e.g., processor 110 and / or processor 122).

[0062] Method 400 may include capturing (box 402) sensor data associated with an individual performing a set of process operations on a set of process materials to manufacture a product. For example, in some examples, the sensor data may include image data, video data, and / or audio data associated with the individual and captured as the individual performs one or more of the process operations in the set. Additionally, in some examples, the sensor data may include data from sensors associated with one or more pieces of equipment involved in the set of process operations. Furthermore, in some examples, the sensor data may include position sensor data associated with process materials involved in the process, equipment involved in the process, individuals involved in the process, etc.

[0063] Additionally, method 400 may include analyzing (box 404) sensor data associated with an individual performing the set of process operations on the set of process materials to manufacture the product. In some examples, analyzing the sensor data may include analyzing audio data to identify words or phrases spoken by the individual while performing one or more of the process operations in the set of process operations.

[0064] In addition, method 400 may include: identifying (box 406) a process definition based on analysis of sensor data associated with an individual performing the set of process operations on the set of process materials to manufacture the product, the process definition including one or more of the following: the set of process materials used to manufacture the product, one or more pieces of equipment used to manufacture the product, the set of process operations applied to the materials to manufacture the product, the sequence of the process operations, the timing of the process operations, or information about the quantity of materials used in the process.

[0065] Figure 5 A flowchart is depicted of an exemplary computer-implemented method 500 according to one embodiment, which is used to define an automatically generated configuration hierarchy structure for an industrial process used to create a product in an industrial plant. One or more steps of method 500 may be implemented as a set of instructions stored on a computer-readable storage medium (e.g., memory 112 and / or memory 124) and executable on one or more processors (e.g., processor 110 and / or processor 122).

[0066] Method 500 may include: analyzing (box 502) sensor data associated with an individual performing a set of process operations on a set of process materials to manufacture a product.

[0067] Additionally, method 500 may include: identifying (box 504) the set of process operations applied to the materials to manufacture the product and one or more pieces of equipment used in each process operation, based on analysis of sensor data associated with an individual performing the set of process operations on the set of process materials to manufacture the product.

[0068] Furthermore, method 500 may include: determining (box 506) a hierarchical structure of the industrial plant associated with each process operation, based on the set of process operations applied to the material to manufacture the product and the one or more pieces of equipment used in each process operation. For example, Figure 3 An example factory hierarchy in an industrial plant is shown.

[0069] Figure 6 A flowchart is depicted of an exemplary computer-implemented method 600 according to one embodiment, which is used to visualize a process definition for an industrial process for creating a product in an industrial plant. One or more steps of method 600 may be implemented as a set of instructions stored on a computer-readable storage medium (e.g., memory 112 and / or memory 124) and executable on one or more processors (e.g., processor 110 and / or processor 122).

[0070] Method 600 may include: analyzing (box 602) sensor data associated with an individual performing a set of process operations on a set of process materials to manufacture a product.

[0071] Furthermore, method 600 may include: generating (box 604) a visualization of performing the set of process operations on the set of process materials to manufacture the product based on captured sensor data associated with an individual performing a set of process operations on a set of process materials to manufacture the product, wherein the visualization exemplifies one or more of the following: the set of process materials used to manufacture the product, one or more pieces of equipment used to manufacture the product, the set of process operations applied to the materials to manufacture the product, the sequence of the process operations, the timing of the process operations, or information about the quantity of materials used in the process.

[0072] Furthermore, method 600 may include providing a visualization, via a user interface, of an individual performing a set of process operations on a set of process materials to manufacture the product. For example, in some examples, providing the visualization of an individual performing the set of process operations on a set of process materials to manufacture the product includes providing an augmented reality (AR) visualization of the individual performing the set of process operations on a set of process materials to manufacture the product. In some examples, the AR visualization of the individual performing the set of process operations on a set of process materials to manufacture the product may be overlaid on a process environment in which sensor data associated with the individual performing the set of process operations on a set of process materials to manufacture the product is captured.

[0073] In some examples, method 600 may further include: receiving input from a user requesting a specific process operation within the set of process operations, and subsequently providing visualization of the specific process operation isolated from the set of process operations via a user interface based on the input from the user.

[0074] In addition, in some examples, method 600 may also include: receiving input from a user indicating a request to increase or decrease the speed associated with the visualization of one or more process operations in the set of process operations, and subsequently providing, via a user interface, a visualization of the slowing down or speeding up of the one or more process operations in the set of process operations based on the input from the user.

[0075] The following additional considerations apply to the above discussion. Throughout the specification, multiple instances can implement operations or structures described as a single instance. Although individual operations of one or more methods are shown and described as separate operations, one or more of the separate operations can be performed simultaneously, and they do not need to be performed in the order shown. These and other variations, modifications, additions, and improvements fall within the scope of this document.

[0076] Unless otherwise specified, the use of words such as “processing,” “computing,” “operation,” “determining,” “presenting,” “displaying,” etc., in this document may refer to the actions or processes by which a machine (e.g., a computer) manipulates or transforms data that is represented as physical (e.g., electronic, magnetic, or optical) quantities within one or more memories (e.g., volatile memory, non-volatile memory, or combinations thereof), registers, or other machine components that receive, store, transmit, or display information.

[0077] As used herein, any reference to “implementation,” “one implementation,” or “some implementations” means that a particular element, feature, structure, or characteristic described in connection with that implementation is included in at least one implementation. The phrases “in one implementation” or “in some implementations” appearing in different places in the specification do not necessarily refer to the same implementation.

[0078] As used herein, the terms “including,” “comprising,” “having,” or any other variation thereof are intended to cover non-exclusive inclusion. For example, a process, method, article, or apparatus that includes a list of elements is not necessarily limited to those elements, but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Furthermore, unless expressly stated to the contrary, “or” refers to inclusive or, not exclusive or. For example, any of the following satisfies condition A or B: A is true (or exists) and B is false (or does not exist); A is false (or does not exist) and B is true (or exists); and both A and B are true (or exist).

[0079] Additionally, the use of "a / an" is used to describe elements and components of the embodiments described herein. This is done merely for convenience and to give a general understanding of the invention. This description should be understood to include one or at least one, and the singular includes the plural, unless otherwise indicated.

[0080] Upon reading this disclosure, those skilled in the art will understand alternative structural and functional designs for automatically generating process definitions for industrial processes used to create products in industrial plants. Therefore, while specific embodiments and applications have been shown and described, it should be understood that the disclosed embodiments are not limited to the precise constructions and components disclosed herein. Various modifications, alterations, and variations, readily apparent to those skilled in the art, may be made to the arrangement, operation, and details of the methods and apparatus disclosed herein without departing from the spirit and scope defined by the appended claims.

Claims

1. A method for automatically generating process definitions for industrial processes used to create products in an industrial plant, the method comprising: Capture sensor data associated with an individual performing a set of process operations on a set of process materials to manufacture a product; Analyze the sensor data associated with the individual performing the set of process operations on the set of process materials to manufacture the product; as well as A process definition is identified based on the analysis of sensor data associated with the individual performing the set of process operations on the set of process materials to manufacture the product. The process definition includes one or more of the following: the set of process materials used to manufacture the product, one or more pieces of equipment used to manufacture the product, the set of process operations applied to the materials to manufacture the product, the sequence of the process operations, the timing of the process operations, or information regarding the quantity of the materials used in the process.

2. The method of claim 1, wherein the sensor data includes one or more of the following: image data, video data, or audio data associated with an individual and captured while the individual is performing one or more of the set of process operations.

3. The method of claim 1, wherein the sensor data includes data from sensors associated with one or more pieces of equipment involved in the set of process operations.

4. The method of claim 1, wherein the sensor data includes position sensor data associated with one or more process materials in the set of process materials, one or more pieces of equipment involved in the set of process operations, or the individual involved in the set of process operations.

5. The method of claim 1, wherein analyzing the sensor data includes analyzing audio data to identify words or phrases spoken by the individual while the individual performs one or more of the set of procedural operations.

6. The method according to claim 1, further comprising: The identified process definition is provided via the user interface; The user interface receives adjustments to one or more of the following: the set of process materials used to manufacture the product, the one or more pieces of equipment used to manufacture the product, the set of process operations applied to the materials to manufacture the product, the sequence of the process operations, the timing of the process operations, or the quantity information of the materials used in the process. as well as The process definition is updated based on the adjustments received via the user interface.

7. A system for automatically generating process definitions for industrial processes used to create products in an industrial plant, the system comprising: One or more sensors, the one or more sensors being configured to capture sensor data associated with an individual performing a set of process operations on a set of process materials to manufacture a product; One or more processors; The memory stores computer-readable instructions that, when executed by the one or more processors, cause the one or more processors to: Analyze the sensor data associated with the individual performing the set of process operations on the set of process materials to manufacture the product; as well as A process definition is identified based on the analysis of sensor data associated with the individual performing the set of process operations on the set of process materials to manufacture the product. The process definition includes one or more of the following: the set of process materials used to manufacture the product, one or more pieces of equipment used to manufacture the product, the set of process operations applied to the materials to manufacture the product, the sequence of the process operations, the timing of the process operations, or information regarding the quantity of the materials used in the process.

8. The system of claim 7, wherein the sensor data includes one or more of the following: image data, video data, or audio data associated with an individual and captured while the individual is performing one or more of the set of process operations.

9. The system of claim 7, wherein the sensor data includes data from sensors associated with one or more pieces of equipment involved in the set of process operations.

10. The system of claim 7, wherein the sensor data includes position sensor data associated with one or more process materials in the set of process materials, one or more pieces of equipment involved in the set of process operations, or the individual involved in the set of process operations.

11. The system of claim 7, wherein analyzing the sensor data includes analyzing audio data to identify words or phrases spoken by the individual while the individual performs one or more of the set of procedural operations.

12. The system of claim 7, further comprising a user interface, and wherein the computer-readable instructions, when executed by the one or more processors, also cause the one or more processors to: The identified process definition is provided via the user interface; The user interface receives adjustments to one or more of the following: the set of process materials used to manufacture the product, the one or more pieces of equipment used to manufacture the product, the set of process operations applied to the materials to manufacture the product, the sequence of the process operations, the timing of the process operations, or information regarding the quantity of the materials used in the process; and The process definition is updated based on the adjustments received via the user interface.

13. A method for automatically generating a configuration hierarchy for an industrial process definition used to create a product in an industrial plant, the method comprising: Analyze sensor data associated with individuals performing a set of process operations on a set of process materials to manufacture a product; Based on the analysis of sensor data associated with the individual performing the set of process operations on the set of process materials to manufacture the product, the set of process operations applied to the materials to manufacture the product and one or more pieces of equipment used in each of the process operations are identified; as well as Based on the set of process operations applied to the material to manufacture the product and the one or more pieces of equipment used in each of the process operations, a hierarchical structure hierarchy of the industrial plant associated with each of the process operations is determined.

14. A system for automatically generating configuration hierarchical structures for process definitions of industrial processes used to create products in an industrial plant, the system comprising: One or more processors; The memory stores computer-readable instructions that, when executed by the one or more processors, cause the one or more processors to: Analyze sensor data associated with individuals performing a set of process operations on a set of process materials to manufacture a product; Based on the analysis of sensor data associated with the individual performing the set of process operations on the set of process materials to manufacture the product, the set of process operations applied to the materials to manufacture the product and one or more pieces of equipment used in each of the process operations are identified; as well as Based on the set of process operations applied to the material to manufacture the product and the one or more pieces of equipment used in each of the process operations, a hierarchical structure hierarchy of the industrial plant associated with each of the process operations is determined.

15. A method for visualizing the process definition of an industrial process for creating a product in an industrial plant, the method comprising: Analyze sensor data associated with individuals performing a set of process operations on a set of process materials to manufacture a product; Based on the analysis of sensor data associated with the individual performing the set of process operations on the set of process materials to manufacture the product, a visualization of performing the set of process operations on the set of process materials to manufacture the product is generated, wherein the visualization exemplifies one or more of the following: the set of process materials used to manufacture the product, one or more pieces of equipment used to manufacture the product, the set of process operations applied to the materials to manufacture the product, the sequence of the process operations, the timing of the process operations, or information about the quantity of the materials used in the process; as well as The visualization provided via a user interface shows the individual performing the set of process operations on the set of process materials to manufacture the product.

16. The method of claim 15, wherein providing the individual with the visualization of the set of process materials performing the set of process operations to manufacture the product comprises: Provides augmented reality (AR) visualization of the individual performing the set of process operations on the set of process materials to manufacture the product.

17. The method of claim 16, wherein providing the individual the AR visualization of the set of process materials performing the set of process operations to manufacture the product comprises: The AR visualization, overlaid on a process environment, shows the individual performing a set of process operations on a set of process materials to manufacture the product, and the sensor data associated with the individual performing the set of process operations on the set of process materials to manufacture the product is captured in the process environment.

18. The method according to claim 15, further comprising: Receive input from the user requesting a specific process operation from the set of process operations; as well as Based on the input from the user, the visualization of the specific process operation that is isolated from the set of process operations is provided via the user interface.

19. The method according to claim 15, further comprising: Receive input from the user, the input indicating a request to increase or decrease the speed associated with the visualization of one or more of the set of process operations; as well as Based on the input from the user, a visualization of slowing down or speeding up one or more process operations in the set of process operations is provided via the user interface.

20. A system for visualizing process definitions of industrial processes used to create products in an industrial plant, the system comprising: user interface; One or more processors; The memory stores computer-readable instructions that, when executed by the one or more processors, cause the one or more processors to: Analyze sensor data associated with individuals performing a set of process operations on a set of process materials to manufacture a product; Based on the analysis of sensor data associated with the individual performing the set of process operations on the set of process materials to manufacture the product, a visualization of performing the set of process operations on the set of process materials to manufacture the product is generated, wherein the visualization exemplifies one or more of the following: the set of process materials used to manufacture the product, one or more pieces of equipment used to manufacture the product, the set of process operations applied to the materials to manufacture the product, the sequence of the process operations, the timing of the process operations, or information about the quantity of the materials used in the process; as well as The user interface provides visualization of the individual performing the set of process operations on the set of process materials to manufacture the product.

21. The system of claim 20, wherein providing the visualization of the individual performing the set of process operations on the set of process materials to manufacture the product comprises: Provides augmented reality (AR) visualization of the individual performing the set of process operations on the set of process materials to manufacture the product.

22. The system of claim 21, wherein providing the individual the AR visualization of the set of process materials performing the set of process operations to manufacture the product comprises: The AR visualization, overlaid on a process environment, shows the individual performing a set of process operations on a set of process materials to manufacture the product, and the sensor data associated with the individual performing the set of process operations on the set of process materials to manufacture the product is captured in the process environment.

23. The system of claim 20, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to: Receive input from the user requesting a specific process operation from the set of process operations; and Based on the input from the user, the visualization of the specific process operation that is isolated from the set of process operations is provided via the user interface.

24. The system of claim 20, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to: Receive input from the user, the input indicating a request to increase or decrease the speed associated with the visualization of one or more process operations in the set of process operations; and Based on the input from the user, a visualization of slowing down or speeding up one or more process operations in the set of process operations is provided via the user interface.

Images