Monitoring and / or controlling equipment

Abstract

The present disclosure relates to the production of cosmetic formulations with specified target properties. Disclosed are methods, apparatuses, systems for determining the stability of the formulation and specified target properties as well as generating formulation data for producing the formulation. Further disclosed are cosmetic products and cosmetic formulations produced based on the generated formulation control data.

Description

[0001] MONITORING AND / OR CONTROLLING EQUIPMENT

[0002] TECHNICAL FIELD

[0003] The present disclosure relates to the intelligent storage, access and use of equipment operation data using large language models. Disclosed are methods, apparatuses, systems for storing operation data associated with operation equipment. Disclosed are methods, apparatuses, systems monitoring and / or controlling operation equipment.

[0004] TECHNICAL BACKGROUND

[0005] Multiple data storage and processing schemes are known. However, with the advent of big data, data lakes and data warehouses, data management has become more cumbersome particular for large cooperations with many thousands of end user or manufacturing equipment to be monitored and controlled.

[0006] SUMMARY OF THE INVENTION

[0007] In one aspect disclosed is a method for monitoring and / or controlling distributed operation equipment(s), the method comprising the steps: providing at least one user instruction and one or more instruction templates relating to monitoring and / or controlling of distributed operation equipment(s); selecting one or more instruction templates based on the user instruction; generating one or more instruction(s) including at least one task instruction related to the monitoring and / or controlling of distributed operation equipment(s); generating at least one operation instruction for monitoring and / or controlling of operation equipment by providing the one or more generated instruction(s) to a data-driven model configured to generate the at least one operation instructions, wherein the data-driven model is a general-purpose model trained at least on unstructured data; providing the at least one operation instruction for monitoring and / or controlling the distributed operation equipment(s).

[0008] In another aspect disclosed is an apparatus for monitoring and / or controlling distributed operation equipment(s), the method comprising: a template store configured to provide at least one user instruction and one or more instruction templates relating to monitoring and / or controlling of distributed operation equipment(s); a selection engine configured to select one or more instruction templates based on the user instruction; an instruction engine configured to generate one or more instruction(s) including at least one task instruction related to the monitoring and / or controlling of distributed operation equipment(s) and configured to generate at least one operation instruction for monitoring and / or controlling of operation equipment by providing the one or more generated instruction (s) to a data-driven model con-figured to generate the at least one operation instructions, wherein the data-driven model is a general-purpose model trained at least on unstructured data; monitoring and / or controlling engine configured to provide the at least one operation instruction for monitoring and / or controlling the distributed operation equipment(s).

[0009] In another aspect disclosed is the use of the operation instruction(s) generated according to the methods or by the apparatus disclosed herein for displaying monitoring and / or controlling instructions to an operator of at least one of the distributed operation equipment(s) and / or for monitoring and / or controlling at least one of the distributed operation equipment(s). In another aspect disclosed is a method for using operation instruction(s) generated according to the methods or by the apparatus disclosed herein by displaying monitoring and / or controlling instructions to an operator of at least one of the distributed operation equipment(s) and / or by monitoring and / or controlling at least one of the distributed operation equipment(s).

[0010] In another aspect disclosed is a method for storing operation data associated with distributed operation equipment(s), the method comprising the steps: providing one or more source identification(s) of one or more data source(s) storing operation data and one or more sink identification(s) of one or more data sink(s) for storing operation data accessing the operation data from one or more data source(s) related to the provided source identification (s) and processing the operation data according to one or more data templates for storage on an intermediate storage; generating one or more instruction(s) including at least one task instruction to generate an operation data description and one or more entries of the operation data to be described; generating the operation data description by providing the one or more instruction(s) to a data-driven model configured to generate the operation data description, wherein the data-driven model is a general-purpose model trained at least on unstructured data; storing the operation data description as metadata of the processed operation data on the intermediate storage and copying the operation data to one or more data sink(s) related to the provided sink identification(s).

[0011] In another aspect disclosed is an apparatus for storing operation data associated with distributed operation equipment(s), the method comprising the steps: input interface configured to provide one or more source identification(s) of one or more data source(s) storing operation data and one or more sink identification(s) of one or more data sink(s) for storing operation data; operation data interface configured to access the operation data from one or more data source(s) related to the provided source identification(s) and processing the operation data according to one or more data templates for storage on an intermediate storage; instruction agent configured to access one or more instruction(s) including at least one task instruction to generate an operation data description and one or more entries of the operation data to be described and configured to generate the operation data description by providing the one or more instruction(s) to a data- driven model configured to generate the operation data description, wherein the data-driven model is a general-purpose model trained at least on unstructured data; storage interface configured to store the operation data description as metadata of the processed operation data on the intermediate storage and configured to copy the operation data to one or more data sink(s) related to the provided sink identification(s).

[0012] In another aspect disclosed is a method for monitoring and / or controlling operation equipment, the method comprising the steps: providing one or more instruction(s) including at least one task instruction related to the monitoring and / or controlling of operation equipment based on access to operation data, wherein the operation data is stored including operation data descriptions as metadata; generating at least one operation instruction for monitoring and / or controlling of operation equipment based on the operation data descriptions by providing the one or more instruction(s) to a data-driven model configured to generate the at least one operation instructions, wherein the data-driven model is a general- purpose model trained at least on unstructured data; providing the at least one operation instruction for monitoring and / or controlling of the distributed operation equipment(s).

[0013] In another aspect disclosed is an apparatus for monitoring and / or controlling operation equipment, the method comprising the steps: an instruction engine configured to provide one or more instruction(s) including at least one task instruction related to the monitoring and / or controlling of operation equipment based on access to operation data, wherein the operation data is stored including operation data descriptions as metadata and configured to generate at least one operation instruction for monitoring and / or controlling of operation equipment based on the operation data descriptions by providing the one or more instruction(s) to a data-driven model configured to generate the at least one operation instructions, wherein the data-driven model is a general-purpose model trained at least on unstructured data; monitoring and / or controlling engine configured to provide the at least one operation instruction for monitoring and / or controlling of the distributed operation equipment(s).

[0014] In another aspect disclosed is computer element with instructions, which when executed by a computing apparatus perform the steps according to the methods or as provided by the apparatuses disclosed herein. EMBODIMENTS

[0015] Any disclosure and embodiments described herein relate to the methods, the apparatuses, the systems, the uses, the formulations, and the computer elements lined out above and below and vice versa. Advantageously, the benefits provided by any of the embodiments and examples equally apply to all other embodiments and examples.

[0016] In the following, embodiments of the present disclosure will be outlined by ways of embodiments and / or example. It is to be understood that the present disclosure is not limited to said embodiments and / or examples. All terms and definitions used herein are understood broadly and have their general meaning.

[0017] The methods, the apparatuses, the systems, the uses, and the computer elements disclosed herein provide an efficient, secure and reliable way for monitoring and / or controlling equipment. In particular, through the generation of operation data description by way of a large language model the processing of operation data for storage and the access to the such stored operation data can be simplified. For example, by including operation data descriptions into the stored operation data, access can be achieved through natural language interaction via a large language model, which allows for simplified and targeted access. This way not only operation data access but also workflows using such operation data for monitoring and / or controlling operation equipment can be triggered and / or executed in a reliable and efficient manner. As a result, the methods, apparatuses and uses disclosed herein allow for more reliable and efficient monitoring and / or controlling of operation equipment.

[0018] Distributed operation equipment(s) includes multiple operation equipment used across different locations and / or in connection with different computing environment(s) such as storage. Operation equipment may be configured to perform operations related to end user services, chemical manufacturing, chemical laboratories, chemical analytics, or combinations thereof. Operation equipment may be configured to generate operation data. Operation data may be associated with operation of the operation equipment. Operation data may be generated by sensors configured to monitor the operation equipment, actuators configured to control the operation equipment, software applications running on or in relation with the operation equipment. Monitoring and / or controlling includes any operation in relation to the operation equipment that may be triggered based on operation data provided in relation to the operation equipment.

[0019] One or more instruction(s) may be configured to be provided to a general purpose data-driven model configured to process at least natural language. One or more instruction (s) may relate to unstructured data specifying instructions in natural language. Task instruction related to the monitoring and / or controlling of distributed operation equipment(s) may include unstructured data specifying the task to be fulfilled by the data-driven model. Task instruction may be provided in natural language. One or more instruction(s) and / or the task instruction related to the monitoring and / or controlling of distributed operation equipment(s) may relate to or include unstructured data provided in relation to an user instruction provided by the operator of the equipment. User instruction may relate to unstructured data provided by an operator of the equipment. One or more instruction templates relating to monitoring and / or controlling of distributed operation equipment(s) may include pre-defined text or unstructured data in natural language. The instruction templates may relate to one or more instruction types. The instruction types may relate to data identification, data retrieval, workflow selection and / or workflow execution. The template instruction may relate to identifying, accessing, using or generating representation of stored or accessible operation data. The template instruction may include one or more operation data descriptions of stored operation data. The template instruction may include one or more machine readable controlling and / or monitoring instruction descriptions of stored or available machine-readable controlling and / or monitoring instructions. The template instruction may include system instructions relating to the output structure of the operation instruction generated by the data-driven model such as conditions to be fulfilled by the output of the data-driven model. The template instruction may relate to the output of operation instruction provided by the data-driven model based on a subsequently used template instruction. The template instruction may include task instruction relating to generation of output by the data-driven model. The template instruction may include task instruction relating to generation of operation instruction(s) by the data-driven model.

[0020] The use of generative artificial intelligence enables novel areas of application. Generative neural networks are a subgroup of artificial intelligence models, that fall into the category of deep learning. Generative neural networks are machine learning models that employ one or more layers of nonlinear units to generate an output for a received input. Some neural networks include one or more hidden layers in addition to an output layer. The output of each hidden layer is used as input to the next layer in the network, e.g., the next hidden layer or the output layer. Each layer of the network generates an output from a received input in accordance with current values of a respective set of parameters. Such complex modelling structures are employed herein for use in industrial use cases that are safety and process performance relevant.

[0021] The data-driven model may be a pre-trained model. The data-driven model may include a transformer-based architecture configured to process at least natural language such as text. The pre-trained model may be a general- purpose model trained or parametrized based on general data sets including input, such as text-based, and output, such as, image-based, data pairs not specific to a task, in particular not specific to generating operation instructions for operation equipment, further in particular not specific to input data including at least one instruction with at least one task instruction.

[0022] Operation instruction for monitoring and / or controlling the distributed operation equipment(s) may include structured or semi structured data. Operation instruction may include machine-readable instruction configured to be executed by a computing device. Operation instruction may include machine readable instruction configured to be displayed to a user. Operation instruction may relate to a status of one or more operation equipment(s). Operation instruction may relate to providing or displaying operation data associated with the operation equipment. Operation instruction may relate to executable code configured to be executed e.g. on the operation device. In an embodiment depending on the user instruction multiple instruction templates may be selected and processed sequentially. The instruction template sequence to be selected may depend on the user instruction. The instruction template sequence may relate to data identification, data access, workflow selection and / or workflow execution in relation to the distributed operation equipment(s). The instruction template may be selected based on a similarity with the user instruction. The template sequence may be pre-defined depending on the user instruction.

[0023] In another embodiment the instruction templates relate to identification of operation data, retrieval of operation data, selection of machine-readable monitoring and / or control instructions, execution of machine-readable monitoring and / or control instructions or combinations thereof. The instruction template sequence may prescribe identification of operation data followed by retrieval of operation data followed by selection of machine-readable monitoring followed by control instructions, execution of machine-readable monitoring and / or control instructions. Depending on the user instruction one or more of the instruction temples may be selected.

[0024] In another embodiment the instruction templates include at least one task instruction specifying the task to be executed by the data-driven model, at least one specification relating to operation data stored including operation data descriptions as metadata and / or at least one specification relating to machine-readable control and / or monitoring instructions stored including descriptions as metadata.

[0025] In another embodiment generating instructions includes providing at least part of the operation instructions generated by the data-driven model from one or more preceding instruction(s) to a subsequent instruction template. Instructions may be generated per instruction template. The instruction templates may be provided to the data driven model in a sequence. The operation instructions generated per preceding instruction template may be used for instruction generation based on a subsequent instruction template. In other words, the operation instruction generated based on a preceding template may be at least in part provided to the instruction generation based on a subsequent instruction template.

[0026] In another embodiment one or more instruction template(s) include at least one task instruction related to the monitoring and / or controlling of distributed operation equipment(s) based on identification of operation data, wherein the operation data is stored including operation data description as metadata, wherein the one or more instruction(s) are generated based on the operation data description, wherein the operation data is identified and / or retrieved based on the operation instruction generated by the data-driven model from the instruction(s) generated based on the operation data description.

[0027] In another embodiment one or more instruction template(s) include at least one task instruction related to the monitoring and / or controlling of distributed operation equipment(s) based on operation data retrieval, wherein the one or more instruction(s) are generated based on the operation data retrieved, wherein the operation instruction is generated by the data-driven model from the instruction(s) generated based on the operation data retrieved. In another embodiment one or more instruction template(s) include at least one task instruction related to the monitoring and / or controlling of distributed operation equipment(s) based on at least one selected machine-readable monitoring and / or control instruction, wherein the one or more instruction(s) are generated based on the at least one selected machine-readable monitoring and / or control instruction, wherein the operation instruction is generated by the data-driven model from the instruction(s) generated based on the at least one selected machine-readable monitoring and / or control instruction.

[0028] In another embodiment the at least one operation instruction is configured to trigger monitoring and / or controlling at least one of the distributed operation equipment(s).

[0029] In another embodiment the at least one operation instruction is configured to display operation data identified and / or retrieved.

[0030] In another embodiment the at least one operation instruction is configured to display machine-readable monitoring and / or control instructions selected and / or to execute machine-readable monitoring and / or control instructions.

[0031] In another embodiment the at least one operation instruction is provided to at least one of the distributed operation equipment(s) for execution and / or the at least one operation instruction is to be executed to provide the result of the execution to at least one of the distributed operation equipment(s).

[0032] BRIEF DESCRIPTION OF DRAWINGS

[0033] In the following, the present disclosure is further described with reference to the enclosed figures:

[0034] Fig. 1 illustrates an example of an intelligent monitoring and / or controlling platform.

[0035] Fig. 2 illustrates an example method for storing operation data associated with operation equipment.

[0036] Fig. 3 illustrates an example apparatus for storing operation data including a prompt template for generating an operation data description.

[0037] Fig. 4 illustrates an example method for monitoring and / or controlling operation equipment.

[0038] Fig. 5 illustrates an example apparatus for monitoring and / or controlling operation equipment including prompts for searching operation data, accessing operation data, selecting machine-readable instructions and executing machine-readable instructions.

[0039] Fig. 6 illustrates an example of method steps chained by the prompts illustrated in Fig. 5. Fig. 7 illustrates example user interfaces for workflow execution.

[0040] Figs. 8-11 illustrate an example transformer architecture of a general-purpose model. Specifically, FIG. 8 illustrates an embodiment of training an embedding layer. FIG. 9A illustrates an embodiment of a transformer encoder architecture. FIG. 9B illustrates an embodiment of a transformer decoder architecture. FIG. 9C illustrates an embodiment of a transformer encoder-decoder architecture. FIG. 10 illustrates an embodiment of training and / or deploying the transformer encoder, the transformer decoder and / or the transformer encoder-decoder. FIG. 11 illustrates an embodiment of input embedding.

[0041] DESCRIPTION OF EMBODIMENTS

[0042] Fig. 1 illustrates an example of an intelligent monitoring and / or controlling platform.

[0043] The monitoring and / or controlling platform may encompass multiple computational layers including operation equipment, operation data pipelines from operation equipment, storages for operation data, data processing layers configured to provide and / or perform services for data processing and equipment monitoring layers configured to provide and / or perform services for monitoring and / or controlling operation equipment. The monitoring and / or controlling platform may be implemented on a cloud infrastructure configured for distributed computing or cloud computing. "Cloud computing” may refer a model for enabling on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services). When distributed, cloud computing environments may be distributed internationally within an organization and / or across multiple organizations. In this example, the distributed cloud computing environment may contain the following computing resources: mobile device(s), operation equipment, applications, databases, data storages and server(s). The cloud computing environment may be deployed as public cloud, private cloud or hybrid cloud. A private cloud may be owned by an organization and only the members of the organization with proper access can use the private cloud, rendering the data in the private cloud at least confidential. In contrast, data stored in a public cloud may be open to anyone over the internet. The hybrid cloud may be a combination of both private and public clouds and may allow to keep some of the data confidential while other data may be publicly available.

[0044] Operation equipment may be configured to perform operations related to end user services, chemical manufacturing, chemical laboratories, chemical analytics, logistics of chemical products, or the like. For example, operation equipment may include an end user device, such as an Augmented / Virtual Reality equipment, a mobile or handheld device, desktop computer, laptop or the like, configured to monitor chemical manufacturing operations, chemical laboratory operations, chemical product logistics to name a few of them. Further for example operation equipment may include manufacturing equipment, such as heat exchangers, reactors, pumps, pipes, distillation or absorption columns or the like, including monitoring sensors and / or controlling actuators configured to monitor chemical manufacturing operations. Further for example operation equipment may include analytics equipment, such as equipment configured for gas chromatography, mass spectroscopy, flow chemistry equipment or the like, including monitoring sensors and / or controlling actuators configured to analyze, monitor and / or control chemical reactions, chemical reaction products and / or the synthesis of molecules. Further for example operation equipment may include robotic equipment, such as (semi-)autonomous trains for transport, (semi-)autonomous warehouse systems, (semi- )autonomous conveyors, (semi-)autonomous sortation systems, or the like, including monitoring sensors and / or controlling actuators configured to monitor and / or control chemical product movements and / or transportation.

[0045] The operation equipment may be configured to provide operation data associated with the operation performed by the equipment. Operation data may relate to manufacturing of chemical products, analysis of chemical products, synthesis of chemical products, storing and / or transporting of chemical products. Operation data may relate to one or more data points provided by one or more sensor(s), actor(s) and / or application (s) of one or more operation equipment(s) or per operation equipment. The operation data may include unstructured data such as text that may be provided by the user monitoring the operation through the end user device or the application running on operation equipment. The operation data may include semi structured data such as machine-readable text data that may be provided by the application running on operation equipment. The operation data may include structured data such as measurement data from sensors or actors monitoring and / or controlling the chemical operation like manufacturing.

[0046] The operation data may be provided to different type(s) of storages such as the local disk of the operation equipment, data bases, data lakes, data warehouses or the like. The source storage may include any type of storage suitable for persistently or non-persistently storing data. Operation data may be provided in structured format particularly relating to data acquired and provided by the operational equipment through applications, sensors and / or actors. Operation data may be provided in structured format with defined data structure such as tables that may be stored, retrieved and / or manipulated using one or more standardized data base languages like SQL or SQL as a Service. Operation data may be provided in unstructured or semi-structured format without or with semi defined data structure. Unstructured or semi-structured data may be transformed to structured data using data transformation techniques such as Natural Language Processing like embedding models configured to transform language or string data to numeric data and using pattern detection or clustering techniques on the numeric data.

[0047] The operation data may be provided to different type(s) of storages for different types of operation equipment. The operation data may be stored in a distributed manner across different storages for different operation equipment. The operation data may be distributed across different storages. For large infrastructures, such as manufacturing sites or warehouses, with distributed operation equipment, such as end user devices for operators, different manufacturing equipment, different robots and so, spread across the operation infrastructure analyzing operation data of the distributed operation equipment is challenging. Moreover, maintenance or trouble shooting in case of operation equipment errors is cumbersome. To enable an interactive system, the operation data may be provided from distributed source storages to intermediate storage(s). The intermediate storage(s) may include object, block and / or file storage. Since object storage allows for dynamic metadata handling associated with each data object, the intermediate storage may be configured as object storage storing operation data in data objects including the data itself, index, and / or data catalogue or metadata.

[0048] The operation data may be processed e.g. manipulated and / or transformed according to one or more data templates. The data templates may include pre-defined data structures and / or rules to process e.g. manipulate and / or transform raw operation data as provided by the operation equipment. The data template may depend on the operation equipment, the operation data types and / or the use of the operation equipment. The data template may be selected based on the operation equipment, the operation data types and / or the use of the operation equipment and retrieved from a template store. The data template may be used to process e.g. manipulate and / or transform the raw operation data as provided by the operation equipment. The raw or processed operation data may further be processed e.g. transformed to a compressed and / or encoded data storage format such as Apache Parquet to enhance performance in handling complex data in bulks. The processed operation data may be provided to sink storage(s) such as databases using different storage architectures configured for data marts, Online Analytical Processing (OLAP), Online Transaction Processing (OLTP), predictive analytics or the like. The selection of storage architecture such as block, object and / or file storage may depend on the latency and performance required for monitoring and / or controlling the distributed operation equipment. For example, the storage architecture may be selected based on data size and / or data transactions per second. The operation data stored in data sinks may be used for monitoring and / or controlling operation equipment.

[0049] To enable monitoring and / or controlling of distributed operation equipment in a simple, reliable and robust manner, the methods, apparatuses, systems and uses disclosed herein are based on a two-prong approach:

[0050] 1) Simplification of operation data retrieval by consolidating data in a way that enables interaction with the data in natural language (see e.g. Figs. 2 and 3).

[0051] 2) Robust and reliable action execution based on the operation data in a way that enables action execution in natural language (see e.g. Figs. 4 and 5).

[0052] Overall operators of operation equipment can hence interact with the operation equipment in an easy and fast manner. This allows operators of the operation equipment to swiftly access the status of the operation system and if deemed needed trigger action execution. For safety and security reasons the operator as the human user of the operation equipment may provide inputs in natural language and the operation equipment may respond in a human like manner in natural language. This way the monitoring and / or controlling of operation equipment can be enhanced over state-of-the-art monitoring and / or controlling systems that analyze and display data with limited interaction possibilities for operators of the operation equipment. Fig. 2 illustrates an example method for storing operation data associated with distributed operation equipment.

[0053] One or more source identification(s) of one or more data source(s) storing operation data and one or more sink identification(s) of one or more data sink(s) for storing processed operation data may be provided.

[0054] The identification(s) of one or more data source(s) storing operation data may relate at least to one or more connector(s), hostname(s), database(s), authentication information and / or authorization information. The identification(s) of one or more data source(s) storing operation data may specify the storage location of operation data e.g. per operation equipment and / or per operation equipment type. The identification(s) of one or more data source(s) storing operation data may relate to one or more source schema(s) and / or source table(s). The identification(s) of one or more data source(s) storing operation data may specify one or more source data packages to be transferred or provided to the sink storage(s).

[0055] The one or more sink identification(s) of one or more data sink(s) may relate at least to one or more connector(s), hostname(s), database(s), authentication information and / or authorization information. The one or more sink identification(s) of one or more data sink(s) may specify the destination location of processed operation data e.g. per operation equipment and / or per operation equipment type. The one or more sink identification(s) of one or more data sink(s) may relate to one or more sink schema(s) and / or sink table(s).

[0056] A relationship between the identification(s) of one or more data source(s) storing operation data and / or one or more sink identification(s) of one or more data sink(s) may be provided. The relationship may group one or more source schema(s) and / or source table(s) to one or more destination schema(s) and / or destination table(s).

[0057] The operation data from one or more data source(s) related to the provided source identification(s) may be accessed and processed according to one or more data templates for storage on an intermediate storage. The data templates may include pre-defined data structures and / or rules for structuring the operation data. The data templates may be stored in a template store and may be used to process raw operation data as provided by the distributed operation equipment e.g. as described in the context of Fig. 1. The processed data may be staged and / or stored on an intermediate storage layer.

[0058] One or more instruction(s) including at least one task instruction to generate an operation data description and one or more entries of the operation data to be described may be generated. The task instruction may relate to a data package or sub-set(s) of the data package of the operation data. The data package or sub-set(s) of the data package may relate to the stored data format, such as compressed or uncompressed formats or encoded or unencoded formats. The data package may include operation data stored in a structured data format such as a table. The subset of the data package may relate to at least one element, e.g. at least one column, and / or at least one row, of the structured data format. The task instruction may include a text in natural language specifying the task to describe the data package or sub-set(s) of the data package. The one or more entries of the operation data to be described may be generated based on or from the data package or sub-set(s) of the data package of the operation data. For example, one or more entries or values of the operation data to be described may be retrieved from the intermediate storage. Further for example, one or more entries or values data package or sub-set(s) of the data package of the operation data to be described may be retrieved from the intermediate storage. The one or more instruction(s) may include at least one system instruction relating to the system context of the task instruction. For example, the system instruction may relate to a characteristic of the system performing the task.

[0059] The one or more instruction(s) may be provided as prompt template. The prompt template may include instruction(s) in natural language and variables to be inserted for: the data package or sub-set(s) of the data package of the operation data as provided by the identification(s) and / or the retrieved operation data and / or one or more entries of the operation data to be described as provided by the retrieved operation data

[0060] The operation data description may be generated by providing the one or more instruction(s) to at least one data- driven model configured to generate the operation data description. The data-driven model may include a general- purpose model trained at least on unstructured data. The one or more instruction (s) may be provided to a data driven model together with configuration parameters for the data-driven model such as maximal token specification or authentication / authorization parameters. The data-driven model may be selected from one or more data-driven models based on a sensitivity and / or confidentiality level of the operation data. Further details of possible data-driven models and their selection are described for example in the context of Fig. 3 or Figs. 8-11 .

[0061] The operation data description may be stored as metadata of the processed operation data on the intermediate storage. The processed operation data may be copied to one or more data sink(s) related to the provided sink identification(s). The processing and the storing of the operation data may be logged and stored in a documentation storage.

[0062] Fig. 3 illustrates an example apparatus for storing operation data including prompt template for generating an operation data description.

[0063] The computing apparatus is configured for storing operation data including prompt template for generating an operation data description, in particular to perform the steps as described in the context of Fig. 2. The computing apparatus may include a processing apparatus communicatively coupled to one or more source storage(s), intermediate storage(s) and / or sink storage(s). The computing apparatus may be communicatively coupled to one or data driven model interfaces configured to access a data driven model and to provide the result of the processing by the data-driven model.

[0064] The processing apparatus may include an operation data interface configured to provide one or more source identification(s) of one or more data source(s) with stored operation data. The processing apparatus may include an operation data interface configured to provide one or more sink identification(s) of one or more data sink(s) for storing operation data.

[0065] The processing apparatus may include an operation data staging interface configured to access the operation data from one or more data source(s) related to the provided source identification(s)

[0066] The processing apparatus may include an operation data preparation interface configured to process the operation data according to one or more data templates for storage on an intermediate storage.

[0067] The processing apparatus may include an instruction agent configured to generate one or more instruction(s) including at least one task instruction to generate an operation data description and one or more entries of the operation data to be described. The processing apparatus may include an instruction agent configured to generate the operation data description by providing the one or more instruction(s) to a data-driven model configured to generate the operation data description.

[0068] The processing apparatus may include a write to storage interface configured to store the operation data description as metadata of the processed operation data on the intermediate storage. The processing apparatus may include a write to storage interface configured to copy the processed operation data from the intermediate storage to one or more data sink(s).

[0069] Fig. 4 illustrates an example method for monitoring and / or controlling operation equipment.

[0070] A user instruction may be provided including natural language or text such as a question.

[0071] One or more instruction templates relating to monitoring and / or controlling of operation equipment may be provided. The instruction templates may be configured for identifying operation data, accessing operation data, selecting machine-readable monitoring and / or controlling instructions and executing machine-readable monitoring and / or controlling instructions. The instruction templates may be stored in template store. The instruction templates may be based on natural language providing instruction(s) such as task instructions relating to identifying operation data, accessing operation data, selecting machine-readable monitoring and / or controlling instructions and executing machine-readable monitoring and / or controlling instructions.

[0072] One or more instruction(s) including at least one task instruction related to the monitoring and / or controlling of operation equipment based on access to operation data may be provided.

[0073] The one or more instruction(s) may include user instruction(s) provided by an operator of the operation equipment.

[0074] The user instruction(s) may be provided by a web-interface configured to retrieve instruction(s) in natural language by an operator of the operation equipment. An example web-interface and examples of operator instructions are illustrated and described e.g. in the context of Fig. 7.

[0075] The one or more instruction(s) may be generated based on template instructions as may be retrieved from a template store. The template instruction may be selected based on the user instruction. The one or more instruction(s) may be generated by embedding the user instruction into the template instruction. The one or more instruction(s) may be generated by embedding the further information as specified by the template instruction into the template instruction.

[0076] For selection of the template instruction, the user instruction may be embedded as described e.g. in the context of Figs. 8 and 9. Embedding in this context may include a mapping of at least part of the user instruction in natural language to a numeric representation of at least part of the user instruction. The numeric representation may also be referred to as embedding. The template instructions may relate to one or more instruction types. The instruction type may relate to identification of data, access to data, selection of machine-readable instructions and / or execution of machine-readable instruction. The template instruction(s) and / or the template instruction type(s) may be provided in natural language and / or as numeric representations of the natural language version. The template instruction(s) and / or the template instruction type(s) may be mapped in at least part from natural language to a numeric representation of at least part of the user instruction.

[0077] For selection of the instruction template the embedding of the user instruction and the embedding of the instruction template(s) and / or type(s) may be compared. The similarity between respective embeddings of the user instruction and the embedding of the instruction template(s) and / or type(s) may be determined. The similarity may be based on any similarity measure known in the art such as Euclidian od cos similarity measures. The embedded template instruction with the lowest distance to the embedded user instruction may be selected.

[0078] Further for example, instructions(s) may be generated including the task instruction to select one template instruction, the user instruction, the template instructions and / or the template instruction types. The generated instructions may be provided to the data-driven model configured to provide the selected template instruction.

[0079] These are only some possibilities for selecting the template instruction. Other possibilities may rely on simple string or semantic comparison. If no suitable template instruction may can be found according to a pre-defined similarity measure, a signal may be sent to the web-interface for asking the operator to further specify the user instruction.

[0080] The template instruction may relate to identifying operation data. The template instruction for identifying data may relate to generating representation of stored or accessible operation data. The template instruction for identifying data may include at least one task instruction relating to the selection of data packages based on the data package identifier and / or the data package schema such as table identifiers and / or schemas. The template instruction for identifying data may include one or more operation data descriptions of stored operation data. The operation data may be stored including operation data description as metadata as e.g. described in the context of Figs. 2 and 3. The template instruction for identifying data may include one or more data package descriptions for stored operation data. The template instruction for identifying data may include task instruction relating to selection of operation data based on the user instruction. The template instruction for identifying data may include system instructions relating to the output structure e.g. as semi structured data in natural language or conditions to be fulfilled by the output such as the instruction to provide output in json format or the instruction to restrict selection to specified one or more operation data descriptions of stored operation data.

[0081] The template instruction may relate to accessing or retrieving operation data. The template instruction for accessing or retrieving data may relate to the task instruction of generating machine-readable instruction in database language format suitable to access operation data. The template instruction for accessing or retrieving data may relate to the output provided by the data-driven model based on the template instruction may relate to accessing or retrieving operation data. The template instruction for accessing or retrieving data may relate to one or more operation data descriptions of stored operation data corresponding to the output provided by the data-driven model based on the template instruction for accessing or retrieving operation data. The template instruction for accessing or retrieving data may relate to one or more examples of machine-readable instruction in database language format suitable to access operation data. The template instruction for accessing data may include task instruction relating to generation of machine-readable instruction in database language format suitable to access operation data based on the user instruction. The template instruction for accessing data may include system instructions relating to the output structure e.g. as semi structured data in data base language such as SQL query, guidance on output generation such as SQL query generation, or conditions to be fulfilled by the output such as limitation to specified one or more operation data descriptions of stored operation data.

[0082] The template instruction may relate to selecting machine-readable monitoring and / or controlling instructions. The template instruction for identifying data may include at least one task instruction relating to selection of a workflow associated with machine-readable monitoring and / or controlling instructions. The template instruction for selecting machine-readable monitoring and / or controlling instructions may include specification of available workflows. The template instruction for selecting machine-readable monitoring and / or controlling instructions may include specification of workflow descriptions e.g. in natural language. The template instruction for selecting machine- readable monitoring and / or controlling instructions may include task instruction relating to selection of workflow based on the user instruction. The template instruction for identifying data may include system instructions relating to the output structure e.g. as semi structured data in natural language or conditions to be fulfilled by the output such as the instruction to provide output in specific format or the instruction to restrict selection to specified one or more workflow descriptions of provided workflows.

[0083] The template instruction may relate to executing machine-readable monitoring and / or controlling instructions. The template instruction may include task instruction to extract attributes required for execution of machine-readable monitoring and / or controlling instructions. The template instruction may include selected specification of machine- readable monitoring and / or controlling instruction(s) corresponding to the output provided by the data-driven model based on the template instruction for selecting machine-readable monitoring and / or controlling instructions. The template instruction may include specification to include historic outputs, e.g. to instructions for identifying or accessing or retrieving operation data. The template instruction for execution of machine-readable monitoring and / or controlling instructions may include task instruction relating to execution of workflow based on the user instruction. The template instruction for execution of machine-readable monitoring and / or controlling instructions may include system instructions relating to the output structure or conditions to be fulfilled by the output or the instruction to restrict execution to fully defined attribute sets.

[0084] At least one operation instruction for monitoring and / or controlling of operation equipment may be generated by providing the one or more generated instruction(s) to a data-driven model configured to generate the at least one operation instructions. The data-driven model may include a general-purpose model trained at least on unstructured data. The one or more instruction(s) may be provided to the data driven model together with configuration parameters for the data-driven model such as maximal token specification or authentication / authorization parameters. The data-driven model may be selected from one or more data-driven models based on a sensitivity and / or confidentiality level of the operation data. Further details of possible data-driven models and their selection are described for example in the context of Fig. 3 or Figs. 8-11 .

[0085] The at least one operation instruction for monitoring and / or controlling of the operation equipment may be provided. If the operation instruction is an execution instruction, the operation instruction may include parsing results to the machine-executable function to be executed and / or triggering execution of the machine-executable function for monitoring and / or controlling of the operation equipment. If the operation instruction is an identification, access or selection instruction, the operation instruction may relate to displaying information to the user or to parsing results to a subsequent instruction or instruction template or function to be executed.

[0086] The at least one operation instruction may trigger monitoring and / or controlling of the operation equipment by performing one or more machine-executable functions. For example, the operation instruction may trigger shutdown of the operation equipment. Further for example, the operation instruction may trigger setting an operation condition of the operation equipment. Further for example, the operation instruction may trigger providing a monitoring signal or status information of the operation equipment to the operator. Further for example, the operation instruction may trigger providing a monitoring signal or status information of the operation equipment to the user of the operation equipment. The user may for example include the end user, while the operator may for example include any other entity responsible for monitoring and / or controlling the operation equipment.

[0087] Fig. 5 illustrates example apparatus for monitoring and / or controlling operation equipment including prompts for identifying operation data, accessing operation data, selecting machine-readable monitoring and / or controlling instructions and executing machine-readable monitoring and / or controlling instructions. The computing apparatus is configured for monitoring and / or controlling operation equipment including prompts for identifying operation data, accessing operation data, selecting machine-readable monitoring and / or controlling instructions and executing machine-readable monitoring and / or controlling instructions, in particular to perform the steps as described in the context of Fig. 2. The computing apparatus may be communicatively coupled to a webinterface for user interaction. The computing apparatus may include a monitoring and / or controlling apparatus communicatively coupled to one or more sink storage(s) storing operation data including descriptions of operation data e.g. in natural language, template store(s) and / or operation equipment. The computing apparatus may be communicatively coupled to one or data driven model interfaces configured to access the data driven model and to provide the result of the processing by the data-driven model.

[0088] The computing apparatus may comprise a selection engine configured to generate on one or more instruction(s) based on template instructions as may be retrieved from a template store. The selection engine may be configured to select template instruction based on the user instruction. The selection engine may be configured to generate the one or more instruction(s) by embedding the user instruction and / or further information as specified by the template instruction into the template instruction. The selection engine may be configured to perform the steps as described in the context of Fig. 4. The selection engine may be configured to provide the operation instructions to the webinterface, to the monitoring and / or controlling engine and / or to the retrieval engine.

[0089] The computing apparatus may comprise a retrieval engine configured to identify and / or retrieve operation data based on template instructions as may be retrieved from a template store. The retrieval engine may be configured to identify operation data by providing instruction(s) generated based on the instruction template for identification of operation data to the data-driven model and by providing the operation instructions received from the data-driven model. The retrieval engine may be configured to parse at least part of the operation instructions to the instruction template for retrieval of operation data. The retrieval engine may be configured to retrieve operation data by providing instruction(s) generated based on the instruction template for retrieval of operation data to the data-driven model and by providing the operation instructions received from the data-driven model. The retrieval engine may be configured to provide the operation instructions to the web-interface, to the selection engine and / or to the monitoring and / or controlling engine.

[0090] The computing apparatus may comprise a monitoring and / or controlling engine configured to select and / or execute machine-readable monitoring and / or controlling instructions based on template instructions as may be retrieved from a template store. The monitoring and / or controlling engine may be configured to select machine-readable monitoring and / or controlling instructions by providing instruction(s) generated based on the instruction template for selecting machine-readable monitoring and / or controlling instructions to the data-driven model and by providing the operation instructions received from the data-driven model. The monitoring and / or controlling engine may be configured to parse at least part of the operation instructions to the instruction template for executing monitoring and / or controlling instructions. The monitoring and / or controlling engine may be configured to execute machine-readable monitoring and / or controlling instructions by providing instruction(s) generated based on the instruction template for executing machine-readable monitoring and / or controlling instructions to the data-driven model and by providing and / or executing the operation instructions received from the data-driven model. The monitoring and / or controlling engine may be configured execute the operation instructions and to provide the result of such execution to the operation equipment. The monitoring and / or controlling engine may be configured provide the operation instructions to operation equipment for execution of the operation instructions.

[0091] The computing apparatus may comprise an instruction engine configured to provide the generated instructions from any of the selection engine, the retrieval engine and / or the monitoring and / or controlling engine to the data-driven model. The instruction engine may be configured to select one or more data-driven models based on execution performance like compute costs, time, and / or accuracy.

[0092] Fig. 6 illustrates an example of method steps chained by the prompts illustrated in Fig. 5.

[0093] The flows of Fig. 6 illustrate the computing flows and system behavior in relation to the inputs from and the outputs to the user. The first flow relates to identifying operation data based on the prompt for identifying operation data as provided by the user. The second flow relates to retrieving operation data based on the prompt for retrieving operation data as provided by the user. The third flow relates to workflow execution based on the prompt for workflow execution as provided by the user.

[0094] Fig. 7 illustrates example user interfaces for workflow execution.

[0095] The user interface may be a web interface accessing the services described in the context of Figs. 1 to 6. The user interface may be configured for interaction by the operator of distributed operation equipment in natural language. The operator of the operation equipment may pose questions in natural language or prompt. The user entry may be provided to the apparatus or methods for monitoring and / or controlling operation equipment as described in the context of Figs. 4-6. The apparatus or methods for monitoring and / or controlling operation equipment may use the user prompt as input to perform the steps of the methods or by the apparatus as described in the context of Figs. 4-6. Based on such processing of the user prompt the system may provide an output in natural language as illustrated in Fig. 7. Depending on the prompt provided by the user the system may provide services for identifying operation data, accessing operation data, selecting machine-readable monitoring and / or controlling instructions and executing machine-readable monitoring and / or controlling instructions.

[0096] Figs. 8-11 illustrate an example transformer architecture of a general-purpose model. Fig. 8 illustrates an embodiment of training an embedding layer.

[0097] The embedding layer may be obtained by training for example a continuous bag of words model (CBOW) or a skipgram model. The embedding layer may be suitable for generating embedded input data based on input data. Generating embedded input data may refer to embedding input data. Embedding input data may result in a representation associated with the input data. Thus, the embedded input 814 may be the representation associated with the input data. The input data may comprise one or more elements. The one or more elements may be represented by the input vector 806. In particular, the embedded input 814 and / or the input vector 806 may be machine-readable and / or processable by a processor. For this purpose, the embedded input 814 and / or the input vector 806 may be a tensor, in particular a first-rank tensor. Specifically, the input vector 806 may be a one hot vector or a summation of a plurality of one hot vectors. A one hot vector may be a vector with one entry unequal to zero. Examples for one hot vectors may be 808, 810 and 812. The entries unequal to zero in the one hot vector and / or in the input vector 806 may indicate the element. For example, a look up table may define the relation between the position of the entries unequal to zero and the element indicated by the one hot vector. The look up table may specify a plurality of different elements. The number of different elements may be equal to the number of entries in the one hot vector. The number of different elements may be referred to as vocabulary size. In an example, the elements may be represented by tokens and a sequence of elements may refer to at least a part of a sentence. The at least a part of the sentence may be represented by a plurality of tokens. A token may represent at least a part of the element and / or word. For example, where one element would be associated with only one word, words such as "embeddings", "embedding” or "embed” would constitute different elements. A first token may represent the stem "embed” and the endings, typically appearing in a plurality of word, may be represented by a second token, a third token and a fourth token. The second token, the third token and the fourth token may be used for representing other words such as "look”, "looking” or the like, preferably together with a fifth token representing the stem "look”. Ultimately, this tokenization of elements associated with a plurality of stems and a plurality of endings results in less tokens to be used for representing a plurality of elements and thus, uses less computational resources.

[0098] A look up table specifying a subset of the vocabulary size e.g. of the English language may comprise 10,000 words or more. The embedded input 814 may be a lower-dimensional representation than the input vector 806. For example, typical embedded inputs 814 may comprise some hundreds of different entries. Followingly, the embedded inputs 814 constitute a densified representation of one or more elements using less computational resources. More than that, the embedded input 814 may represent a relation between two or more elements. For example, the words "Italy” and "Germany” may be similar or may be more closely related since they both define european countries, whereas the word "embodiment” may be very different from the two respective words. The smaller the dot product between two embedded inputs 814 may be the more similar the two elements associated with the embedded inputs 814 may be. Hence, the embedded inputs 814 may represent one or more elements accurately and lead to accurate results based on processing the embedded inputs 814.

[0099] For transforming the input vector 806 into the embedded input 814, the embedding layer may comprise a number of neurons equal to the number of entries in the embedded input 814. Based on the embedded inputs 814, the output layer may generate the output vector 816. The output vector may be a vector and / or may indicate one or more elements. The output vector 816 may indicate one or more elements different from the input vector 806 and / or the one hot vectors associated with the input vector 806. For this purpose, the output layer may comprise a number of neurons equal to the number of entries of the input vector 806 and / or the output vector 816. The output layer may apply a softmax function to the embedded inputs 814. By doing so, the output vector may comprise the probabilities associated with the elements associated with the entries of the output vector 816 unequal to zero. Hence, from the output vector 816 one or more elements may be obtained with a corresponding probability. Where the input vector 806 may specify one or more sequence(s) of elements, the output vector 816 may specify one or more elements corresponding to the sequence(s) of elements specified by the input vector 806. In the example of FIG. 8, the element associated with vector 818 may correspond to the input vector with a probability of 71 %. Additional or alternative elements may correspond to the input vector as indicated by the output vector with lower probability. By defining a threshold to which the probability may be compared, the selection of the corresponding elements may be tailored to the needs of the user. The elements generated by the model comprising the embedding layer 802 and the output layer 804 may refer to the most probable elements indicated by the output vector 816. Hence, the model depicted in FIG. 8 may generate the element associated with the vector 818 with a confidence score of 71 %.

[0100] The model of FIG. 8 may be continuous bag of words (CBOW) model. The CBOW model may be trained based on a training data set comprising a plurality of input vectors and corresponding output vectors. As the training data set may not be labeled, the training of the CBOW model may be referred to as self-supervised. Before training of the CBOW model, the CBOW model may be initialized with random values assigned to the weights of the neurons. During the training of the CBOW model, the input vectors may be passed through the initialized embedding layer and the output layer and a loss may be determined by comparing the output vector obtained by passing the input vector 806 through the model to the output vector corresponding to the input vector 806 as specified by the training data set. Based on the determined loss, backpropagation may be applied to determine the gradients associated with the neurons of the embedding layer 802 and the output layer 804 to lower the loss. According to the determined gradients, the weights of the neurons may be updated by using a gradient descent algorithm. If a predetermined loss may be achieved by the CBOW model, the training may be terminated and a trained CBOW model may be obtained. From the trained CBOW model, the embedding layer 802 may be suitable for embedding input data comprising one or more elements. This embedding layer 802 may be used in other machine-learning architectures requiring an embedding layer 802 such as a transformer encoder, transformer decoder or transformer encoder decoder architecture as described within the context of FIG. 9A, FIG. 9B and FIG. 9C. For training these architectures, a trained embedding layer 802 may be required. Hence, a model such as a CBOW model may be trained prior to training the transformer encoder, transformer decoder or transformer encoder decoder architecture.

[0101] Fig. 9A illustrates an embodiment of a transformer encoder architecture.

[0102] The transformer encoder comprises an encoder input 978, one or more encoder blocks 974, 914 and an encoder output. The transformer encoder architecture may be derived from the transformer encoder-decoder architecture as known in the art and shown in FIG. 9C. In particular, the transformer encoder may be referred to as X-former. The transformer encoder architecture may correspond to the encoder architecture associated with the transformer encoder-decoder architecture with an additional encoder output instead of connecting the encoder block directly to the decoder of the transformer encoder-decoder architecture. A plurality of transformer encoder architectures are available in the art such as the bi-directional encoder representations from transformers (BERT).

[0103] The input data may be received at the encoder input 978. The input data may be contextualized chemical product data. The encoder input 978 may apply an input embedding 902. Applying the input embedding 902 may refer to passing the input data through an embedding layer e.g. as described within the context of FIG. 8. Applying the input embedding 902 to the contextualized chemical product data may result in embedded contextualized chemical product data.

[0104] The encoder input 978 may apply positional encoding 904. Applying positional encoding 904 may refer to adding a positional factor to the embedded input obtained via input embedding. Preferably, the input data may specify a sequence of elements. The positional factor ?i»« may be indicative of the position of the elements within the sequence. For example, the positional factor may be obtained based on the following equation: where pos may refer to the position of the element within the sequence, I may refer to the dimension associated with the input embedding and d may refer to the dimension of the model, e.g. transformer decoder, transformer encoder or transformer encoder-decoder. This may be referred to as absolute positional embeddings. Alternatively, the positional encoding may be based on rotary positional embeddings (RoPE). Positional encoding is beneficial since it enables the processing of sequential data without requiring further dimensions indicating the position of each element. Followingly, the positional encoding 904 reduces the computational resources needed for embedding the input data. By passing the input data through the encoder input, the input data may be transformed into a second-rank tensor representing the sequence of elements. This second-rank tensor may be referred to as embedded input data. The embedded input data may be processed by the encoder block. The embedded input data may be provided to the layer normalization 908 by a residual connection. Multi-head self attention 906 may be applied to the embedded input data. Multi-head self attention 906 may comprise the two components multi-head and self-attention. Self-attention may be understood as being a filter applied to the embedded input data. By applying the filter to the embedded input data, the elements associated with the embedded input data contributing to the to be generated output data may be identified for generating the output data. Hence, the filter may represent the degree of contributing to the to be generated output data by the elements associated with the embedded input data. Applying the filter may be referred to as weighting the elements associated with the embedded input data. This is advantageous specifically regarding long sequences of elements. The filter may be learned and improved during the training by learning to identify the contribution of elements associated with the embedded input data. For example, in the partial sentence "I went to the bakery to buy a” the last word may be generated by the data-driven model such as the transformer encoder. The self attention may focus the transformer encoder to attend to the word "bakery” and "buy” mostly to generate the word "bread”. Self attention may refer to attention generated based on the input data. Hence, the filter may be determined based on the input data, preferably the embedded input data. The embedded input data may serve as query Q, key K and value V with respect to the self attention operation. The self attention may refer to attention based on the received input data. Hence, the filter may be calculated based on the following formula by inserting the respective tensors based on the embedded input data: where dfc corresponds to the dimension of the key.

[0105] For improving the efficiency of the transformer encoder further, the multiple heads are used to apply the filter resulting in the multi-head self attention 906. Multi-head self attention 906 may comprise applying the filter to two or more parts of the embedded input data. Hence, the tensor may be split into two or more parts and the filter may be applied to the two or more parts separately by two or more heads according to the following equation:head i =Attention(QWiQ, KWiK, VWiv) with parameter matrices where i may refer to the number of heads, dy, dg and ^<3 may refer to the dimensions of the value, key and query.

[0106] The result of the two or more head may be concatenated according to the following equation: MultiHead(Q, K, F) = Concat (head 1, . . . , headhW° j xd and h may refer to the number of heads.

[0107] The embedded input data may be transformed via the multi-head self attention 906 into a context tensor. The context tensor may represent the sequence of elements and the relation between two or more elements of the input data. Transforming the embedded contextualized chemical product data may result in chemical product and environmental property data context tensor 622. Transforming the embedded contextualized chemical product data may result in chemical product data context tensor 620. Hence, the context tensor may be chemical product data context tensor 620 and / or chemical product and environmental property data context tensor 622. The context tensor may be a second rank tensor and / or may comprise one or more first rank tensor(s). After the multi-head self attention 906 layer normalization 908 may be applied based on the context tensor and / or the embedded input data from the residual connection. Applying layer normalization 908 may refer to normalizing the context tensor. Normalizing the context tensor may lower the values of the entries of the context tensor. This reduces the computational cost associated with processing the context tensor. Layer normalization 908 may be followed by passing the context tensor to a feed forward layer 910 again followed by layer normalization 912 based on the residual connection to the context tensor and / or the output of the feed forward layer 910. The feed forward layer 910 may be a feed-forward neural network. The feed-forward neural network may comprise of a plurality of fully connected neurons. Passing the context tensor through the feed-forward neural network may result in transforming the context tensor linearly. Additionally or alternatively, the neural network may comprise one or more activation functions such as a rectified linear unit (ReLU). Hence, the neural network may be configured for performing one or more non-linear operations to the context tensor and / or transforming the context tensor non-li nearly . After the context tensor has been transformed and / or normalized by the feed forward layer 910 and the layer normalization 912, the context tensor may be provided to one or more further encoder blocks 914. Having passed the context tensor through the feed forward layer 910 may adapt the context tensor for the processing by a further attention layer of the one or more further encoder blocks 914 for applying a self attention filter, preferably multi-head self attention 906. The context vector after being transformed by the layer normalization 912 and the feed forward layer 910 may be referred to as hidden state.

[0108] The encoder output 976 comprises of a linear layer 916 and a softmax layer 918. The linear layer 916 may transform the context vector into a logits vector. The linear layer may be fully-connected. The logits vector obtained by passing the context tensor through the linear layer 916 may be passed through the softmax layer 918. Passing the logits vector through the softmax layer 918 may refer to applying the softmax function to the logits vector. Applying the softmax function to the logits vector may result in a probability distribution of one or more elements corresponding to the sequence of elements in the input data. From the probability distribution based on predefined selection criteria, one or more elements may be chosen. The one or more chosen elements may be referred to as the one or more elements generated by the transformer encoder. The one or more generated elements may be provided to the encoder input for generating further one or more elements corresponding to the sequence of the input data and the one or more elements generated by the transformer encoder as described within the context of FIG. 10.

[0109] The output data from the encoder output 976 may be chemical product production and / or processing data. Hence, the result of transforming the chemical product data context tensor 620 may be chemical product production and / or processing data.

[0110] FIG. 9B illustrates an embodiment of a transformer decoder architecture. Input data, embedded input data, context tensor and / or output data may be as defined within the context of FIG. 9A.

[0111] The transformer decoder comprises a decoder input 984, one or more decoder blocks 980, 932 and a decoder output 992. The transformer decoder architecture may be derived from the transformer encoder-decoder architecture as known in the art and shown in FIG. 9C. The transformer decoder may be referred to as X- former. The transformer decoder architecture may correspond to the decoder architecture associated with the transformer encoder-decoder architecture independent of receiving one or more hidden states from the encoder of the transformer encoder-decoder. A plurality of transformer decoder architectures are available in the art such as the generalized pretrained transformers (GPT).

[0112] The decoder input 984 may apply input embedding 920 and positional encoding 922 analogous to analogous to the input embedding 902 and the positional encoding 904 as described within the context of FIG. 9A.

[0113] The decoder block 980 may comprise the layer normalizations 926, the masked multi-head self attention 924, the feed forward layers 928 and / or the layer normalization 930. The embedded input data resulting from passing the input data through the decoder input 984 may be provided to the layer normalization 926 via a residual connection. Further, masked multi-head self attention 924 may be applied to the embedded input data. Masked multi-head self attention 924 corresponds to the multi-head self attention 906 as described within the context of FIG. 9A with additionally masking a part of the embedded input data associated with elements later in the sequence than the element to be generated. Additionally or alternatively, the part of the input data associated with elements later in the sequence than the element to be generated may not be received and / or transformed into the embedded input data. Thus, the transformer decoder may be suitable for generating a subsequent element to a sequence, whereas the transformer encoder may be suitable for generating a missing element in within one sequence and / or between two or more sequences. Therefore, the transformer encoder may be configured for classification tasks. The transformer decoder may be configured for text generation.

[0114] Similar to the transformer encoder as described within the context of FIG. 9A, a context tensor may be generated by applying the masked multi-head self attention 924 and the layer normalization 926. The context tensor may be provided to the layer normalization 930 via a residual connection. Further, the feed forward layer 928 and the layer normalization 930 may be analogous to the feed forward layer 910 and the layer normalization 912 as described within the context of FIG. 9A. The context tensor may be provided to one or more further decoder blocks 932.

[0115] The decoder output 992 may comprise of a linear layer 934 and a softmax layer 936. The linear layer 934 and the softmax layer 936 may be analogous to the linear layer 916 and the softmax layer 918 as described within the context of FIG. 9A.

[0116] FIG. 9C illustrates an embodiment of a transformer encoder-decoder architecture. Input data, embedded input data, context tensor and / or output data may be as defined within the context of FIG. 9A.

[0117] The transformer encoder-decoder may comprise the encoder input 988, the one or more encoder blocks 986, 964, the decoder input 994, the decoder block 990 and the decoder output 992. The encoder input 988 may correspond to the encoder input 978 of FIG. 9A. The one or more encoder block 986, 964 may correspond to the one or more encoder blocks 974, 914 of FIG. 9A. The decoder input 994 may correspond to the decoder input 984 of FIG. 9B.

[0118] The decoder block 990 may comprise a masked multi-head self attention 970, a layer normalization 972, a feed forward layer 938 and a layer normalization 940 analogous to the masked multi-head self attention 924, the layer normalization 926, the feed forward layer 928 and the layer normalization 930 as described within the context of FIG. 9B. The decoder block 990 may further comprise a multi-head self attention 950 and a layer normalization 948. Analogous to the description of FIG. 9B, the context tensor may be obtained from the masked multi-head self attention 970 and the layer normalization 972. Multi-head self attention 950 analogous to the multi-head self attention 906 of FIG. 9A may be applied to the context vector obtained from the layer normalization 972 and the hidden states of the one or more encoder blocks 986, 964. Layer normalization 948 may be applied to the context vector obtained from the multi-head self attention 950 and the context vector obtained from the layer normalization 972 provided via a residual connection. The context vector resulting from the layer normalization 948 may be processed via the feed forward layer 938 and the layer normalization 940 analogous to the description of FIG. 9B. The context vector resulting from the layer normalization 940 may be provided to further decoder blocks 942 analogous to the decoder block 990. The context vector obtained from the one or more decoder blocks 990, 942 may be provided to the decoder output 992. The decoder output 992 may correspond to the decoder output 982 of FIG. 9B.

[0119] With the above-described architecture, the transformer encoder-decoder may receive and process input data at the encoder input 988 and the one or more encoder blocks 986, 964 and the decoder block 990 and the decoder output 992. Based on the input data, the transformer encoder-decoder may generate output data part by part or sequentially. The sequentially generated output data may be provided to and / or may be processed by the decoder input 994, the one or more decoder blocks 990, 942 and the decoder output 992. Preferably, a sequence may be provided to the encoder input 988 and after having generated at least a part of the output data, the decoder input 994 may be provided with at least the part of the elements of the output data already generated. By doing so, the next elements of the output data may be generated with a higher accuracy by taking the input data and the generated output data into account since more data is received by the transformer encoder-decoder may be received over time.

[0120] Because of the transformer encoder-decoder architecture, the transformer encoder-decoder may be configured for transforming a sequence into another representation of the sequence. An example for transforming one sequence into another representation may be translation of one sentence into another language. A plurality of transformer encoder-decoders are available in the art such as BART, T5 or the like.

[0121] In an embodiment, the layer normalization 908, 912 may be applied prior to the masked multi-head self attention 924, multi-head self attention 906 and / or the feed forward layer 910 in the transformer decoder, the transformer encoder and / or the transformer encoder-decoder. By doing so, the computational resources for applying the multi-head self attention 906 and / or the feed forward layer 910 to the embedded input data and / or the context tensor may be decreased as the entries of the respective tensors may be lower after normalization.

[0122] In an embodiment, the decoder output 992 may comprise of a classification neural network, further feedforward layers, convolutional layers, fully connected layers or the like. For example, the transformer encoder-decoder may be configured for choosing between a plurality of options. For this purpose, the transformer encoder-decoder may be provided with three different input data sets and may classify the context vectors obtained from the one or more decoder blocks 990 via one or more linear layers. Fol lowingly , the architecture may be extended depending on the use case to be solved.

[0123] FIG. 10 illustrates an embodiment of training and / or deploying the transformer encoder, the transformer decoder and / or the transformer encoder-decoder.

[0124] The encoder / decoder / encoder-decoder architecture 1002 may correspond to the transformer decoder, the transformer encoder and / or the transformer encoder-decoder as describe within the context of FIG. 9A- FIG. 9C. Input data, embedded input data, context tensor and / or output data may be as defined within the context of FIG. 9A.

[0125] The output data generated by the encoder / decoder / encoder-decoder architecture 1002 may comprise of one or more elements, in particular a sequence of elements. The previously generated elements of the output data may be provided as input for generating the next element in the sequence of the output data.

[0126] In the example of FIG. 10, the input data may comprise of N elements, in particular input tokens. An input token may be a token dedicated to be inputted into a data-driven model such as the transformer decoder, the transformer encoder or the transformer encoder-decoder. The output data to be generated may comprise of M elements. The encoder / decoder / encoder-decoder architecture 1002 may generate one element of the output data based on receiving the input data and optionally previously generated elements of the output data at a timestep. Hence, for generating M elements M time steps are required. A time step comprises of providing input 1010, 1012, 1014 to the encoder / decoder / encoder-decoder architecture 1002 and receiving output data 1004, 1008, 1006 from the encoder / decoder / encoder-decoder architecture 1002. In a first timestep, the input 1010 may comprise of N input tokens. The N input tokens may be associated e.g. with N words, stems or endings. Preferably, the N input tokens may specify a question. One or more input tokens may specify the beginning of the sequence of tokens and / or the end of the sequence of tokens. The input 1010 may be processed by the encoder / decoder / encoder-decoder architecture 1002. Based on the input 1010 at least a part of the output data 1004 may be generated. The at least a part of the output data may comprise a first output token. In the next timestep, the generated first output token may be provided together with the input 1012. Specifically, where the input 1012 may be received by a transformer encoder-decoder the input tokens may be received at the encoder input 988 and the first output token may be received at the decoder input 994. Where the input 1012 may be received by the transformer encoder, the input 1012 may be received by the encoder input 978 and analogously regarding the transformer decoder and the decoder input 984. Based on the input 1012, the output data 1008 comprising the first output token and a second output token may be generated. Generating the output data 1008 based on the input 1012 may refer to generating the second token based on the first token and the N input tokens, wherein the first token may have been generated based on the N input tokens. This process may be repeated until the last token in the sequence of the output data 1006 may be generated. Preferably, the last token may be an end token. The end token may terminate the generation of a further output token.

[0127] Similarly, to the data processing during deployment of the encoder / decoder / encoder-decoder architecture 1002, the encoder / decoder / encoder-decoder architecture 1002 may be trained. The training data set may comprise a plurality of sequences comprising a plurality of elements. The sequences may be associated with the input data and / or the output data. Additionally or alternatively, the sequences may be independent of the input data and / or the output data. For example, where the input data and the output data may refer to chemical compositions represented via text, the training data set may comprise sequential text data independent of chemical compositions. In this example, the training data set may comprise sequences of words originating from a conversation. In an embodiment, the training data set may comprise at least partially input data sets and / or output data sets.

[0128] The training may be initialized by initializing the encoder / decoder / encoder-decoder architecture 1002. In an embodiment, the parameters associated with the encoder / decoder / encoder-decoder architecture 1002 may be initialized randomly. Additionally or alternatively, the input embedding of the encoder / decoder / encoder- decoder architecture 1002 may be obtained by training a CBOW model or a skip gram model as described within the context of FIG. 8. The trained embedding layer may be used during training. The parameters associated with the embedding layer may be kept constant and / or may be updated after a predefined number of training epochs. By doing so, the number of parameters to be updated is lower enabling a faster and less computational resources-consuming training. Further, the accuracy associated with the embedding layer may be constant and / or may be increased by avoiding error compensation in relation to the just initialized encoder / decoder / encoder-decoder architecture 1002.

[0129] During the training of the encoder / decoder / encoder-decoder architecture 1002, at least a part of the sequences of the training data set may be provided to the encoder / decoder / encoder-decoder architecture 1002 one by another and one or more elements may be generated based on the sequences of the training data set one by another. The elements generated based on the sequences may follow the elements of the parts of sequences the encoder / decoder / encoder-decoder architecture 1002 may have been provided with. The generated one or more elements may be compared to the one or more elements following the at least a part of the sequences provided to the encoder / decoder / encoder-decoder architecture 1002 as specified by the training data set. Hence, during the training the encoder / decoder / encoder-decoder architecture 1002 may generate a guess on the next element and the guess on the next element in a sequence may be compared to the ground truth specifying the actual next element according to the training data set. Based on the guess on the next element and the ground truth a loss may be determined. The loss may define the similarity between the guess on the next element and the ground truth. The loss may be determined by forming a vector dot product between the token associated with the one or more elements and the token associated with the ground truth. A loss unequal to zero may result in updating the parameters associated with encoder / decoder / encoder-decoder architecture 1002. Preferably the parameters associated with the encoder / decoder / encoder-decoder architecture 1002 may be independent of the embedding layer. For example, the parameters associated with the encoder / decoder / encoder-decoder architecture 1002 may be weights of the neurons of the encoder / decoder / encoder-decoder architecture 1002.

[0130] Based on the determined loss, backpropagation may be applied to determine the gradients associated with the parameters of the parameters associated with encoder / decoder / encoder-decoder architecture 1002 to lower the loss. According to the determined gradients, the parameters associated with the encoder / decoder / encoder-decoder architecture 1002, preferably the weights of the neurons associated with the encoder / decoder / encoder-decoder architecture 1002, may be updated by using a gradient descent algorithm.

[0131] The training data set may be unlabeled. The sequences of elements within the training data set may inherently comprise the ground truth for determining the loss with respect to the one or more elements generated during the training of the encoder / decoder / encoder-decoder architecture 1002. Hence, the encoder / decoder / encoder- decoder architecture 1002 may be trained self-supervised. This is advantageous since time and resources for creating a labeled training data set may be saved. Furthermore, this enables the usage of large training data sets associated with a size of several tera bytes. Consequently, the data-driven model may be accurate in generating elements of a sequence. In addition, the large training data set enables few shot predictions or even zero shot predictions. Hence, the data-driven models trained as described above are versatile contributing to saving resources needed for training and / or hosting a plurality of purpose-driven models such as CNNs. The training described above may be referred to as pretraining. The data-driven model may be configured for performing few shot or even zero shot predictions with respect to a plurality of use cases after pretraining. The performance of the data-driven model may be increased further by additional training referred to as finetuning.

[0132] FIG. 11 illustrates an embodiment of input embedding. Input data, embedded input data, context tensor and / or output data may be as defined within the context of FIG. 9A.

[0133] Where the sequence of elements associated with the input data, preferably comprised in the input data, may be of one type, the input embedding 902, 920, 952, 966 as described within the context of FIG. 9A - 2C may be used. For example, a type of input data may be text where the elements may be associated with at least a part of a word, a punctuation character, a start token specifying the beginning of one or more sequences associated with the input data and / or the end token. In another example, the input data may be at least partially numerical. Hence, the input data may comprise a plurality of numbers. Numerical input data may be for example tabular data. Tabular data may specify one or more rows and / or one or more columns. Hence, the tabular data may comprise one or more cells, wherein the cells may be associated with one or more numerical values.

[0134] Numerical input data may require a different embedding than text input data. Input embeddings for numerical input data may comprise a token embedding, a positional embedding, a column embedding, a row embedding or a combination thereof.

[0135] Applying a token embedding to one or more elements, in particular tokens associated with the input data may result in a machine-processable representation associated with the one or more elements, in particular tokens. Applying the token embedding to one or more elements may refer to passing the one or more elements through the embedding layer, e.g. as described within the context of FIG. 8. Hence, token embeddings may specify the one or more elements, in particular tokens in a machine-processable representation. For example, the token embedding may transform a numerical value into a vector. This is advantageous since this representation can be enriched by further information such as the position of the token within the sequence and / or within a table associated with the sequence of tokens. The positional embedding may be analogous to the positional embedding as described within the context of FIG. 8, FIG. 9A- 2C. Where the input data may be tabular data, column embedding may be applied. Applying a column embedding to one or more elements, in particular tokens associated with the input data may result in a machine-processable representation specifying the location of the one or more elements within a table 1102, preferably within the columns of the table 1102. Applying the column embedding may refer to adding a column factor to the input data embedded via token embeddings, in particular the embedded input data. The column factor may be the same for elements associated with the same column and / or may differ between two or more elements associated with different columns. Analogous, row embeddings may be applied where the input data may be tabular data. Applying a row embedding to one or more elements, in particular tokens associated with the input data may result in a machine-processable representation specifying the location of the one or more elements within a table 1102, preferably within the rows of the table 1102. Applying the row embedding may refer to adding a column factor to the input data embedded via token embeddings, in particular the embedded input data. The row factor may be the same for elements associated with the same row and / or may differ between two or more elements associated with different rows.

[0136] In an embodiment, input data may be at least partially numerical and at least partially text. Hence, the input data may comprise two or more types of data. A type of data may refer to a modality. Followingly, different embeddings may be applied to the input data. To parts of the input data comprising text the input embedding referred to in FIG. 8, FIG. 9A-2C may be applied. To parts of the input data being numerical token embeddings, positional embeddings, column embeddings and row embeddings may be applied. Further, segment embeddings may be applied to the input data independent of the type of input data. The segment embedding may specify the type of input data one or more elements may be associated to. For example, if the input data comprises of text and numbers, the input data may comprise of two types of input data. Applying the segment embedding to the input data may refer to adding a segment factor to the input data, preferably the embedded input data and / or the input data after having applied the token embedding. The segment factor may specify the type of data associated with the one or more elements. The segment factor may be the same for one or more elements associated with the same type of input data and / or may differ between two or more elements associated with different types of input data.

[0137] Applying the token embedding, the positional embedding, the segment embedding, the column embedding, the row embedding, or a combination thereof may result in embedded input data and / or may be the output of any one of the encoder input 978, 984, 988 or decoder input 984, 994. The data obtained by applying the token embedding, the positional embedding, the segment embedding, the column embedding, the row embedding, or a combination thereof may be processed by the encoder block 974, 986, decoder block 980, 990, encoder output 976, decoder output 992, 982.

[0138] The present disclosure has been described in conjunction with preferred embodiments and examples as well. However, other variations can be understood and effected by those persons skilled in the art and practicing the claimed invention, from the studies of the drawings, this disclosure and the claims.

[0139] Any steps presented herein can be performed in any order. The methods disclosed herein are not limited to a specific order of these steps. It is also not required that the different steps are performed at a certain place or in a certain computing node of a distributed system, i.e. each of the steps may be performed at different computing nodes using different equipment / data processing.

[0140] As used herein ..determining" also includes ..initiating or causing to determine", "generating" also includes ..initiating and / or causing to generate" and "providing” also includes "initiating or causing to determine, generate, select, send and / or receive”. "Initiating or causing to perform an action” includes any processing signal that triggers a computing node or device to perform the respective action.

[0141] In the claims as well as in the description the word "comprising” or "including” or similar wording does not exclude other elements or steps and shall not be construed limiting to the elements or steps lined out. The indefinite article "a” or "an” does not exclude a plurality. A single element or other unit may fulfill the functions of several entities or items recited in the claims. The mere fact that certain measures are recited in the mutual different dependent claims does not indicate that a combination of these measures cannot be used in an advantageous implementation or further elements may be included. Providing in the scope of this disclosure may include any interface configured to provide data. This may include an application programming interface, a human-machine interface such as a display and / or a software module interface. Providing may include communication of data or submission of data to the interface, in particular display to a user or use of the data by the receiving entity.

[0142] Any disclosure and embodiments described herein relate to methods, systems, apparatuses, devices, chemicals, materials, services, uses, computer program elements lined out above and vice versa. Advantageously, the benefits provided by any of the embodiments and examples equally apply to all other embodiments and examples and vice versa.

[0143] All terms and definitions used herein are understood broadly and have their general meaning.

Claims

CLAIMS1 . A computer-implemented method for monitoring and / or controlling distributed operation equipment(s), the method comprising the steps: providing at least one user instruction and one or more instruction templates relating to monitoring and / or controlling of distributed operation equipment(s); selecting one or more instruction templates from the provided one or more instruction templates based on the user instruction; generating one or more instruction (s) based on the selected one or more instruction templates, including at least one task instruction related to the monitoring and / or controlling of distributed operation equipment(s); generating at least one operation instruction for monitoring and / or controlling of operation equipment by providing the one or more generated instruction(s) to a data-driven model configured to generate the at least one operation instructions, wherein the data-driven model is a general-purpose model trained at least on unstructured data; providing the at least one operation instruction for monitoring and / or controlling the distributed operation equipment(s).

2. The method of claim 1, wherein depending on the user instruction multiple instruction templates may be selected and processed sequentially.

3. The method of any of the preceding claims, wherein the instruction templates relate to identification of operation data, retrieval of operation data, selection of machine-readable monitoring and / or control instructions, execution of machine-readable monitoring and / or control instructions or combinations thereof.

4. The method of any of the preceding claims, wherein the instruction templates include at least one task instruction specifying the task to be executed by the data-driven model, at least one specification relating to operation data stored including operation data descriptions as metadata and / or at least one specification relating to machine-readable control and / or monitoring instructions stored including descriptions as metadata.

5. The method of any of the preceding claims, wherein generating instructions includes providing at least part of the operation instructions generated by the data-driven model from one or more preceding instruction(s) to a subsequent instruction template.

6. The method of any of the preceding claims, wherein the one or more instruction template(s) include at least one task instruction related to the monitoring and / or controlling of distributed operation equipment(s) based on identification of operation data, wherein the operation data is stored including operation data description asmetadata, wherein the one or more instruction (s) are generated based on the operation data description, wherein the operation data is identified and / or retrieved based on the operation instruction generated by the data-driven model from the instruction(s) generated based on the operation data description.

7. The method of any of the preceding claims, wherein the one or more instruction template(s) include at least one task instruction related to the monitoring and / or controlling of distributed operation equipment(s) based on operation data retrieval, wherein the one or more instruction(s) are generated based on the operation data retrieved, wherein the operation instruction is generated by the data-driven model from the instruction(s) generated based on the operation data retrieved.

8. The method of any of the preceding claims, wherein the one or more instruction template(s) include at least one task instruction related to the monitoring and / or controlling of distributed operation equipment(s) based on at least one selected machine-readable monitoring and / or control instruction, wherein the one or more instruction(s) are generated based on the at least one selected machine-readable monitoring and / or control instruction, wherein the operation instruction is generated by the data-driven model from the instruction(s) generated based on the at least one selected machine-readable monitoring and / or control instruction.

9. The method of any of the preceding claims, wherein the at least one operation instruction is configured to trigger monitoring and / or controlling at least one of the distributed operation equipment(s).

10. The method of any of the preceding claims, wherein the at least one operation instruction is configured to display operation data identified and / or retrieved.11 . The method of any of the preceding claims, wherein the at least one operation instruction is configured to display machine-readable monitoring and / or control instructions selected and / or to execute machine-readable monitoring and / or control instructions.

12. The method of any of the preceding claims, wherein the at least one operation instruction is provided to at least one of the distributed operation equipment(s) for execution and / or the at least one operation instruction is to be executed to provide the result of the execution to at least one of the distributed operation equipment(s).

13. An apparatus for monitoring and / or controlling distributed operation equipment(s), the method comprising: a template store configured to provide at least one user instruction and one or more instruction templates relating to monitoring and / or controlling of distributed operation equipment(s); a selection engine configured to select one or more instruction templates from the provided one or more instruction templates based on the user instruction;an instruction engine configured to generate one or more instruction(s) based on the selected one or more instruction templates including at least one task instruction related to the monitoring and / or controlling of distributed operation equipment(s) and configured to generate at least one operation instruction for monitoring and / or controlling of operation equipment by providing the one or more generated instruction(s) to a data-driven model con-figured to generate the at least one operation instructions, wherein the data-driven model is a general-purpose model trained at least on unstructured data; monitoring and / or controlling engine configured to provide the at least one operation instruction for monitoring and / or controlling the distributed operation equipment(s).

14. Use of the operation instruction generated according to the methods of any of claims 1 to 12 or be the apparatus of claim 13 for displaying monitoring and / or controlling instructions to an operator of at least one of the distributed operation equipment(s) and / or for monitoring and / or controlling at least one of the distributed operation equipment(s).

Images