Mediator for generating and verifying engineering artifacts

Abstract

An engineering computing system can generate engineering artifacts that are correct in a reliable manner. In particular, various users with little effort and expertise can use chatbots based on large language models (LLMs), with the aid of a mediator module communicatively coupled to the chatbots, to generate various engineering artifacts, such as designs, configurations, diagrams, or program code.

Description

MEDIATOR FOR GENERATING AND VERIFYING ENGINEERING ARTIFACTSBACKGROUND

[0001] Engineering design can be generally characterized as a series of decisions that lead to a final prototype. Generative artificial intelligence (Al) chatbots are often based on large language models (LLMs). In some cases, such chatbots aid in the generation of code, but such code typically needs to be verified by a human user. Furthermore, such code generation is typically not the result of a single request to the chatbot, but rather the result of a dialog that drives the chatbot to produce the desired outcome by providing feedback to the chatbot concerning problems that are identified by a human in the generated code. Thus, a certain level of expertise is required from a person interacting with the chatbot. Furthermore, current chatbots arc currently limited to generating code.SUMMARY

[0002] Embodiments of the invention address and overcome one or more of the described- herein shortcomings or technical problems by providing methods and systems such for generating engineering artifacts that are correct in a reliable manner. In particular, various users with little effort and expertise can use chatbots based on large language models (LLMs), with the aid of a mediator module communicatively coupled to the chatbots, to generate various engineering artifacts, such as designs, models, configurations, diagrams, or program code.

[0003] In an example aspect, an engineering computing system includes one or more processors and a memory having a plurality of application modules stored thereon. The modules can include a large language module (LLM) chatbot configured to receive prompts and, based on the prompts, generate outputs using an LLM database. The modules can further include a mediator module communicatively coupled to the LLM chatbot. The mediator module can be configured to perform various operations. For example, the mediator module can receive an input from a user, wherein the input defines a request for an engineering artifact. Based on the request for the engineering artifact, the mediator module can generate a prompt and send the prompt to the LLM chatbot. Responsive to the prompt, the mediator module can receive a resultfrom the LLM chatbot. The operations can further include performing a validation of the result from the LLM chatbot. For example, the mediator module can make a determination that the validation of the result from the LLM chatbot is successful. Based on the determination, the mediator module can send the result to the user, wherein the result includes the engineering artifact.

[0004] In an example, the engineering artifact defines program code for a programmable logic controller. In the example, responsive to the determination, the mediator module can install the program code on the programmable logic controller. Operations of the mediator module can further include verifying that the result of the LLM chatbot is in compliance with a plurality of constraints imposed by at least one of another engineering artifact, guidelines, or regulations. In yet another example, the mediator module can make a determination that the validation of the result from the LLM chatbot is at least partially unsuccessful. Based on this determination, the mediator module can send feedback to the LLM chatbot. Responsive to the feedback, the mediator module can receive an updated result from the LLM chatbot. In some cases, the result defines program code for a programmable logic controller; the feedback defines comments corresponding to the program code; and the updated result defines new code that updates the program code based on the comments. The mediator module can perform a second validation of the updated result. The mediator module can be further configured continue to provide further feedback, based at least in part on the second validation, to the LLM chatbot until the mediator module validates an output of the LLM chatbot.BRIEF DESCRIPTION OF THE DRAWINGS

[0005] The foregoing and other aspects of the present invention are best understood from the following detailed description when read in connection with the accompanying drawings. For the purpose of illustrating the invention, there is shown in the drawings embodiments that are presently preferred, it being understood, however, that the invention is not limited to the specific instrumentalities disclosed. Included in the drawings are the following Figures:

[0006] FIG. 1 is a block diagram of an example engineering computing system that includes a mediator module in communication with a user device and a large language module (LLM) chatbot according to an example embodiment.

[0007] FTG. 2 depicts an example industrial production system for which the engineering computing system can perform operations so as to generate engineering artifacts associated with the industrial production system.

[0008] FIG. 3 shows an example of a computing environment within which embodiments of this disclosure may be implemented.DETAILED DESCRIPTION

[0009] As an initial matter, it is recognized herein that chatbots based on large language models (LLMs) are not currently used for various engineering applications, but have been used for some software development in which users consists of developers who must examine the generated code to make corrections. For example, users might interact with the chatbot directly by providing input in the form of questions or context information (prompting), and the chatbot might generate a response for the user (e.g., a configuration, a design, code) using its LLM. Continuing with the example, the user might then check the response and request the chatbot to correct any errors that were found. This process can be repeated and iterated until the user is satisfied with the generated result.

[0010] Referring now to FIG. 1, in accordance with various example embodiments, an engineering computing system 100 can include an LLM chatbot 102 and a mediator module 104 communicatively coupled to the LLM chatbot 102. The mediator module 104 can be configured to guide the chatbot 102 that can include or communicatively be coupled to an LLM or LLM database 106. In an example, a user can provide a high-level description of an engineering artifact, for instance a design, configuration, diagram, program, or the like, to the mediator module 104. Based on the high-level description, the mediator module 104 can guide the chatbot 102 instead of the user iteratively providing feedback to the chatbot 102. For example, at each iteration, the mediator module 104 can provide input to the chatbot 102, validate the result that the LLM chatbot 102 generates, and perform new iterations where generated results need to be corrected or revised. When a given result or artifact is validated successfully by the mediator module 104, the mediator 104 can provide the result or artifact to the user.

[0011] Still referring to FIG. 1, the engineering computing system 100 can include one or more processors and memory having stored thereon applications, agents, and computer program modules including, for example, the chatbot 102, the mediator module 104, and the LLM 106.Similarly, the mediator module 104 can include one or more processors and memory having stored thereon applications, agents, and computer program modules. It will be appreciated that the program modules, applications, computer-executable instructions, code, or the like depicted in FIG. 1 are merely illustrative and not exhaustive, and that processing described as being supported by any particular module may alternatively be distributed across multiple modules or performed by a different module. In addition, various program module(s), script(s), plug-in(s), Application Programming Interface(s) (API(s)), or any other suitable computer-executable code may be provided to support functionality provided by the program modules, applications, or computer-executable code depicted in FIG. 1 and / or additional or alternate functionality. Further, functionality may be modularized differently such that processing described as being supported collectively by the collection of program modules depicted in FIG. 1 may be performed by a fewer or greater number of modules, or functionality described as being supported by any particular module may be supported, at least in part, by another module. In addition, program modules that support the functionality described herein may form part of one or more applications executable across any number of systems or devices in accordance with any suitable computing model such as, for example, a client-server model, a peer-to-peer model, and so forth. In addition, any of the functionality described as being supported by any of the program modules depicted in FIG. 1 may be implemented, at least partially, in hardware and / or firmware across any number of devices.

[0012] Still referring to FIG. 1, as described further herein, the mediator module 104 can use reinforcement learning to learn behavior of a human designer. Further, in an example, the chatbot 102 can interface with external databases, such as the LLM database 106, to abstract various knowledge graphs and ontologies for various applications.

[0013] Referring in particular to FIG. 1, at 103, an example user 101 can provide an input, for instance via their computer or user device, to the mediator module 104. The mediator module 104 can analyze the input and generate a validation process from the input. In various examples, the input at 103 can include a request for an engineering artifact, and the validation process that is determined by the mediator module 104 can depend on the engineering artifact that is requested. In particular, for example, the input might request properties of a given artifact, and the validation process might depend on such properties that are requested about the artifact. In some cases, the mediator module 104 performs formal verification and a novel regretminimization strategy, so as to engage a dual validation process. Formal verification can involve a Finite State Machine representation of the generated engineering model. The model can be checked by exhaustively verifying that the Finite State Machine meets a set of design specifications, for example, using first order logic. When correctness is established, the mediator module 104 can declare that the generated engineering model (e.g., PLC program) to be valid on the target input space. To check the quality (not correctness) of the produced engineering artifacts, the regret minimization strategy can prompt the mediator module 104 to generate alternate engineering models that pass the formal verification step within an identified limit. The mediator module 104 can select the engineering model that minimizes the regret on a set of chosen quality metrics as the engineering model with the best design / implementation. In an example, regret is defined as the difference between the best possible performance on a set of chosen quality metrics and how a generated engineering model (artifact) performs.

[0014] By way of example, at 103, the user 101 might make a request from the mediator module 104 for a programmable logic controller (PLC) program. By way of further example, and without limitation, the user 101 might request a PLC program that controls the level of a tank by controlling its output valve. Continuing with the example, based on the request, the mediator module 104 can determine that the validation process associated with the request can include loading the PLC program in an engineering tool, compiling and downloading the program to a controller (or controller simulator), and running tests with a simulation of the tank and its valve. By way of example, the mediator module 104 might only load the program into a test PLC programming tool such as TIA Portal if the generated engineering model fails formal verification. Loading a generated PLC program into a test programming tool (e.g., TIA Portal) can result in the mediator module 104 obtaining the STACKTRACE from the RUNTIME or the error messages from the compiler, which can be sent back to the LLM chatbot 102.Consequently, the LLM chatbot 102 can improve on the generated PLC program, or more generally, engineering model or artifact.

[0015] Validation processes may include static analysis or formal verification of the generated artifact. In some cases, when the validation of the artifact fails, the mediator module 104, at 105, can translate the failure to feedback for the chatbot 102. For example, the mediator module 104 might generate feedback for the chatbot 102 that explains a problem encountered during validation. Furthermore, at 105, the mediator module 104 might instruct the chatbot 102to improve the artifact (or solution). For example, to translate the validation failure, the mediator module 104 can use a structured correction template to iteratively communicate failures to the LLM chatbot 102. Such iterative communication, in some cases, can continue until a version of the engineering artifact (e.g., PLC code, etc.) is generated that passes formal verification. Thus, the communications at 105 between the mediator module 104 and the chatbot 102 can define multiple iterations with no involvement from the user 101. By way of example, and without limitation, the structured correction template can include various data, such as the original design specification, the engineering artifact or model generated at the prior iteration, errors, and further directives. The further directives data might define additional instructions to guide the LLM chatbot 102 to better refine the generated engineering artifact. In various examples, when the chatbot 102 (at 105) returns an artifact (model) or solution that is successfully validated by the mediator module 104, the mediator module 104 can return the artifact (or final result) to the user 101, at 107.

[0016] Referring also to FIG. 2, the engineering computing system 100 can be implemented in an example production system, for instance an example simple salt production system 200. The example production system 200 can define various pumps, valves, tanks, pipes, pressure regulators, pressure transducers, and flow meters, among other industrial components. For example, the production system 100 can include a supply of a first chemical compound 202a (e.g., NaOH) and a supply of a second chemical second compound 202b (e.g., HCL) coupled to first and second pumps 204a and 204b, respectively. The pumps 204a and 204b can be coupled to first and second fast-acting valves 206a and 206b, respectively, that are each connected to a mixer 208 and a production or mixing tank 210. In the example, the pumps 204a-b, fast-acting valves 206a-b, and mixer 208 can be actuated so as to regulate the production of NACL and remove H2O. It will be understood that the simple salt production system 200 is presented for purposes of example, and the engineering computing system 100 can be implemented in alternative or additional industrial settings or systems and all such settings or systems are contemplated as being within the scope of this disclosure.

[0017] Continuing with the example illustrated in FIG. 2, the user 101 can enter an input x that defines an LLM prompt. In particular, for example, the input x can represent a prompt by the user for generating a control algorithm of a programmable logic controller (PLC) to operate the system 200, for instance “Can you generate a control algorithm of a PLC to operate theplant?”). At 103, the mediator module 104 can receive the input x. The mediator module 104 can determine a validation v of the input x, which can be represented as v = f (x) . In particular, for example, at 105, the mediator module 104 can generate a prompt for the chatbot 102 to generate a control algorithm in structured text for a particular controller (e.g., Siemens Simatic S7-300 controller) that operates the system 200. The verification can be performed external to the chatbot 102, such that the LLM chatbot 102 is not engaged. In some examples, the validation involves various mathematical operations, such as formal verification and regret minimization.

[0018] Based on the validation prompt from the mediator module 104, the chatbot 102, using the LLM 106, can generate an LLM output y that can be represented as y = g'(f). By way of example, the LLM output can be based on an industrial control logic program that includes the example variables in Table 1.

[0019] By way of further example, the program might perform various operations on the variables, such as the following: / / Calculate the concentrations of NaCl and H2ONaCIConcentration := NaOHFlowRate I (NaOHFlowRate + HCIFlowRate);H2OConcentration := HCIFlowRate / (NaOHFlowRate + HCIFlowRate); / / Check the concentrations and adjust the valves 206a and 206b accordinglyIF mixingTankLevel >= mixingTankMaxLevel THEN / / Tank 210 is full, close the valves 206a and 206bNaCIValve := FALSE;H2OValve := FALSE;ELS IF mixingTankLevel <= mixingTankMinLevel THEN / / Tank 210 is empty, open the valves 206a and 206bNaCIValve := TRUE;H2OValve := TRUE;ELSE / / Adjust valves 206a and 206b based on concentration errorIF NaCIConcentration > desiredNaCIConcentration THEN / / NaCl concentration is too high, close NaCl valve 206aNaCIValve := FALSE;ELSIF NaCIConcentration < desiredNaCIConcentration THEN / / NaCl concentration is too low, open NaCl valve 206aNaCIValve := TRUE;ELSE / / NaCl concentration is within acceptable range, maintain current position of valve 206a END_IF; / / Remove excess H2OIF H2OConcentration > 0.0 THEN / / H2O is present, open H2O valve 206bH2OValve := TRUE;ELSE / / No H2O, close H2O valve 206bH2OValve := FALSE;END_IF;END_IF;END_METHOD

[0020] Continuing with the example, the mediator module 104 can validate the LLM output y, for instance the industrial control logic program illustrated above that can be provided by the chatbot 102 at 105. Such a validation can be represented as LLM output u = / i(y). Inparticular, for example, the mediator module 104 can submit the program code to a compiler, for instance an internal TIA Portal Advanced Structured Text Compiler, to determine compilation errors. By way of further example, when there are no compilation errors, the validation can further include determining, by the mediator module 104, implications of the assumed constants (e.g., mixingTankVolume = 100.0; mixingTankMaxLevel = 90.0; mixingTankMinLevel = 10.0). In an example, the mediator module 104 might determine that the constants are limiting, such the constants should be parameters that are read from a PLC plant configuration file. The example values can be defined as limiting because they are defined as constants in the PLC program. For example, a variable can take on any value, and it can be more flexible to structure the PLC program with variables. These variables can then take on values specified in an external file. Thus, the same PLC program can be used to run multiple plants (similar) by varying the production parameters or variables.

[0021] As another example, the validation can include running code from a quality checker to determine if the best function result is achieved. In some cases, after the running the quality checker code, the mediator module 104 might determine that there is zero operational regret if the program code is executed as-is. Thus, even with the hardcoded constraints or constants, the PLC can operate the production system 200 within the defined limits. Based on such a determination, the mediator module 104 might execute the program code on a virtual PLC simulator. If the virtual simulator successfully operates the virtual system, the mediator module 104 can determine that the validation u is successful, thereby passing validation, for validation settings that arc not strict. Strict validation can involve formal verification with regret minimization, which can examine the quality. By way of example, referring to the example PLC program, the PLC program with constants might pass the formal verification for the specified plant, but might be better with variables used in the place of the constants. Non-strict verification can refer to only formal verification (e.g., without regret minimization). The mediator module 104 can pass on the other validation settings. Thereafter, the mediator module 104 can install the code on the PLC for the validation settings that are not strict. In some cases, the mediator module 104 can also perform the downstream installation.

[0022] Alternatively, when the virtual simulator does not successfully operate the virtual system, the mediator module 104 can determine that the validation u is not successful, thereby failing validation. In an example, when the validation u fails, the mediator module 104 canrespond back to the LLM chatbot 104, at 105. Tn particular, for example, the mediator module 104 can augment the program code, for instance with the comments on hardcoded values tuple, and send such augmented code back to the LLM chatbot 102. In an example, based on the response from the mediator module 104, for instance based on the augmented code, the LLM chatbot 102 can generate an improved or updated result or output y ' and pass that back to the mediator module 104.

[0023] Still continuing with the example, the improved result y ' might define updated code as compared to the original program code that is generated by the LLM chatbot 102. In particular, for example, the updated code might define the variables mixingTankVolume, mixingTankMaxLcvcl, and MixingTankMinLcvcl without initialization values, such that the variables can be set and configured in the PLC configuration file or by assigning values to them in a PLC program or human-machine interface (HMI) software. Just as the mediator module 104 validates the original LLM output y, the mediator module 104 can validated the updated result or output y' , which can be represented as u' = / i(y'). When the updated validation u' passes the validation, the mediator module 104 can return the updated result y', for instance the new code, to the user 101, at 107. Additionally, or alternatively, the mediator module 104 can execute the new code on the PLC. When the updated validation u' fails the validation, the mediator module 104 can again augment the code or otherwise respond to the LLM chatbot 102 with feedback or instructions until the validation is passed or the number of interactions between the mediator 104 and the LLM chatbot 102 exceeds a predetermined threshold.

[0024] Thus, as described herein, the mediator module 104 can perform as a gatekeeper to check and verify output of the LLM chatbot 102. The mediator module 104 can check outputs of the LLM chatbot 102 against various relevant guidelines, such as regulations, policies, design recommendations, prevailing sanctions, and the like. Additionally, or alternatively, the mediator module 104 can check outputs of the LLM chatbot 102 against constraints imposed by other engineering or product artifacts. When a given check does not comply with a given guideline or constraint, the mediator module 104 can generate an error. Errors can be associated with various review flags that can define signals that an error has occurred in the corresponding output of the chatbot 102. In some cases, when the mediator module 104 raises a review Hag, the output is automatically rejected, and the mediator module 104 can provide feedback to the chatbot 102 based on the review flag, so that the chatbot 102 can improve its output based on the feedbackfrom the mediator module 104. The feedback process can repeat until the validation is passed, and then the mediator module 104 can return the output (c.g., engineering artifact) associated with the successful validation to the user 101.

[0025] Thus, as described herein, an engineering computing system (e.g., engineering computing system 100) can include one or more processors and a memory having a plurality of application modules stored thereon. The modules can include a large language module (LLM) chatbot configured to receive prompts and, based on the prompts, generate outputs using an LLM database. The modules can further include a mediator module communicatively coupled to the LLM chatbot. The mediator module can be configured to perform various operations. For example, the mediator module can receive an input from a user, wherein the input defines a request for an engineering artifact. Based on the request for the engineering artifact, the mediator module can generate a prompt and send the prompt to the LLM chatbot. Responsive to the prompt, the mediator module can receive a result from the LLM chatbot. The operations can further include performing a validation of the result from the LLM chatbot. For example, the mediator module can make a determination that the validation of the result from the LLM chatbot is successful. Based on the determination, the mediator module can send the result to the user, wherein the result includes the engineering artifact.

[0026] In an example, the engineering artifact defines program code for a programmable logic controller. In the example, responsive to the determination, the mediator module can install the program code on the programmable logic controller. Operations of the mediator module can further include verifying that the result of the LLM chatbot is in compliance with a plurality of constraints imposed by at least one of another engineering artifact, guidelines, or regulations. In yet another example, the mediator module can make a determination that the validation of the result from the LLM chatbot is at least partially unsuccessful. Based on this determination, the mediator module can send feedback to the LLM chatbot. Responsive to the feedback, the mediator module can receive an updated result from the LLM chatbot. In some cases, the result defines program code for a programmable logic controller; the feedback defines comments corresponding to the program code; and the updated result defines new code that updates the program code based on the comments. The mediator module can perform a second validation of the updated result. The mediator module can be further configured continue toprovide further feedback, based at least in part on the second validation, to the LLM chatbot until the mediator module validates an output of the LLM chatbot.

[0027] Without being bound by theory, the engineering computing system 100 can ensure the quality of content or artifacts that are generated by the LLM chatbot 102. Additionally, the computing system 100, in particular the mediator module 104, can ensure that artifacts generated by the LLM chatbot 102 are in compliance with various operational, performance, safety, and ethical standards. Furthermore, the mediator module 104 can enable users with less expertise and effort to use the LLM chatbot 102 to generate engineering artifacts as compared to using the LLM chatbot 102 without the mediator 104 because, for example, the mediator module 104 can validate results of the chatbot 102 and can provide feedback to the chatbot 102 to improve such results.

[0028] FIG. 3 illustrates an example of a computing environment within which embodiments of the present disclosure may be implemented. A computing environment 300 includes a computer system 310 that may include a communication mechanism such as a system bus 321 or other communication mechanism for communicating information within the computer system 310. The computer system 310 further includes one or more processors 320 coupled with the system bus 321 for processing the information.

[0029] The processors 320 may include one or more central processing units (CPUs), graphical processing units (GPUs), or any other processor known in the art. More generally, a processor as described herein is a device for executing machine-readable instructions stored on a computer readable medium, for performing tasks and may comprise any one or combination of, hardware and firmware. A processor may also comprise memory storing machine-readable instructions executable for performing tasks. A processor acts upon information by manipulating, analyzing, modifying, converting or transmitting information for use by an executable procedure or an information device, and / or by routing the information to an output device. A processor may use or comprise the capabilities of a computer, controller or microprocessor, for example, and be conditioned using executable instructions to perform special purpose functions not performed by a general purpose computer. A processor may include any type of suitable processing unit including, but not limited to, a central processing unit, a microprocessor, a Reduced Instruction Set Computer (RISC) microprocessor, a Complex Instruction Set Computer (CISC) microprocessor, a microcontroller, an Application SpecificTntegrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), a System-on-a-Chip (SoC), a digital signal processor (DSP), and so forth. Further, the proccssor(s) 320 may have any suitable microarchitecture design that includes any number of constituent components such as, for example, registers, multiplexers, arithmetic logic units, cache controllers for controlling read / write operations to cache memory, branch predictors, or the like. The microarchitecture design of the processor may be capable of supporting any of a variety of instruction sets. A processor may be coupled (electrically and / or as comprising executable components) with any other processor enabling interaction and / or communication there-between. A user interface processor or generator is a known element comprising electronic circuitry or software or a combination of both for generating display images or portions thereof. A user interface comprises one or more display images enabling user interaction with a processor or other device.

[0030] The system bus 3 1 may include at least one of a system bus, a memory bus, an address bus, or a message bus, and may permit exchange of information (e.g., data (including computer-executable code), signaling, etc.) between various components of the computer system 310. The system bus 321 may include, without limitation, a memory bus or a memory controller, a peripheral bus, an accelerated graphics port, and so forth. The system bus 321 may be associated with any suitable bus architecture including, without limitation, an Industry Standard Architecture (ISA), a Micro Channel Architecture (MCA), an Enhanced ISA (EISA), a Video Electronics Standards Association (VESA) architecture, an Accelerated Graphics Port (AGP) architecture, a Peripheral Component Interconnects (PCI) architecture, a PCI- Express architecture, a Personal Computer Memory Card International Association (PCMCIA) architecture, a Universal Serial Bus (USB) architecture, and so forth.

[0031] Continuing with reference to FIG. 3, the computer system 310 may also include a system memory 330 coupled to the system bus 321 for storing information and instructions to be executed by processors 320. The system memory 330 may include computer readable storage media in the form of volatile and / or nonvolatile memory, such as read only memory (ROM) 331 and / or random access memory (RAM) 332. The RAM 332 may include other dynamic storage device(s) (e.g., dynamic RAM, static RAM, and synchronous DRAM). The ROM 331 may include other static storage device(s) (e.g., programmable ROM, erasable PROM, and electrically erasable PROM). In addition, the system memory 330 may be used for storing temporary variables or other intermediate information during the execution of instructions by theprocessors 320. A basic input / output system 333 (BIOS) containing the basic routines that help to transfer information between elements within computer system 310, such as during start-up, may be stored in the ROM 331. RAM 332 may contain data and / or program modules that are immediately accessible to and / or presently being operated on by the processors 320. System memory 330 may additionally include, for example, operating system 334, application modules 335, and other program modules 336. Application modules 335 may include aforementioned modules described for FIG. 1 and may also include a user portal for development of the application program, allowing input parameters to be entered and modified as necessary.

[0032] The operating system 334 may be loaded into the memory 330 and may provide an interface between other application software executing on the computer system 310 and hardware resources of the computer system 310. More specifically, the operating system 334 may include a set of computer-executable instructions for managing hardware resources of the computer system 310 and for providing common services to other application programs (e.g., managing memory allocation among various application programs). In certain example embodiments, the operating system 334 may control execution of one or more of the program modules depicted as being stored in the data storage 340. The operating system 334 may include any operating system now known or which may be developed in the future including, but not limited to, any server operating system, any mainframe operating system, or any other proprietary or non-proprietary operating system.

[0033] The computer system 310 may also include a disk / media controller 343 coupled to the system bus 321 to control one or more storage devices for storing information and instructions, such as a magnetic hard disk 341 and / or a removable media drive 342 (e.g., floppy disk drive, compact disc drive, tape drive, flash drive, and / or solid state drive). Storage devices 340 may be added to the computer system 310 using an appropriate device interface (e.g., a small computer system interface (SCSI), integrated device electronics (IDE), Universal Serial Bus (USB), or FireWire). Storage devices 341, 342 may be external to the computer system 310.

[0034] The computer system 310 may include a user input interface or graphical user interface (GUI) 361, which may comprise one or more input devices, such as a keyboard, touchscreen, tablet and / or a pointing device, for interacting with a computer user and providing information to the processors 320.

[0035] The computer system 310 may perform a portion or all of the processing steps of embodiments of the invention in response to the processors 320 executing one or more sequences of one or more instructions contained in a memory, such as the system memory 330. Such instructions may be read into the system memory 330 from another computer readable medium of storage 340, such as the magnetic hard disk 341 or the removable media drive 342. The magnetic hard disk 341 and / or removable media drive 342 may contain one or more data stores and data files used by embodiments of the present disclosure. The data store 340 may include, but are not limited to, databases (e.g., relational, object-oriented, etc.), file systems, flat files, distributed data stores in which data is stored on more than one node of a computer network, peer-to-peer network data stores, or the like. Data store contents and data files may be encrypted to improve security. The processors 320 may also be employed in a multi-processing arrangement to execute the one or more sequences of instructions contained in system memory 330. In alternative embodiments, hard-wired circuitry may be used in place of or in combination with software instructions. Thus, embodiments are not limited to any specific combination of hardware circuitry and software.

[0036] As stated above, the computer system 310 may include at least one computer readable medium or memory for holding instructions programmed according to embodiments of the invention and for containing data structures, tables, records, or other data described herein. The term “computer readable medium” as used herein refers to any medium that participates in providing instructions to the processors 320 for execution. A computer readable medium may take many forms including, but not limited to, non-transitory, non-volatile media, volatile media, and transmission media. Non-limiting examples of non-volatile media include optical disks, solid state drives, magnetic disks, and magneto-optical disks, such as magnetic hard disk 341 or removable media drive 342. Non-limiting examples of volatile media include dynamic memory, such as system memory 330. Non-limiting examples of transmission media include coaxial cables, copper wire, and fiber optics, including the wires that make up the system bus 321. Transmission media may also take the form of acoustic or light waves, such as those generated during radio wave and infrared data communications.

[0037] Computer readable medium instructions for carrying out operations of the present disclosure may be assembler instructions, instruction- set- architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-settingdata, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) may execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present disclosure.

[0038] Aspects of the present disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, may be implemented by computer readable medium instructions.

[0039] The computing environment 300 may further include the computer system 310 operating in a networked environment using logical connections to one or more remote computers, such as remote computing device 380. The network interface 370 may enable communication, for example, with other remote devices 380 or systems and / or the storage devices 341, 342 via the network 371. Remote computing device 380 may be a personal computer (laptop or desktop), a mobile device, a server, a router, a network PC, a peer device or other common network node, and typically includes many or all of the elements described above relative to computer system 310. When used in a networking environment, computer system 310 may include modem 372 for establishing communications over a network 371, such as the Internet. Modem 372 may be connected to system bus 321 via user network interface 370, or via another appropriate mechanism.

[0040] Network 371 may be any network or system generally known in the art, including the Internet, an intranet, a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a direct connection or series of connections, a cellular telephone network, or any other network or medium capable of facilitating communication between computer system 310 and other computers (e.g., remote computing device 380). The network 371 may be wired, wireless or a combination thereof. Wired connections may be implemented using Ethernet, Universal Serial Bus (USB), RJ-6, or any other wired connection generally known in the art. Wireless connections may be implemented using Wi-Fi, WiMAX, and Bluetooth, infrared, cellular networks, satellite or any other wireless connection methodology generally known in the art. Additionally, several networks may work alone or in communication with each other to facilitate communication in the network 371.

[0041] It should be appreciated that the program modules, applications, computer-executable instructions, code, or the like depicted in FIG. 3 as being stored in the system memory 330 are merely illustrative and not exhaustive and that processing described as being supported by any particular module may alternatively be distributed across multiple modules or performed by a different module. In addition, various program module(s), script(s), plug-in(s), Application Programming Interface(s) (API(s)), or any other suitable computer-executable code hosted locally on the computer system 310, the remote device 380, and / or hosted on other computing device(s) accessible via one or more of the network(s) 371, may be provided to support functionality provided by the program modules, applications, or computer-executable code depicted in FIG. 3 and / or additional or alternate functionality. Further, functionality may be modularized differently such that processing described as being supported collectively by the collection of program modules depicted in FIG. 3 may be performed by a fewer or greater number of modules, or functionality described as being supported by any particular module may be supported, at least in part, by another module. In addition, program modules that support the functionality described herein may form part of one or more applications executable across any number of systems or devices in accordance with any suitable computing model such as, for example, a client-server model, a peer-to-peer model, and so forth. In addition, any of the functionality described as being supported by any of the program modules depicted in FIG. 3 may be implemented, at least partially, in hardware and / or firmware across any number of devices.

[0042] It should further be appreciated that the computer system 310 may include alternate and / or additional hardware, software, or firmware components beyond those described or depicted without departing from the scope of the disclosure. More particularly, it should be appreciated that software, firmware, or hardware components depicted as forming part of the computer system 310 are merely illustrative and that some components may not be present or additional components may be provided in various embodiments. While various illustrative program modules have been depicted and described as software modules stored in system memory 330, it should be appreciated that functionality described as being supported by the program modules may be enabled by any combination of hardware, software, and / or firmware. It should further be appreciated that each of the above-mentioned modules may, in various embodiments, represent a logical partitioning of supported functionality. This logical partitioning is depicted for ease of explanation of the functionality and may not be representative of the structure of software, hardware, and / or firmware for implementing the functionality. Accordingly, it should be appreciated that functionality described as being provided by a particular module may, in various embodiments, be provided at least in part by one or more other modules. Further, one or more depicted modules may not be present in certain embodiments, while in other embodiments, additional modules not depicted may be present and may support at least a portion of the described functionality and / or additional functionality. Moreover, while certain modules may be depicted and described as sub-modules of another module, in certain embodiments, such modules may be provided as independent modules or as sub-modules of other modules.

[0043] Although specific embodiments of the disclosure have been described, one of ordinary skill in the art will recognize that numerous other modifications and alternative embodiments are within the scope of the disclosure. For example, any of the functionality and / or processing capabilities described with respect to a particular device or component may be performed by any other device or component. Further, while various illustrative implementations and architectures have been described in accordance with embodiments of the disclosure, one of ordinary skill in the art will appreciate that numerous other modifications to the illustrative implementations and architectures described herein are also within the scope of this disclosure. In addition, it should be appreciated that any operation, element, component, data, or the like described herein as being based on another operation, element, component, data, or the like canbe additionally based on one or more other operations, elements, components, data, or the like. Accordingly, the phrase “based on,” or variants thereof, should be interpreted as “based at least in part on.”

[0044] The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the Figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustration, and combinations of blocks in the block diagrams and / or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts or carry out combinations of special purpose hardware and computer instructions.

Claims

CLAIMSWhat is claimed is:

1. An engineering computing system comprising: a memory having a plurality of application modules stored thereon; and a processor for executing the application modules, the application modules comprising: a large language model (LLM) chatbot configured to receive prompts and, based on the prompts, generate outputs using an LLM database; and a mediator module coupled to the LLM chatbot, the mediator module configured to: receive an input from a user, the input defining a request for an engineering artifact; based on the request for the engineering artifact, generate a prompt and send the prompt to the LLM chatbot; responsive to the prompt, receive a result from the LLM chatbot; and perform a validation of the result from the LLM chatbot.

2. The engineering computing system as recited in claim 1, wherein the mediator module is further configured to: make a determination that the validation of the result from the LLM chatbot is successful; based on the determination, send the result to the user, wherein the result includes the engineering artifact.

3. The engineering computing system as recited in claim 2, wherein the engineering artifact defines program code for a programmable logic controller, and the mediator module is further configured to: responsive to the determination, install the program code on the programmable logic controller.

4. The engineering computing system as recited in claim 2, wherein the mediator module is further configured to verify that the result of the LLM chatbot is in compliance with a plurality of constraints imposed by at least one of another engineering artifact, guidelines, or regulations.

5. The engineering computing system as recited in claim 1, wherein the mediator module is further configured to: make a determination that the validation of the result from the LLM chatbot is at least partially unsuccessful; based on the determination, send feedback to the LLM chatbot; and responsive to the feedback, receive an updated result from the LLM chatbot.

6. The engineering system as recited in claim 5, wherein the result defines program code for a programmable logic controller; the feedback defines comments corresponding to the program code; and the updated result defines new code that updates the program code based on the comments.

7. The engineering computing system as recited in claim 5, wherein the mediator module is further configured to perform a second validation of the updated result.

8. The engineering computing system as recited in claim 6, wherein the mediator module is further configured to continue to provide further feedback, based at least in part on the second validation, to the LLM chatbot until the mediator module validates an output of the LLM chatbot.

9. A method performed by a mediator module of an engineering computing system that further comprises a large language model (LLM) chatbot communicatively coupled to the mediator module, the method comprising: receiving an input from a user, the input defining a request for an engineering artifact;based on the request for the engineering artifact, generating a prompt and sending the prompt to the LLM chatbot; responsive to the prompt, receiving a result from the LLM chatbot; and performing a validation of the result from the LLM chatbot.

10. The method as recited in claim 9, the method further comprising: making a determination that the validation of the result from the LLM chatbot is successful; based on the determination, sending the result to the user, wherein the result includes the engineering artifact.

11. The method as recited in claim 10, wherein the engineering artifact defines program code for a programmable logic controller, the method further comprising: responsive to the determination, installing the program code on the programmable logic controller.

12. The method as recited in claim 10, the method further comprising: verifying that the result of the LLM chatbot is in compliance with a plurality of constraints imposed by at least one of another engineering artifact, guidelines, or regulations.

13. The method as recited in claim 9, the method further comprising: making a determination that the validation of the result from the LLM chatbot is at least partially unsuccessful; based on the determination, sending feedback to the LLM chatbot; and responsive to the feedback, receiving an updated result from the LLM chatbot.

14. The method as recited in claim 13, wherein the result defines program code for a programmable logic controller; the feedback defines comments corresponding to the programcode; and the updated result defines new code that updates the program code based on the comments.

15. The method as recited in claim 13, the method further comprising: performing a second validation of the updated result.

Images