System and method for determining process scheduling information by using multi-agents

Abstract

A schedule providing method according to an embodiment of the present disclosure comprises the steps of: generating a first GUI for acquiring user input information; acquiring the user input information on the basis of the first GUI; specifying a target period; on the basis of the user input information and the target period, determining whether a schedule can be generated; generating the schedule during the target period when it is identified that the schedule can be generated; and controlling a display to output a second GUI for providing the schedule during the target period, wherein the schedule during the target period includes a first schedule pertaining to the operation of a warehousing tank during the target period, a second schedule pertaining to the operation of a blending tank during the target period, and a third schedule pertaining to the operation of a cracking furnace during the target period, and at least one among the first schedule, the second schedule, and the third schedule may be determined by one or more AI agents for performing reinforcement learning.

Description

System and method for determining process scheduling information using multi-agents

[0001] The present disclosure relates to a process scheduling system, and more specifically, to a system and method in which an artificial intelligence-based multi-agent asynchronously generates a schedule in a complex continuous process consisting of multiple stages.

[0002] Large-scale chemical processes, such as naphtha cracking centers (NCCs), and semiconductor manufacturing processes are representative multi-stage continuous processes involving numerous facilities and complex constraints. Scheduling these processes requires that each stage, from raw material intake to blending, cracking, and purification, be organically connected, and various variables such as fluctuations in market conditions, equipment failures, and safety regulations must be considered in real time.

[0003] Conventional process scheduling has been performed by relying on the experience of skilled engineers or using mathematical optimization tools such as linear programming. However, methods relying on engineer experience have the disadvantage of significant individual variability and slow response times to unexpected situations. Furthermore, mathematical optimization methods have limitations in deriving optimal solutions in real time due to the curse of dimensionality problem, where the amount of computation increases exponentially as the scale of the process grows.

[0004] Recently, research on scheduling using reinforcement learning has been attempted; however, single-agent methods face the problem of difficulty in learning vast state spaces, while simple multi-agent methods suffer from unstable convergence due to communication overhead between agents and goal mismatch issues. In particular, achieving overall optimization by synchronizing processes with different time scales, such as receiving, mixing, and disassembly, remains a challenging problem.

[0005] Therefore, there is a need for new AI-based scheduling technology that can effectively handle the complexity of large-scale processes while flexibly and rapidly deriving optimal schedules by reflecting the characteristics of each process stage.

[0006] One embodiment of the present disclosure aims to provide a system and method in which an artificial intelligence-based multi-agent asynchronously generates a schedule in a complex continuous process consisting of a plurality of steps.

[0007] A system according to one embodiment of the present disclosure comprises at least one processor; and at least one memory storing instructions that cause the system to perform operations when executed by the at least one processor, wherein the operations may include: an operation of generating a first GUI for obtaining user input information; an operation of obtaining user input information based on the first GUI; an operation of identifying a target period; an operation of identifying whether to generate a schedule based on the user input information and the target period; an operation of generating a schedule for the target period when it is identified that the schedule can be generated; and an operation of controlling a display to output a second GUI for providing a schedule for the target period.

[0008] In one embodiment, the operations may include an operation to perform a simulation of a schedule for the target period; and an operation to generate a third GUI to provide an evaluation result of the simulation along with the schedule for the target period.

[0009] In one embodiment, the operations may include: an operation of identifying that the schedule cannot be generated based on the user input information and the target period; an operation of generating a fourth GUI for obtaining additional information based on the identification that the schedule cannot be generated; an operation of obtaining the additional information based on the fourth GUI, which includes an operation of controlling a display to output the fourth GUI; and an operation of identifying whether the schedule can be generated based on the user input information and the additional information.

[0010] In one embodiment, based on the user input information and the target period, the method may include: an operation to identify the need for additional information, even though schedule generation is possible; an operation to generate a message regarding the need for additional information based on the identification of the need for additional information; and an operation to control the display to output the message regarding the need for additional information.

[0011] In one embodiment, the user input information may include at least one of simulation information, AI model information, process data, and constraint information.

[0012] In one embodiment, the process data may include at least one of a raw material receiving schedule during a first period, a first process operation schedule during the first period, a second process operation schedule during the first period, a third process operation schedule during the first period, a raw material inventory amount per raw material storage tank, a history of characteristics per raw material storage tank, a history of daily target production volume for each product during the first period, a history of daily prices of raw materials during the first period, and a history of daily prices of products during the first period.

[0013] In one embodiment, the first period may be longer than the target period.

[0014] In one embodiment, the schedule during the target period includes a schedule for a first process and a schedule for a second process, and at least one of the schedule for the first process and the schedule for the second process may be determined by one or more AI agents performing reinforcement learning.

[0015] In one embodiment, one or more AI agents performing the reinforcement learning are characterized by being trained using a single reward, and the single reward may be determined based on profit, whether a required constraint is satisfied, and whether a preference constraint is satisfied.

[0016] In one embodiment, the schedule during the target period includes at least one of a schedule for a first process determined by a first AI agent, a schedule for a second process determined by a second AI agent, and a schedule for a third process determined by a third AI agent, and the first AI agent, the second AI agent, and the third AI agent may be asynchronous multi-agents.

[0017] In one embodiment, the first AI agent may include a plurality of first AI agents provided for each type of raw material input, and the third AI agent may include a plurality of third AI agents provided for each type of decomposition furnace.

[0018] In one embodiment, the first AI agent determines an incoming tank to store incoming raw materials, the second AI agent determines one or more incoming tanks to be moved to a mixing tank and determines the number of pumps to be connected to the determined incoming tanks, and the third AI agent can determine the supply rate of raw materials supplied to each decomposition furnace, the coil outlet temperature, and the dilution steam ratio.

[0019] In one embodiment, the operations include controlling a display to provide a plurality of schedules for the target period through the second GUI, and the plurality of schedules for the target period may include at least two of the following: an optimal schedule based on reinforcement learning reward criteria, an optimal schedule based on criteria that comprehensively consider target production volume and profit, an optimal schedule based on profit criteria, an optimal schedule based on target production volume criteria, and an optimal schedule based on operating conditions criteria.

[0020] In one embodiment, the operations may include: an operation of obtaining a schedule reflection option based on the user selecting one of the following: an optimal schedule based on reinforcement learning reward criteria, an optimal schedule based on criteria that comprehensively consider target production volume and profit, an optimal schedule based on profit criteria, an optimal schedule based on target production volume criteria, and an optimal schedule based on operating conditions criteria; and an operation of generating a schedule for a target period corresponding to the schedule reflection option.

[0021] In one embodiment, the schedule during the target period may include an operation schedule of the receiving tank during the target period, an operation schedule of the mixing tank during the target period, and an operation schedule of the disassembly during the target period.

[0022] In one embodiment, the user input information includes process data and constraint information, the process data includes at least one of a target ethylene production volume, a target propylene production volume, and a cracking furnace decoking schedule, and the constraint information may include at least one of a property constraint information, information on whether recycled naphtha applies, and information on whether high-content oxygenate operation applies.

[0023] In one embodiment, the characteristic constraint information may include characteristic-specific concentration hard reference information and characteristic-specific concentration soft reference information.

[0024] In one embodiment, the user input information includes an option to reflect the opening inspection schedule and an option to maintain continuity, and the operations may include: an operation to identify the opening inspection period, the tank to be inspected, the tank to be transferred, and the remaining amount transfer period based on the option to reflect the opening inspection schedule; an operation to generate a schedule for the target period to transfer the inventory of the tank to be inspected to the tank to be transferred from the start of emptying during the opening inspection period until the inventory of the tank to be inspected reaches a preset remaining amount; an operation to generate a schedule for the target period to transfer the inventory of the tank to be inspected corresponding to the preset remaining amount to the tank to be transferred; and an operation to generate a schedule for the target period based on the option to maintain continuity such that the difference from the currently executing schedule is minimized.

[0025] A method for providing a schedule according to one embodiment of the present disclosure may include: a step of generating a first GUI for obtaining user input information; a step of obtaining user input information based on the first GUI; a step of identifying a target period; a step of identifying whether a schedule can be generated based on the user input information and the target period; a step of generating a schedule for the target period if it is identified that the schedule can be generated; and a step of controlling a display to output a second GUI for providing a schedule for the target period.

[0026] One embodiment of the present disclosure includes a program stored on a recording medium to execute a method according to one embodiment of the present disclosure on a computer.

[0027] One embodiment of the present disclosure includes a computer-readable recording medium having a program for executing a method according to one embodiment of the present disclosure on a computer.

[0028] One embodiment of the present disclosure includes a computer-readable recording medium that records a database used in one embodiment of the present disclosure.

[0029] FIG. 1 is a diagram illustrating training buffers in MacDec-POMDP according to one embodiment of the present disclosure.

[0030] FIG. 2 is a diagram showing a centralized critique using joint history encoders according to one embodiment of the present disclosure.

[0031] FIG. 3 is a diagram illustrating an Agent-Oriented Centralized Critic (AOCC) method according to one embodiment of the present disclosure.

[0032] FIG. 4 is a drawing showing a reinforcement learning system according to one embodiment of the present disclosure.

[0033] FIG. 5 is a diagram showing an agent-centric encoder of an agent-centric centralized critic according to one embodiment of the present disclosure.

[0034] FIG. 6 is a diagram illustrating a method for calculating a value function by integrating agent-centered history according to one embodiment of the present disclosure.

[0035] FIGS. 7a to 7c are drawings illustrating an Agent-Centric Actor-Critic (ACAC) method according to one embodiment of the present disclosure.

[0036] FIG. 8 is a diagram illustrating an Agent-Centric Actor-Critic (ACAC) method according to one embodiment of the present disclosure.

[0037] FIG. 9 is a schematic diagram of a system according to one embodiment of the present disclosure.

[0038] FIG. 10 is a block diagram illustrating the internal structure of an agent-centered history encoding module according to one embodiment of the present disclosure.

[0039] FIG. 11 is a block diagram illustrating the internal structure of a centralized critique module according to one embodiment of the present disclosure.

[0040] FIG. 12 is a schematic diagram of a product production process of a naphtha cracking center according to one embodiment of the present disclosure.

[0041] FIG. 13 is a diagram illustrating a reinforcement learning method using multiple agents according to one embodiment of the present disclosure.

[0042] FIGS. 14a and FIGS. 14b are drawings illustrating a reinforcement learning method using multiple agents according to one embodiment of the present disclosure.

[0043] FIG. 15 is a drawing showing the schedule of a naphtha cracking center according to one embodiment of the present disclosure.

[0044] FIG. 16 is a diagram illustrating a method for determining information about a process consisting of a plurality of steps according to one embodiment of the present disclosure.

[0045] FIG. 17 is a flowchart illustrating a method for determining information about a process consisting of a plurality of steps according to one embodiment of the present disclosure.

[0046] FIG. 18 is a block diagram of a system for determining information about a process consisting of a plurality of steps according to one embodiment of the present disclosure.

[0047] FIG. 19 is a drawing showing a sequential multi-agent according to one embodiment of the present disclosure.

[0048] FIG. 20 is a drawing showing a supervisory agent according to one embodiment of the present disclosure.

[0049] FIG. 21 is a drawing showing a hierarchical agent system according to one embodiment of the present disclosure.

[0050] FIG. 22 is a drawing showing a discussion-type multi-agent system according to one embodiment of the present disclosure.

[0051] FIG. 23 is a drawing showing a mixed agent system according to one embodiment of the present disclosure.

[0052] FIG. 24 is a drawing showing a ReAct agent system according to one embodiment of the present disclosure.

[0053] FIG. 25 is a drawing showing a CodeAct agent system according to one embodiment of the present disclosure.

[0054] FIG. 26 is a drawing showing a modern tool-using agent system according to one embodiment of the present disclosure.

[0055] FIG. 27 is a drawing showing a self-reflective agent system according to one embodiment of the present disclosure.

[0056] FIG. 28 is a drawing illustrating a multi-agent workflow system according to one embodiment of the present disclosure.

[0057] FIG. 29 is a drawing showing an Agentic RAG (Retrieval-Augmented Generation) system according to one embodiment of the present disclosure.

[0058] FIG. 30 is a drawing showing a Multi-Agent Debate (MAD) system according to one embodiment of the present disclosure.

[0059] FIG. 31 is a diagram showing an A2A (Agent2Agent) protocol system according to one embodiment of the present disclosure.

[0060] FIG. 32 is a drawing showing an Agentic RAG (Retrieval-Augmented Generation) system according to one embodiment of the present disclosure.

[0061] FIG. 33 is a schematic diagram of an AI agent system according to one embodiment of the present disclosure.

[0062] FIG. 34 is a schematic diagram of a Large Language Model (LM) chatbot system according to one embodiment of the present disclosure.

[0063] FIG. 35 is a schematic diagram of a Robotic Process Automation (RPA) system according to one embodiment of the present disclosure.

[0064] FIG. 36 is a schematic diagram of a RAG (Retrieval-Augmented Generation) system according to one embodiment of the present disclosure.

[0065] FIG. 37 is a schematic diagram of a Learning-Augmented Mechanism (LAM) system according to one embodiment of the present disclosure.

[0066] FIG. 38 is a diagram showing the memory structure of an AI agent according to one embodiment of the present disclosure and a question-and-answer process utilizing the same.

[0067] FIG. 39 is a diagram illustrating a text generation process of a first language model according to one embodiment of the present disclosure.

[0068] FIG. 40 is a diagram illustrating the operation process of a second language model according to one embodiment of the present disclosure.

[0069] FIG. 41 is a diagram illustrating a method of operation of a third language model according to one embodiment of the present disclosure.

[0070] FIG. 42 is a diagram illustrating a method of operation of a fourth language model according to one embodiment of the present disclosure.

[0071] FIG. 43 is a diagram illustrating a method of operation of a fifth language model according to one embodiment of the present disclosure.

[0072] FIG. 44 is a diagram illustrating the operation method of a Large Action Model (LAM) according to one embodiment of the present disclosure.

[0073] FIG. 45 is a diagram illustrating the operation method of a Hierarchical Reasoning Model (HRM) according to one embodiment of the present disclosure.

[0074] FIG. 46 is a drawing showing a ToolFormer (Tools-trained Model) according to one embodiment of the present disclosure.

[0075] FIGS. 47 to 52 are drawings illustrating vulnerabilities of an MCP according to one embodiment of the present disclosure.

[0076] FIG. 53 is a diagram illustrating a context engineering structure in an AI agent system according to one embodiment of the present disclosure.

[0077] FIG. 54 is a block diagram of a system according to one embodiment of the present disclosure.

[0078] FIG. 55 is a drawing showing a Graphical User Interface (GUI) of a schedule providing system according to one embodiment of the present disclosure.

[0079] FIG. 56 is a drawing showing a scheduling range selection interface according to one embodiment of the present disclosure.

[0080] FIG. 57 is a drawing showing an inspection success scenario according to one embodiment of the present disclosure.

[0081] FIG. 58 is a drawing illustrating an inspection failure scenario according to one embodiment of the present disclosure.

[0082] FIG. 59 is a diagram illustrating a method for providing an inspection warning message according to one embodiment of the present disclosure.

[0083] FIG. 60 is a drawing illustrating a method for setting a target period according to one embodiment of the present disclosure.

[0084] FIG. 61 is a drawing showing a GUI for setting a soft upper limit of concentration by characteristic according to one embodiment of the present disclosure.

[0085] FIG. 62 is a drawing illustrating a GUI showing concentration constraint information by characteristic according to one embodiment of the present disclosure.

[0086] FIG. 63 is a drawing showing a GUI for selecting a scheduling reflection option according to one embodiment of the present disclosure.

[0087] FIG. 64 is a drawing showing a second GUI for providing a generated schedule according to one embodiment of the present disclosure.

[0088] FIG. 65 is a flowchart illustrating a method for providing a schedule according to one embodiment of the present disclosure.

[0089] FIG. 66 is a diagram schematically illustrating the configuration of a reinforcement learning-based scheduling system according to one embodiment of the present disclosure.

[0090] To clarify the technical concept of the present disclosure, embodiments of the present disclosure will be described in detail with reference to the attached drawings. In describing the present disclosure, detailed descriptions of related known functions or components will be omitted if it is determined that such detailed descriptions would unnecessarily obscure the essence of the present disclosure. Components having substantially the same functional configuration among the drawings have been assigned the same reference numerals and symbols as much as possible, even if they are shown in different drawings. For convenience of explanation, devices and methods are described together where necessary. Each operation of the present disclosure does not necessarily need to be performed in the order described and may be performed in parallel, selectively, or individually.

[0091] The terms used in the embodiments of this disclosure have been selected to be as widely used and general as possible, taking into account the functions of this disclosure; however, these terms may vary depending on the intent of those skilled in the art, case law, the emergence of new technologies, etc. Additionally, in specific cases, terms have been selected at the applicant's discretion, and in such cases, their meanings will be described in detail in the description of the relevant embodiments. Therefore, terms used in this specification should be defined not merely by their names, but based on their meanings and the overall content of this disclosure.

[0092] Throughout this disclosure, singular expressions may include plural expressions unless the context clearly indicates otherwise. Terms such as “comprising” or “having” are intended to specify the presence of features, numbers, steps, actions, components, parts, or combinations thereof, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof. That is, throughout this disclosure, when a part is described as “comprising” a certain component, it means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0093] Expressions such as "at least one" modify the entire list of components and do not modify the components of the list individually. For example, "at least one of A, B, and C" and "at least one of A, B, or C" refer to only A, only B, only C, both A and B, both B and C, both A and C, all of A, B, and C, or any combination thereof.

[0094] Additionally, terms such as “...part,” “...module,” etc., as described in this disclosure refer to a unit that processes at least one function or operation, and may be implemented in hardware or software, or a combination of hardware and software.

[0095] Throughout the entire disclosure, when a part is described as being “connected” to another part, this includes not only cases where they are “directly connected” but also cases where they are “electrically connected” with other elements interposed between them. Furthermore, when a part is described as “comprising” a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0096] As used throughout this disclosure, the expression “configured to” may be replaced, depending on the context, with, for example, “suitable for,” “having the capacity to,” “designed to,” “adapted to,” “made to,” or “capable of.” The term “configured to” may not necessarily mean only “specifically designed to” in hardware. Instead, in some situations, the expression “system configured to” may mean that the system is “capable of” together with other devices or components. For example, the phrase “a processor configured (or set) to perform A, B, and C” may mean a dedicated processor for performing said operations (e.g., an embedded processor), or a generic-purpose processor (e.g., a CPU or an application processor) capable of performing said operations by executing one or more software programs stored in memory.

[0097] In one embodiment, an artificial intelligence (AI) agent refers to autonomous and intelligent software or system designed to perform specific tasks, and may mean a system equipped with the capability to make decisions and perform actions based on user instructions or changes in the environment. An AI agent is composed of five core elements: perception, knowledge base, reasoning and planning, action and actuation, and learning; each element interacts to enable the agent to perform given goals and continuously improve its performance. Perception may refer to the AI ​​agent detecting input data and interpreting it to recognize the environment. In the perception stage, sensors or external data

[0098] AI agents are computer system implementations of human decision-making and behavioral processes, and they have established themselves as a core component of current AI systems. The basic structure of an AI agent follows a cyclical pattern of 'Perception → Knowledge Base → Reasoning & Planning → Action & Actuation → Learning Loop.' Through this structure, AI agents perceive their environment, make decisions based on collected information, execute the results, and learn from those results to continuously improve.

[0099] AI agents can process various input data, make decisions based on it, and generate appropriate outputs or execute actions as a result. Furthermore, AI agents can possess the characteristic of continuously learning and improving by incorporating real-time feedback to achieve given goals. As part of a multi-agent system, AI agents can divide and optimize complex tasks through collaborative work and self-improvement.

[0100] No. Workflow Type Description Key Use Cases 1. Prompt Chaining: A method that breaks down a single task into steps, where the output of each step becomes the input for the next step. - Chatbot applications - AI agent-based tools 2. Parallelization: A method that executes identical tasks in parallel or subdivides them to generate a merged output. - Implementation of guardrails (safety devices) - Automated Evaluation Systems (EVALs) 3. Orchestrator-Worker: A central LLM (Orchestrator) breaks down the task, distributes processing to multiple worker LLMs, and integrates the results. - Agentic RAG - Coding Agent 4. Evaluator-Optimizer: An iterative structure where one LLM generates a result, and another LLM evaluates and modifies it. - Data science agent - Real-time monitoring system 5. Routing: A structure that classifies inputs into specialized sub-tasks and delivers them to different LLMs or paths. - Customer support agent - Multi-Agent Discussion System (MAD) 6. Autonomous Workflow: A loop-based structure where agents perform actions autonomously based on feedback from the external environment. - Autonomous robot agent - Computer-Aware Agent (CUA) 7. Reflection: Verbal A structure that learns and iteratively improves through feedback and self-reflection. - Complex data monitoring - Full-process app creation agent 8Rewoo A structure where a multi-stage planner creates tasks and drives efficient task execution through variable substitution. - Deep research agent - Multi-stage question-and-answer system 9Plan and Execute A planner creates sub-tasks, specialized agents process each task, and then integrate the results. - Business process automation - Data pipeline orchestration

[0101] In one embodiment, most of the structure may consist of a hierarchical structure of "Input → Plan / Evaluate / Branch → Output". Additionally, due to the agentic nature, complex problem solving may be possible through patterns such as iteration, parallelism, and collaboration rather than a single LLM call. Furthermore, since each structure is designed to suit a specific business purpose, selecting the optimal structure depending on the purpose may be important. For example, a workflow such as [Table 2] may be recommended depending on the business purpose.

[0102] Business Purpose Recommendation Workflow Conversational AI (e.g., Chatbots, Consultation) Prompt Chaining, Routing Real-time Analysis and Evaluation Evaluator-Optimizer, Reflection Complex Business Planning and Automation Plan and Execute, Rewoo Autonomous Behavior-based Systems Autonomous Workflow Large-scale Parallel Processing or Synthesis Parallelization, Orchestrator-Worker

[0103] Throughout this disclosure, reward shaping refers to a method of inducing learning by providing additional rewards when an agent performs a specific action or reaches a specific state. This may include manual reward shaping, where a person directly designs a reward function and grants additional rewards to the agent, and automatic reward shaping, where rewards are automatically formed according to specific rules or algorithms. A reinforcement learning system according to one embodiment is a learning method performed by an agent acquiring observations characterizing the current state of the environment and using those observations to select actions to be performed by the agent. In response to the agent performing a selected action, the environment transitions to a new state and the reinforcement learning system receives a reward. While interacting with the environment, the reinforcement learning system may attempt to maximize the long-term reward received by the agent in response to actions performed by the agent. A Macro-Action Decentralized Partially Observable Markov Decision Process (MacDec-POMDP) ​​may integrate an option framework into the Dec-POMDP by defining a set of macro-actions for each agent. In one embodiment, MacDec-POMDP is a tuple It can be expressed as. Here, is a set of identifiers for agents, S is a state space, and is a combined primitive behavior space, and is a combined macro behavior space, and is a combined primitive observation space, and is a combined macro-observation space, and is the probability of a state change, R is the reward shared by all agents, and is a combined observability, may be a combined macro-observability. Throughout this disclosure □ i represents an element or space of agent i, and is for all agents □ i It can imply the combination or multiplication of. Each macro action has a termination condition , initial set , and lower-level policies A tuple consisting of It could be. Here, (or ) is the observation history space of primitive (or macro) actions. In one embodiment, the goal of the MacDec-POMDP is a joint higher-level policy that maximizes the expected return from an initial state s0 for given lower-level policies (e.g., macro actions). It may be to find. For example, the joint upper-level policy can be determined by the following [Equation 1].

[0104] [Mathematical Formula 1]

[0105]

[0106] FIG. 1 is a diagram illustrating training buffers in MacDec-POMDP according to one embodiment of the present disclosure.

[0107] Referring to FIG. 1, the training buffers of Mac-CERTs (Macro-Action Concurrent Experience Replay Trajectories) (110) and Mac-JERTs (Macro-Action Joint Experience Replay Trajectories) (120) are illustrated. In FIG. 1, z i wa m i can mean the macro observation and macro action of agent i, respectively.

[0108] Referring to the example in FIG. 1, multiple agents may include Agent 1, Agent 2, and Agent 3. Agent 1 may act at time steps 4 and 6, Agent 2 at time steps 1 and 6, and Agent 3 at time step 5, so that a state transition may occur. In one embodiment, Mac-CERTs (110) may create a training buffer by collecting time steps where a state transition occurred for each agent. Accordingly, as in FIG. 1, a buffer containing information at time points t=0, 4, and 6 may be formed for Agent 1, a buffer containing information at time points t=0, 1, and 6 may be formed for Agent 2, and a buffer containing information at time points t=0 and 5 may be formed for Agent 3. Alternatively, Mac-JERTs (120) may create a training buffer by collecting time steps where a state transition occurred for any one of the agents. Accordingly, a buffer containing information for time points t=0, 1, 4, 5, and 6 can be formed. This will be explained in more detail below.

[0109] In Mac-CERTs according to one embodiment, the state transition experience of each agent i is a tuple It can be expressed as. Here, r i is the timestep t i Starting from macro behavior m that persists during a time step i It can mean accumulated rewards for. Accordingly, r i It can be defined as [Equation 2] below.

[0110] [Mathematical Formula 2]

[0111]

[0112] In Mac-JERTs according to one embodiment, the state transition experience is a tuple It can be expressed as. Here, joint macro action With the accumulated rewards, It is possible. Unlike Mac-CERTs, in Mac-JERTs, is the time step when an agent performs the corresponding macro action, and may refer to the end time step when the agent finishes the macro action. In the example of Fig. 1, Agent 2 starts the action at time step 1 and performs a joint macro action This is generated, and the next macro action is initiated by Agent 1 at timestep 4, and the next joint macro action is It could be.

[0113] Multi-Agent Reinforcement Learning is being actively developed and applied in various fields. Using Multi-Agent Reinforcement Learning, distributed agents can effectively collaborate to complete assigned tasks. Currently used Multi-Agent Reinforcement Learning adopts a method where all agents decide on their respective actions simultaneously due to identical durations between actions. However, in real-world scenarios, agents often decide and execute actions at different times, resulting in asynchronous decision-making and execution between agents. One embodiment of the present disclosure aims to provide a framework for modeling multi-agent decision-making where the decision-making and execution of actions between agents occur asynchronously over time using MacDec-POMDP. In one embodiment, in MacDec-POMDP, the start time of each agent's action and the duration of the action may differ.

[0114] FIG. 2 is a diagram showing a centralized critique using joint history encoders according to one embodiment of the present disclosure.

[0115] Referring to FIG. 2, a common history encoder (210) focused on the common history of all agents along the time axis is illustrated. In one embodiment, the training data (220) input to the common history encoder (210) is common macro observation data, which can be generated by collecting observations of time steps in which a state transition occurred for any one agent. For example, When represents the k-th macro observation of agent i, as agent 1 undergoes a state transition at time steps t=4 and 6, agent 2 undergoes a state transition at time steps t=2 and 6, and agent 3 undergoes a state transition at time step t=5, training data (220) can be generated to include time steps t=1, 4, 5, and 6 where a state transition occurs in any one of agents 1, 2, and 3. Accordingly, the training data (220) includes the observation of agent 1 at t=0 , Agent 2's observation , Agent 3's observation It may include, and since Agent 1 has no state transition at t=1, the same observation as at t=0. , Agent 2's observation , Agent 3 also has no state transition, so the same observation as at t=0. It may include, and at t=4, Agent 1's observation , Agent 2 has no state transition, so the same observation as at t=1 , Agent 3 also has no state transition, so the same observation as at t=0 or 1. It may include, and since Agent 1 has no state transition at t=5, the same observation as at t=4. , Agent 2 has no state transition, so the same observation as at t=1, 4. , Agent 3's observation It may include, and at t=6, the observation at Agent 1 , Agent 2's observation , Agent 3 has no state transition, so the same observation as at t=5 It may include. This training data (220) can be input into a joint history encoder (210).

[0116] In one embodiment, the joint history encoder (210) may include a first Multi-Layer Perceptron (MLP) (230) and a Gated Recurrent Unit (GRU) (240). The first MLP (230) may include an input layer, one or more hidden layers, and an output layer. The first MLP (230) uses training data (220) The output information is determined by taking the input, and the output information of the first MLP (230) can be input to the GRU (240). The GRU (240) is a model used to process sequential data and can be operated using a reset gate and an update gate. The reset gate determines how much of the previous state should be forgotten, and the update gate determines how much of the previous state information should be retrieved. The output information of the GRU (240) can be input to the second MLP (250).

[0117] A centralized critique focusing on the combination of the histories of all agents along this time axis can use a joint history encoder to abstract the combined macro observations. However, consecutive joint macro observations may contain duplicate observation data at time steps where one agent begins performing a macro action and other agents do not, and consequently, these duplicate observations may risk leading to inaccurate inferences. For example, at t=5 in Fig. 2, duplicate observation data for Agents 1 and 2 is included even though only Agent 3 undergoes a state transition, and such duplicate observation data can lead to inappropriate inference results.

[0118] In one embodiment, in the example of Figure 2 of the MacDec-POMDP, the combined macro observation history abstracted by the joint history encoder at time step t=6 can be expressed by the following [Equation 3].

[0119] [Mathematical Formula 3]

[0120]

[0121] As shown above, since the shared history encoder uses the same macro observation multiple times (for example, (used three times) can interfere with accurately determining the local history of all agents and reasoning between agents.

[0122] Therefore, because the common centralized critique structure for multi-agent reinforcement learning focuses on histories along a common time axis, redundant macro observations are introduced in the centralized critique for agents capable of non-macro actions at timesteps where one agent begins performing a macro action and other agents are unable to perform actions. However, such redundant macro observations can lead to inaccurate evaluations in the centralized critique. To address the limitations of this approach and enable efficient asynchronous learning, a special architecture is required that can effectively collect individual agent histories and facilitate inference between agents. To this end, one embodiment of the present disclosure aims to provide an agent-oriented centralized critique. This will be described in more detail later with reference to FIG. 3.

[0123] FIG. 3 is a diagram illustrating an Agent-Oriented Centralized Critic (AOCC) method according to one embodiment of the present disclosure.

[0124] Referring to FIG. 3, an Agent-Oriented Centralized Critic (AOCC) method for MacDec-POMDP is illustrated. In one embodiment, the agent-oriented reinforcement learning structure may include an agent-oriented history encoder (310, 320, 330) provided for each agent performing positional encoding, and an aggregation module (340) for integrating the histories of all agents. The agent-oriented history encoder (310, 320, 330) provided for each agent may receive the latest local macro observation of the corresponding agent as input and output a local macro observation history. Since this agent-oriented history encoder is not a centralized encoder like FIG. 2, it may not use the same observations redundantly. According to one embodiment, the agent-oriented history encoder may accurately capture local history by focusing only on determining the history of the corresponding agent.

[0125] In one embodiment, a GRU for an agent-oriented history encoder may be used without sharing parameters between agents. That is, a first agent-oriented history encoder (310) corresponding to a first agent may include a first GRU, a second agent-oriented history encoder (320) corresponding to a second agent may include a second GRU, and a third agent-oriented history encoder (330) corresponding to a third agent may include a third GRU.

[0126] In one embodiment, the history of agent i can be expressed as [Equation 4] as follows.

[0127] [Mathematical Formula 4]

[0128]

[0129] Here, Enc iis the agent-oriented history encoder of agent i, and can be the latest local macro observation of agent i.

[0130] In one embodiment, an agent-oriented history encoder may break the temporal alignment between agents because it encodes only each agent's local macro-observations rather than joint observations. In one embodiment of the present disclosure, sinusoidal position encoding may be performed to introduce time markers for each agent's macro-observations in order to account for temporal order. However, position encoding is not limited to the sinusoidal method, and various methods capable of representing time information, such as learnable embedding methods, may be applied. The time step of a local macro-observation is the position encoding vector p i It is encoded as, and the encoded result is observed by the local macro z i By connecting with it, the temporal order can also be considered. Accordingly, the history for agent i can be expressed as follows [Equation 5].

[0131] [Mathematical Formula 5]

[0132]

[0133] In one embodiment, location encoding may allow the chronological order and duration of macro actions across all agents to be taken into account.

[0134] In one embodiment, agent-oriented histories derived from an agent-oriented history encoder may be further processed by an aggregation module to approximate a value function. The aggregation module may include attention networks, an MLP following concatenation, a summing unit, etc. Alternatively, for simplification, only concatenation with MLP layers may be performed.

[0135] In one embodiment, the input of the aggregation module is a combination of the latest local histories, which can be expressed as [Equation 6] below.

[0136] [Mathematical Formula 6]

[0137]

[0138] The subsequent MLP layers use the shared history as their own values It can be switched to.

[0139] According to one embodiment of the present disclosure, the observation history and time information of each agent can be independently encoded through position encoding and then aggregated to enable efficient central critique learning in an asynchronous setting. That is, the observation history of each agent is encoded with the timestep information at the time when each agent begins to perform a macro action, and the time information is also encoded separately so that it can be explicitly associated for agent-oriented critique learning.

[0140] In addition, according to one embodiment of the present disclosure, reinforcement learning can be performed to derive more accurate results by deleting duplicate macro observations.

[0141] In one embodiment, the system inputs observation data of each of the multiple agents into an encoder module corresponding to each of the multiple agents to generate history information, and can generate output data based on the history information of each of the multiple agents. Here, the encoder module performs position encoding and may be provided for each of the multiple agents.

[0142] FIG. 4 is a drawing showing a reinforcement learning system according to one embodiment of the present disclosure.

[0143] Referring to FIG. 4, a reinforcement learning system performed by three agents, a first agent (410), a second agent (420), and a third agent (430), is illustrated as an example of a multi-agent reinforcement learning system. However, this is merely an example, and unlike FIG. 3, reinforcement learning may be performed by two agents, and it is obvious that reinforcement learning may be performed by more than three agents. For the convenience of explanation, the following will be described using the case of three agents as an example.

[0144] In one embodiment, the environment (405) may be changed by a first action (470) determined by a first agent (410), a second action (480) determined by a second agent (420), and a third action (490) determined by a third agent (430). Accordingly, the first agent (410) may obtain a first observation (440) based on the change in the environment, the second agent (420) may obtain a second observation (450) based on the change in the environment, and the third agent (430) may obtain a third observation (460) based on the change in the environment.

[0145] In one embodiment, a multi-agent reinforcement learning system may include a memory for storing one or more instructions and at least one processor for executing the one or more instructions stored in the memory. By executing one or more instructions, the at least one processor may input observation data corresponding to a first observation (440) of a first agent (410) into a first history encoder for reinforcement learning to generate first history information of the first agent (410). Additionally, the at least one processor may input observation data corresponding to a second observation (450) of a second agent (420) into a second history encoder to generate second history information of the second agent (420). Additionally, the at least one processor may input observation data corresponding to a third observation (460) of a third agent (430) into a third history encoder to generate third history information of the third agent (430). Such history encoders may be provided for each agent of the multi-agents. That is, if there are n agents, n history encoders can be provided.

[0146] In one embodiment, the first history encoder, the second history encoder, and the third history encoder may each perform encoding by including observation data of the corresponding agent and time information related to the observation. If time information is not included, a separate history encoder is provided for each agent, and inaccurate information may be generated because state transition times are not reflected; however, according to one embodiment of the present disclosure, accurate training can be enabled by encoding by matching time information to the observation data of each agent. In addition, the first history encoder, the second history encoder, and the third history encoder may each include a Multi-Layer Perceptron (MLP) and a Gated Recurrent Unit (GRU).

[0147] In one embodiment, a multi-agent reinforcement learning system may include an aggregation module that receives and processes one or more history information corresponding to the output of a history encoder provided for each agent of the multi-agents. That is, a single aggregation module that processes multiple history information may be provided. Additionally, the output value of the aggregation module may be processed by an MLP.

[0148] In one embodiment, the reward (400) may be determined by changing the environment (405) according to the first action (470) determined by the first agent (410), the second action (480) determined by the second agent (420), and the third action (490) determined by the third agent (430). Each agent may cooperate with one another to maximize the value of the reward.

[0149] Multi-agent reinforcement learning can face challenges in sparse reward environments where agents struggle to learn effective strategies. Macro-behaviors, which are a series of actions executed as a single decision, facilitate long-term planning; however, varying durations among agents introduce asynchronous behavior, which can complicate Centralized Training with Decentralized Execution (CTDE).

[0150] In the CTDE (Centralized Training with Decentralized Execution) framework, centralized critiques can generally be trained using the combined histories of all agents. The observation history of the i-th agent at each time step t can be expressed as follows.

[0151] [Mathematical Formula 7]

[0152]

[0153] Here, represents the sequence of all observations by Agent i up to time step t. The critique takes the set of all agents' histories as input and the value at time step t It can predict.

[0154] In an asynchronous setup, all agents observe the macro at every timestep t Since it is not possible to obtain, missing macro observations are the most recent valid macro observations when generating history It can be replaced with.

[0155] This mechanism critiques across time steps even when macro observations are missing It can be ensured that the input to is maintained consistently. By utilizing the latest valid macro observations, centralized critiques can effectively adapt to asynchronous environments while maintaining compatibility with the training process.

[0156] In a decentralized execution, each agent independently receives local macro observations and can select a macro action based on the current local macro observation and historical data. Unlike a critic, since each agent operates independently, there may be no structural difference between synchronous and asynchronous configurations. The macro action of the i-th agent at timestep t. Is It can be sampled as follows.

[0157] Currently studied multi-agent reinforcement learning methods utilize padding to handle asynchronous characteristics, but this padding process can misalign asynchronous experiences, leading to incorrect correlations.

[0158] According to one embodiment of the present disclosure, an Agent-Centric Actor-Critic (ACAC) method that resolves asynchronous behavior without relying on padding may be provided.

[0159] In one embodiment, a Proximal Policy Optimization (PPO) method incorporating a Generalized Advantage Estimate (GAE) method suitable for asynchronous settings may be used to train centralized critics and distributed actors. GAE is a technique that balances the variance of empirical returns and the bias of Temporal Difference (TD) estimation in estimating the advantage function using a hyperparameter λ. When λ=0, reliance is placed on a single-step TD, and when λ=1, the empirical returns over the entire interval excluding the baseline may be used. When Λ is a value between 0 and 1, a compromise balance between the two extremes may be provided. In asynchronous multi-agent reinforcement learning, the interval between consecutive macro observations can vary significantly depending on the agent, situation, etc. If the existing micro-timestep-based λ discount is applied as is, an excessive discount accumulates on the future reward of the decision as the duration of the macro action increases, which can lead to a problem where the relative importance of the macro decision is underestimated. Consequently, policy updates at the time of selecting a long-lasting macro action may be distorted, and learning stability and final performance may degrade.

[0160] According to one embodiment, a modified GAE method may be used to switch the criterion for the λ discount from a micro time step to a macro time step. A micro time step refers to a raw step of the simulator, and a macro time step may refer to an indicator that counts, in chronological order, a set of points in time when any agent acquires a new macro observation. Accordingly, according to one embodiment, the λ discount may be applied to the TD error based on the number of times a macro decision occurred, rather than the elapsed time (number of micro steps).

[0161] In one embodiment, using an actor-critic algorithm, each Distributed actors parameterized by r and centralized critics parameterized by ψ can be trained. The training procedure can be expressed as follows in [Equation 8].

[0162] [Mathematical Formula 8]

[0163]

[0164] Here, τ is the trajectory sampled using the current actors, and r t represents the reward at time step t, and the advantage function A t It can be defined as follows [Equation 9].

[0165] [Mathematical Formula 9]

[0166]

[0167] Here, t next is the next time step in which a new macro observation for an agent becomes possible.

[0168] In synchronous multi-agent reinforcement learning, it is common practice to achieve a shared history abstraction by collecting shared observations from all agents at every time step. However, in an asynchronous setup, only some agents may provide observations at a given time step, while observations are absent for the remaining agents. Conventionally, the shared history was constructed by filling in missing observations with padding data (e.g., copies of the previous observation); however, this padding causes redundant information and spurious correlations, which undermines the accuracy of the shared history representation.

[0169] FIG. 5 is a diagram showing an agent-centric encoder of an agent-centric centralized critic according to one embodiment of the present disclosure.

[0170] Throughout this disclosure, i represents the agent index and t represents the global time step of the environment. represents a macro-observation or equivalent state feature acquired by agent i at time step t, and represents the time information (time position) associated with the observation. is the hidden history representation of Agent i. The output V of the central critique t is the value function estimate at time step t.

[0171] The history of each agent can be abstracted from macro observations using recurrent networks. However, in asynchronous multi-agent reinforcement learning, it is essential to consider the elapsed time between consecutive macro observations. Without this duration information, the history of each agent may be learned inaccurately due to varying durations, as there is a lack of context regarding how long the interval between consecutive macro observations is—for example, whether it is 1 time step or 10 time steps.

[0172] Referring to FIG. 5, in order to solve these problems, one embodiment of the present disclosure involves macro observation and the corresponding time step An agent-centered encoder (500) integrating may be provided. The agent-centered encoder (500) is a time step It is embedded using sinusoidal position encoding, and this embedded time step can be associated with a macro observation. The combined information is processed through a multilayer perceptron (MLP) and a gated recurrence unit (GRU) to effectively abstract the history of each agent. The recurrence module can be implemented as an LSTM or a Transformer Encoder (including a time mask) instead of a GRU.

[0173] In one embodiment, the agent-centered encoder (500) may include an input unit (510), a time embedding unit (520), a combination node (530), a multi-layer perceptron (MLP) module (540), a GRU (550), etc.

[0174] In one embodiment, the input unit (510) and Can receive. may be a compressed vector of a vectorized observation or feature map. The time embedding unit (520) is It can be converted into sine / cosine-based position embeddings, learning-based embeddings, or log scale / clipping, etc. The resulting vector is TE( It can be expressed as ). In one embodiment, the joining node (530) is wa TE( ) can be connected. If necessary, additive combinations may be used. The MLP (multi-layer perceptron) module (540) can perform non-linear transformations by mapping the combination vectors to multiple perceptrons. Activation functions may include ReLU, GELU, etc. The GRU (Gated Recurrent Unit, 550) is the previous time step hidden state With the output of the MLP module (540) as input, the current hidden state It can be calculated. In FIG. 5, a GRU is shown as a recurrent module, but this is merely an example, and LTSM, RNN, etc. may be used. In one embodiment, the output unit (560) is the calculated It can be provided to Agent i's actors and centralized critique. The hidden state is at the start of the episode. Or initialized with a learning vector, optionally reset at macro-episode boundaries (e.g., completion of a specific process step), or damping coefficient Multiply by Memory reduction can be applied.

[0175] According to one embodiment, the combined information is processed by the MLP module (540) and the GRU (550) so that the history of each agent can be effectively abstracted. In addition, according to one embodiment, time step information macro observation By integrating with, the agent-centric encoder (500) can ensure that the history of each agent is learned accurately and consistently even in asynchronous multi-agent reinforcement learning.

[0176] According to one embodiment, the hidden state of the GRU (550) is updated only for agents that have received a new observation, and for agents that have not received an observation, the previous hidden state It can be maintained as is. Therefore, the history of each agent is preserved continuously without the injection of padding data, and redundancy and distortion of the history representation can be suppressed.

[0177] According to one embodiment, since the interval between observations is directly expressed without padding injection, history distortion caused by asynchronous duration inconsistency is reduced, the hidden state is maintained continuously and consistently even with imbalance in learning data between agents, and the bias in value estimation in time-sensitive rewards is lowered so that policy convergence can be stabilized.

[0178] In one embodiment, the parameters of the agent-centered encoder (500) are shared among all agents to increase data efficiency, while additionally connecting agent identification embeddings can be used to correct individual characteristics. In another embodiment, the parameters of the agent-centered encoder (500) may be partially shared only among sets of agents with different roles, or may not be shared at all.

[0179] FIG. 6 is a diagram illustrating a method for calculating a value function by integrating agent-centered history according to one embodiment of the present disclosure.

[0180] Referring to FIG. 6, the agent-centered centralized critique may include agent-centered encoders (500-1, 500-2, ..., 500-N) that abstract the history of each agent and an attention-based aggregation module (640) that combines these abstracted histories. Each of the agent-centered encoders (500-1, 500-2, ..., 500-N) may correspond to the agent-centered encoder (500) described above with reference to FIG. 5.

[0181] In one embodiment, an agent-centric encoder may be provided for each agent. The agent-centric encoder for agent i is and Current hidden state with as input It can output the last valid hidden state, even for agents with no observations at the last time step t. At time step t, the centralized critique can be calculated according to the following [Equation 10].

[0182] [Mathematical Formula 10]

[0183]

[0184] Here, refers to timestep information for agent i's macro observations up to timestep t.

[0185] In one embodiment, the self-attention module (610) is in a hidden state Using the input, a first aggregate expression can be produced by weighting the importance of interactions between agents. For example, the self-attention module (610) can convert input vectors into queries, keys, and values, and produce a first aggregate expression by weighting the importance of interactions between agents. The self-attention module (610) learnably reflects the relative importance of interactions between agents, enabling more precise co-inference compared to fixed-weight summation. Additionally, since the hidden state of unobserved agents is maintained without updating, duplicate inflow due to padding can be structurally blocked.

[0186] The average pooling module (620) can generate a global integrated representation by performing average pooling on the output of the self-attention module (610). In one embodiment, the output of the self-attention module (610) and the output of the average pooling module (620) may be combined or weighted summed. The MLP module (630) processes the integrated representation corresponding to the output of the average pooling module (620) using multiple perceptrons to perform a high-dimensional non-linear transformation, thereby generating a value function V t It can output. If necessary, layer normalization or scalar scale parameters may be added to stabilize the output scale.

[0187] That is, one embodiment of the present disclosure may organize the history of each agent on an agent-by-agent basis and then integrate these agent-by-agent histories into a common representation. Specifically, an agent-centric history encoder may be used to capture the macroscopic observation history of each agent. By aggregating this agent-centric history using an attention-based module, a more accurate common history representation for value estimation may be possible. By focusing on encoding the histories of each agent before combining them, according to one embodiment, cases where macroscopic observations are lacking can be effectively handled, and the limitations of padding-based methods can be overcome.

[0188] FIGS. 7a to 7c are drawings illustrating an Agent-Centric Actor-Critic (ACAC) method according to one embodiment of the present disclosure.

[0189] Referring to FIGS. 7a through 7c, at time step t=t1, the first agent (710) can acquire a new macro observation and, accordingly, select a new macro action. On the other hand, the second agent (720) and the third agent (730) cannot acquire a new macro observation because the previous macro action is still in progress. In this case, the first agent (710) that has acquired a new macro observation can input the macro observation into the encoder centered on the first agent to generate an updated history. Conversely, the second agent (720) and the third agent (730) that have not acquired a new macro observation can use the existing history as input to the aggregation module. Through this approach, central critics can accurately estimate the value function using the latest history information of each agent, even in an asynchronous multi-agent reinforcement learning setup.

[0190] For example, the first agent (710) obtains an observation at time step t=t1 and time step information indicating the time of acquisition of the observation or the location of the decision It can be input into the first agent-centered history encoder (500-1). The first agent-centered history encoder (500-1) is by the time embedding unit Converts into an embedding vector, and the embedding vector converted by the time embedding unit and Combines them, performs a non-linear transformation by the MLP module, and then the hidden state from all time steps by GRU Update to the current hidden state It can generate. The generated It can be provided to the policy output of the first actor and used to select a new macroscopic action, and at the same time, it can also be used in a centralized aggregation path.

[0191] On the other hand, since there is no observation for the second agent (720) and the third agent (730) at time step t=t1, each history encoder does not update the hidden state, and = , = The existing history can be maintained. The hidden representation maintained in this way can be input directly into the central aggregation path. In this case, padding may not be performed.

[0192] In one embodiment, at time step t=t1, the agent-specific history hiding representation{ , , } can be input to the aggregation module. The aggregation module can generate critique input features by weighting the relative importance of interactions between agents using a self-attention module, generating a global integrated representation using an average pooling module, and performing non-linear transformations using an aggregation MLP. Based on these features, the central critique has a value function V t1 It can generate.

[0193] In one embodiment, the update history of the first agent (710) that has acquired a new observation and the maintenance history of the second agent (720) and the third agent (730) that have no observation are considered simultaneously, so that the contribution of each history can be dynamically reflected in the aggregation module. Accordingly, even in an asynchronous multi-agent environment, the central critic can perform accurate value estimation by utilizing the latest and highly relevant history information.

[0194] FIG. 8 is a diagram illustrating an Agent-Centric Actor-Critic (ACAC) method according to one embodiment of the present disclosure.

[0195] Referring to FIG. 8, the ACAC system may include agent-centered distributed actors (810), agent-centered central critics (820), etc. Each agent may acquire macro observations at different times in an asynchronous environment.

[0196] In one embodiment, a system may be provided that includes an agent-specific history encoder and a central aggregation module to effectively process macroscopic observations acquired at different times by agent in an asynchronous multi-agent reinforcement learning environment. Even if some agents fail to acquire new observations at a specific time, previous history hiding is maintained without padding, so that a value function can be calculated precisely through aggregation based on self-attention and mean pooling.

[0197] In one embodiment, each agent may be equipped with an agent-centric history encoder. That is, a history encoder may be provided for each of the multiple agents. For example, if the multiple agents include a first agent and a second agent, the system may include a first history encoder corresponding to the first agent and a second history encoder corresponding to the second agent.

[0198] In one embodiment, an agent-oriented history encoder embeds temporal information, such as observation time and decision location, using positional encoding, combines the observation and temporal embeddings with an MLP, and updates the previous hidden state with a GRU to obtain the current history hidden representation h. i It can generate. The generated h i It is passed to the upper MLP policy head, and Agent i's policy π(h i A macro action based on ) can be selected. Only agents that have acquired a new observation update their GRU hiding, and the hiding of agents without observations is maintained, so that observation padding is not performed.

[0199] For example, when the first agent receives the first observation data at the first time point, the system can update the first agent's history information by inputting the first observation data and the corresponding time information into the first agent's history encoder. On the other hand, if the second agent's observation data is not acquired at the first time point, padding is not performed, and the previous hiding is maintained as is and used as the second agent's current history. That is, the second agent's history information can be maintained. Here, multiple agents may have different action start and / or duration times and may be asynchronous multi-agents that determine different information at different times.

[0200] In one embodiment, the history information of a plurality of agents is input into an aggregation module, and a value function can be calculated by performing self-attention and average pooling.

[0201] In one embodiment, h calculated by each agent encoder 1 , h 2 , h 3Hidden expressions such as can be input into the critique path. The critique may include another agent-centered history encoder and can generate the latest history by reflecting time information in a manner similar to the actor. In one embodiment, the actor and the critique may share a history encoder or have independent parameters. This set of histories can be input into the aggregation module (830).

[0202] In one embodiment, the aggregation module (830) is a module that aggregates the history information of each of a plurality of agents and may include a self-attention module and an average pooling module. The self-attention module may map input history into queries, keys, values, etc., to increase the relative importance of interactions between agents. The average pooling module may generate an aggregated representation by globally aggregating attention outputs. The MLP module may output a value function based on the aggregated representation.

[0203] In one embodiment, the history encoder of each agent may include an MLP module that processes observation data and time information corresponding to the observation data by combining them, and a GRU that processes the hidden state of the previous time point and the hidden state of the current time point in sequence.

[0204] In one embodiment, based on the output of the history encoder of the first agent, the policy of the first agent is determined, and the macro action of the first agent corresponding to the policy of the first agent can be determined.

[0205] In one embodiment, the system defines a set of macro decision time points l(0), l(1), ... and the advantage estimation at macro time point l(0) It can be calculated as. Here λ is the TD error, γ is the depreciation rate, and λ is a value between 0 and 1. The λ discount is accumulated based on the number of macro decision-making cycles, so that even if the duration of a macro action is long, the importance may not decrease excessively.

[0206] In one embodiment, policy updates can be performed using the clipping purpose of Proximal Policy Optimization (PPO). The value function is learned with a standard loss such as mean squared error, and if necessary, normalization (e.g., PopArt) can be applied to stabilize the value scale.

[0207] In one embodiment, an agent-centric actor-critic method can process the trajectory of each agent individually by using an agent-centric history encoder and perform attention-based centralized criticism to integrate this agent-specific information. According to one embodiment, by eliminating the redundancy inherent in padding-based methods, more accurate time abstraction, faster learning, and higher quality policies can be provided.

[0208] According to one embodiment, learning convergence is accelerated under sparse rewards, and higher accuracy can be achieved than with conventional padding-based approaches.

[0209] FIG. 9 is a schematic diagram of a system according to one embodiment of the present disclosure.

[0210] Referring to FIG. 9, the system may include a structure for performing multi-agent reinforcement learning. For example, the system may include a plurality of agents (910a to 910n), an agent-centric history encoder module (920), a centralized critique module (930), an actor-critic learning module (940), etc. The plurality of agents (910a to 910n) are entities that observe a state within an environment and perform actions, and each agent can select and execute a high-level action, i.e., a macro-action. Each macro-action is broken down into a plurality of low-level actions, i.e., micro-actions, and executed, and the execution time may vary from agent to agent. Accordingly, action selection between agents can be performed asynchronously. The system may collect history data including past state, action, and reward information of each agent.

[0211] In one embodiment, the agent-centric history encoder module (920) acquires history data including past state, behavior, and reward information of each agent, and can encode the collected history data including time point information. According to one embodiment, asynchronousness can be reflected by applying time point embeddings to each history data. To this end, the time point information may include behavior duration information, for example, the execution time of a macro behavior. Encoding can be performed using a Recurrent Neural Network (RNN), Long Short Term Memory (LSTM), Gated Recurrent Unit (GRU), or Transformer-based network. The agent-centric history encoder module (920) can naturally handle asynchronousness to enable learning without padding. Therefore, by processing the independent history of each agent without using padding, incorrect correlations occurring in the prior art can be prevented.

[0212] In one embodiment, the centralized critique module (930) can integrate history data encoded from multiple agents based on a self-attention mechanism. Through this, the centralized critique module (930) can reflect the importance of each agent as a learnable weight and estimate a co-value function. The self-attention may include multi-head attention, thereby allowing the interaction between agents to be effectively modeled. The centralized critique module (930) can estimate a co-value function by applying learnable weights that reflect the importance of each agent. Through this, the system can perform accurate value estimation even in an asynchronous environment.

[0213] In one embodiment, the actor-critic learning module (940) can update the policy of each agent based on the co-value function estimated by the centralized critique module (930). The policy update is performed based on the Proximal Policy Optimization (PPO) algorithm, and clipping and entropy regularization may be applied. Additionally, the actor-critic learning module (940) can strengthen long-term strategies using a modified Generalized Advantage Estimation (GAE) that applies a discount rate at the macro action level.

[0214] In one embodiment, the system can prevent spurious correlation and improve learning stability and efficiency by performing learning without padding while considering the asynchronous nature between agents.

[0215] In one embodiment, the system can identify whether the environment is a sparse reward environment and, if identified as such, apply reward shaping to improve learning efficiency. For example, the system can monitor the variance or frequency of rewards collected over a certain period and, if they fall below a threshold, automatically determine that it is a sparse reward environment and activate the reward shaping module. Thus, the system can achieve rapid convergence and high performance even in a sparse reward environment. Additionally, the system can dynamically change the reward shaping strategy according to the characteristics of the environment. For example, in a sparse reward environment, it can maximize learning efficiency by setting an intermediate goal to provide rewards, while in a dense reward environment, it can maintain the existing reward structure. Furthermore, by applying an adaptive algorithm that automatically adjusts reward shaping rules according to changes in the environment, the system can support stable learning in various simulation environments.

[0216] In one embodiment, the system can accelerate large-scale multi-agent learning through GPU-based parallel processing. The model can be lightweighted to enable execution on mobile devices or edge computing environments. To this end, the system can reduce model size and computational load by applying techniques such as network pruning, quantization, and knowledge distillation. This lightweighting can enable real-time learning and inference in environments with limited hardware resources, such as robot control, IoT devices, and autonomous vehicles.

[0217] In one embodiment, the system can operate not only on a single computing node but also in a distributed learning environment where multiple nodes are connected via a network. To this end, the system applies a parameter server structure to centrally aggregate and synchronize model parameters calculated at each node, thereby improving the learning speed in a large-scale multi-agent environment. This distributed processing method can enable efficient learning even in complex simulation environments containing thousands or more agents.

[0218] In one embodiment, the attention mechanism used in the centralized critique module may apply a hierarchical attention structure in addition to basic multi-head attention. Agent groups may be clustered based on physical distance, role, or functional similarity, and attention within the group may be performed in the first step, and attention between group representatives may be performed in the second step. For example, by first performing attention at the agent group level and then additionally applying attention between groups, the system can reduce computational complexity and effectively model interactions even in environments with a very large number of agents. This structure can be particularly useful in large-scale team-based simulations performing collaborative tasks.

[0219] FIG. 10 is a block diagram illustrating the internal structure of an agent-centered history encoding module according to one embodiment of the present disclosure.

[0220] Referring to FIG. 10, the agent-centered history encoder module (920) performs the function of encoding history data collected from multiple agents while reflecting asynchronousness, and may include a history data input unit (1010), a time point embedding processing unit (1020), and an encoding network (1030).

[0221] In one embodiment, the history data input unit (1010) may receive history data including past state, action, and reward information of each agent. The history data is collected independently for each agent and may be recorded in units of macro-actions. Since each macro-action is broken down into multiple micro-actions and executed, the history data may have asynchronous characteristics. In one embodiment, the input may be processed without data loss by taking this asynchronous nature into account.

[0222] In one embodiment, the time embedding processing unit (1020) may apply a time embedding to reflect time information in the input history data. The time embedding processing unit (1020) may model the difference in execution time between agents by adding an embedding that indicates the time at which each state and action occurred. In addition to time information, the time embedding processing unit (1020) may additionally include the duration of the action to reflect the execution length of the macro action. The embedding method may be implemented as a positional embedding or a learnable embedding.

[0223] In one embodiment, the encoding network (1030) can convert history data reflecting time information into a high-dimensional representation. The encoding network (1030) may use a Recurrent Neural Network (RNN), Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), or Transformer-based structure. When the encoding network (1030) applies a Transformer-based structure, it can model long-term dependencies through a self-attention mechanism. Additionally, the encoding network (1030) can effectively process asynchronous data sequences and learn agent-specific behavior patterns.

[0224] Here, the Self-Attention used when the encoding network (1030) is implemented based on a Transformer models the influence of a specific past point in time on current decision-making within the time-series data of a single agent, and can be distinguished from the attention that models the interaction between agents in the centralized critique module (930) described later.

[0225] In one embodiment, history data received from the history data input unit (1010) can be converted into a high-dimensional vector in the encoding network (1030) after asynchronous behavior is reflected through the time point embedding processing unit (1020). The result of this encoding is transmitted to a centralized critique module and can be used to model interactions between multiple agents.

[0226] FIG. 11 is a block diagram illustrating the internal structure of a centralized critique module according to one embodiment of the present disclosure.

[0227] Referring to FIG. 11, the centralized critique module (930) may include an encoding network (1110), a self-attention module (1120), a GAE computation unit (1130), a macro action processing unit (1140), etc. to reflect asynchronousness in multi-agent reinforcement learning and to efficiently perform policy updates.

[0228] In one embodiment, the encoding network (1110) receives a high-dimensional vector transmitted from an agent-centered history encoding module and can convert it into a form usable by a centralized critique module. The encoding network (1110) may apply a Transformer-based structure and may also apply recurrent structures such as RNN, LSTM, and GRU. The encoding network (1110) can normalize the input for modeling interactions between agents and prepare it to be combined with an attention mechanism.

[0229] In one embodiment, the self-attention module (1120) can model interactions between agents by integrating encoded history data. For example, when multi-head attention is used, relationships between agents can be learned from various perspectives using multiple attention heads. The self-attention module (1120) can estimate a co-value function by applying learnable weights that reflect the importance of each agent. Additionally, the self-attention module (1120) can enable efficient computation even in a large-scale agent environment by applying hierarchical attention.

[0230] In one embodiment, the GAE calculation unit (1130) can perform reward calculations suitable for an asynchronous environment based on Generalized Advantage Estimation (GAE). The GAE calculation unit (1130) can support stable learning by adjusting the discount rate according to the execution length of each macro action to strengthen long-term strategies and prevent reward distortion caused by asynchronousness.

[0231] In one embodiment, the macro action processing unit (1140) can identify the duration and end time during which a selected macro action is broken down into a plurality of micro actions and executed, and determine the policy update timing based thereon. Additionally, the macro action processing unit (1140) can adjust the learning schedule based on the execution time (duration) of the macro action.

[0232] In one embodiment, the system processes inputs in an encoding network (1110), models interactions between agents through a self-attention module (1120), calculates rewards in a GAE computer (1130), and performs policy updates in a macro action processor (1140). This process can be combined with a Proximal Policy Optimization (PPO) algorithm to apply clipping and entropy regularization.

[0233] In one embodiment, a new type of encoder combining a position encoding technique may be used in the process of encoding and aggregating observation history collected from multiple agents. This allows time-stamped information to be effectively reflected in the history data of each agent and prevents information loss caused by asynchronous action execution. Additionally, the encoder according to one embodiment is designed so that actors and critics can share the same type of encoding network, thereby maintaining consistency in the network structure and improving learning efficiency.

[0234] In addition, in one embodiment, a self-attention mechanism may be applied during the process of abstracting the joint history. This allows for encoding by considering the interrelationships between the histories of each agent, and enables more sophisticated interaction modeling compared to simple data merging methods. In particular, by applying multi-head attention, interactions between agents can be learned from various perspectives, and if necessary, the hierarchical attention structure can be extended to support efficient computation even in large-scale agent environments.

[0235] In addition, in one embodiment, a new reinforcement learning model and algorithm framework based on an Agent-Centric Actor-Critic structure may be utilized based on encoding and aggregation methods. This structure can independently encode the observation history of each agent and then train a Centralized Critic Network and a Decentralized Actor Network through an aggregation module. Compared to existing simple centralized learning methods, this structure is robust to asynchronous operations and can provide high learning stability in a cooperative multi-agent environment.

[0236] In addition, in one embodiment, a Proximal Policy Optimization (PPO)-based algorithm may be applied instead of the existing Naive Actor-Critic algorithm to learn the agent's policy function. PPO ensures the stability of policy updates through clipping and entropy regularization and can significantly improve sample efficiency. Furthermore, the system can strengthen long-term strategies and prevent reward distortion caused by asynchronous behavior by modifying Generalized Advantage Estimation (GAE) to apply a discount rate at the macro-action level.

[0237] In the following description of an embodiment of the present disclosure, for convenience of explanation, a method for determining the schedule of a naphtha cracking center (NCC) will be described as an example. However, the embodiments of the present disclosure are not limited to a method for determining the schedule of a naphtha cracking center, and can, of course, be applied to a method for determining the schedule of other processes or to a method for determining information other than a schedule. An embodiment of the present disclosure can be applied to a method for determining information about a process in multiple steps.

[0238] FIG. 12 is a schematic diagram of a product production process of a naphtha cracking center according to one embodiment of the present disclosure.

[0239] Referring to Fig. 12, at a Naphtha Cracking Center (NCC), naphtha, which is a gasoline fraction obtained from an atmospheric distillation unit of crude oil, is thermally cracked in a high-temperature cracking furnace, and then through processes such as rapid cooling, compression, and refining, ethylene, propylene, butylene, and BTX (Benzene, Toluene, Xylene), which are basic raw materials for petrochemical products, can be produced.

[0240] In other words, naphtha can be converted into substances with high industrial utility, such as ethylene, propylene, benzene, toluene, and xylene, through steam cracking or thermal cracking. For example, ethylene serves as a raw material for making polyethylene and polystyrene, propylene serves as a raw material for making polypropylene, and butane or butylene can be used to make synthetic rubber. These substances serve as raw materials for the plastics processing, textile, rubber, paint, and detergent industries, and can become final products such as daily necessities, adhesives, dyes, pesticides, pharmaceuticals, industrial products, and interior materials.

[0241] A naphtha cracking center is a core facility that produces petrochemical raw materials through a complex process and consists of a receiving stage for unloading naphtha, a mixing stage for blending naphtha, and a cracking stage for producing marketable products. More specifically, naphtha is initially transported from various geographically distributed refineries via vessels and unloaded into receiving tanks; various types of naphtha from these receiving tanks are then supplied to mixing tanks; and the naphtha blended in the mixing tanks is heated in a cracking furnace to produce marketable products of the desired quality. That is, the product production process of the naphtha cracking center may include a receiving process of storing naphtha supplied from one or more vessels (1210) or companies (e.g., other oil companies) in one or more receiving tanks (1220), a mixing process of transferring the naphtha from the receiving tanks (1220) to a mixing tank (1230) for a naphtha cracking process, and a cracking process of thermally cracking the naphtha supplied from the mixing tanks (1230) at high temperature in a furnace (1240). Here, the mixing tanks (1230) may also be referred to as blending tanks or feed tanks.

[0242] In one embodiment, the product production process of the naphtha cracking center may further include a process of measuring the paraffin content of naphtha supplied from a vessel (1210) or a company, and a process of measuring the paraffin content of naphtha stored in an incoming tank (1220), a mixing tank (1230), etc.

[0243] In one embodiment, the constraints may include a range for the paraffin content for each tank. For example, the paraffin content of the mixing tank (1230) may be limited to a range of about 80 to 83% based on the total weight of the naphtha. Since naphtha has different properties depending on the country of origin or company, the naphtha stored in the receiving tank also has different properties, and the receiving and mixing processes must be performed so that the paraffin content of the mixing tank (1230) satisfies the range of the constraints. However, the constraints are not limited to paraffin content and may include various physical / chemical constraints such as specific gravity, sulfur content, delivery date, and tank level limit.

[0244] Considering these constraints, determining the optimal schedule for the naphtha cracking center is crucial for profitability and efficiency. Generally, experts decide based on their experience and know-how which incoming tank to store naphtha, at what ratio to mix it into the mixing tank, and to what extent to heat it using which cracking furnace. However, relying on human experience to determine the schedule has limitations in predicting complex chemical reactions and actual results; results vary significantly depending on the level of the expert's experience and know-how; it is difficult to verify whether all constraints have been satisfied; and it is difficult to respond to sudden changes in circumstances.

[0245] Accordingly, the present disclosure aims to provide a method for determining optimal information (e.g., a schedule) using artificial intelligence. For example, multi-agent reinforcement learning may be utilized. A naphtha cracking center can be operated autonomously using multi-agent reinforcement learning, in which real-world constraints are overcome and each agent takes responsibility for a step and cooperates to achieve a common goal.

[0246] In the present disclosure, each furnace or each tank is modeled as an individual agent, capable of deciding its own actions and cooperating.

[0247] FIG. 13 is a diagram illustrating a reinforcement learning method using multiple agents according to one embodiment of the present disclosure.

[0248] Referring to FIG. 13, a naphtha cracking center can be operated according to an optimal schedule using scheduling information determined through reinforcement learning using multiple agents (1310, 1320, 1330). A simulator using reinforcement learning can take actions (1340, 1350, 1360) from agents (1310, 1320, 1330) and provide the next observation and reward (1380) based on the current action. In one embodiment, each agent is responsible for a specific process and can cooperate with each other to achieve goals such as profit maximization while complying with real-world constraints. For example, there may be realistic constraints such as performing the transfer process from the receiving tank to the mixing tank for at least 8 hours, within a range that does not exceed the minimum naphtha storage capacity of the receiving tank and the maximum naphtha storage capacity of the mixing tank.

[0249] In one embodiment, each agent can determine the information necessary to generate scheduling information for naphtha cracking centers for a predetermined future period based on current information and various constraints, such as the inventory status of each tank, ship arrival plans, naphtha supply plans from other companies, and prices of naphtha and marketable products.

[0250] In one embodiment, each of the multiple agents may produce different results (e.g., durations) at different times. For example, the first agent (1310) is an agent managing incoming goods and may decide actions such as selecting an incoming tank to store naphtha when a ship arrives irregularly and determining the amount to store in that incoming tank, and the second agent (1320) is an agent for mixing naphtha and may decide actions such as determining an incoming tank to bring naphtha to a mixing tank and determining the amount to bring from that incoming tank when the level of a certain incoming tank reaches a threshold (e.g., 90% of the tank capacity). Additionally, the third agent (1330) is an agent managing a cracking furnace and may decide actions such as receiving naphtha from a mixing tank and determining variables to operate the cracking furnace when the product inventory is below a predetermined amount. A virtual NAFTA operating environment (1370) can be created using actions (1340, 1350, 1360) determined at different times. The simulation device can determine expected profits in the virtual NAFTA operating environment (1370) and determine a reward (280) based thereon. This reward can be delivered to multiple agents (1310, 1320, 1330) and used by the agents to perform reinforcement learning. That is, multiple agents (1310, 1320, 1330) can be trained using the same reward during reinforcement learning. However, it is also possible for multiple agents to be trained using different rewards.

[0251] In one embodiment, the reward (1380) may be determined based on total revenue, facility operating costs, naphtha purchase costs, costs according to constraints, etc. For example, the reward may be determined by the following [Equation 11].

[0252] [Mathematical Formula 11]

[0253]

[0254] In [Equation 11], Constraints are constraint conditions, and w c is the weight per constraint, and Cost c can refer to the cost incurred per constraint. Accordingly, the more constraints are violated, the higher the Cost c The value can increase. For example, if the constraint includes the stability of the paraffin component—that is, the condition that the paraffin component must be maintained to a certain extent—the change in the paraffin component stored in the mixing tank can be used as variable c.

[0255] In addition, in [Equation 11], the profit can be calculated by subtracting the estimated production cost of marketable products from the estimated revenue generated from the sale of naphtha, taking into account facility operating costs (e.g., energy usage costs) and naphtha purchase costs. For example, the profit can be determined according to [Equation 12], which subtracts facility operating costs and naphtha purchase costs from total revenue as follows.

[0256] [Mathematical Formula 12]

[0257]

[0258] In one embodiment, Revenue can be calculated as, for example, "CH4 production volume * CH4 product price + PSA OFF GAS production volume * PSA OFF GAS price + RC2 production volume * ethane product price + C3 LPG production volume * propane product price + ethylene production volume * C2 product price + propylene production volume * C3 product price + H2 (99%) production volume * 99% H2 product price + HRPG production volume * HRPG product price + PFO production volume * PFO product price + Raw C5 production volume * Raw C5 product price + (Mixed C4 production volume * Mixed C4 product price) + (RPG production volume - Raw C5 production volume) * RPG product price".

[0259] In one embodiment, energy usage can be calculated as, for example, "[(Naphtha input amount + C3 LPG input amount + C4 LPG input amount) * A + Mixed C4 production amount * B + (RPG production amount Raw C5 production amount) * C] * C3 LPG price / C3 LPG calorific value / 1000", where A is the average energy unit value of a naphtha cracking center plant, B is the average energy unit value of a BD plant, and C is the average energy unit value of a BTX plant (for example, a plant that produces aromatic products using pyrolysis gasoline produced from an ethylene plant).

[0260] In one embodiment, hh can be calculated as "Total naphtha feed input * naphtha price + C3 LPG input * C3 LPG price + C4 LPG input * C4 LPG price + RC2 input * ethane product price".

[0261] In one embodiment, when an optimal scheduling is determined through such reinforcement learning, the naphtha cracking center can be operated according to the generated optimal scheduling. For example, if multiple schedules are formed and provided to a user, the user can operate the naphtha cracking center based on one of them.

[0262] FIGS. 14a and FIGS. 14b are drawings illustrating a reinforcement learning method using multiple agents according to one embodiment of the present disclosure.

[0263] Referring to FIG. 14a, an asynchronous multi-agent system is illustrated in which the start and duration times of the actions of each agent are different. For example, if the multi-agents consist of three first agents (1310), second agents (1320), and third agents (1330), each agent determines a different action at a different time, and the determined action vector The changed state vector is transmitted to the environment (1370) and the actions are applied to the environment (1370). It can be determined. Also, the action termination vector The reward generated as a result of applying actions to the environment (1370) can be provided to the agents. In addition, the reward generated as a result of applying actions to the environment (1370) can be provided. It can also be provided to each agent. The actions of each agent can be determined asynchronously as shown in Fig. 14b.

[0264] In another embodiment, the actions of each agent may be transmitted to their respective environment (1370) whenever an action is determined. For example, each action may be reflected in their respective environment (340) such that at the first time when the first agent (1310) determines the first action, the first action is reflected in the environment (1370), at the second time when the second action is determined, the second action is reflected in the environment (1370), and at the third time when the second agent (1320) determines the third action, the third action is reflected in the environment (1370). However, since the reward is determined by assuming that the product is ultimately produced and sold, the reward may be determined and provided to each agent after the actions of the first agent (1310), the second agent (1320), and the third agent (1330) are all reflected in the environment (1370).

[0265] In one embodiment, the state, behavior, and reward of each agent may include the following information.

[0266] 1. First Agent

[0267] - Status: Naphtha receiving schedule (e.g., vessel schedule or receiving plans from other companies), current status of receiving tanks (e.g., naphtha inventory and properties per tank), constraints related to receiving tanks

[0268] - Action: Identifier of the receiving tank to store naphtha upon receipt, the amount of naphtha to be stored in the corresponding receiving tank, or the pipeline connection schedule (e.g., pipeline connection with Vessel A to Receiving Tank No. 1 from 2:00 PM to 10:00 PM)

[0269] - Reward: Returns considering whether constraints are satisfied

[0270] 2. Second agent

[0271] - Status: Stock and properties of each receiving tank, stock and properties of the mixing tank

[0272] - Action: Identifier of the receiving tank to hold at least some naphtha in the mixing tank, the amount of naphtha to be brought from that receiving tank to the mixing tank, or the pipe connection schedule (e.g., pipe connection from Receiving Tank No. 1 to the mixing tank from 8:00 AM to 2:00 PM)

[0273] Compensation: Returns considering whether constraints are satisfied

[0274] 3. Third agent

[0275] Status: Inventory and properties of mixing tanks, operating status by disassembly furnace

[0276] Action: Feed rate, COT, DS ratio, etc. to feed naphtha from the mixing tank into each cracking furnace

[0277] Compensation: Returns considering whether constraints are satisfied

[0278] However, this is merely one example, and it goes without saying that the state, behavior, and rewards of each agent can be adjusted differently.

[0279] In one embodiment, the naphtha cracking center scheduling system can acquire input information. For example, the input information can be acquired through user input based on a User Interface (UI).

[0280] In one embodiment, the input information may include constraints, naphtha receiving schedule information, tank inventory information, naphtha property information within the tank, mixing tank operation information, cracking furnace operation plan information, target production volume information for each specific product, raw material unit price information, product unit price information, etc. In another embodiment, some of the information, such as constraints, may be pre-set information, and in that case, may not be included in the input information because it has been set in advance.

[0281] In one embodiment, constraints may include physical constraints such as tank storage capacity criteria to be satisfied and the number of pipes that can be connected at once, stability constraints regarding stability, and operational constraints for complying with a set target production volume during a specific period (e.g., weekly or monthly). Additionally, naphtha receiving schedule information may include a vessel receiving schedule, tank information of another company, a naphtha receiving schedule for a specific future period, scheduled receiving date and time, receiving rate, receiving quantity, naphtha property information, and identification information based on the naphtha receiving method (e.g., vessel identifier, tank identifier of another company, pipe identifier by company, etc.).

[0282] In one embodiment, at least one of the input information may include information for a specific period or information at a specific point in time. For example, the tank inventory information and the naphtha properties information within the tank, respectively, may each include the naphtha inventory information of the corresponding tank at the time of the scheduling start and the naphtha properties information of the corresponding tank at the time of the scheduling start.

[0283] In one embodiment, the blending tank operation information may include one or more blending schedules, such as a blending start time, a blending end time, the name of the receiving tank to be blended, and a blending speed per receiving tank. For example, the blending schedule may include a recent blending schedule.

[0284] In one embodiment, the decomposition furnace operation plan information may include schedule information for each decomposition furnace for a future specified period. For example, the decomposition furnace operation plan information may include schedule information determined for each decomposition furnace for the next 30 days. In one embodiment, the decomposition furnaces may exist in various types. For example, if there are a first decomposition furnace, a second decomposition furnace, and a third decomposition furnace of different types, the decomposition furnace operation plan information may include schedule information determined for each of the first decomposition furnace, the second decomposition furnace, and the third decomposition furnace for the next 30 days. The decomposition furnace operation plan information may include a decomposition start time, a decomposition end time, an operating mode (or feed mode), decoking schedule information, COT (coil outlet temperature), coil outlet pressure, a predetermined speed (e.g., feed rate), a DS (Dilution Steam) ratio, etc.

[0285] In one embodiment, the target production volume information for a specific product may include the target production volume or rate for a specific product over a specific period in the future. For example, the daily target production volume of ethylene for the next 30 days, the daily target production volume of propylene for the next 30 days, etc., may be included in the target production volume information for a specific product.

[0286] In one embodiment, the raw material unit price information may include raw material unit price information at the time of information input, raw material unit price information for a specific period prior to the time of information input, and expected raw material unit price information for a specific period after the time of information input. For example, the raw material unit price information may include the expected daily price of raw materials for the next 30 days.

[0287] In one embodiment, the product unit price information may include unit price information at the time of inputting information for each product, unit price information for a specific period prior to the time of inputting information for each product, and expected unit price information for a specific period after the time of inputting information for each product. For example, the product unit price information may include the expected daily price of naphtha products for the next 30 days.

[0288] However, the above input information is merely an example and is not limited thereto, and various input information for scheduling the naphtha cracking center may be included.

[0289] In one embodiment, a naphtha cracking center scheduling system may determine receiving tank information using a first agent based on input information. In one embodiment, the first agent may be an agent trained using reinforcement learning. Based on input information including a ship receiving schedule, a naphtha receiving schedule from another company, real-time receiving tank inventory, naphtha properties information within the tank, cracking furnace operation plan information, etc., the first agent may determine a receiving tank to receive naphtha from at least one of the tanks of a ship and another company, and determine the amount, ratio, or schedule information of naphtha to be stored in the corresponding receiving tank. For example, based on the input information, the first agent may determine an identifier for at least one receiving tank to store naphtha among a plurality of receiving tanks, and determine naphtha receiving ratio or amount information for each tank corresponding to each identifier. Additionally, based on the input information, the first agent may determine naphtha receiving schedule information and information on the period for storing naphtha in the corresponding receiving tank for each tank corresponding to each identifier. The receiving schedule information may include date or time information for connecting the receiving tank to a vessel or another company's device via a pipe (e.g., receiving naphtha from Vessel B to Receiving Tank A from 2:00 PM to 6:00 PM). In one embodiment, pipes may be connected as a method to transfer naphtha from a vessel or another company's device to a receiving tank; however, since connecting pipes and performing other tasks is inconvenient if the schedule changes frequently, constraints such as a minimum connection time of n hours per pipe may exist. The first agent may determine the receiving tank information by taking these constraints into account.

[0290] In one embodiment, the naphtha cracking center scheduling system can obtain naphtha property information corresponding to each receiving tank after a predetermined amount of naphtha has been distributed to the receiving tank.

[0291] In one embodiment, all receiving tank information may be determined using a single first agent, receiving tank information may be determined using a different first agent for each receiving tank, or receiving tank information may be determined using a different first agent for each receiving tank group. That is, there may be one or more first agents.

[0292] In one embodiment, the naphtha cracking center scheduling system can determine mixing tank combination information using a second agent. In one embodiment, the second agent may be an agent trained using reinforcement learning. The second agent can determine the mixing tank combination information based on the inventory of each receiving tank, the properties of the naphtha stored in each receiving tank, etc.

[0293] In one embodiment, the blending tank combination information may include an identifier of at least one receiving tank among a plurality of receiving tanks to transfer naphtha into the blending tank, naphtha ratio information (or amount information) to be transferred to the blending tank for each of the at least one receiving tank identifiers, blending schedule information with the blending tank for each of the at least one receiving tanks, naphtha blending ratio information for each of the at least one receiving tanks, blending execution date information, etc. The naphtha blending ratio information may include ratio information or amount information of naphtha to be taken from each receiving tank.

[0294] In one embodiment, all mixing tank combination information may be determined using a single second agent, receiving tank information may be determined using a different second agent for each mixing tank, or receiving tank information may be determined using a different second agent for each mixing tank group. That is, there may be one or more second agents.

[0295] In one embodiment, the naphtha cracking center scheduling system may determine cracking furnace operation information using a third agent. In one embodiment, the third agent may be an agent trained using reinforcement learning. The third agent may determine cracking furnace operation information based on the inventory information of the mixing tank, the properties of the mixing tank, cracking furnace status information, etc. In one embodiment, the cracking furnace operation information may include cracking furnace mode information, cracking furnace identifier, feed rate, COT (Coil Outlet Temperature), DSR (Dilution Steam Ratio), heating time, cracking furnace operation schedule information, and one or more variables for cracking furnace operation.

[0296] In one embodiment, there may be multiple third agents. For example, different agents may be used for each cracking furnace mode. For example, the third agents include a 3-1 agent reinforced learning for the cracking furnace of mode A, a 3-2 agent reinforced learning for the cracking furnace of mode B, and a 3-3 agent reinforced learning for the cracking furnace of mode C, and the naphtha cracking center scheduling system may determine the operation information for the cracking furnace of mode A using the 3-1 agent for the cracking furnace of mode A, determine the operation information for the cracking furnace of mode B using the 3-2 agent for the cracking furnace of mode B, and determine the operation information for the cracking furnace of mode C using the 3-3 agent for the cracking furnace of mode C. In another embodiment, a single third agent may determine all cracking furnace operation information.

[0297] In one embodiment, the naphtha cracking center scheduling system can determine one or more scheduling information of a naphtha cracking center based on receiving tank information generated by a first agent, mixing tank combination information generated by a second agent, and cracking furnace operation information generated by a third agent. In one embodiment, the scheduling information may include receiving scheduling information, mixing scheduling information, cracking furnace scheduling information, expected production volume information, expected profit information, expected naphtha inventory information, expected properties information, constraint satisfaction test result information, scheduling graph, etc.

[0298] In one embodiment, the receiving scheduling information may include receiving schedule information for a predetermined period in the future. For example, the receiving scheduling information may include tank identification information to be received for the next two weeks, the start time of receiving for the tank, the end time of receiving for the tank, etc.

[0299] In one embodiment, the mixing scheduling information may include mixing schedule information for a predetermined period in the future. For example, the mixing scheduling information may include mixing tank identification information to be used for the next two weeks, the mixing start time of the mixing tank, the mixing end time of the mixing tank, and mixing speed information of the tank (e.g., mixing speed of Tank A: about 100 Ton / hour).

[0300] In one embodiment, the decomposition furnace scheduling information may include schedule information for each decomposition furnace for a predetermined period in the future. For example, the decomposition furnace identification information to be used for the next two weeks, the start time of decomposition for the corresponding decomposition furnace, the end time of decomposition for the corresponding decomposition furnace, the decomposition speed (e.g., target control speed determined by artificial intelligence, feed rate, etc.), COT, DS ratio, etc. may be included in the decomposition furnace scheduling information.

[0301] In one embodiment, the expected production volume information, expected revenue information, expected naphtha inventory information, expected properties information, etc., may also be expected information for a predetermined period in the future. For example, the expected production volume information may include daily expected production volume provided by product for the next two weeks. Additionally, the expected naphtha inventory information may include naphtha inventory or naphtha change information provided by tank for the next two weeks, and the expected properties information may include properties change information provided by tank for the next two weeks.

[0302] In one embodiment, the constraint satisfaction test result information may include evaluation information on how well the generated schedule satisfies the predetermined constraints.

[0303] Furthermore, the naphtha cracking center scheduling system can provide one or more scheduling information to the user through a UI / UX. For example, an overview of each of the above one or more scheduling information can be displayed and provided in the form of a graph or figure through the UI / UX, and summary information such as cumulative profit and constraint satisfaction can also be provided.

[0304] According to one embodiment of the present disclosure, a naphtha cracking center scheduling system may determine scheduling information for a naphtha cracking center using an asynchronous multi-agent system comprising a first agent, a second agent, and a third agent. For example, each agent may determine different information at different times.

[0305] FIG. 15 is a drawing showing the schedule of a naphtha cracking center according to one embodiment of the present disclosure.

[0306] Referring to FIG. 15, scheduling at a naphtha cracking center may mean planning a continuous and multi-stage process that converts raw naphtha into high-value-added products such as ethylene. For example, the process at the naphtha cracking center may include the following three interdependent stages: 1) Unloading stage (1510): a stage of unloading naphtha transported from a vessel, etc., into an incoming tank; 2) Blending stage (1530): a stage of mixing the naphtha from the selected incoming tank in a blending tank to achieve a target composition ratio; 3) Cracking stage (1550): a stage of producing ethylene, etc., by high-temperature cracking of the mixed raw materials in a furnace.

[0307] These stages are so closely linked that the entire process can be halted if even one is out of schedule; therefore, a scheduling strategy that precisely coordinates the timing of each stage is essential.

[0308] To explain using the example in FIG. 15, when a new vessel arrives, an unloading step (1510) must be performed. When the first vessel arrives, unloading from the first vessel into the first tank must be performed, and when the second vessel arrives, unloading from the second vessel into the second tank must be performed. Accordingly, the processor must schedule the unloading step (1510) by anticipating the arrival time of the first vessel and the arrival time of the second vessel. In addition, based on the determined receiving tank information, the processor must determine the time to move from the receiving tank to the blending tank, the amount to be mixed, etc., and based on this blending information, determine the start time of operation of the disassembly furnace, the operating period, the temperature, etc.

[0309] That is, the processor must determine the timing and actions to be performed at each stage based on the fact that the unloading stage (1510), blending stage (1530), and cracking stage (1550) are interconnected. However, these stages are so closely linked that if even one of them is out of schedule, the entire process may be halted; therefore, a scheduling strategy that precisely coordinates the timing of each stage is essential. Furthermore, each stage of the naphtha cracking center process is also affected by external factors such as operational constraints (e.g., equipment tolerances, safety standards), demand fluctuations, and ship arrival schedules, requiring long-term and robust scheduling. In one embodiment, when determining the schedule, the plant operating status (e.g., current tank inventory, equipment utilization rate, maintenance schedule, etc.), the ship arrival schedule (e.g., arrival time and shipment volume based on prior shipment data (shipment plan)), tank capacity (e.g., remaining space in each receiving tank and blending tank), raw material quality objectives (e.g., composition ratio after blending, sulfur content, evaporation point, etc.), and market conditions (e.g., external economic variables such as oil price, ethylene price, contract delivery date, etc.) must be comprehensively considered.

[0310] In one embodiment, the processor determines which receiving tank will receive the naphtha unloaded from each vessel in the unloading step (1510), sets the tank combination and mixing ratio to satisfy the standard quality in the blending step (1530), determines the blending sequence, and determines operating conditions such as the feed rate and coil outlet temperature per cracking furnace in the cracking step (1550). The schedule determined in this way must be designed to maximize profitability while preventing conflicts between processes, and it is important that it possesses robust characteristics capable of responding to real-time fluctuations (vessel delays, equipment failures, etc.).

[0311] However, the mainstream approach has traditionally been to optimize by separating steps such as unloading, blending, and cracking. This step-by-step optimization method fails to adequately reflect the interactions between processes, making it difficult to fully resolve issues such as cascading delays and quality degradation that occur during actual operations. Furthermore, the NCC scheduling environment is extremely sensitive, where a single seemingly minor error can invalidate the entire schedule and ultimately halt operations. For instance, failing to start blending on time can cause the incoming tank to overflow, while an inappropriate blending ratio can generate off-spec feed, leading to a cascading impact on downstream processes. Due to these vulnerabilities, there is a problem in that reinforcement learning agents alone are insufficient to reliably generate a consistently valid and safe schedule.

[0312] In addition, reinforcement learning agents are generally trained based on a single scalar reward function, but in actual petrochemical processes, there may be a need to prioritize conflicting goals differently depending on the situation, such as profit maximization, process stability, and compliance with operating constraints. Since priorities fluctuate frequently depending on external factors such as market prices, delays in raw material arrival, and changes in equipment status, there is a problem in that a fixed reward function according to the method described with reference to FIGS. 12 to 14b is insufficient to adequately reflect these dynamic trade-offs.

[0313] One embodiment of the present disclosure aims to provide a method for determining information regarding a process consisting of multiple steps that solves these problems. Specifically, one embodiment of the present disclosure aims to provide a method for supporting an operator's decision-making in both long-term and short-term planning by generating, evaluating, and selecting multiple candidate schedules while simultaneously considering complex constraints.

[0314] That is, the present disclosure relates to a system / method for generating and updating a schedule to reduce the risk of overall failure due to interdependence between stages and to maximize process efficiency in a process consisting of multiple stages (e.g., manufacturing, logistics, IT pipeline, model learning pipeline, etc.). More specifically, the present disclosure relates to a technology in which artificial intelligence sequentially analyzes stage status, result information, etc., to branch and expand a pivot schedule, and combines reinforcement learning-based agent macro operations with a fitness estimator network (evaluator) to update the pivot schedule at each synchronization point and determine the final schedule. In the present disclosure, a pivot schedule refers to an intermediate schedule that is fixed or hypothetically set up to a specific point in time (branching point) during the entire process period, and various future schedules can be derived (branched) from this point. In one embodiment, the fitness estimator may correspond to the centralized critique module (930) described above.

[0315] FIG. 16 is a diagram illustrating a method for determining information about a process consisting of a plurality of steps according to one embodiment of the present disclosure.

[0316] Referring to FIG. 16, at the initialization point (1610), the processor can initialize the pivot schedule of each group into a blank macro operation sequence, reflecting the initial operating state of the NCC system.

[0317] In one embodiment, the processor may determine a planning period for creating a schedule. That is, the processor may determine a reference time for starting to create the schedule and a deadline for not extending the schedule further. For example, the processor may determine a planning period of three weeks from a specific point in time as the planning period for creating the schedule.

[0318] In one embodiment, the processor may create a plurality of groups. Referring to the example of FIG. 16, the processor may create group 1-1, group 1-2, group 2-1, group 3-1, etc.

[0319] In one embodiment, the processor may generate a plurality of groups corresponding to specific operating scenarios, including operating criteria, operating objectives, operating levels, constraints, etc. Operating objectives may include increased profitability, increased process stability, increased energy efficiency, compliance with quality specifications, savings in computing costs, reduced training time, reduced latency, reduced memory usage, and increased model accuracy. Operating criteria may include indicators representing process performance, such as profitability and process stability, and may be evaluated by a scalar goodness-of-fit function. Additionally, operating levels may include indicators representing the strictness of constraints and may be expressed as conservative, moderate, or stressed. Conservative may be selected when the range of constraints is narrow and safety is prioritized, while stressed may be selected under high-load operating requirements approaching equipment limits. Furthermore, constraints may include conditions regarding equipment operating limits, storage capacity, material property specifications, quality specifications, safety standards, memory capacity, the number of available server instances, network bandwidth, target inference latency, etc.

[0320] In one embodiment, the processor may configure multiple groups such that, as one moves toward lower groups, it satisfies higher (stricter) operating standards and operating levels. Since a schedule satisfying a higher level, i.e., a stricter level, will naturally also satisfy lower-level constraints, promising schedules can be transitioned between groups through these structured groups. For example, Group 1-2 may correspond to a group that must satisfy stricter operating standards and operating levels than Group 1-1, Group 2-1 than Group 1-2, and Group 3-1 than Group 2-1.

[0321] According to one embodiment of the present disclosure, such a hierarchical structure directly reflects real-world conditions in the model and can improve search efficiency through propagation from upper levels to lower levels. For example, in actual NCC operations, the intensity of constraints changes frequently depending on the state of the plant, market demand, etc., and the processor can select a stress level during a surge in demand and a maintenance level during stable operation.

[0322] - Pivot-based Branching operation

[0323] In one embodiment, the processor may duplicate the pivot schedule of each of the plurality of groups into a plurality of branch schedules for each of the plurality of groups. For example, the processor may duplicate the pivot schedule (1620) of the first-1 group in parallel to determine that all branch schedules of the first-1 group are identical to the pivot schedule (1620) of the first-1 group. Additionally, the processor may duplicate the pivot schedule of the first-2 group in parallel to determine that all branch schedules of the first-2 group are identical to the pivot schedule of the first-2 group.

[0324] - Scenario-based Rollout Action

[0325] In one embodiment, the processor may perform a rollout process for each quarter schedule, taking into account the operational scenario of the corresponding group. A rollout process may refer to a stage of creating a complete schedule by deploying and simulating candidate schedules (or policies) for a predetermined planning period. This rollout process may be performed by an agent trained based on an artificial intelligence model.

[0326] That is, an agent trained based on an artificial intelligence model can extend a schedule for each of multiple quarter schedules by reflecting operational scenarios corresponding to each of multiple groups. For example, the processor can have the agent trained based on an artificial intelligence model continue to extend the schedule until the schedule for a predetermined planning period is fully completed. The processor can have the agent trained based on an artificial intelligence model construct a complete schedule by sequentially attaching macro actions to quarter schedules that are still empty or only partially defined. For example, the processor can complete a three-week production plan by having the agent trained based on an artificial intelligence model simulate actions in the unloading, blending, and cracking phases.

[0327] In one embodiment, the processor can generate a complete schedule in which all timesteps up to a predetermined planning period are filled, by taking as input the branch schedule up to the current point in time, the operating scenario of the group (objective, constraint, etc.), and macro action candidates (pre-learned policy, etc.).

[0328] In one embodiment, the processor may calculate a reward or penalty score using a fitness function of the group during or at the time of completion of the rollout process. Such reward or penalty score may be used when selecting a schedule to replicate during the synchronization evaluation phase.

[0329] - Synchronized Evaluation & Update operation

[0330] In one embodiment, the processor may update the group's pivot schedule by performing an evaluation at a predefined synchronization point (1630, 1640, 1650). The synchronization points (1630, 1640, 1650) may include a time when a common event occurs. For example, the processor may determine the time of a ship's arrival at the NCC as the synchronization point (1630, 1640, 1650). Based on the fact that the candidate schedules have progressed to the same operational time, the processor may select the superior schedule up to that time as the pivot schedule.

[0331] In one embodiment, the processor may evaluate the branch schedule of a first group and the branch schedule of another group with the same or higher constraint level as the first group at a first synchronization time using a fitness estimator network, and update the pivot schedule of the first group according to the evaluation result. In one embodiment, the fitness estimator network (e.g., a multilayer perceptron or a transformer) may receive various information as input and output a scalar fitness. For example, the input information may include a summary of the state of each branch schedule (e.g., inventory, capacity utilization, quality margin, safety margin, etc.), resource allocations, energy estimates, cost estimates, constraint violation penalties, scenario levels (e.g., maintenance, medium, stress, etc.), and operational purpose weight vectors. Learning is performed by regression on labels (e.g., realized revenue, total penalty, etc.) generated from a simulator or historical operating data, and may be fine-tuned by mini-batch based on the results of recent executions in the online phase. If the processor retains the pivot schedule of the first group at the first synchronization time when the score of the pivot schedule of the first group is the highest, and if the score of the pivot schedule of the first group is not the highest at the first synchronization time, it may replace the pivot schedule of the first group with the schedule having the highest score.

[0332] In one embodiment, at predefined synchronization points (1630, 1640, 1650), each group may evaluate (i) its own group's branch schedule and (ii) the branch schedule of other groups having the same or higher operating level using its own fitness function. As a result of the evaluation, the best schedule may be updated as the new pivot schedule for that group.

[0333] - Multilayer recovery mechanism

[0334] In one embodiment, a recovery mechanism may be used. If a failed schedule exists, the processor may immediately replace it with a superior schedule within the group or a higher-level group and continue the search. That is, if any schedule is determined to have failed because it can no longer proceed throughout the rollout process, the recovery mechanism may be executed.

[0335] In one embodiment, if a specific branch schedule fails, the processor may replace the schedule with the schedule of highest fitness within the same group and continue the rollout process (Intra-group recovery). For example, the pivot schedule of Group 1-1 is replicated to become the first branch schedule and the second branch, and the schedule is expanded by reflecting the operational scenario of Group 1-1 for each of the first and second branch schedules. If, at some point, the first branch schedule fails, the processor evaluates the branch schedule of Group 1-1 and, if it determines that the second branch schedule has the highest fitness among the multiple branch schedules of Group 1-1, can replace the first branch schedule with the second branch schedule.

[0336] If all schedules belonging to a group fail, the processor may replace the entire group with a copy of the schedule with the highest fitness among other groups having the same or a higher level (inter-group recovery). For example, if the processor replicates the pivot schedule of Group 1-1 into the first and second branch schedules, and replicates the pivot schedule of Group 1-2 into the third and fourth branch schedules, and all branch schedules of Group 1-1 fail, the processor may determine the schedule with the highest fitness among the branch schedules of Group 1-2 as the pivot schedule of Group 1-1. In this case, Group 1-2 may be a group having requirements of a higher level (stricter level) than Group 1-1.

[0337] In one embodiment, if the schedule of all groups fails, the processor may perform a re-search starting from the pivot schedule of each group at the previous synchronization point.

[0338] This hierarchical recovery mechanism can increase the robustness of the plan by preventing premature search termination.

[0339] In one embodiment, after a predetermined planning period has ended, the processor may apply post-processing evaluation criteria to the completed pivot schedules to select one or more final schedules (1660).

[0340] For example, when the schedule is completed by a predetermined planning period, the processor may re-evaluate the completed pivot schedule based on final evaluation criteria. Unlike the fitness function used in the rollout process, the final evaluation criteria may include elements that can be precisely calculated only after the entire schedule is completed (e.g., cumulative profit, overall safety indicator, etc.). Based on the results of the re-evaluation, the processor may determine the final schedule. This final schedule (1660) may include a schedule consisting of multiple stages and may be multiple. For example, the top three schedules may be provided to the user.

[0341] For example, when one embodiment of the present disclosure is used to determine the schedule of an NCC, the final schedule includes the operational schedule of the NCC, and the synchronization point may include the time of vessel arrival, the time of completion of a process step, the time of simulation time elapsed, etc. Additionally, the processor may perform a rollout process by extending the schedule based on a pre-learned reinforcement learning policy, by having at least one agent perform at least one of a naphtha receiving operation, a blending operation, and a cracking operation.

[0342] A candidate-based multi-scenario planning method according to one embodiment may structure candidates into multiple groups to overcome the limitations of existing single-population-based methods. According to one embodiment of the present disclosure, by maintaining various candidates, multiple pieces of information optimized under different objective functions (profit, stability, etc.) and constraint levels can be searched and preserved in parallel. Furthermore, according to one embodiment of the present disclosure, by utilizing a scenario hierarchy, a schedule satisfying high (strict) constraint levels automatically satisfies low constraint levels as well, thereby facilitating transitions between schedules.

[0343] In one embodiment, a structured candidate population can enable targeted search for various operational scenarios while maintaining the advantages of candidate-based techniques such as parallel exploration and escaping local optima. As a result, a set of information with superior quality and diversity compared to a single population explored uniformly can be generated. For example, in NCC scheduling, a schedule can be provided that simultaneously ensures comprehensiveness and robustness of production planning.

[0344] According to one embodiment of the present disclosure, by always maintaining a set of schedules capable of responding quickly to changes in priority or unexpected situations, robustness and adaptability that are difficult to achieve with an agent alone can be secured.

[0345] In addition, according to one embodiment of the present disclosure, the driver can immediately select the option most suitable for the current situation among a plurality of scenario-based schedules, thereby reducing the burden of decision-making.

[0346] In addition, according to one embodiment of the present disclosure, the risk of a single selection error disrupting the entire process can be significantly reduced.

[0347] In addition, according to one embodiment of the present disclosure, the schedule can be flexibly reconfigured even when abnormal events occur, such as market price fluctuations or equipment malfunctions.

[0348] In addition, according to one embodiment of the present disclosure, the limitations of a fixed reward structure can be compensated for by multi-objective, multi-constraint search according to one embodiment of the present disclosure while effectively utilizing behavior candidates generated by a reinforcement learning agent.

[0349] In other words, one embodiment of the present invention serves as a key element that bridges the potential of reinforcement learning-based scheduling with real-world plant requirements, and by simultaneously improving usability, safety, and efficiency, it can enable the introduction of various multi-stage processes into industrial sites.

[0350] Although an NCC scheduling method has been described as an example to explain one embodiment of the present disclosure, the method according to one embodiment of the present disclosure is not applicable only to NCC scheduling. For example, one embodiment of the present disclosure may also be used for data preprocessing operations, distributed model training operations, fine-tuning operations, inference operations, etc.

[0351] FIG. 17 is a flowchart illustrating a method for determining information about a process consisting of a plurality of steps according to one embodiment of the present disclosure.

[0352] In operation 1710, the processor may create a plurality of groups including a first group and a second group. In one embodiment, the plurality of groups may be created hierarchically. For example, the second group may be formed as a group having a higher level of operational scenario than the first group.

[0353] In one embodiment, each group can be mapped to an operational scenario defined by an objective function (e.g., profit, stability, latency minimization) and a constraint level (conservative, moderate, stress, etc.). This allows the processor to simultaneously explore different operational goals and constraints even within the same system.

[0354] In operation 1730, the processor may duplicate pivot information for each of the multiple groups into multiple branch information for each of the multiple groups. In one embodiment, for each group, the processor may duplicate multiple branch information in parallel based on the pivot information of the group—a macro operation sequence that initially represents a blank or system initial state. For example, the processor may duplicate the pivot information of the first group into first branch information and second branch information, and duplicate the pivot information of the second group into third branch information and fourth branch information. This provides a basis for extensively exploring various alternative decision-making options at the same time.

[0355] In this case, the information may include schedules, logistics delivery routes, generator load curves, LLM training pipelines, etc. Additionally, macro actions may include unloading actions, blending actions, cracking actions, warehouse management actions, GPU training actions, validation actions, container deployment actions, etc.

[0356] In operation 1750, the processor can expand information for each of multiple branch information by reflecting the operational scenario corresponding to each of multiple groups. That is, the processor can roll out information by reflecting the operational scenario of the corresponding group for each branch information. For example, in a Large Language Model (LM) pipeline, the processor can expand the steps in the order of data preprocessing, distributed learning, checkpoint saving, and inference deployment.

[0357] In one embodiment, if the processor cannot proceed with the scenario using the branch information of the first group, the first branch information may be replaced with the second branch information having the highest suitability among the plurality of branch information of the first group.

[0358] If there is no branch information to replace the multiple branch information in the first group, that is, if scenario progression is impossible with all the branch information in the first group, the processor may replace the first branch information with the third branch information having the highest suitability among the multiple branch information in the second group. In this case, the second group may be a group having a higher level of constraint than the first group.

[0359] If scenario progression is impossible with any of the information from all groups, including the first and second groups, the processor may perform a re-search based on the pivot information of each of the multiple groups at the previous synchronization point.

[0360] In one embodiment, the operation scenario may include operation objectives, constraints, etc. Operation objectives include increased profitability, increased process stability, increased energy efficiency, compliance with quality standards, reduced computing costs, reduced training time, reduced latency, reduced memory usage, increased model accuracy, etc., and constraints may include conditions regarding at least one of equipment operating limits, storage capacity, material property specifications, quality specifications, safety standards, memory capacity, the number of available server instances, network bandwidth, and target inference latency.

[0361] In operation 1770, the processor may update the pivot information of the first group at the first synchronization point based on the expanded information. In one embodiment, when the expanded branch information reaches the first synchronization point, the processor may evaluate all branch information of the first group and branch information of other groups having the same or higher constraint level using a fitness function. Additionally, the processor may update the pivot information of the first group to the best branch information based on the result evaluated by the fitness function. The synchronization point may include, for example, common system-wide events such as ship arrival, facility turnaround, or completion of a learning epoch.

[0362] In operation 1790, the processor may determine final information based on the updated pivot information of the first group. In one embodiment, after a predetermined planning period has ended, the processor may select one or more final information by applying post-processing evaluation criteria to the completed pivot information.

[0363] In one embodiment, when the time of plan completion is reached, the processor may re-evaluate the completed pivot information, including the updated pivot information of the first group, using a separate final evaluation criterion. Unlike the fitness function used during rollout, the final criterion may include elements that can be precisely calculated only at completion, such as cumulative revenue, total delay, and energy consumption. Information with the highest performance results from the evaluation may be selected as Final Info and transmitted to an operator UI (User Interface) or an automated execution module. In one embodiment, the Final Info may include information consisting of multiple steps.

[0364] FIG. 18 is a block diagram of a system for determining information about a process consisting of a plurality of steps according to one embodiment of the present disclosure.

[0365] Referring to FIG. 18, a system (1800) (the system may be referred to as a server or device) may include a transceiver (1810), memory (1820), a database (1830), and a processor (1840). However, not all components shown in FIG. 18 are essential components of the system (1800). The system (1800) may be implemented by more components than those shown in FIG. 18, or by fewer components than those shown in FIG. 18. Furthermore, the transceiver (1810), memory (1820), and processor (1840) may be implemented in the form of a single chip.

[0366] In one embodiment, the transceiver (1810) may communicate with a terminal or other electronic device connected to the system (1800) via wired or wireless connection. For example, the transceiver (1810) may receive user input information from a user terminal. In one embodiment, the user input information may include simulation information, AI model information, process data, pharmaceutical information, naphtha receiving plan information, production target quantity for each product (e.g., ethylene production target quantity), status information, scheduling start time, etc. Process data may include a raw material receiving schedule for a first period, a first process operation schedule for the first period, a second process operation schedule for the first period, a third process operation schedule for the first period, raw material inventory quantity by raw material storage tank, characteristics history by raw material storage tank, daily target production volume history for each product during the first period, daily price history of raw materials during the first period, daily price history of products during the first period, etc. Naphtha receiving planning information may include the scheduled receiving time, receiving rate, receiving quantity, naphtha properties, and receiving type information (e.g., whether it is a vessel or a tank belonging to another company). Status information may include a predetermined blending schedule and a predetermined cracking schedule. The blending or cracking schedule may include the start time, end time, tank name, and blending or cracking rate for each tank; furthermore, the cracking schedule may include various variable information such as pressure and temperature. Additionally, status information may include the naphtha inventory quantity and properties for each tank at the start of scheduling.

[0367] Various types of data, such as programs and files, such as applications, can be installed and stored in the memory (1820). The processor (1840) may access and use the data stored in the memory (1820) or store new data in the memory (1820). Additionally, one or more instructions may be stored in the memory (1820). The processor (1840) may execute one or more instructions stored in the memory.

[0368] The processor (1840) controls the overall operation of the system (1800) and may include at least one processor, such as a CPU, GPU, etc. The processor (1840) may control other components included in the system (1800) to perform operations for operating the system (1800). For example, the processor (1840) may determine receiving tank information using a first agent based on input information, determine mixing tank combination information using a second agent, and determine cracking furnace operation information using a third agent. However, according to one embodiment of the present disclosure, not all process steps (receiving, mixing, cracking) of the naphtha cracking center must necessarily be determined by multiple agents. In one embodiment, the processor (1840) may receive a fixed schedule for some process steps from a user and determine scheduling information using an agent only for the remaining process steps.

[0369] For example, if a user directly determines and inputs incoming tank information (such as a ship unloading schedule) based on their own experience or external circumstances, the processor (1840) may set the incoming tank information as a hard constraint or an environmental state. In this case, the second agent (responsible for mixing) and the third agent (responsible for disassembly) recognize the incoming tank information determined by the user as an observation and determine the mixing tank combination information and disassembly operation information optimized for it.

[0370] Conversely, if the user predetermines and inputs operation information for the disassembly furnace based on the final product production goal, the first and second agents may determine the optimal receiving and mixing schedules by working backward to match the operation schedule for the disassembly furnace.

[0371] That is, the system (1800) of the present disclosure can support not only a fully automated mode but also a hybrid scheduling mode that combines user decision-making and artificial intelligence recommendations. Through this, it can flexibly reflect unexpected variables at the site or the operator's intuition, while receiving assistance from artificial intelligence in complex computational areas that are difficult for humans to calculate (e.g., calculation of optimal mixing ratios).

[0372] In one embodiment, the processor (1840) may generate one or more scheduling information for the naphtha cracking center based on the receiving tank information, the mixing tank combination information, and the cracking furnace operation information. Additionally, the transceiver (1810) may transmit the scheduling information to the user terminal so that the scheduling information is displayed on the display of the user terminal. In one embodiment, the output scheduling information may include a receiving schedule for a future predetermined period (e.g., receiving start and end times, receiving tank identifier, etc.), a mixing schedule for a future predetermined period (e.g., mixing start and end times, mixing tank identifier, mixing speed, etc.), a cracking furnace schedule for a future predetermined period (e.g., cracking start and end times, target speed (feed rate) determined by an algorithm, COT, DS ratio, etc.), daily production volume and expected profit information by product for a future predetermined period, naphtha inventory quantity and change in properties for a future predetermined period, constraint check result information for the generated schedule, and a plot visualizing the generated schedule.

[0373] The process of generating and outputting scheduling information in this manner can be performed through the UI / UX of a user terminal. For example, when the processor (1840) obtains input information entered by a user, it verifies whether there is sufficient data in the input information to generate output data, and if it is determined that the input information is valid, it can generate one or more scheduling information using an artificial intelligence scheduler based on the scheduling start date entered by the user. Additionally, the processor (1840) can graph one or more scheduling information to provide information, and the user terminal can display this information in the form of a UI / UX.

[0374] Additionally, the processor (1840) can execute logic to generate a plurality of group-specific pivot schedules according to program instructions loaded into memory (1820) and to update the pivot schedules through a fitness estimator at each synchronization point.

[0375] The database (1830) can store various training data for training a learning model. Additionally, the database (1830) may store material information, phase information, simulation result information, etc., and in various embodiments, output data produced by the learning model may be stored. Although FIG. 18 shows the system (1800) including the database (1830), the database (1830) may be provided outside the device. In this case, the database (1830) may be connected to the system (1800) via wired or wireless connection.

[0376] Additionally, the learning model may be implemented outside the system (1800) (e.g., cloud-based) or included inside the system (1800).

[0377] FIG. 19 is a drawing showing a sequential multi-agent according to one embodiment of the present disclosure.

[0378] Referring to FIG. 19, a sequential multi-agent may include a user agent, a writing agent, a style agent, etc. In one embodiment, the sequential agents communicate sequentially and can perform a single task in order. Additionally, each agent can receive the result of the previous step and perform the next task. For example, the user agent may obtain user input from the user to generate a first processing result, the writing agent may generate a second processing result including the written text by generating the first processing result from the user agent, and the style agent may generate a third processing result by receiving the second processing result from the writing agent and applying a style. Accordingly, the third processing result may finally be output. Such a sequential multi-agent has a linear flow and may be suitable for processing a single task. For example, the sequential multi-agent may be used in the field of creative writing.

[0379] According to one embodiment of the present disclosure, the aforementioned first agent (responsible for receiving), second agent (responsible for mixing), and third agent (responsible for disassembly) can operate in a sequential multi-agent structure as illustrated in FIG. 19.

[0380] As explained with reference to FIG. 19, a sequential agent can perform a task by using the result of the previous step as the input for the next step. When this is applied to an NCC scheduling system, a first agent (corresponding to a user agent) can first determine and generate an entry schedule (first processing result) based on ship entry information, etc.

[0381] Next, the second agent (corresponding to the creation agent) receives the above-determined receiving schedule as an unchangeable environmental state and can generate a mixing tank combination and blending schedule optimized thereto (second processing result). Finally, the third agent (corresponding to the style agent) can generate a cracking furnace operation schedule (third processing result) based on the above-determined blending schedule and output a final NCC operation schedule.

[0382] This sequential structure can be effectively utilized when dependencies between stages are clear (linear flow) and when a top-down scheduling method is required where decisions at higher stages have an absolute influence on lower stages (e.g., situations where receiving plans must be prioritized due to unstable raw material supply).

[0383] FIG. 20 is a drawing showing a supervisory agent according to one embodiment of the present disclosure.

[0384] Referring to FIG. 20, a supervised agent may refer to an agent in which a centrally located Supervisor Language Model (LM) coordinates the entire process. In one embodiment, the supervised agent may direct the appropriate agent to perform necessary tasks and synthesize the results according to the user's request. That is, the supervised agent may manage communication between agents. Accordingly, the supervised agent may enable flexible task distribution. For example, the supervised agent may request research from a research agent and request calculations from a mathematics agent. Such a supervised agent may be used in fields such as deep research.

[0385] According to another embodiment of the present disclosure, a naphtha cracking center scheduling system may adopt a supervisor agent structure as illustrated in FIG. 20.

[0386] Referring to Fig. 20, a supervising agent (or orchestrator) coordinates the entire process and directs tasks to subordinate agents. When applied to an NCC system, the supervising agent can be the entity performing pivot schedule management and synchronization evaluation as described with reference to Fig. 16.

[0387] Specifically, the supervisory agent monitors the current process status (e.g., occurrence of urgent orders, equipment failure, etc.) and, accordingly, can dynamically distribute tasks, such as requesting the first agent (corresponding to the research agent) to establish a receiving plan or requesting the second agent (corresponding to the math agent) to calculate the optimization of blending ratios.

[0388] In particular, when a supervisory agent performs the multi-layer recovery mechanism of Fig. 16, if a specific subordinate agent reports that it has failed to generate a schedule, the agent can perform a central control role by immediately instructing an agent in another group to generate an alternative schedule or by relaxing higher-level constraints to re-instruct. This structure enables flexible work distribution and error recovery in complex and variable NCC processes.

[0389] FIG. 21 is a drawing showing a hierarchical agent system according to one embodiment of the present disclosure.

[0390] Referring to FIG. 21, a hierarchical agent system can refer to a system in which a meta-agent controls and coordinates subordinate agents. For example, the meta-agent can obtain user input from a user, request tasks from subordinate agents such as a research agent or a data analysis agent, and generate outputs by receiving task results from each agent. This hierarchical agent system has a hierarchical control structure, allowing tasks to be divided and managed in a more complex manner. In other words, a meta-agent acting as an intermediate manager can be utilized. This hierarchical agent system is suitable for complex systems or coding, and can be used as a coding agent, etc.

[0391] According to one embodiment of the present disclosure, a naphtha cracking center scheduling system may adopt a hierarchical agent system structure as shown in FIG. 21 to efficiently manage large-scale facilities.

[0392] Referring to Fig. 21, the meta-agent distributes tasks to subordinate agents and aggregates the results. When applied to an NCC system, the meta-agent can perform the role of a plant manager that determines the target production volume for the entire plant (e.g., 3,000 tons of ethylene / day) and the overall energy budget.

[0393] Furthermore, sub-agents can be subdivided by function or facility group. For example, a first sub-agent group corresponding to a research agent performs a precise analysis of the characteristics of each incoming tank and reports it, while a second sub-agent group corresponding to a data analysis agent is assigned to each individual furnace (Furnace #1 to #N) to optimize detailed operating variables (COT, Feed Rate) to achieve the quota assigned by the meta-agent.

[0394] This hierarchical structure reduces the computational load on the central meta-agent, which must handle even complex detailed controls in large-scale NCC processes involving dozens of tanks and cracking furnaces, and enables precise control tailored to the characteristics of each facility.

[0395] FIG. 22 is a drawing showing a discussion-type multi-agent system according to one embodiment of the present disclosure.

[0396] Referring to FIG. 22, a discussion-based multi-agent system is a system in which multiple agents present different opinions and select the most appropriate result by voting or evaluating it, thereby deriving the optimal solution based on discussion. That is, multiple agents perform discussion and evaluation based on user input and can output the optimal answer among them. Such a multi-agent discussion-based system can make the best choice by comparing various perspectives in a competitive structure and can be used in fields such as world simulation.

[0397] According to another embodiment of the present disclosure, a naphtha cracking center scheduling system may adopt a Debate-based Multi-Agent System structure as illustrated in FIG. 22 to coordinate conflicting goals.

[0398] Referring to Fig. 22, multiple agents present and evaluate different opinions to derive the optimal solution. This can be applied to the synchronization evaluation and pivot schedule update steps of the NCC system (see Fig. 16).

[0399] Specifically, the system may include multiple agents with different personas. For example, a profit-seeking agent may propose an aggressive schedule to maximize production, a safety management agent may propose a conservative schedule to minimize equipment overcrowding, and a maintenance agent may propose a schedule that considers equipment lifespan.

[0400] They undergo a virtual debate process as shown in Fig. 22. Each agent points out weaknesses in the opponent's schedule (e.g., "Your schedule is profitable, but the risk of pipe blockage increases by 80%), and the system (or a separate referee agent) aggregates these debate results and votes or assigns weights to finally select the optimal pivot schedule that balances profitability and safety. This can maximize the reliability of the schedule by mutually compensating for blind spots that an agent trained with a single objective function might miss.

[0401] FIG. 23 is a drawing showing a mixed agent system according to one embodiment of the present disclosure.

[0402] Referring to FIG. 23, in a hybrid agent system, multiple agents perform parallel processing layer by layer, and an aggregator can integrate the results at the end. For example, a first agent and a second agent may perform a process in parallel at the first layer, and then the first agent and the second agent may perform a process in parallel at the second layer, and an aggregator may synthesize these to generate an output. In one embodiment, the hybrid agent system may use a multi-stage approach for complex problems. The hybrid agent system has a hierarchical and parallel structure and is characterized by the distribution and combination of expertise, so it can be used for medical research, etc.

[0403] According to one embodiment of the present disclosure, a naphtha cracking center scheduling system may adopt a hybrid agent system structure as shown in FIG. 23 to manage the complexity of a large-scale process.

[0404] Referring to Fig. 23, agents can perform parallel processing layer by layer, and an aggregator can integrate them. When applied to an NCC system, in the first layer, multiple receiving agents can generate unloading schedules in parallel for different vessels and receiving tanks. At this time, the first aggregator of the first layer can aggregate the receiving schedules generated in parallel, verify whether there is a pipeline conflict, and produce an integrated receiving plan.

[0405] Next, in the second layer, based on the integrated receiving plan, multiple mixing agents optimize the blending ratio for each mixing tank in parallel, and the second aggregator can comprehensively evaluate the uniformity of the properties of the entire raw material.

[0406] This hybrid structure can solve the NCC scheduling problem, which involves thousands of variables, by decomposing it into facility units (parallel) and process stages (hierarchy), thereby dramatically improving computational speed compared to a single agent method and enabling specialized optimization.

[0407] FIG. 24 is a drawing showing a ReAct agent system according to one embodiment of the present disclosure.

[0408] Referring to FIG. 24, ReAct is a compound word of "Reason + Act," and a ReAct agent can refer to an agent that solves problems by repeating reasoning (Reason) and action (Act). For example, if a user asks, "What is the weather like in New York these days?", the LLM can interpret the meaning of the question, search for the current weather in New York through a search engine's search tool, summarize the results, and deliver them back to the user. Such a ReAct agent can be used in AI chatbots, etc. According to one embodiment, the ReAct agent has excellent tool usage capabilities and can generate more accurate responses through the repetition of reasoning and action.

[0409] According to another embodiment of the present disclosure, each agent of the naphtha cracking center scheduling system (in particular, a third agent directly related to safety) may adopt a ReAct (Reason+Act) agent system structure as illustrated in FIG. 24.

[0410] Referring to Fig. 24, the ReAct agent solves problems by repeating reasoning and action. When applied to an NCC environment, the agent can make decisions by going through a reasoning stage and a tool use stage, rather than simply outputting an action regarding the current state. For example, if an abnormal sign is detected where the temperature of the cracking furnace rises rapidly, the ReAct-based third agent can send a reduction command to the actual cracking furnace control system through a reasoning stage where "it is necessary to verify whether the cause of the temperature rise is a change in the raw material properties or an over-fuel injection," an action stage where the "NCC simulator tool" is called to simulate the temperature change after 10 minutes if the current fuel injection amount is maintained, an observation stage where the simulation results predict that the critical temperature will be exceeded after 10 minutes, and a reasoning stage where "the fuel injection amount must be immediately reduced by 5%."

[0411] By introducing the ReAct structure in this way, the system can go beyond simple numerical optimization and respond to unexpected situations through logical thinking processes, utilizing virtual simulations as a tool just like a skilled engineer.

[0412] FIG. 25 is a drawing showing a CodeAct agent system according to one embodiment of the present disclosure.

[0413] Referring to Fig. 25, the CodeAct agent system can perform complex data processing and logic by executing Python code based on a query received from a user. Unlike existing JSON-based data processing systems, the CodeAct agent has the advantage of being able to handle more flexible and complex calculations and analysis tasks by using Python. JSON is primarily used as a data exchange format and can be advantageous for clearly defining the structure of data. However, JSON itself does not provide functions for logic execution or data processing. Therefore, to perform complex data analysis or calculations, it is necessary to define the data structure using JSON and then write additional code. On the other hand, since Python code is executable code capable of data processing and logic implementation, it can directly perform more complex and diverse tasks. By executing this Python, the CodeAct agent enables immediate data processing and calculation, and can process tasks dynamically and in real time.

[0414] For example, when the LLM receives input from a user such as "Analyze sales data for the last 3 months," it analyzes the request, and a Python structured data preprocessing library (e.g., Pandas) can be utilized. Pandas is a Python-based data analysis library that provides tools for easily manipulating and analyzing large datasets. For instance, if a user requests "Analyze sales data for the last 3 months," the CodeAct agent analyzes this query, loads the relevant data in CSV file format, and can efficiently process the data using the Pandas library. Pandas allows for easy handling of tabular data using DataFrame structures and enables simple execution of various data processing tasks, such as filtering, grouping, and aggregation.

[0415] For example, using Pandas, you can group "sales data for the last three months" and analyze sales trends through statistical calculations. In addition, complex tasks such as loading CSV files, handling missing values, filtering, and generating summary reports can also be easily executed using Python code and Pandas.

[0416] According to one embodiment, the CodeAct agent can process complex data analysis and calculation tasks in real time using Python. This can provide much more powerful and flexible capabilities than systems that use only data formats such as JSON.

[0417] In addition, according to one embodiment, CodeAct implements an automated data processing process to analyze user requests and automatically execute corresponding analysis tasks using Python code. For example, it can automatically perform data loading, filtering, and statistical calculations using Pandas, and provide the results in a form that the user can understand.

[0418] Furthermore, Pandas is a tool capable of efficiently handling advanced data analysis within a Python environment, and the CodeAct agent system leverages it to simplify the analysis of complex datasets and statistical processing. This enhances the technical efficiency required for large-scale data analysis. For example, if a user requests, "Analyze sales data for the past three months," the CodeAct agent first loads sales data from a CSV file using the Pandas library. It then analyzes the data using various Pandas functions, performing tasks such as calculating average sales volume, identifying sales trends, and generating graphs. All of these processes are executed using Python code, allowing users to obtain analysis results in real time.

[0419] In this way, the CodeAct agent system is highly useful for handling real-time data analysis and complex calculations, and can provide powerful capabilities utilizing Python and Pandas. Through this, users can experience more efficient and flexible task execution.

[0420] According to one embodiment of the present disclosure, a naphtha cracking center scheduling system may include a CodeAct (Code-Action) agent system as illustrated in FIG. 25 to process unstructured queries from an operator.

[0421] Referring to Fig. 25, the CodeAct agent converts a user's query into code and executes it. When this is applied to an NCC system, the system can be equipped with a data analysis agent. For example, if an operator queries in natural language, "Analyze the correlation between the paraffin content of naphtha unloaded from Vessel A and the ethylene production yield over the past month," it may be impossible to process this with an existing JSON-based rule engine.

[0422] However, the CodeAct agent according to the present embodiment can analyze the query using LLM and immediately generate Python code. The generated code may include (1) code to load the last month's incoming data and production data from the NCC database in the form of CSV or DataFrame (using Pandas), (2) code to calculate the correlation coefficient between the two data (using SciPy or NumPy), and (3) code to visualize the results and output them as a graph (using Matplotlib).

[0423] In one embodiment, the system can execute this code in a sandbox environment and output the results to an operator terminal. This allows the operator to perform a deep dive into complex process data in real time without the assistance of a developer.

[0424] According to another embodiment of the present disclosure, the compensation calculation unit or the constraint verification unit of the naphtha cracking center scheduling system may be implemented based on CodeAct.

[0425] As explained in Fig. 25, JSON does not provide logic execution capabilities, whereas Python can perform complex calculations. In the NCC process, the calculation logic for the expected reward may change frequently depending on fluctuations in crude oil prices, exchange rates, complex chemical reaction yield curves, etc.

[0426] Therefore, instead of using fixed formulas, the system can dynamically generate or update the profit calculation logic itself in the form of Python functions through the CodeAct agent. For example, if a command such as "The carbon tax emission factor has changed starting this month, so recalculate the profit to reflect this" is entered, the CodeAct agent can automatically modify and execute the carbon tax calculation function within the existing profit calculation Python script.

[0427] This goes beyond simple parameter changes and ensures flexibility in the algorithm itself, thereby providing a scheduling system that can respond agilely to rapidly changing market environments and environmental regulations.

[0428] FIG. 26 is a drawing showing a modern tool-using agent system according to one embodiment of the present disclosure.

[0429] Referring to FIG. 26, a modern tool usage agent may be an agent system that interprets a natural language-based request received from a user, calls various external SaaS tools, APIs, etc. through a Multi-Channel Processing (MCP) server, and provides the execution results to the user. In one embodiment, the agent receives a request in the form of text, voice, or other forms from a user, analyzes the intent of the request, and determines whether an external tool call is necessary. The agent may establish a plan to call multiple tools sequentially or in parallel, in addition to a single tool call.

[0430] In one embodiment, the agent may communicate with one or more MCP servers. An MCP server may be a component that acts as middleware between the agent and external tools or services. In one embodiment, the MCP server may perform functions such as providing a standardized interface to external SaaS tools or APIs, managing authentication information (API keys, tokens, etc.), converting agent requests to match the call specifications of each tool, and normalizing tool execution results into a form understandable by the agent. For example, as illustrated in FIG. 8, the first MCP server may be integrated with a search service or web search API (e.g., Brave Search), and the second MCP server may be integrated with a cloud service. By separating the MCP servers by function in this way, the agent can use various tools in a consistent manner without directly being aware of the detailed implementation of each external tool.

[0431] For example, upon receiving a text request from a user such as "Stop the EC2 instance on my cloud service," the agent analyzes the user input to determine that the request is a task related to controlling cloud resources. If it determines that an API call to the cloud service is required, it can select the corresponding MCP server. Additionally, it transmits the request regarding the EC2 instance stop to the MCP server, which can convert it into a cloud service API call format and execute it. The agent receives the execution result (e.g., whether the instance stop was successful) from the MCP server and can provide it to the user in the form of a natural language message. Agents utilizing modern tools can be used in developer IDE-integrated AI, and they offer the advantages of enabling tool control with minimal code and facilitating easy integration with various cloud or web functions.

[0432] According to one embodiment of the present disclosure, a naphtha cracking center scheduling system may include a Modern Tool Use Agent and a Model Context Protocol (MCP) server structure as illustrated in FIG. 26 to reflect external environmental variables in real time.

[0433] Referring to Fig. 26, the agent can communicate with external SaaS tools through the MCP server. When applied to the NCC system, the market analysis agent or logistics management agent can be linked with a global crude oil price information API (e.g., Bloomberg API) or a maritime logistics tracking API (e.g., MarineTraffic API) through the first MCP server.

[0434] For example, if a scheduling agent requests, "Tell me the estimated delay time for naphtha transport vessels due to the current typhoon," the MCP server can convert this into calls to the marine weather information API and vessel location API, execute them, normalize the results, and return them to the agent.

[0435] Through this, the agent can call external data as if it were an internal function via a standardized interface provided by the MCP server, without the need to directly handle authentication or data formats (JSON, XML, etc.) of complex external APIs, and immediately reflect it in the scheduling.

[0436] According to another embodiment of the present disclosure, an agent of the system can utilize an MCP server as middleware to integrate and control heterogeneous legacy systems within a factory.

[0437] Within the NCC plant, ERP systems for material management, DCS (Distributed Control System) for equipment control, and MES for process monitoring are mixed together, and these systems use different communication protocols. The MCP server according to the present embodiment can serve as a standardized interface for these legacy systems.

[0438] For example, an NCC operator can issue a maintenance ticket for an emergency inspection of Unit 3's disassembly furnace and request the release of spare parts from the materials room using natural language. In this case, the agent can interpret the natural language request and establish a parallel planning process to (1) call the parts release API from the first MCP server (for ERP integration) and (2) call the work ticket creation API from the second MCP server (for maintenance system integration). This means that factory operations can be automated using natural language through an AI agent without having to learn the usage of complex internal systems one by one.

[0439] FIG. 27 is a drawing showing a self-reflective agent system according to one embodiment of the present disclosure.

[0440] Referring to FIG. 27, the self-reflective agent system may be an agent system that includes a metacognition-based iterative processing structure that evaluates and modifies results generated by a large language model (LLM). In one embodiment, the self-reflective agent system may include a main language model that interprets user input and generates a draft or intermediate result, a generator that generates a response based on the draft generated by the main language model, a critic that evaluates the generated draft or response and identifies areas requiring improvement, a memory that stores previous generation results, evaluation results, modification history, etc., and a tool that performs auxiliary functions such as grammar checking, keyword consistency analysis, and style rule application.

[0441] In one embodiment, the self-reflective agent system may acquire user input in the form of natural language. The main language model may analyze the user input and generate a corresponding draft response. The result generated at this stage may not be a completed final output, but an intermediate output subject to evaluation. The generated draft may be passed to a Critique for evaluation. The Critique may evaluate the draft by considering logical consistency, contextual appropriateness, consistency with user requests, clarity of expression or sentence structure, and suitability for the purpose (e.g., resume, report, explanatory text, etc.). Based on the Critique results, it may be determined whether the draft has been sufficiently improved. In one embodiment, if the Critique results do not meet the criteria, the system may determine that modifications are necessary. If modifications are determined, the Critique results are passed back to the main language model or the generator, and an improved draft may be regenerated. This process may be repeated until a preset number of iterations or quality criteria are satisfied. If the Critique results are determined to meet the criteria, the result may be finalized as the final output and provided to the user.

[0442] In one embodiment, the self-reflective agent system may utilize memory to store previous drafts and revision history, critique results and evaluation criteria, user preferences, or past request patterns. Additionally, tools may be implemented as grammar checkers, keyword analyzers, style rule engines, etc., and may be utilized as auxiliary tools during the critique or revision phases.

[0443] According to one embodiment of the present disclosure, a naphtha cracking center scheduling system may include a self-reflection agent system as illustrated in FIG. 27 to ensure the safety and feasibility of the generated schedule.

[0444] Referring to Fig. 27, the system generates a draft and modifies it by critiquing it. When this is applied to the NCC system, the main language model (generator) can generate a primary draft schedule as the reinforcement learning agent described earlier.

[0445] Next, the Critique module can perform physical and logical verification on the draft schedule. The Critique can identify not only simple constraint violations but also deep logical flaws, such as, "It was stated that the water would be transferred from Tank A to Tank B at 2:00 PM, but isn't there a risk that the water level in Tank A at that time will be lower than the minimum pump suction height?"

[0446] If a defect is identified, the critic can provide specific feedback to the generator, such as "delay the start time of transfer for Tank A by 30 minutes or reduce the transfer speed by 10%." The generator regenerates a modified schedule reflecting this feedback, and this process can be repeated until the critic makes a pass judgment.

[0447] Through this, the system can prevent hallucinations or safety regulation violations that may occur during the numerical optimization process, thereby generating a high-reliability schedule that can be immediately applied to the field.

[0448] According to another embodiment of the present disclosure, the system may utilize a self-reflection function to explain the basis of a determined schedule or to generate an operation report. Complex NCC scheduling results consist of thousands of data points, making them difficult for humans to understand intuitively. Accordingly, the reporting agent of the system may utilize the structure of FIG. 27 to generate an explanation of "why this schedule was determined."

[0449] For example, if a reporting agent drafts a statement saying, "Unit 3 was operated at full capacity for profitability," a critic may point out that "an explanation regarding compliance with safety regulations is missing." In response, the agent may modify the wording to, "Unit 3 was operated at full capacity for profitability, but safety was ensured by controlling the Coil Outlet Temperature (COT) to be maintained below the safety threshold of 850 degrees."

[0450] This self-reflective reporting capability can transform the decision-making process of AI, which can be regarded as a black box, into an explainable AI (XAI) form, thereby providing a high level of trust to factory operators.

[0451] FIG. 28 is a drawing illustrating a multi-agent workflow system according to one embodiment of the present disclosure.

[0452] Referring to FIG. 28, a multi-agent workflow system may be a system in which multiple specialized agents share roles to collaboratively perform a single task and aggregate the results to generate a final output. In one embodiment, the multi-agent workflow system may include a central agent that receives user input, analyzes the entire task to break it down into sub-tasks, and assigns each sub-task to an appropriate sub-agent; a sub-agent that performs the sub-task assigned by the central agent as an agent specialized in a specific function or domain; an aggregator that receives results generated by multiple sub-agents and generates a final output by synthesizing, organizing, and structuring them; a memory that stores intermediate results generated during the task execution process, sub-agent outputs, previous task history, etc.; a search engine; a database; an analysis tool; an external API; and tools that the sub-agent or central agent can call to perform the task.

[0453] In one embodiment, a user inputs a task request in the form of natural language, and a central agent analyzes the user input to break down the task into multiple sub-tasks. For example, the central agent may separate the task into sub-tasks such as market trend research, competitor analysis, and investment trend analysis. The central agent assigns the decomposed sub-tasks to each sub-agent, and the sub-agents may perform their respective tasks in parallel or sequentially. For example, a first sub-agent may collect the latest market trends, a second sub-agent may analyze competitor information, and a third sub-agent may summarize investment trends. The results of each sub-agent's work may be stored in memory or managed as intermediate outputs before being transmitted to an aggregator. An aggregator (Aggregator LLM) may synthesize the results received from multiple sub-agents to generate a final result having a logical structure. During this process, the removal of duplicate information, standardization of representation, and correction of contextual consistency may be performed. The aggregated result may be output to the user in the form of a report, summary, analysis results, etc.

[0454] According to one embodiment of the present disclosure, a naphtha cracking center scheduling system may include a multi-agent workflow system as illustrated in FIG. 28 at an upper layer to establish an optimal operation goal.

[0455] Referring to Fig. 28, the central agent breaks down tasks into sub-tasks and assigns them to sub-agents. When this is applied to the NCC system, when a user requests, "Please formulate the optimal production strategy for next month," the central agent (strategy planning agent) can break this down into the following sub-tasks.

[0456] 1. 1st Sub-Agent (Market Analyst): Analyzes international oil and naphtha price trends and spread (margin) trends of major products (ethylene, propylene, BD, etc.).

[0457] 2. Second Sub-Agent (Competitor Analyst): Collects news regarding the turnaround schedule and shutdowns of nearby competitor NCCs to predict the possibility of supply shortages.

[0458] 3. Third Sub-Agent (Inventory Analyst): Analyzes the current tank inventory levels in the factory and the planned volume of raw materials to be introduced.

[0459] The analysis reports generated by each sub-agent can be transmitted to the Aggregator LLM. The Aggregator can synthesize this information to generate a final strategy report stating, "Since ethylene margins are declining but a surge in propylene demand is expected, adopt a Mode-B operation strategy to increase propylene yield by 5% and maintain inventory at 80%." This final strategy is not limited to a simple text report but can be automatically converted and applied as weights or constraints of the reward function transmitted to the lower-level reinforcement learning scheduling agents (the agents in FIGS. 1 to 18).

[0460] According to another embodiment of the present disclosure, the system may be equipped with an automated reporting system utilizing a multi-agent workflow to synthesize complex process data and provide insights to the user. In an NCC process, a vast amount of data may be generated from different domains, such as receiving, mixing, and disassembly. By applying the structure of FIG. 28, a central agent may direct analysis for each part to generate periodic reports. For example, a first sub-agent may summarize the number of vessel delays and the status of demurrage incurred for the receiving part, a second sub-agent may analyze the standard deviation and tank operation efficiency of the mixing part, and a third sub-agent may analyze the energy intensity and production achievement rate of the disassembly part. An aggregator may combine these individual reports, unify the context, remove duplicates, and generate a Daily / Weekly Plant Operation Report. Beyond simple summation, the aggregator can provide the operator with a final result featuring a logical structure that explains the causal relationships between processes, such as, "Although the margin rate of the mixing process decreased due to incoming delays, the overall production target was achieved through the adjustment of the operating rate of the disassembly process."

[0461] FIG. 29 is a drawing showing an Agentic RAG (Retrieval-Augmented Generation) system according to one embodiment of the present disclosure.

[0462] Referring to FIG. 29, the Agentic RAG system may be an agent-based search augmentation generation system in which an agent actively searches for external information sources and contextually integrates the searched information to generate a response. The Agentic RAG system may include an agent that receives a user query, determines whether external information search is required for the query, and controls the search and generation process overall; one or more search tools for searching for information from external knowledge sources such as a vector-based search system, a document database, and a web-based information source; a vector database in which documents collected and processed in advance or information collected in real time is stored in an embedding form and which can perform a search based on similarity with the query; a generator that combines the searched information and the user query to generate a natural language response; and a result output unit that provides the generated response to the user.

[0463] In one embodiment, a user may input a question in the form of natural language. The agent analyzes the user query to determine whether a response can be provided using only internal knowledge or if external information retrieval is required. If the agent determines that a search is necessary, it may invoke one or more search tools to retrieve relevant information from external sources. In one embodiment, the search may include vector embedding-based similarity search, document database search, real-time web content search, etc. The retrieved information is not used as is, but may be configured into a context suitable for response generation through summarization, sorting, duplicate removal, or confidence-based selection processes. The generator receives the user query and the configured context as input and can generate a description or answer in the form of natural language. At this time, the agent controls the entire generation process and may perform additional searches or reconstruction as necessary. The generated response may be provided to the user as the final result. Unlike a structure that simply attaches search results to a prompt, the Agentic RAG system according to this embodiment allows the agent to actively control the search and generation processes.

[0464] In one embodiment, the agent can generate a response with high accuracy and up-to-dateness depending on the nature of the query by determining the necessity of a search, selecting a suitable source among multiple sources, iteratively supplementing search results, and adjusting the contextual construction method.

[0465] According to one embodiment of the present disclosure, a naphtha cracking center scheduling system may include an Agentic RAG (Retrieval-Augmented Generation) system as illustrated in FIG. 29 to utilize text-based knowledge information as well as numerical optimization.

[0466] Referring to Fig. 29, the agent can retrieve and integrate information from external knowledge sources, such as the Vector database. When applied to an NCC system, the system can be equipped with a facility technical support agent. This agent can utilize the vast amount of disassembly operation manuals, accident / failure reports from the past 10 years, and maintenance history data stored in the Vector database. For example, if the third agent (responsible for disassembly) detects a condition where "the coil temperature of Unit 3 is rising abnormally," the Agentic RAG system can be automatically triggered. The RAG agent can search the Vector DB by converting the context of "coil temperature rise and vibration occurrence" into a query.

[0467] At this time, the agent can present to the operator a specific natural language guide by combining (1) searched past similar cases (similar accident in May 2023) and (2) emergency measures from the manufacturer's manual, stating, "The current situation is presumed to be due to intensified fouling, so immediately reduce the feed input by 10% and increase the steam ratio."

[0468] According to another embodiment of the present disclosure, an Agentic RAG system can be utilized to update scheduling constraints by collecting external unstructured information in real time.

[0469] NCC scheduling is sensitive to environmental regulations or market conditions. By applying the structure of Fig. 29, the regulatory monitoring agent can actively search for changes in government environmental regulations (e.g., carbon tax increase, stricter emission standards for specific chemicals) or news from the global petrochemical market (e.g., news of major port strikes) through a web-based information source (Web Search Tool).

[0470] If an agent searches for news stating that "the allowable benzene emission limit will be reduced by 5% starting next month" and determines it to be reliable information, it can interpret this and automatically modify the constraint parameters of the NCC scheduling system.

[0471] This ensures that the AI ​​can generate safe schedules free from compliance risk by autonomously learning and reflecting the latest external information, without the operator having to manually check news and change system settings.

[0472] FIG. 30 is a drawing showing a Multi-Agent Debate (MAD) system according to one embodiment of the present disclosure.

[0473] Referring to FIG. 30, the MAD system may be a system that derives a final response through a discussion-based structure in which multiple small language models (SLMs) present opinions from different perspectives on the same user query and perform mutual refutation and verification. In one embodiment, the MAD system may include an aggregator that receives a user query, controls the entire discussion, and determines a final response (Verdict) by synthesizing the discussion results generated by multiple small language models; multiple small language models participating in the discussion process, which are language models configured to present different inference paths, perspectives, or hypotheses for the same query or intermediate judgment; tools including a search tool, a vector database, a semantic-based database, etc., which some small language models or aggregators can call for fact verification or evidence reinforcement during the discussion process; and a memory that stores statements at each discussion stage, intermediate judgments (Verdict), previous discussion history, etc.

[0474] In one embodiment, a user inputs a question or request in the form of natural language, and the query may be transmitted to an aggregator. The aggregator may analyze the user query to generate an initial response or hypothesis to initiate a discussion. This initial response may serve as a starting point for the discussion, rather than being a final answer. Multiple small language models may present their respective answers to the initial response or user query. For example, a first small language model may directly present a specific conclusion, a second small language model may present a different conclusion while providing supporting evidence, and a third small language model may present supplementary or rebuttal opinions regarding existing answers. The aggregator may compare and analyze the multiple opinions generated in the first discussion stage to derive an interim judgment. This interim judgment may be used as input for the next discussion stage. Based on the interim judgment, the small language models may participate in the discussion once again. In this stage, some small language models may call a tool to perform fact verification or present additional supporting data. The discussion and judgment process may be repeated until a preset number of times or quality criteria are met. Through the iterative process, the focus of the discussion can gradually narrow, and uncertainty can be reduced. The aggregator can synthesize the results of the repeated discussions, determine the response deemed most appropriate as the final response, and provide it to the user.

[0475] A MAD system according to one embodiment may have a discussion structure that includes mutual refutation and verification between models, unlike a method of simply averaging or selecting multiple model outputs. That is, each small language model operates as an independent decision-making entity, and the aggregator can dynamically adjust the flow of the discussion and the judgment criteria.

[0476] According to one embodiment of the present disclosure, a naphtha cracking center scheduling system may adopt a Multi-Agent Debate (MAD) system as illustrated in FIG. 30 to ensure data security and computational efficiency. Since NCC process data is a core corporate secret, transmission to an external cloud may be restricted. Accordingly, the system may utilize multiple Small Language Models (SLMs) that can be run on an internal factory server (on-premise).

[0477] Referring to Fig. 30, each SLM presents opinions from different perspectives. When applied to an NCC system, the first SLM (production optimization model) can propose an aggressive schedule for profit maximization, while the second SLM (equipment maintenance model) can propose a conservative schedule considering equipment lifespan. For example, if the first SLM suggests raising the reactor temperature to 860 degrees to increase production, the second SLM can rebuttal by stating, "In that case, the coke generation rate will accelerate, requiring operation to be stopped within 3 days, which is a loss in the long run." The aggregator synthesizes this discussion process to derive a balance point between profit and stability (operation at 855 degrees and decoking after 5 days) as the final response (Verdict), which is difficult to derive with a single SLM. This enables high-level decision-making in a closed network environment through the collective intelligence of lightweight models, without the need for high-cost, large models.

[0478] According to another embodiment of the present disclosure, the system may utilize the MAD system as an intelligent alarm system to improve the accuracy of judgment regarding signs of process anomalies. A single model may misidentify noise from a specific sensor as an actual hazardous situation. Accordingly, when an anomaly is detected, a plurality of SLMs may initiate a discussion as follows.

[0479] 1. 1st SLM: "The pressure at Sensor A has surged, posing a risk of explosion. Requesting an emergency shutdown." (Initial Hypothesis)

[0480] 2. 2nd SLM: "I refute this. The readings for flow meter B and thermometer C, which are linked to sensor A, are within the normal range. This suggests that sensor A is likely the only failure." (Rebuttal and presentation of evidence)

[0481] 3. 3rd SLM: "After searching the recent maintenance history using a tool, it was found that Sensor A was replaced yesterday and there is a possibility of a calibration error." (Fact check)

[0482] Based on their discussion, the aggregator can make a final decision to "require inspection of Sensor A (maintain process operation)" instead of halting the factory. Through this cross-validation process, the system can prevent unnecessary factory downtime and maximize operational efficiency.

[0483] FIG. 31 is a diagram showing an A2A (Agent2Agent) protocol system according to one embodiment of the present disclosure.

[0484] Referring to Fig. 31, the A2A protocol system may have a structure in which multiple AI agents (AI Agent 1, AI Agent 2, etc.) cooperate organically with each other through the A2A protocol (Agent2Agent Protocol). First, the central A2A protocol can provide communication protocols to support efficient and secure collaboration between agents. Specifically, the A2A protocol may have the following characteristics. First, the system can provide a function that allows communication by exchanging only necessary information without directly sharing each other's source data. This enables collaboration while protecting the data privacy of each agent. Second, the system can support the distribution of complex tasks among multiple agents and the performance of negotiations to derive optimal results. Third, through state maintenance characteristics, the system can manage agents so that they can maintain and update shared context and state information in real time.

[0485] In one embodiment, the first AI agent (AI Agent 1) may primarily be responsible for local-based data processing and search tasks in the system. The first AI agent may have an independent technology stack and, for example, may run based on a language model (LLM) on the ADK framework and may use a database for data management. The first AI agent may be connected to various external function servers via the Model Context Protocol (MCP). In the example of FIG. 31, the first AI agent may access local files (documents, images, videos, etc.) or perform search tasks through a specific platform via the MCP server.

[0486] In one embodiment, the second AI agent (AI Agent 2) may be configured to primarily handle cloud infrastructure utilization and external communication tasks. The second AI agent may also have a technology stack independent of the first agent. The second AI agent may also connect to various MCP servers using the MCP protocol. Through this, the second AI agent can control the resources of a cloud service provider or perform communication tasks through collaboration and messaging tools.

[0487] In a system according to one embodiment, the Model Context Protocol (MCP) can maximize the scalability and flexibility of the system by separating each detailed function (file access, search, cloud control, communication, etc.) into an independent server form to communicate with the agent. As a result, the A2A protocol system illustrated in FIG. 31 can ensure that each agent operates independently while maintaining its own language model, framework, and database. At the same time, the system can provide the advantage of simultaneously securing strong security and high-level task processing capabilities by enabling agents to coordinate necessary tasks and collaborate without direct data exposure between them.

[0488] According to one embodiment of the present disclosure, a naphtha cracking center scheduling system may adopt an Agent-to-Agent (A2A) protocol system as illustrated in FIG. 31 for secure collaboration with external partners (e.g., shipping companies, raw material suppliers). In the NCC process, the supply of raw materials from external sources is a key element of scheduling, but each company is reluctant to share internal inventory levels or detailed vessel operation logs for security reasons. Referring to FIG. 31, the A2A protocol allows for the exchange of only necessary information without sharing source data.

[0489] The first AI agent (the receiving agent on the NCC side) and the second AI agent (the logistics agent on the shipping company side) can conduct negotiations through the A2A protocol. For example, based on the internal context that "receiving is required within 10 days due to a current inventory shortage," the first agent can query the second agent solely regarding "the possibility of arrival on X." The second agent can internally analyze vessel location data in the cloud without directly exposing it, and instead propose a negotiation proposal stating, "Day X is impossible, but Day X+1 is possible, and in this case, a freight rate discount is available." The first agent can receive this proposal, run internal simulations, and then decide whether to accept it. By introducing the A2A protocol in this manner, the system can achieve cooperative scheduling that optimizes the efficiency of the entire supply chain while perfectly protecting the privacy of sensitive data between companies.

[0490] FIG. 32 is a drawing showing an Agentic RAG (Retrieval-Augmented Generation) system according to one embodiment of the present disclosure.

[0491] Referring to FIG. 32, the Agentic RAG system may include a system that extracts data from external sources, such as websites, stores it in a Vector Database (Vector DB), searches for optimal information regarding user queries, and generates intelligent responses through a Large Language Model (LLM). In particular, the Agentic RAG system may support advanced question-response by including agentic elements such as memory, tools, and planning, going beyond simple search and generation functions. The pipeline of the Agentic RAG system according to one embodiment may be broadly divided into a data extraction stage, a search stage, and a generation stage.

[0492] In one embodiment, the data extraction step is a step of building a knowledge base from an external source, and the system may collect data from a designated website or various web content. The content collected may include not only text but also multimodal data in various formats, such as images, audio, and video. The system extracts text and metadata from the collected content and may chunk it into small units for efficient processing. Each chunked piece is vectorized through an embedding model and then stored in a vector database (Vector DB) to be utilized for future searching.

[0493] In one embodiment, when a user's query is input during the search phase, the system may perform a search optimization process. For example, the system may perform query rewriting to clarify the user's intent, followed by a query rewriting and embedding step that vectorizes the query; a similarity search step that searches for information fragments most similar to the user query within a vector database; and a context configuration and ranking step that selects and configures the most suitable context for response generation by performing relevance-based ranking based on the search results.

[0494] In one embodiment, during the generation phase, the system can derive a final response by combining the extracted context and agent functions. For example, an input prompt for the LLM can be generated by sophisticatedly combining the user query and the retrieved context. Furthermore, the Large Language Model (LLM) can generate an optimal response by performing functions such as memory functions (maintaining past conversational context), tool calling (utilizing external APIs), and planning (designing complex query resolution procedures), rather than simply generating text. The finally generated intelligent response can be delivered through a user interface.

[0495] The Agentic RAG system according to the present disclosure enables the generation of contextually accurate answers through architecture-based search accuracy enhancement, and can be utilized for automating complex tasks that are difficult to solve through simple search by utilizing planning and external tool calling functions. Furthermore, it allows for the processing of various forms of content, such as images and videos, in addition to text, thereby enabling broad information utilization. Additionally, since the Agentic RAG system can collect and utilize real-time information from websites, it can be widely applied to technical support requiring up-to-date information, search engine enhancement, personal assistant services, and the like.

[0496] According to another embodiment of the present disclosure, the system may include an Agentic RAG (Retrieval-Augmented Generation) system as illustrated in FIG. 32 to incorporate web-based real-time market information into scheduling.

[0497] Referring to Fig. 32, the system builds a knowledge base by collecting data from external sources such as websites. When applied to an NCC environment, the system can collect "global petrochemical news," "status of major port strikes," "reports on competitor shutdowns," etc., in real time and embed them into a vector database (Vector DB).

[0498] The Agentic RAG pipeline can be activated when a central agent responsible for scheduling performs the task of "establishing an ethylene production strategy for the next month." The agent queries "recent ethylene supply issues in the East Asia region" to search for relevant news in the Vector DB (Search stage), obtains information from the search results stating that "Competitor A's scheduled maintenance has been extended by one week" (Planning stage), and based on this, can generate a strategic decision such as "Generate an aggressive schedule to increase the cracking furnace operating rate from 95% to 100%, as a temporary price increase is predicted due to the competitor's supply disruption" (Generation stage). This enables the implementation of a Market-Adaptive AI system that goes beyond simple numerical optimization to interpret vast amounts of unstructured information (text, news videos, etc.) on the web in real time and reflect it in scheduling.

[0499] FIG. 33 is a schematic diagram of an AI agent system according to one embodiment of the present disclosure.

[0500] Referring to Fig. 33, the AI ​​agent system may include an advanced architecture that goes beyond a simple question-and-answer structure to autonomously interpret and execute the user's intent.

[0501] In one embodiment, a user can transmit commands to an agent system through system prompts and user prompts. Here, the system prompt defines the agent's role, persona, behavioral guidelines, and constraints, while the user prompt may include specific queries or requests for performing specific tasks. Based on the input prompts, the AI ​​agent system can generate an optimal response by organically utilizing complex internal functions.

[0502] In one embodiment, the AI ​​agent system may support a planning function that identifies sub-tasks necessary to achieve an input complex goal and autonomously establishes execution priorities and sequences; a reasoning function that performs logical judgments based on the established plan and given data and derives the optimal solution for problem solving; a tool calling function that handles actual tasks by dynamically selecting and executing various tools such as external API integration, software execution, and search engine utilization; a memory storage function that ensures information consistency and supports intelligent judgment by storing and maintaining the results of previous steps or contextual information of conversations; a function that queries necessary information from large-scale data sources or systematically stores and manages generated data; and a feedback collection function that analyzes results generated during the execution process and forms a feedback loop to improve performance through continuous learning and behavior correction.

[0503] An agent system according to one embodiment of the present disclosure can generate an advanced final response by repeating the actions of planning, calling tools, storing in memory, and collecting feedback. Since such a system can dynamically select tools through an autonomous planning and execution structure and enables continuous improvement through a feedback loop, it can be widely used for complex multi-step tasks, business process automation, professional research, etc. Consequently, the system can function as an intelligent autonomous agent that goes beyond a simple input-output model.

[0504] According to one embodiment of the present disclosure, a naphtha cracking center scheduling system may adopt an Autonomous AI Agent System structure as shown in FIG. 33 to autonomously perform complex multi-stage process control.

[0505] Referring to Fig. 33, the system generates an advanced response by repeating planning, tool calling, and feedback collection. When applied to an NCC environment, the user can assign a persona to "maximize profit while complying with safety constraint Grade A" via a system prompt and request "create next week's schedule" via a user prompt.

[0506] Accordingly, the AI ​​agent system breaks down the entire scheduling task into sub-tasks such as raw material supply analysis, blending optimization, and load distribution for decomposition, establishes a sequence (planning), dynamically selects and executes external tools such as an NCC process simulator, a profit calculator, and a constraint verifier according to the plan (tool calling), stores simulation results (intermediate outputs) in memory, logically determines that "profit is high but an inventory shortage is expected on the third day" (inference and storage), modifies the plan itself to solve the expected problem, and can call the tool again with measures such as increasing the incoming quantity on the third day (feedback). In this way, the present embodiment can implement an autonomous scheduling system that corrects errors and finds the optimal solution through a feedback loop, rather than a simple input / output model.

[0507] FIG. 34 is a schematic diagram of a Large Language Model (LM) chatbot system according to one embodiment of the present disclosure.

[0508] Referring to Fig. 34, an LLM chatbot system may have a basic structure that receives user input and generates an output through Large Language Models. A query entered by a user can be transmitted to a Large Language Model. At this time, the model can improve the accuracy of the response by referencing Enterprise Data in various formats, such as documents, images, and videos. The system may be composed of a single LLM call process without complex agent functions (planning, tool calling, etc.). In other words, the flow from input to output is unidirectional, and the system may not include separate external tool integration or complex work pipelines. A system with such a simple structure can provide efficiency suitable for simple question-and-answer, FAQ systems, and basic customer service, where fast and accurate information delivery is key rather than complex logical reasoning. However, since the LLM chatbot system lacks a process for collecting feedback or iterative plan modification, it may lack the ability to maintain detailed context during long conversations or may have limitations in autonomously performing complex tasks that require multiple steps.

[0509] According to another embodiment of the present disclosure, the system may provide an LLM chatbot system as illustrated in FIG. 34 as a separate interface to provide rapid and accurate process information to the operator.

[0510] Referring to Fig. 34, the LLM chatbot system can provide immediate answers to queries by referencing corporate data without a complex planning process. When applied to an NCC system, the system can include a process monitoring chatbot.

[0511] When an operator inputs, "Tell me the current coil exit temperature (COT) and trend of Unit 3," the chatbot skips complex reasoning processes, immediately accesses the NCC real-time sensor log to extract the corresponding value, and can generate and output an accurate answer such as, "The current COT of Unit 3 is 845 degrees, and it has been trending upward by 2 degrees over the past hour."

[0512] Unlike the autonomous agent of Fig. 33, this structure has very low latency and fast response speed, so it can be specialized for FAQ systems, daily briefings, and field inspection support that require urgent situation identification or simple information retrieval, thereby increasing operational efficiency.

[0513] FIG. 35 is a schematic diagram of a Robotic Process Automation (RPA) system according to one embodiment of the present disclosure.

[0514] Referring to FIG. 35, the RPA system may include a structure that performs structured tasks according to predefined rules using user input as a trigger. In one embodiment, the RPA system can execute rule-based triggers. When user input occurs, the RPA system may initiate an operation according to a pre-set logical structure (such as If-Then). At this stage, rather than flexible judgment, a fixed tool section may be invoked to perform a defined procedure. In one embodiment, the RPA system may use an LLA secondarily. Data that has passed through the rule-based triggers may be passed to a Large Language Model (LLM). Here, the LLM may be utilized in a limited, secondary role of refining results or performing simple processing by referencing a fixed tool section or a database. In one embodiment, since the RPA system operates consistently according to defined rules, it can be optimized for handling repetitive automation tasks or simple back-office tasks. The RPA system can demonstrate high efficiency in workflows where data input and output are relatively clear.

[0515] In other words, the RPA system can implement an automation architecture that ensures business stability and repeatability by leveraging the language processing capabilities of LLM while retaining overall control over fixed rules.

[0516] According to one embodiment of the present disclosure, a naphtha cracking center scheduling system may include a Robotic Process Automation (RPA) system, as illustrated in FIG. 35, as a post-processing module to automate the execution procedure of a confirmed schedule. Referring to FIG. 35, the RPA system can perform standardized tasks according to predefined rules. When applied to an NCC system, the RPA system can be triggered at the moment an AI agent generates and confirms an optimal receiving / production schedule. Specifically, the RPA system can execute tool sections such as ERP input, which parses the generated schedule data and automatically enters it into the production planning module of the in-house ERP system; sending purchase orders, which drafts and sends an 'Order Email' to be sent to a supplier according to the determined raw material receiving plan; and creating daily reports, which converts a summary of today's production plan into a PDF and sends it via messenger to the plant manager and relevant department heads. At this time, an auxiliary LLM within the RPA system can perform an auxiliary role, such as politely refining the email wording or generating ERP input comments. By combining RPA in this way, the system can maximize work efficiency by entrusting scheduling requiring high-level judgment to AI and repetitive administrative processing to RPA.

[0517] FIG. 36 is a schematic diagram of a Retrieval-Augmented Generation (RAG) system according to one embodiment of the present disclosure.

[0518] Referring to Fig. 36, the RAG system is broadly composed of three stages—Retrieval, Augmentation, and Generation—to derive accurate responses based on external knowledge. When user input occurs during the retrieval stage, the RAG system can embed and vectorize it. Subsequently, the RAG system can search for data highly relevant to the query among the numerous information rules stored within the Vector DB and Database. The Augmentation stage is a step in which the input is reinforced based on the retrieved data; the RAG system can utilize external tools and combine the retrieved knowledge data with existing System Prompts and User Prompts to generate an augmented prompt in a form most suitable for processing by the LLM. The augmented prompt can be passed to the Large Language Model (LLM) during the Generation stage. The LLM can generate a final response (Output) by adding the retrieved external context to the learned internal knowledge and provide it to the user.

[0519] In one embodiment, the RAG system generates correct answers by referencing external knowledge (documents, etc.) in real time, so it can be utilized to improve the accuracy of simple LLM or for professional QA systems, document search, and summarization services. Additionally, the RAG system can reflect the latest information without retraining the model and can suppress hallucination because it finds the basis for the answer in external data.

[0520] The system may include a Retrieval-Augmented Generation (RAG) system as illustrated in FIG. 36 to provide the basis for scheduling results and to accurately answer operator queries. Referring to FIG. 36, the RAG system retrieves relevant information from a vector database to augment the response. When applied to an NCC environment, the system can vectorize and store 'Standard Operating Procedures (SOPs)', 'Material Safety Data Sheets (MSDS)', 'Safety and Environmental Regulations', etc.

[0521] If an operator asks, "Does increasing the pressure of Unit 3 by 5% according to the currently determined schedule violate safety regulations?", the RAG system does not answer solely based on the LLM's internal knowledge (prevention of hallucinations), but can search for 'Chapter 3, Section 4 of the High Pressure Operation Safety Guide' in the Vector DB. The RAG system then configures the retrieved document into an augmented prompt, enabling the LLM to generate an accurate, evidence-based answer such as, "According to Chapter 3, Section 4 of the Safety Guide, the safety valve setpoint must be readjusted upon pressure increase; therefore, failure to take such action constitutes a potential violation of regulations." This suppresses the hallucination phenomenon where AI provides plausible answers based on false information, thereby ensuring reliability in chemical processes where safety is paramount.

[0522] FIG. 37 is a schematic diagram of a Learning-Augmented Mechanism (LAM) system according to one embodiment of the present disclosure.

[0523] Referring to FIG. 37, the LAM system may include an intelligent automation architecture that combines user intent with learned behavioral patterns to lead to the actual execution of a task. In the training data stage, when user input occurs, the LAM system may refer to pre-trained data. The training data utilized at this time includes screenshots of specific task execution or tool usage data, which can serve as a basis for the agent to learn how to operate in a real user environment. In one embodiment, a large language model may process the task based on the training data. The LLM may call external tools or access a database to refine the task if necessary, and can derive executable commands beyond simply generating language. The optimal path determined by the model leads to actual action, and finally, a response indicating the completion of the task can be output to the user.

[0524] In one embodiment, the LAM system uses a model trained by directly utilizing tool usage data, thereby enabling behavior-based learning and execution in an actual interface environment. Additionally, because the LAM system learns user operation patterns, it can demonstrate high efficiency in the field of repetitive GUI (Graphical User Interface) task automation.

[0525] According to one embodiment of the present disclosure, a naphtha cracking center scheduling system may include a Learning-Augmented Mechanism (LAM) system as illustrated in FIG. 37 as an execution module to automatically control legacy control systems that are difficult to integrate with APIs. Referring to FIG. 37, the LAM system performs tasks by learning user input (screenshots, tool usage). When applied to an NCC environment, the system can visually learn action patterns for a skilled operator to adjust valves or set temperatures on the GUI (Graphical User Interface) screen of the control room. For example, if a scheduling agent issues a command to "set the temperature of Unit 3 to 850 degrees," but the equipment does not support APIs, the LAM system can operate. The LAM agent can automatically perform a series of GUI operations by capturing and analyzing the current control screen, moving the mouse cursor to the 'temperature setting field' according to the learned pattern, entering 850, and clicking the confirmation button. This allows aging process equipment to be perfectly integrated with the latest AI scheduling system through visual learning alone, without the need for separate hardware replacement or high-cost system integration projects.

[0526] A summary of the AI ​​system mentioned above is as shown in [Table 3] below.

[0527] Item Structure Key Features Limitations LLM Chatbot Single LLM call, simple response generation, lack of context retention RPA Rule-based + fixed tool, repetitive automation, lack of flexibility RAG Search-based + prompt augmentation, utilizes external knowledge, lack of information other than search LAM Behavior learning-based GUI automation, requires learning, limited versatility AI Agents Planning + memory + tool + repetition, performs autonomous tasks, complex structure, consumes resources

[0528] FIG. 38 is a diagram illustrating the memory structure of an AI agent and a question-and-answer process utilizing the same according to an embodiment of the present disclosure. Referring to FIG. 38, the AI ​​agent can utilize a memory unit to perform advanced decision-making on an input query and generate an optimal response. The memory is divided into short-term and long-term memory and operates hierarchically, supporting complex decision-making and task execution through various types of memory. Short-term memory is a temporary memory space for focusing on the currently ongoing workflow and may include a working memory that tracks and manages the inference process and real-time work flow for each workflow, and a cache memory that supports immediate access to data or result values ​​frequently used in the current workflow. Long-term memory is a knowledge and experience-based memory that is continuously preserved and may include episodic memory, which manually or automatically stores specific events or incidents that occur during workflow execution; semantic memory, which stores conceptual and factual knowledge such as "Paris is the capital of France"; and procedural memory, which stores methodologies and procedural knowledge for performing specific actions or tasks, such as "How to reset a server," which stores methods for performing tasks or procedural knowledge (e.g., "How to reset a server"). In one embodiment, a central memory controller may operate in organic linkage with a Language Model Framework. When a query is input, language models within the framework may refer to memory to analyze the query, search for additional relevant information in memory if necessary based on the analyzed information, and generate a final response through a decision loop.The input query and output response can support various data formats, including text, documents, images, and videos.

[0529] In one embodiment, the MCP server is responsible for interfacing with external knowledge and tools and may include a Vector DB that stores search-based embedding information, a Semantic DB that is a conceptual knowledge base, and the like. Additionally, the MCP server can perform functions by exchanging data in real time with external services, such as payment, communication, and web search, through a Third Party API integration unit. Consequently, an AI agent equipped with the memory structure of the present disclosure can autonomously perform complex multi-step tasks through the harmonious utilization of short-term and long-term memory, and can achieve practical business automation through integration with external tools.

[0530] In one embodiment, the process can be handled as follows.

[0531]

[0532] One embodiment of the present disclosure can systematically integrate design elements essential for implementing an agent system that can continuously learn and make decisions according to the situation, as shown in [Table 4] below.

[0533] Item Description Integration of Memory Structures Combines short-term (Working / Cache) and long-term (Episodic / Semantic / Procedural) memory to enhance context awareness and learning. Includes decision logic for prompt adjustment and response generation based on a framework linked to memory. External Information Extensibility Supports information enrichment and real-time integration via databases and APIs. User-Customized Responses Capable of generating responses that reflect past experiences and current context.

[0534] Next, we would like to explain the types of language models utilized by the AI ​​agent system. Each model has a specific processing method and role, and a suitable model can be selected and utilized depending on the nature of the task. FIG. 39 is a diagram illustrating the text generation process of a first language model according to an embodiment of the present disclosure.

[0535] Referring to FIG. 39, the first language model is a general-purpose language model pre-trained based on large-scale text data, and can generate text through the following step-by-step process.

[0536] (1) Pretraining stage (Pretrain): The first language model can acquire the structure and knowledge of the language by training the model in advance using a large text corpus.

[0537] (2) Prompt encoding step: The first language model can convert the input prompt into a vector form that the model can process through tokenization and embedding processes.

[0538] (3) Transformer layer processing step: The first language model can process the encoded input through a transformer layer to generate a hidden state containing context information.

[0539] (4) Logit and probability calculation step: The first language model can calculate the logit and occurrence probability for the next token based on the generated hidden state.

[0540] (5) Token sampling and selection step: The first language model may sequentially sample or select one of the top K (Top-k) tokens based on the calculated probability.

[0541] (6) Decoding and output step: The first language model can decode the selected tokens into text form and output the final response text.

[0542] According to one embodiment of the present disclosure, a naphtha cracking center scheduling system may include a first language model (Pre-trained LLM) as illustrated in FIG. 39 for natural language communication with an operator and for providing general engineering knowledge. Referring to FIG. 39, the first language model pre-trains a large-scale text corpus. When applied to an NCC system, the corpus may include chemical engineering textbooks, NCC facility operation manuals, petrochemical-related patents and papers, etc. By utilizing the first language model pre-trained with such domain-specific data, the system can understand the context through prompt encoding and transformer processing, even if an operator inputs an ambiguous query such as, "What are the expected problems when the specific gravity of naphtha is 0.75?" Subsequently, through a token selection process, the model can generate an answer with clear chemical causal relationships, such as, "If the specific gravity is high, there is a possibility that the aromatic content is high, which may accelerate coking." The model can explain and guide general phenomena of the process based on pre-trained knowledge without the need for separate rule input.

[0543] FIG. 40 is a diagram illustrating the operation process of a second language model according to one embodiment of the present disclosure.

[0544] Referring to FIG. 40, the second language model may include a Mixture of Experts (MoE) model. The Mixture of Experts (MoE) model may include a distributed model structure that selectively activates only some experts (sub-networks) of the entire network depending on the input data. The second language model can perform a network training step in which multiple sub-networks and a gating or routing network for assigning each token to an appropriate expert are trained together; an input encoding step in which the input is tokenized and embedding to convert it into a vector form that the model can process; an expert selection step in which the gating network selects the top-k most suitable expert sub-models for each token in real time; a conditional execution step in which input data is forwarded only to the expert sub-models selected by the gating network to perform operations; a result merging step in which the individual output values ​​derived from the selected experts are merged into one using a weighted average or aggregation method; and a final decoding step in which decoding is performed based on the merged results to generate final text data. This MoE model has the advantage of very high computational efficiency relative to the number of parameters because it maintains a large overall parameter scale while activating only some expert networks during actual inference.

[0545] To efficiently process complex and diverse NCC process data, the system may adopt a second language model based on Mixture of Experts (MoE), as illustrated in Fig. 40, as a core inference engine. The expertise required for the NCC process varies depending on the receiving, mixing, and decomposition stages, and the optimal operating conditions differ according to the characteristics of the feed. Referring to Fig. 40, the MoE model can select appropriate experts through a gating network. When applied to NCC scheduling, the system's gating network can identify what data needs to be processed at the moment.

[0546] For example, if the input is a ship unloading schedule, the system can activate the logistics expert sub-network (Expert A); if the input is the thermal cracking of paraffinic naphtha, the system can activate the paraffin reaction expert sub-network (Expert B); and if the input is the thermal cracking of naphthenic naphtha, the system can activate the naphthenic reaction expert sub-network (Expert C).

[0547] In this way, instead of using all parameters, the system can perform computations by activating only the top K experts that are perfectly suited to the current situation through conditional computation. This allows for the handling of large-scale NCC scheduling problems with tens of thousands of variables while drastically reducing inference latency and obtaining precise prediction results optimized for each process characteristic.

[0548] FIG. 41 is a diagram illustrating a method of operation of a third language model according to one embodiment of the present disclosure.

[0549] Referring to Fig. 41, the third language model may include a Large Reasoning Model (LRM). Beyond simply generating results, the LRM can autonomously generate and verify a chain of thought to solve complex problems. The LRM recognizes the input query and related context by dividing them into tokens that the model can process. In the initial stage, it internally generates chain of thought steps, which are logical steps for problem solving. Through a reflection layer, it evaluates and prioritizes various possible reasoning paths generated. If the evaluation results are insufficient or logical errors are found, it can selectively regenerate the reasoning steps to enhance logical completeness. Based on the verified optimal reasoning path, the LRM determines the final logical answer and outputs the final correct answer to the user, which is sufficiently supported by internal reasoning processes. Through this deep thinking process, the LRM can provide high reliability in fields such as solving high-difficulty mathematical and coding problems and logic-based question answering, which were difficult for existing language models to solve.

[0550] According to one embodiment of the present disclosure, a naphtha cracking center scheduling system may include a third language model based on a Large Reasoning Model (LRM), as illustrated in FIG. 41, to analyze the root cause of complex process anomalies. Referring to FIG. 41, the LRM generates a chain of thought and verifies it through a reflection layer. This can be applied to a fault diagnosis agent of an NCC system. For example, in a situation where the yield of the cracking furnace drops sharply, a conventional model may simply issue a one-dimensional prescription of raising the temperature. The LRM can provide a highly reliable solution in high-difficulty process troubleshooting situations by not only performing numerical optimization but also inferring engineering causal relationships step-by-step and re-verifying its own judgment.

[0551] FIG. 42 is a diagram illustrating a method of operation of a fourth language model according to one embodiment of the present disclosure.

[0552] Referring to FIG. 42, the fourth language model may include a Vision Language Model (VLM). The VLM is a multimodal model capable of integrally understanding and processing data of different formats, such as images and text. It performs image encoding to encode input image data and extract visual feature information that the model can process, performs text tokenization to recognize text inputs such as user queries or context by dividing them into tokens, combines the encoded visual information and the tokenized text information to generate a single integrated embedding, and the generated integrated embedding passes through a joint transformer layer to learn deep correlations between the two modalities. The VLM can generate complex inference by cross-referencing between visual information and language information. Additionally, the VLM can decode the final answer based on the integrated representation and output it to the user.

[0553] A VLM according to one embodiment can demonstrate superior performance compared to existing language models in the field of advanced question answering and image description generation including visual information through a multimodal integration mechanism.

[0554] According to another embodiment of the present disclosure, the system may include a fourth language model based on a Vision Language Model (VLM) as illustrated in FIG. 42 to utilize visual information of the site that sensor data cannot capture. Referring to FIG. 42, the VLM combines images and text to generate an integrated embedding. This can be applied to a visual monitoring agent of the NCC system. The condition inside the NCC furnace is difficult to fully determine using only temperature sensors, and the color or shape of the burner flame can serve as an important indicator.

[0555] The VLM can receive visual images captured internally through disassembly and real-time temperature sensor data (text input). The VLM performs image encoding by extracting visual features such as the flame being close to red and fluctuating irregularly. It learns through the Joint Transformer layer that while the temperature appears normal in text, there are clear visual signs of incomplete combustion. Consequently, in situations where the system would be determined to be normal based solely on sensor data, it can generate a sophisticated diagnosis based on visual information, stating, "Inspection required due to suspected burner tip blockage."

[0556] FIG. 43 is a diagram illustrating a method of operation of a fifth language model according to one embodiment of the present disclosure.

[0557] Referring to FIG. 43, the fifth language model may include a Small Language Model (SLM). As a language model that operates efficiently even in environments with limited computational resources through a lightweight structure, it performs input tokenization by dividing input data into token units that the model can process, performs low-dimensional embedding by projecting the divided tokens into a low-dimensional space rather than a high-dimensional space to perform embedding, processes a lightweight transformer that improves computational speed by processing context through a relatively small number of transformer layers, calculates the probability of occurrence for the next token, samples or selects the optimal token based on the calculated probability, and decodes and outputs the final text. Through this optimized structure, such an SLM can enable low-power and high-speed inference in edge devices or on-device AI environments.

[0558] According to one embodiment of the present disclosure, a naphtha cracking center scheduling system may install a fifth language model based on a Small Language Model (SLM) as shown in FIG. 43 on an edge device to ensure real-time control without communication delay and safety against network failures.

[0559] Referring to Fig. 43, the SLM can perform low-power, high-speed inference with a lightweight structure. When applied to an NCC site, while the central server's massive model (LLM) establishes the overall 24-hour schedule, specialized SLMs can be installed in each decomposition controller or smart valve at the site to operate independently. For example, in an emergency situation where communication with the central server is lost and the pressure in Unit 3 rises rapidly, the SLM installed in Unit 3 can make a decision on its own without waiting for instructions from the central server. The SLM can immediately infer, "Although communication is lost, the current pressure data pattern is 99% similar to signs of an explosion," and generate and execute a local command to "open the emergency pressure relief valve." This overcomes the communication dependency that is a weakness of centralized AI and enables the establishment of a distributed AI control ecosystem where thousands of facilities possess independent intelligence.

[0560] FIG. 44 is a diagram illustrating the operation method of a Large Action Model (LAM) according to one embodiment of the present disclosure.

[0561] Referring to FIG. 44, LAM is a model designed to perform specific actions in a real or virtual environment beyond simple text generation. It receives inputs of task descriptions and current environmental state information, tokenizes the input descriptions and environmental information, and performs context embedding. Additionally, LAM autonomously designs a sequence of actions to achieve a goal based on the Chain-of-Thought technique, generates API calls, command calls, or control signals necessary to execute planned actions, and executes actions within the real environment based on the generated signals. LAM monitors execution results and performs modification and iteration procedures as needed to derive optimal results by modifying the plan or revising the task as necessary. In one embodiment, LAM can be used in robot control, game agents, automation systems, etc.

[0562] FIG. 45 is a diagram illustrating the operation method of a Hierarchical Reasoning Model (HRM) according to one embodiment of the present disclosure.

[0563] Referring to Fig. 45, the HRM is a model that maximizes problem-solving ability and efficiency by hierarchically separating and processing complex inference tasks. It encodes input data and task contexts to convert them into internal representations, establishes an abstract and macroscopic plan for overall task resolution in the upper layer, the H-module, and performs specific and detailed operations based on the high-level plan in the lower layer, the L-module. The HRM can repeat a feedback loop between the H-module and the L-module until the task is completely completed, undergoing an iterative coordination process to ensure that the high-level plan and the results of the low-level operations match and converge (Hierarchical convergence loop). Based on the converged final state, the HRM can decode and output a logical final answer. Through this hierarchical structure, the HRM can demonstrate outstanding performance in complex multi-step planning and high-difficulty inference tasks that are difficult for a single-layer model to solve.

[0564] According to one embodiment of the present disclosure, a naphtha cracking center scheduling system may include a Hierarchical Reasoning Model (HRM) as illustrated in FIG. 45 as a core reasoning engine to convert abstract management goals into specific facility operation variables. For example, the H-module (strategy layer) may establish a macroscopic and abstract plan such as "the goal for this quarter is to increase ethylene production by 10% and reduce energy costs by 5%," and the L-module (execution layer) may receive the plan from the H-module and perform specific operations such as "raising the temperature of cracking furnace No. 3 from 840 degrees to 850 degrees and increasing the quenching pump speed by 3%." If the L-module calculates that "raising the temperature to 850 degrees increases energy costs by 7%, violating the H-module's goal (5% reduction)," this feedback is passed back to the H-module, and the H-module can modify the goal (relaxing the energy cost reduction goal) or guide the L-module to find other alternatives (utilizing other decomposition), thereby ultimately establishing a final schedule in which management goals and physical control align without contradiction.

[0565] FIG. 46 is a drawing showing a ToolFormer (Tools-trained Model) according to one embodiment of the present disclosure.

[0566] Referring to Fig. 46, ToolFormer is a language model trained to use various external tools. It starts based on a pre-trained LLM, samples examples of tool calls, determines valid tool usage through test and evaluation, and performs fine-tuning with filtered data. ToolFormer has advantages in integration with calculators, searches, API calls, etc.

[0567] Table 5 summarizes the four major types of artificial intelligence systems—basic LLM workflow, RAG (Retrieval-Augmented Generation), single AI agent, and multi-agent based Agentic AI—in terms of structure, function, characteristics, and use cases.

[0568] Item LLM Workflow RAGAI Agent Agentic AI Functionality Input-based next token prediction Answer search and augmentation through external knowledge Autonomous execution + component combination Autonomous tasks based on multi-agent collaboration Representative Use Cases Text generation, summarization Accurate Q&A tools and planning Workflow Large-scale tasks, collaboration-based problem solving Strengths Fast, simple, easy to deploy Improved accuracy through external knowledge Planning + reasoning-based automation Flexible division of labor, solving complex problems Weaknesses Limited contextual understanding Sensitivity to data quality Requires access to clear goals and tools Increased design and control complexity Examples Chatbots, email generators Graph RAG, Modular RAG ReAct Agent, Rewoo Agent CUA (Computer Using Agent), Embodied Agents

[0569] FIGS. 47 to 52 illustrate vulnerabilities of an MCP according to an embodiment of the present disclosure. Referring to FIG. 47, a command injection problem may occur in an MCP system. An attacker may insert a hidden command that an agent can interpret into a prompt entered by a user. The agent mistakes this for a legitimate request and accesses external resources through the MCP server. In this process, unauthorized access to databases and external servers may occur. Referring to FIG. 48, a tool poisoning vulnerability may exist in an MCP environment. An attacker may register a malicious tool containing malicious code on the MCP server or contaminate an existing tool. When the tool is executed, the agent performs actions intended by the attacker. For example, access rights to a trusted service may be granted to the attacker, thereby allowing API keys or personal information to be leaked externally.

[0570] Referring to Fig. 49, structural problems related to Server-Sent Events (SSE), the communication method of MCP, may occur. Since the SSE method transmits data by dividing it into multiple segments, the communication connection must remain open for a long time. When interoperating with external services, system delays may occur as the connection is maintained excessively, and there is a risk of exposure to additional security risks, such as interception through the open connection channel.

[0571] Referring to Fig. 50, a privilege escalation issue caused by a malicious tool may occur. A malicious server installed within the MCP server may intercept calls directed to other trusted tools or override their functions. Even if a user calls a legitimate tool, a malicious tool overridden by an attacker may actually be executed, thereby stealing the user's trust privileges.

[0572] Referring to Fig. 51, there may be security threats related to persistent context maintained between sessions. MCP records and maintains conversational context and context across sessions to improve user experience, but if an attacker tamperes with this recorded context, the session information of the linked tools may be corrupted, causing the agent to make incorrect decisions or the security settings to be neutralized.

[0573] Referring to Fig. 52, a problem of server data theft may occur on the client side. There is a risk that an attacker who has infiltrated the client may use a malicious MCP tool to gain control of connected tool servers, thereby transmitting or stealing sensitive credential information, such as user data or chat passwords stored in external services, to another malicious server.

[0574] FIG. 53 is a diagram illustrating a context engineering structure in an AI agent system according to one embodiment of the present disclosure.

[0575] Referring to Fig. 53, the AI ​​agent system may have context engineering and a memory hierarchy to generate situation-optimized responses by sophisticatedly configuring and utilizing various contextual information generated during the interaction process. When a user inputs a query or request in various forms, such as text, voice, or images, the agent can analyze the input information and the current context to formulate a plan and determine the direction of the task. The RAG module collects relevant information by searching for similar documents in a vector database when necessary, and the system calls action tools according to the requested task to obtain specific execution results. It can also dynamically configure or update the final prompt by integrating the collected search results, tool execution values, and the agent's inference content. Additionally, the system can generate a final response based on the configured prompt and deliver it to the user. All contextual elements within the session (input, tool usage history, inference, etc.) are recorded in 'Chat History,' and specific contexts or results with high reusability can be added to long-term memory based on an MCP server or database to be utilized for future interactions.

[0576] In one embodiment, the prompt may be composed of user input corresponding to a query or command, tool usage results including API call results, calculation results, etc., search-based context including documents retrieved from the RAG, agent reasoning including internal reasoning and planning, user information including preferences, ID, status, etc., conversation history including the previous conversation context, etc.

[0577] In one embodiment, the prompt may be deleted based on priority when the maximum prompt length is exceeded. For example, the priority may be configured in the order of agent inference > search context > tool results > user information > past conversation history. In one embodiment, result values ​​after a tool call may be inserted into the prompt in the form of a "contextual tag." Additionally, the search context may be inserted along with a summary and confidence score, rather than the original text.

[0578] In one embodiment, short-term memory is intended to maintain the entire context within a session and may store user input, prompt components, reasoning processes, tool usage results, etc. Short-term memory is deleted upon session termination, but important information may be transferred to long-term memory.

[0579] In one embodiment, the long-term memory may be composed of an MCP server (Agent Metadata) and a domain knowledge DB (Structured Knowledge), etc. The long-term memory may be updated when the Add to memory command is executed or when automatic saving conditions are satisfied. For example, information such as "User A prefers tools related to 'data visualization'" and "On August 7, 2025, the 'RAG + Tools' path was used in the 'Context Engineering' flow" may be stored in the long-term memory.

[0580] In one embodiment, the agent can determine whether to call a tool, the necessity of searching, and the possibility of repeated calls. Additionally, the agent can perform priority-based reasoning. For example, priorities may proceed in the order of user goal → environment state → available resources → execution strategy. In one embodiment, the agent can support parallel calls to multiple tools and support feedback-based iterative execution after execution.

[0581] For example, in the case of multimodal question and answer, when a user image is uploaded, a corresponding description is generated, and a date corresponding to the user image can be calculated via a tool call.

[0582] As another example, in the case of report generation, the search and summarization process proceeds based on user instructions, and templates can be inserted and edited.

[0583] As another example, in the case of automated schedule coordination, the calendar API is invoked based on natural language requests, and schedule recommendations can be provided.

[0584] According to one embodiment, a multi-agent-based Agentic AI extension structure may be supported. Additionally, according to one embodiment, a prompt dynamic optimization (auto-slimming) algorithm may be implemented. Furthermore, a memory vectorization-based summary storage module may be constructed, and user-specific customized context weighting profiling may be performed.

[0585] In particular, AI agents are characterized by autonomy and independence. Autonomous agents act according to their own rules without external intervention; a representative example is Automated Guided Vehicles (AGVs) in the robotics field, which recalculate their own paths based on real-time order data. Furthermore, they possess decentralized characteristics, allowing agents to make independent decisions without central control. This is implemented in the robotics field by multi-robot systems establishing collaborative path planning using only local sensor data.

[0586] FIG. 54 is a block diagram of a system according to one embodiment of the present disclosure.

[0587] Referring to Fig. 54, the AI ​​agent acts from a local perspective. Local perspective is an important concept in AI agent technology, referring to the characteristic where the agent perceives only the part related to its mission rather than the entire system. This is a key element that enables efficient decision-making and resource utilization in distributed systems; the agent perceives only the part related to its mission rather than the entire system, and in cyber-physical systems (CPS), this can be implemented in a way where each agent monitors only a specific section of the power grid. This local perspective enables efficient decision-making and resource utilization in distributed systems.

[0588] A distributed reinforcement learning-based AI agent system according to one embodiment can implement autonomy through a cyclical structure of 'Perception → Knowledge Base → Reasoning & Planning → Action & Actuation → Learning Loop'. Instead of simultaneously grasping the state of the entire system, each agent in the system can adopt a local perspective, focusing only on the specific area or task under its charge. This can provide the effect of saving computational resources and reducing the cognitive load of individual agents in large-scale, complex systems, such as power grid monitoring in cyber-physical systems (CPS) or path planning in multi-drone systems. The local perspective can serve as a core principle for enhancing the decentralization, autonomy, and scalability of the system.

[0589] In one embodiment, the system may adopt a distributed reinforcement learning framework so that each agent can learn independently while achieving a global goal. The sensing unit (5420) can generate local state information by integrating and processing multimodal sensor data such as cameras, lidar, and radar. The memory unit (5450) includes short-term and long-term memory structures and can manage persistent knowledge such as perceived environmental information, traffic regulations, and road structures. The inference unit (5430) can plan an optimal path and perform decision-making by utilizing a Transformer-based neural network architecture. In particular, computational efficiency can be enhanced through a multi-token prediction technique. The execution unit (5440) can reflect the determined action in an external environment, such as robot arm, drone movement, or vehicle engine control. The learning module (5460) can update the policy by receiving feedback on the reward value based on the execution result.

[0590] In one embodiment, game-theoretic methods may be utilized to address the complexity of cooperative reward design arising from each agent learning independently. By using a reward scheduler that considers Nash Equilibrium or Pareto optimization, conflicts of interest between agents can be prevented, and overall system optimization can be achieved. Communication between agents is conducted via a standardized Agent-to-Agent (A2A) protocol, which can coordinate model information sharing and collaborative learning. A safety monitoring module may be included to address the issue of unpredictability associated with autonomy. The system may have a redundancy structure in which control is automatically transferred to a manual control mode or transferred to a central control unit when risk indicators exceed a threshold. Additionally, the reliability of the system can be enhanced by clearly presenting the basis for the agents' decisions to the user through an explainable AI module.

[0591] In one embodiment, a distributed reinforcement learning-based AI agent system that autonomously determines a path and performs cooperative actions based on multi-modality sensor data input from an environment may include a plurality of agents that perform decisions independently without central control. Each agent may include a sensing unit (5420) that collects sensor data within the agent's assigned area to generate local state information, a memory unit (5450) that stores and manages local state information, an inference unit (5430) that performs decision-making and path planning for achieving a goal based on local state information and includes a transformer-based neural network architecture, an execution unit (5440) that reflects the determined path or action to an external environment, and a learning module (5460) that receives a reward value from the execution result and updates a policy. The plurality of agents may be implemented to achieve a global goal for the entire system by cooperating with each other through a game theory-based cooperative reward structure.

[0592] A system according to one embodiment may include a cyclic structure for autonomously optimizing policies and improving performance through interaction with an external environment. A sensing unit (5420) may collect state information from an environment (5410). The sensing unit (5420) may receive multi-modality sensor data, such as a camera, lidar, or radar, and convert it into local state information in a form that can be processed by an agent. An inference unit (5430) may determine the optimal action for achieving a goal and plan a path based on the state information transmitted from the sensing unit (5420). At this time, the inference unit (5430) may improve the accuracy of decision-making by referring to existing knowledge, experience, and learned policy models stored in a memory unit (5450). An execution unit (5440) may reflect the determined action command to the environment (5410). The execution unit (5440) may perform the role of actually executing the determined action, such as the movement of a robot arm, the movement of a drone, or the control of a vehicle. The learning module (5460) can receive feedback on a reward value from the environment (5410) as a result of executing a reward-based policy update action. The learning module (5460) can update the policy parameters of the neural network within the inference unit by analyzing the correlation between the reward value and the state change. The learning module (5460) can increase learning efficiency by utilizing multi-token prediction techniques, etc., and can perform game theory-based reward scheduling so that the performance of individual agents contributes to the reward of the entire system. The memory unit (5450) can store the learning results updated by the learning module (5460). The stored data can be referenced again during the inference process of the next cycle to continuously enhance the intelligence of the agent.

[0593] In one embodiment, each agent can reduce the computational cost of the entire system by maintaining a local perspective through processing only sensor information within its assigned area, without accessing information outside its mission scope. In one embodiment, the game theory-based cooperative reward structure may include a cooperative reward scheduler that adjusts rewards to approach a Nash Equilibrium state based on each agent's individual utility function. In one embodiment, each agent may use an A2A communication protocol to send and receive messages for sharing model information, collaborative learning, and resource exchange between agents. In one embodiment, the AI ​​agent system may further include a federated learning-based parameter aggregator that receives parameters individually learned by multiple agents in an encrypted form and updates the policy of the entire system. In one embodiment, the AI ​​agent system may further include an explainable AI module that analyzes the output and internal state of the inference unit to generate explanatory information regarding the reasons for specific action choices. In one embodiment, the system may include a redundant control structure that includes a Safety Supervisor module that monitors the system's risk indicators and switches to a manual control mode or transfers control to a central unit when a threshold is exceeded. In one embodiment, the inference unit (5430) can improve computational efficiency by predicting multiple tokens simultaneously using a multi-token prediction technique. In one embodiment, the learning module (5460) can support a dual learning mode capable of switching between an on-device learning mode and a federated learning mode. In one embodiment, each agent can be implemented as at least one of a power grid management agent, an autonomous vehicle, a logistics robot, and a drone.

[0594] According to one embodiment of the present disclosure, a naphtha cracking center scheduling system may adopt a distributed reinforcement learning-based AI agent system structure as illustrated in FIG. 54 for efficient control of a large-scale process. Referring to FIG. 54, each agent has a local perspective that recognizes only its assigned area, rather than the entire system. When applied to an NCC environment, this is as follows. The cracking agent can collect temperature / pressure sensor data only for the 'No. 3 cracking furnace' under its charge, rather than the entire plant. This can reduce unnecessary data communication load. The agent can immediately decide and execute actions, such as adjusting the fuel injection amount, based solely on the collected local information. If No. 3 operates excessively to increase its own efficiency and overloads the downstream process (compression process), a game theory-based reward scheduler can detect this and reduce the reward for No. 3. Through this, each agent autonomously does their best within their own area, but as a result, can reach a Nash Equilibrium state that achieves the global goal of the entire factory: 'total profit maximization and zero accidents.'

[0595] According to another embodiment of the present disclosure, the system may be equipped with a redundant control structure including a Safety Supervisor to prevent accidents caused by the unpredictability of the autonomous agent. The system transfers control when a risk index exceeds a threshold. For example, if an autonomous scheduling agent issues an aggressive command to "raise reactor pressure to 99% of the design limit to maximize profits," a separate, independent Safety Supervisor monitors this, and if it is determined that the Risk Index exceeds the Safety Threshold, it can immediately interlock the agent's control signal. Then, the control mode can be automatically switched to Manual Mode, or a predefined Emergency Shutdown logic can be executed to ensure the safety of the plant as a top priority.

[0596] In addition, the explainable AI (XAI) module reports to the operator the reasoning behind why the agent made such a dangerous judgment (e.g., "It was safe for one hour even at 99% pressure in past data"), which can be used as training data to prevent future recurrence.

[0597] FIG. 55 is a drawing showing a Graphical User Interface (GUI) of a schedule providing system according to one embodiment of the present disclosure.

[0598] Referring to FIG. 55, the processor generates a first GUI for acquiring user input information and can acquire user input information based on the first GUI. The first GUI may include a version selection interface (5510) for selecting a simulation version, an AI model, etc., required for schedule generation simulation, and a scheduler execution preparation interface (5520) for receiving necessary information from the user to generate a schedule. In one embodiment, the Advanced Planning & Scheduling (APS) simulator may include a simulation simulator that predicts the yield of a production product by simulating a thermal cracking reaction based on key operating variables of a naphtha cracking furnace and the properties of raw naphtha. The APS simulator can adjust bias values ​​that correct the prediction error with the actual process on a quarterly basis. Such an APS simulator is managed as a new version at each update point of the bias values, and the user can use the desired version of the simulator through the version selection interface (5510). In one embodiment, as the APS simulator is periodically updated and the changes accumulate, the simulation results may differ from the previous version; therefore, there is a need to train and provide a new AI model accordingly. Therefore, the user can select the AI ​​model best suited for the simulator version through the version selection interface (5510).

[0599] The scheduler execution preparation interface (5520) is an area for inputting and setting specific data for schedule generation, and according to one embodiment, it may proceed through a three-step process of 'input file generation step', 'input file verification step', and 'option selection step'. In the input file generation step, the system can support the user in easily generating complex input files. When the user determines the time to retrieve initial tank information through 'baseline time selection' and uploads the target production files for ethylene and propylene (e.g., target_production.csv) and the decomposition furnace decoking schedule file (e.g., furnace_decoking_schedule.xlsx), the processor can automatically generate the entire input file by linking with an internal database to extract the remaining necessary data. If the user already possesses a completed input file, they can move immediately to the next step and upload the file directly through the 'skip' function. In the input file verification step, the system can set the scope of scheduling and verify the necessary input data.

[0600] In one embodiment, input information corresponding to a period longer than the target period (e.g., 2 weeks) of the schedule that the system intends to generate (e.g., 4 weeks) may be obtained. For example, to generate an optimal schedule for 2 weeks, the system may require receiving, operation, and production target data for the next 4 weeks. This is because relying solely on 2 weeks of information results in short-term optimization (myopic optimization), which fails to prepare for inventory shortages or process issues that may occur after 2 weeks, thereby hindering long-term efficiency. Therefore, by using information that includes the next 2 weeks as the basis for learning and judgment, the system can ensure the robustness of the schedule by enabling the AI ​​algorithm to make decisions at the present time while looking further into the future.

[0601] In one embodiment, the system may obtain a plurality of input files necessary for schedule generation. The input files may include naphtha receiving schedules (receipt_events.csv), blending operation schedules (blending_events.csv), operation and decoking schedules for each cracking furnace (MS, RE, SC), inventory and properties history for each naphtha storage tank (TNKINV.csv), daily target production history for each product (target_production.csv), and price history of raw materials / products (price_history.csv), etc.

[0602] Additionally, the processor can perform an inspection operation on uploaded input files to check for format errors, missing values, outliers, etc. The inspection results can be broadly classified into success, warning, or failure. If the input file passes the inspection without errors, the processor can visualize the contents of the input file at the bottom and display them, and allow progress to the next step. If the contents of the input file allow for schedule generation but exceed predefined safety regulations or recommended ranges (e.g., exceeding software limits), the processor can generate a warning message to alert the user while allowing progress to the next step. Conversely, if critical errors are identified that make schedule generation impossible, such as missing essential data, the processor can display an inspection failure message along with the specific cause of the failure and block entry to the next step. In this case, the user can perform a re-inspection by directly modifying the data or re-uploading the file via the web interface.

[0603] In one embodiment, after the inspection is completed, the processor can generate a schedule for a target period, generate a second GUI (5530) to provide the schedule for the target period, and control the display to output the second GUI (5530).

[0604] FIG. 56 is a drawing showing a scheduling range selection interface according to one embodiment of the present disclosure.

[0605] Referring to FIG. 56, the user may select at least one of the following as a scheduling range: an overall mode, a receiving-limited mode, a mixing-limited mode, or a disassembly-limited mode. The overall mode is a mode in which an AI agent determines all processes of raw material storage (receiving), mixing between tanks, and disassembly production. The receiving-limited mode is a mode in which the mixing and disassembly schedules follow a fixed schedule specified in an input file, but only the schedule for receiving control is optimized and determined by the AI ​​agent. The mixing-limited mode is a mode in which the receiving and disassembly schedules follow a fixed schedule specified in an input file, but only the schedule for mixing control is optimized and determined by the AI ​​agent. The disassembly-limited mode is a mode in which the storage and mixing schedules follow a fixed schedule specified in an input file, but only the schedule for disassembly control is optimized and determined by the AI ​​agent. According to one embodiment of the present disclosure, not only is it possible to generate a schedule using an AI agent at every stage, but a disassembly-limited scheduling mode may also be provided, which allows for the generation of an optimal schedule only at the receiving, mixing, or disassembly unit level even in abnormal operation situations.

[0606] FIG. 57 is a drawing showing an inspection success scenario according to one embodiment of the present disclosure.

[0607] Referring to Fig. 57, the processor can perform an inspection process to verify the validity of data based on an input file (user input information) obtained from the user and a set target period. Fig. 57 is a diagram showing the state where the inspection was successful, with all data identified normally as a result of the input file inspection. Referring to Fig. 57, the processor can inspect the format, date range, and presence of required data of multiple uploaded files (e.g., receipt_events.csv, TNKINV.csv, etc.). If it is identified that all items satisfy predefined criteria and schedule generation is possible, the processor can output a visual indication on the display such as 'Input file inspection completed', 'Inspection successful', or 'Pass'. Additionally, upon successful completion of the inspection, the processor can visualize and display the data contained in the input file. For example, naphtha inventory trends and receiving schedule graphs based on the input data may be displayed, allowing the user to intuitively verify the consistency of the basic data before AI scheduling is performed. At this point, the 'Next' button is activated, allowing the user to proceed to the option selection stage.

[0608] FIG. 58 is a drawing illustrating an inspection failure scenario according to one embodiment of the present disclosure.

[0609] FIG. 58 is a diagram showing a state where the inspection fails because an error is found that makes schedule generation impossible as a result of the input file inspection. The processor may identify that the schedule cannot be generated based on user input information and the target period. For example, this may apply if a required file is missing, the date format of the data is incorrect, or a logical contradiction between the data (e.g., the inventory quantity is entered as a negative number) is found. If it is identified that schedule generation is impossible, the processor may generate a fourth GUI (5810) to obtain additional information (modified file or corrected data) and control the display to output it. The fourth GUI (5810) includes the filename of the file where the error occurred and the specific cause of the failure, and may provide an interface that allows the user to delete the file, upload a new file, or directly modify the data. If the user re-uploads the modified input file (additional information) through the fourth GUI (5810), the processor may perform the inspection logic again to re-identify whether the schedule can be generated. If the error is resolved by the modified information, the screen switches to an inspection success state as shown in FIG. 57.

[0610] FIG. 59 is a diagram illustrating a method for providing an inspection warning message according to one embodiment of the present disclosure.

[0611] FIG. 59 is a diagram showing a state in which, as a result of inspecting the input file, an inspection warning message is displayed because an item requiring caution was found, even though schedule creation is possible. Based on user input information and a target period, the processor may identify that while schedule creation itself is possible, additional verification (need for additional information) is required because specific data falls outside the recommended range or is close to safety regulations. For example, this may apply when the water level of a storage tank does not exceed the physical hard limit but exceeds the soft limit recommended for operation, or when the concentration of a specific characteristic approaches a standard value. In this case, the processor generates a warning message (5910) to alert the user and controls the display to output it. The warning message (5910) may include specific warning content, such as "The inventory quantity of Tank 5401 has exceeded the recommended level." Unlike the inspection failure in FIG. 58, the schedule creation process is not interrupted in the situation of FIG. 59, and the user can choose to proceed to the next step after confirming the warning or modify the data as necessary.

[0612] FIG. 60 is a drawing illustrating a method for setting a target period according to one embodiment of the present disclosure.

[0613] Referring to FIG. 60, the processor can identify a target period for which a schedule is to be generated based on user input information. The method for identifying the target period can be implemented in various embodiments depending on the degree of user intervention and the level of automation. In one embodiment, the target period may be determined by explicit input from the user. The user can directly set both the start date and the period of the schedule through the first GUI, and the processor can identify the target period accordingly. In another embodiment, the period may be fixed or automatically set for the user's convenience. If the user sets only the schedule start date, the processor can calculate the target period by automatically applying a pre-set standard period (e.g., 2 weeks). This reflects the field situation where typical process operation plans are established in specific cycles (e.g., 2-week units). In yet another embodiment, the system can actively propose a period considering data availability or factory conditions. The processor presents a start date automatically recommended by the AI ​​to the user, and if the user approves it, it can identify a pre-set period (e.g., 2 weeks) from that start date as the target period. Furthermore, the processor may automatically recommend or determine both the optimal start date and the period through an AI algorithm. For example, AI can identify times when urgent maintenance is required or when there are significant fluctuations in raw material supply, and suggest an optimal scheduling period that includes such times.

[0614] FIG. 61 is a drawing showing a GUI for setting a soft upper limit of concentration by characteristic according to one embodiment of the present disclosure.

[0615] The user can set specific constraints to be considered when generating a schedule. FIG. 61 is a diagram illustrating a GUI for setting a soft upper limit of concentration by property according to one embodiment of the present disclosure. Referring to FIG. 61, the processor can obtain user input information including constraint information through the first GUI. In one embodiment, the constraint information may include property constraint information regarding the chemical characteristics of the raw material, information on whether it is recycled naphtha, information on whether it is a high-content oxygenate operation, etc. Such constraint information may include data essential for ensuring process safety and maintaining the quality of the produced product.

[0616] In one embodiment, the system can manage property constraint information by classifying it into hard reference information for concentration by property and soft reference information for concentration by property. Hard reference information consists of values ​​predefined by the physical limits of process equipment or absolute safety regulations, and is a mandatory constraint that must be observed when generating a schedule. If a generated schedule violates the hard reference, the schedule is considered invalid. On the other hand, soft reference information consists of recommended constraints that the user can flexibly set according to operating conditions. Through the GUI shown in FIG. 61, the user can select a specific property to be applied to the feed tank and set a soft upper limit value for that property.

[0617] In one embodiment, when generating a schedule, the AI ​​agent may perform scheduling to ensure that the corresponding characteristic value within the feed tank remains smaller than a soft upper limit set by the user. However, unlike hard criteria, this is not an absolute constraint; therefore, if a trade-off occurs with other important goals (e.g., achieving production volume, complying with essential constraints, etc.), the soft upper limit may be temporarily exceeded at the discretion of the AI. In other words, the soft criteria act as a penalty factor during the optimization process of the AI ​​model, thereby inducing learning and generation to adhere to them as much as possible.

[0618] FIG. 62 is a drawing illustrating a GUI showing concentration constraint information by characteristic according to one embodiment of the present disclosure.

[0619] Referring to FIG. 62, the processor can respond to a user's request (e.g., clicking the "Concentration Constraint Information by Specific Type" button) by outputting detailed reference information for the concentration of each specific type currently applied to the system in the form of a pop-up window or a separate layer. The GUI can visualize and provide Hard Limit and Soft Limit values ​​set for each of multiple types (e.g., Specific Gravity, Sulfur, Paraffin, etc.) in the form of a table or list.

[0620] In terms of the user interface, when a user selects a property constraint they wish to add, the processor displays information on the minimum and maximum input ranges for that property at the bottom of the input box to prevent user input errors. Additionally, if the user clicks the "Concentration Constraint Information by Property" button or searches the corresponding area, the processor displays both the hard upper limit currently set in the system and the soft upper limit set by the user, helping the user compare the two standards and identify the appropriate operating range.

[0621] The GUI of Fig. 62 displays the criteria for hard and soft constraints side by side, allowing the user to intuitively understand how much margin the currently set soft upper limit has compared to the hard upper limit (safety threshold). Typically, the soft upper limit is set to a more conservative (stricter) value than the hard upper limit. If the soft upper limit entered by the user exceeds the hard upper limit or is logically contradictory, the processor can display the value in red or provide a warning message to the user via a warning icon. Consequently, the interface of Fig. 62 supports the user in clearly distinguishing and controlling between "safety that must be observed" and "optimal operating points that are good to observe" in complex chemical process scheduling.

[0622] FIG. 63 is a drawing showing a GUI for selecting a scheduling reflection option according to one embodiment of the present disclosure.

[0623] Referring to FIG. 63, the processor can obtain a scheduling reflection option from the user through the first GUI. The scheduling reflection option may include an open inspection schedule reflection option and a continuity maintenance option.

[0624] In one embodiment, the open inspection schedule reflection option may be used when a specific tank needs to be emptied according to the maintenance cycle of the storage tank. Typically, naphtha storage tanks must be opened for a thorough inspection at regular intervals (e.g., every 10 years) in accordance with safety regulations. To do this, the inventory in the tank must be completely depleted and the tank excluded from process operations during the inspection period. When the user activates the open inspection schedule reflection option, the processor can identify the tank to be inspected, the emptying start date, the inspection end date (i.e., the open inspection period), the tank to receive the remaining amount, and the period for transferring the remaining amount.

[0625] A system according to one embodiment of the present disclosure may generate a schedule by dividing the inventory depletion process into two stages for efficient tank emptying. The first stage is a natural depletion or priority supply stage, in which the processor may generate a schedule to prioritize the input of the inventory in the tank into the process (transfer to a feed tank) from the start of emptying during the open inspection period until the inventory in the tank reaches a preset remaining amount (e.g., 1,000 tons). That is, the inventory is naturally reduced by using it as process raw material up to a level where the pump can normally draw it in. The second stage is a forced transfer of the remaining amount stage, in which, when the tank level drops to a preset remaining amount (e.g., 1,000 tons) or lower, causing a decrease in pump efficiency or making process input difficult, the processor may generate a schedule to forcibly transfer the remaining inventory to a receiving tank during a user-specified remaining amount transfer period. This enables the target tank to be completely emptied before the start of the inspection.

[0626] In one embodiment, the continuity maintenance option is a function designed to ensure the stability of plant operations. Due to the nature of the naphtha cracking process, which is a continuous process, sudden valve manipulation or rapid changes in operating conditions can cause safety accidents. Therefore, if the user selects the continuity maintenance option, the processor can control the process to ensure that the newly generated future schedule does not deviate significantly from the schedule currently being executed (or the schedule executed just prior to it). Specifically, the AI ​​agent recognizes the current operating status (tank level, flow rate, etc.) as the initial state and can generate a schedule for the target period with a smooth transition, minimizing the difference from the previous schedule by methods such as imposing penalties for sudden changes. Once the user completes these option settings and presses the schedule generation button, the processor inputs the finalized user input information, constraint information, and option information into the AI ​​model to derive the optimal schedule.

[0627] FIG. 64 is a drawing showing a second GUI for providing a generated schedule according to one embodiment of the present disclosure.

[0628] Referring to FIG. 64, when the computation through the AI ​​agent is completed, the processor intuitively provides the result to the user. The processor generates a second GUI (5530) to provide a schedule for a target period and controls the display to output it. The second GUI (5530) visualizes the schedules generated by the AI ​​and displays them in the form of graphs or charts, and may also display a summary of key indicators for each schedule (e.g., expected profit, production achievement rate, constraint satisfaction, etc.). In particular, the system according to one embodiment of the present disclosure does not provide only a single result, but can guarantee the user's right to choose by simultaneously providing multiple schedules generated by applying different optimization criteria (Objective Functions). This is because, in factory operations, profitability may be the top priority depending on the situation, or production achievement or operational stability may be more important.

[0629] Multiple schedules provided through the second GUI (5530) may include at least one of the following five types. First, there is the 'AI Recommendation' schedule. This is the optimal schedule based on reinforcement learning reward criteria, which is the result derived by the AI ​​model in a direction that maximizes the reward function defined during the learning process (combining profit, compliance with constraints, etc.). It is generally the schedule that demonstrates the most balanced performance. Second, there is the 'Smart Ethylene Optimization' schedule. This is the optimal schedule based on criteria that comprehensively consider target production volume (especially ethylene) and profit. For example, it may be the result of exploring operating conditions that can generate maximum profit within that range, while assuming the basic premise of achieving the ethylene production target. Third, there is the 'Profit Optimization' schedule. This is literally the optimal schedule based on profit. It refers to an aggressive operation plan that can maximize economic benefits even if it sacrifices some constraints or operational convenience to some extent. Fourth, there is the 'Ethylene Optimization' schedule. This is the optimal schedule based on target production volume, which is suitable for cases where market demand surges and the maximum ethylene production volume must be secured. Fifth, there is the 'Operational Stability Optimization' schedule. This is the optimal schedule based on operating conditions. It is a conservative schedule that prioritizes stable factory operations by avoiding abrupt changes in operation, minimizing equipment load, and securing a comfortable margin.

[0630] In one embodiment, the user can compare summary information of the five schedules on the second GUI (5530) and select the one that best fits the current factory's management strategy or field conditions. Based on the user selecting (clicking) one of these, the processor can obtain a schedule reflection option and confirm or load schedule data for a specific target period corresponding to the option.

[0631] Additionally, the second GUI (5530) may include a result download button. When the user clicks this button, detailed data (flow rate by time zone, temperature set value, tank level change, etc.) for the selected schedule (or the entire schedule) may be provided in the form of a document file such as Excel or CSV.

[0632] Furthermore, the processor can automatically save the generated result data to the 'schedule creation history' database instead of treating it as volatile. This allows users to later query past creation records or utilize the data to compare and analyze previously selected schedules with actual operational results.

[0633] FIG. 65 is a flowchart illustrating a method for providing a schedule according to one embodiment of the present disclosure.

[0634] Referring to FIG. 65, in operation 6510, the processor generates a first GUI for obtaining user input information, and in operation 6520, the processor can obtain user input information necessary for generating a schedule from the user based on the first GUI. In one embodiment, the user input information may include simulation information for setting the version and environment of the simulation, AI model information for selecting an algorithm to apply, process data to be a specific scheduling target, and constraint information for defining the limits of process operation.

[0635] In one embodiment, process data may include a raw material receiving schedule during a first period, a first process operation schedule during a first period (e.g., a naphtha receiving schedule for 4 weeks), a second process operation schedule during a first period (e.g., a mixing schedule for 4 weeks), a third process operation schedule during a first period (e.g., a cracking furnace operation schedule for 4 weeks), raw material inventory quantity per raw material storage tank, characteristics history per raw material storage tank, daily target production history for each product during a first period, daily price history of raw materials during a first period, daily price history of products during a first period, etc. Here, the first period may be a longer period than the target period for which a schedule is to be generated. For example, if the target period of the schedule that the user intends to generate is 2 weeks in the future, the processor may request and receive process data corresponding to 4 weeks in the future for this purpose.

[0636] The technical reason for receiving data for a period longer than the target period is to avoid the pitfalls of short-term optimization and secure long-term efficiency. If an AI model is provided with only data for the target period of two weeks, there is a risk that the AI ​​will generate a short-sighted schedule that does not consider operations beyond the third week, such as emptying tank inventory or operating equipment excessively on the last day of the second week, solely to maximize rewards (profit, etc.) within that two-week period. Consequently, this undermines process continuity and causes a sharp decline in operational efficiency from a long-term perspective. Therefore, a processor according to one embodiment of the present disclosure induces the AI ​​algorithm to make current decisions with a look-ahead perspective by identifying information in advance regarding an additional period (e.g., the next two weeks) beyond the target period (e.g., two weeks). Through this, a more robust and continuous schedule for the target period can be generated, which ensures stable inventory levels and equipment conditions even after the target period has ended.

[0637] In operation 6530, the processor can identify the target period.

[0638] In operation 6540, the processor can identify whether to generate a schedule based on user input information and a target period. In this specification, this process may be referred to as an input file inspection process or a validation process.

[0639] In one embodiment, the processor may identify that a schedule can be generated normally based on user input information. For example, the processor may identify that a schedule can be generated normally if the format of the input data is correct, all required data is present, and there are no logical contradictions between the data. In this case, the processor may perform operation 6550 to proceed the process to the next step.

[0640] In another embodiment, the processor may identify that a schedule cannot be generated based on user input information and a target period. Cases where a schedule cannot be generated may include cases where essential files are missing or fatal data errors are detected. In such cases, based on the identification that a schedule cannot be generated, the processor may generate a fourth GUI to obtain additional information and control a display to output the fourth GUI. Here, the fourth GUI may include an interface that guides the cause of the error and allows the user to re-upload files or modify data. Subsequently, the processor may obtain additional information (such as modified data) through user input based on the fourth GUI. Once additional information is obtained, the processor may return to operation 6540 to re-identify whether a schedule can be generated.

[0641] In another embodiment, the processor may identify a state where, based on user input information, schedule generation itself is possible but additional verification is required in relation to safety regulations or recommended ranges. Based on identifying the need for such additional information, the processor may generate a message (warning message) regarding the need for additional information. Subsequently, the processor may control the display to output the message regarding the need for additional information. In this case, if there is user confirmation input or if a certain amount of time has elapsed, the processor may proceed to operation 6550, just as in the case of successful inspection.

[0642] If, in operation 6540, the processor identifies that it has passed the verification process and can generate a schedule, the processor enters operation 6550 and can generate an optimal schedule for a target period set by the user. In one embodiment, the processor can generate a schedule for a target period based on user input information, such as an AI model selected by the user or a simulation selected by the user.

[0643] In one embodiment, the schedule for the target period may include a schedule for multiple interconnected processes rather than a single process. For example, the schedule for multiple processes may include a schedule for a first process, a schedule for a second process, a schedule for a third process, and so on. More specifically, for example, in the case of a naphtha cracking center (NCC), the schedule for multiple processes may include a schedule for a first process (e.g., receiving process) for receiving raw materials, a schedule for a second process (e.g., mixing process) for mixing stored raw materials, and a schedule for a third process (e.g., cracking process) for feeding the mixed raw materials into a cracking furnace to produce a product.

[0644] In one embodiment, at least one of the schedules for a plurality of processes may be determined by one or more AI agents performing reinforcement learning. That is, the scope of involvement of the AI ​​agent can be flexibly determined according to the user's selection or process operation strategy. For example, the processor may have the AI ​​agent determine the schedules for the first, second, and third processes. This corresponds to a full mode in which the AI ​​integrally optimizes the entire value chain from the receipt of raw materials to final cracking. In another embodiment, the processor may have the AI ​​agent determine the process schedules for only some of the plurality of processes, while the remaining process schedules follow a fixed plan pre-entered by the user (expert) or separate rule-based logic. For example, when the processes of a naphtha cracking center include a receiving process, a mixing process, and a cracking process, one can assume a case where the user selects a mode limited to cracking. In this case, the schedules for the receiving process and the mixing process will follow the schedules pre-determined by the expert as described in the input file. On the other hand, only the schedule for the disassembly process (such as disassembly furnace operating conditions), which has the greatest impact on final production efficiency, can be actively determined by the AI ​​agent. This flexible configuration offers the advantage of being able to respond to both situations: whether one wishes to optimize only the disassembly process without altering the already established receiving / mixing plan based on on-site conditions, or conversely, to pursue extreme efficiency by re-examining the entire process from receiving to disassembly from scratch.

[0645] In one embodiment, an AI agent performing reinforcement learning may correspond to the agent described above in the description of at least one of FIGS. 1 to 54. For example, multiple AI agents performing reinforcement learning to generate a schedule for a target period may operate in an asynchronous multi-agent manner. That is, each agent interacts individually with the simulation environment to observe the state and perform actions, but can achieve optimization of the entire system by sharing the resulting experiences or asynchronously updating learning parameters. Although multiple AI agents perform different roles, they may be trained using a single i...

Claims

1. As a system, At least one processor; It includes at least one memory that stores instructions that cause the system to perform operations when executed by the above-mentioned at least one processor, and The above operations are, The operation of creating a first GUI to obtain user input information; An operation to acquire user input information based on the above-mentioned first GUI; Action to identify the target period; An operation to identify whether to generate a schedule based on the above user input information and the above target period; If it is identified that the above schedule can be generated, the action of generating a schedule for the above target period; and It includes an operation to control the display to output a second GUI for providing a schedule during the above-mentioned target period, and The schedule for the above target period is, It includes a first schedule regarding the operation of the receiving tank during the above target period, a second schedule regarding the operation of the mixing tank during the above target period, and a third schedule regarding the operation of the disassembly furnace during the above target period, and At least one of the first schedule, the second schedule, and the third schedule is, A schedule providing system determined by one or more AI agents performing reinforcement learning.

2. In paragraph 1, the above operations are, The operation of performing a simulation of the schedule during the above target period; A schedule providing system comprising the operation of generating a third GUI to provide evaluation results for the simulation along with a schedule for the above target period.

3. In paragraph 1, the above operations are, An action of identifying that the schedule cannot be generated based on the above user input information and the above target period; Based on the identification that the above schedule cannot be generated, the operation of generating a fourth GUI to obtain additional information; operation of controlling the display to output the above-mentioned fourth GUI Based on the above-mentioned fourth GUI, the operation of obtaining the above-mentioned additional information; and A system comprising an operation to identify whether to generate a schedule based on the above user input information and the above additional information.

4. In Paragraph 1, Based on the above user input information and the above target period, an action that identifies the need for additional information even though schedule generation is possible; An operation to generate a message regarding the necessity of the additional information based on identifying the necessity of the additional information; A system comprising an operation to control the display to output a message regarding the need for the additional information mentioned above.

5. In paragraph 1, the user input information is, A system comprising at least one of simulation information, AI model information, process data, and constraint information.

6. In paragraph 5, the above process data is, A system comprising at least one of a raw material receiving schedule during a first period, a first process operation schedule during the first period, a second process operation schedule during the first period, a third process operation schedule during the first period, a raw material inventory quantity per raw material storage tank, a history of characteristics per raw material storage tank, a history of daily target production volume for each product during the first period, a history of daily prices of raw materials during the first period, and a history of daily prices of products during the first period.

7. In Paragraph 6, the above-mentioned first period is, A system characterized by a period longer than the above-mentioned target period.

8. In paragraph 1, one or more AI agents performing the reinforcement learning are, It is characterized by being learned using a single reward, The above-mentioned compensation is, A system determined based on profit, whether mandatory constraints are satisfied, and whether preferred constraints are satisfied.

9. In Paragraph 1, the schedule during the above target period is, It includes at least one of the first schedule determined by the first AI agent, the second schedule determined by the second AI agent, and the third schedule determined by the third AI agent, and The above-mentioned first AI agent, the above-mentioned second AI agent, and the third AI agent are, A system characterized by being an asynchronous multi-agent.

10. In Paragraph 9, The above-mentioned first AI agent is, It includes multiple first AI agents provided for each type of raw material input, and The above-mentioned third AI agent is, A system comprising a plurality of third AI agents provided for each type of decomposition.

11. In paragraph 9, the first AI agent is, Determine the receiving tank to store the incoming raw materials, and The above-mentioned second AI agent is, Determine one or more receiving tanks to be transferred to the mixing tank, determine the number of pumps to be connected to the determined receiving tanks, and The above-mentioned third AI agent is, A system that determines the feed rate of raw materials supplied to each decomposition furnace, the coil outlet temperature, and the dilution steam ratio.

12. In paragraph 1, the above operations are, It includes an operation to control a display to provide multiple schedules for the target period through the second GUI, and Multiple schedules during the above target period are, A system comprising at least two of the following schedules: an optimal schedule based on reinforcement learning reward criteria, an optimal schedule based on criteria that comprehensively consider target production volume and profit, an optimal schedule based on profit criteria, an optimal schedule based on target production volume criteria, and an optimal schedule based on operating conditions criteria.

13. In paragraph 1, the above operations are, An action of acquiring a schedule reflection option based on the user selecting one of the following: the optimal schedule based on reinforcement learning reward criteria, the optimal schedule based on criteria comprehensively considering target production volume and profit, the optimal schedule based on profit criteria, the optimal schedule based on target production volume criteria, or the optimal schedule based on operating conditions criteria; A system comprising the operation of generating a schedule for a target period corresponding to the above-mentioned schedule reflection option.

14. In paragraph 1, the user input information is, Includes process data and pharmaceutical information, The above process data is, It includes at least one of an ethylene target production volume, a propylene target production volume, and a cracking furnace decoking schedule, The above constraint information is, A system comprising at least one of the following: information on properties, information on whether recycled naphtha applies, and information on whether high-content oxygenation operation applies.

15. In Paragraph 14, the above-mentioned characteristic restriction information is, A system including hard reference information on concentration by characteristic and soft reference information on concentration by characteristic.

16. In paragraph 1, the user input information includes an option to reflect the open inspection schedule and an option to maintain continuity, and The above operations are, An operation to identify the open inspection period, the tank to be inspected, the tank to be transferred, and the remaining amount transfer period based on the above open inspection schedule reflection option; The operation of generating a schedule during the target period to transfer the inventory of the tank to be inspected to a receiving tank from the start of emptying during the opening inspection period until the inventory of the tank to be inspected reaches a preset remaining amount; The operation of generating a schedule during the target period to transfer the inventory of the inspection target tank corresponding to the above-set remaining amount to the tank to be transferred; and A system comprising an operation to generate a schedule for the target period such that the difference from the currently executing schedule is minimized based on the above continuity maintenance option.

17. Regarding the method of providing the schedule, A step of generating a first GUI for obtaining user input information; A step of obtaining user input information based on the above-mentioned first GUI; Step to identify the target period; A step of identifying whether to generate a schedule based on the above user input information and the above target period; If it is identified that the above schedule can be generated, the step of generating a schedule for the above target period; and It includes the step of controlling a display to output a second GUI for providing a schedule during the above-mentioned target period, and The schedule for the above target period is, It includes a first schedule regarding the operation of the receiving tank during the above target period, a second schedule regarding the operation of the mixing tank during the above target period, and a third schedule regarding the operation of the disassembly furnace during the above target period, and At least one of the first schedule, the second schedule, and the third schedule is, A method determined by one or more AI agents performing reinforcement learning.

18. A computer-readable recording medium having a program recorded thereon for performing the method of paragraph 17 on a computer.

Images