System and method for determining process scheduling information by using multi-agent

Abstract

One embodiment of the present disclosure relates to a method performed by a computer, the method comprising the operations of: generating, on the basis of an initial state of a process, a pivot schedule including an initial macro operation sequence; duplicating the pivot schedule so as to generate a plurality of branch schedules; using an artificial intelligence agent so as to reflect different operation scenarios or probabilistic behavior policies for the respective branch schedules and sequentially add macro operations, thereby expanding the schedule; evaluating fitness for the respective expanded branch schedules when a predefined synchronization point is reached; and, on the basis of the evaluation result, selecting at least one from among the plurality of branch schedules so as to repeatedly perform a process of updating the pivot schedule, thereby determining final scheduling information.

Description

System and method for determining process scheduling information using multi-agents

[0001] The present disclosure relates to a process scheduling system, and more specifically, to a system and method in which an artificial intelligence-based multi-agent asynchronously generates a schedule in a complex continuous process consisting of multiple stages.

[0002] Large-scale chemical processes, such as naphtha cracking centers (NCCs), and semiconductor manufacturing processes are representative multi-stage continuous processes involving numerous facilities and complex constraints. Scheduling these processes requires that each stage, from raw material intake to blending, cracking, and purification, be organically connected, and various variables such as fluctuations in market conditions, equipment failures, and safety regulations must be considered in real time.

[0003] Conventional process scheduling has been performed by relying on the experience of skilled engineers or using mathematical optimization tools such as linear programming. However, methods relying on engineer experience have the disadvantage of significant individual variability and slow response times to unexpected situations. Furthermore, mathematical optimization methods have limitations in deriving optimal solutions in real time due to the curse of dimensionality problem, where the amount of computation increases exponentially as the scale of the process grows.

[0004] Recently, research on scheduling using reinforcement learning has been attempted; however, single-agent methods face the problem of difficulty in learning vast state spaces, while simple multi-agent methods suffer from unstable convergence due to communication overhead between agents and goal mismatch issues. In particular, achieving overall optimization by synchronizing processes with different time scales, such as receiving, mixing, and disassembly, remains a challenging problem.

[0005] Therefore, there is a need for new AI-based scheduling technology that can effectively handle the complexity of large-scale processes while flexibly and rapidly deriving optimal schedules by reflecting the characteristics of each process stage.

[0006] One embodiment of the present disclosure aims to provide a system and method in which an artificial intelligence-based multi-agent asynchronously generates a schedule in a complex continuous process consisting of a plurality of steps.

[0007] A system executed by a computer according to one embodiment of the present disclosure comprises at least one processor; and at least one memory that stores instructions that cause the system to perform operations when executed by the at least one processor, wherein the operations may include: an operation of generating a pivot schedule including an initial macro operation sequence based on an initial state of a process; an operation of generating a plurality of branch schedules by replicating the pivot schedule; an operation of expanding the schedules by sequentially adding macro operations to each of the plurality of branch schedules using an artificial intelligence agent, reflecting different operating scenarios or probabilistic behavior policies; an operation of evaluating the suitability of each of the expanded plurality of branch schedules when a predefined synchronization point is reached; and an operation of determining final scheduling information by repeatedly performing a process of selecting at least one of the plurality of branch schedules and updating the pivot schedule based on the evaluation result.

[0008] In one embodiment, the process is a product production process of a naphtha cracking center, and the artificial intelligence agent includes at least one of: a first agent determining a schedule for unloading naphtha from a ship or external supply chain and storing it in a receiving tank; a second agent determining a ratio and schedule for transferring the naphtha stored in the receiving tank to a mixing tank and blending it; and a third agent (cracking agent) determining operating conditions for feeding the raw material from the mixing tank into a cracking furnace and pyrolyzing it, and the operations may include an operation to update the pivot schedule by integrating macro operations asynchronously generated by at least one of the first agent, the second agent, and the third agent.

[0009] In one embodiment, the operations may include: collecting observation information and operation history of at least one of the first agent, the second agent, and the third agent, respectively, and storing them in an agent-centered history buffer; applying a time-point embedding to the data in the history buffer using a transformer-based encoding network to convert it into a high-dimensional vector; inputting the converted high-dimensional vector into a centralized critique network to estimate a value function reflecting the interaction between agents; and learning a policy of at least one of the first agent, the second agent, and the third agent based on the value function.

[0010] In one embodiment, at least one of the first agent, the second agent, and the third agent includes a Mixture of Experts (MoE) model for performing specialized inference according to the properties or operating mode of the input raw material, and the MoE model may include a gating network that analyzes the characteristics of the input raw material data and selects a suitable expert among a plurality of expert sub-networks; and a decoding layer that integrates the computational results of the selected expert sub-networks to predict the yield or coking speed to the decomposition furnace.

[0011] In one embodiment, the at least one processor includes a Retrieval-Augmented Generation (RAG) module, and the RAG module may include a vector database that vectorizes and stores at least one of standard operating procedures related to the process, past accident case reports, and external market news data; and a search agent that, when a constraint violation or anomaly is detected during the process of determining the scheduling information, searches the vector database to extract a corresponding solution guide or constraint update information and provides it as an input context for the agent.

[0012] In one embodiment, the at least one processor includes a non-criteria module that verifies physical feasibility and safety compliance with respect to a schedule draft generated by the artificial intelligence agent, and the operations may include an operation to perform a self-reflection loop that controls the artificial intelligence agent to modify the schedule draft by providing feedback when the non-criteria module identifies a logical error or risk factor in the schedule draft.

[0013] In one embodiment, the artificial intelligence agent may determine the necessity of an action through an inference step before determining a macro action in the current state, call a process simulator as a tool to simulate the expected result of the action, and then execute a final action based on the result.

[0014] In one embodiment, the at least one processor further includes an action module that converts the final scheduling information into a physical control signal or an administrative processing command and executes it when the final scheduling information is determined, and the action module may include at least one of a Large Action Model (LAM) that recognizes the control screen of legacy equipment that does not support APIs and generates mouse or keyboard input to control the equipment; and a Robotic Process Automation (RPA) system that inputs the determined scheduling information into an Enterprise Resource Planning (ERP) system or sends an email to an external partner.

[0015] In one embodiment, the first agent supports an Agent-to-Agent (A2A) protocol for communicating with an agent of an external raw material supplier, and can optimize the receiving schedule by exchanging negotiation messages regarding the timing of receiving and price conditions with the agent of the external raw material supplier while concealing accurate inventory data of the receiving tank.

[0016] In one embodiment, the operations may include: intra-group recovery, which replaces a specific branch schedule with another valid branch schedule within the same group when a failure occurs due to a constraint violation in that specific branch schedule; and inter-group recovery, which replaces a specific branch schedule by replicating a valid branch schedule from another group having a higher constraint level when all branch schedules within the same group fail.

[0017] In one embodiment, the system further includes an interface for receiving a fixed schedule for a process step handled by some of the agents among the first agent, the second agent, or the third agent from a user, and the operations may include, when the fixed schedule is received, the remaining agents recognizing the fixed schedule as an unchangeable environmental constraint and determining scheduling information for the remaining process steps based thereon.

[0018] In one embodiment, the system is,

[0019] The system further includes a small language model that operates independently and is mounted on the equipment or edge device of the above process; and a safety monitoring module that monitors in real time whether a schedule generated by the artificial intelligence agent exceeds a risk threshold, wherein the safety monitoring module can block control of the artificial intelligence agent and transfer control to the small language model or a predefined emergency logic when an exceedance of the risk threshold is detected.

[0020] A method performed by a computer according to one embodiment of the present disclosure may include: generating a pivot schedule including an initial macro operation sequence based on an initial state of a process; generating a plurality of branch schedules by replicating the pivot schedule; expanding the schedules by sequentially adding macro operations to each of the plurality of branch schedules using an artificial intelligence agent, reflecting different operating scenarios or probabilistic behavior policies; evaluating the suitability of each of the expanded plurality of branch schedules when a predefined synchronization point is reached; and determining final scheduling information by repeatedly performing a process of selecting at least one of the plurality of branch schedules and updating the pivot schedule based on the evaluation result.

[0021] One embodiment of the present disclosure includes a program stored on a recording medium to execute a method according to one embodiment of the present disclosure on a computer.

[0022] One embodiment of the present disclosure includes a computer-readable recording medium having a program for executing a method according to one embodiment of the present disclosure on a computer.

[0023] One embodiment of the present disclosure includes a computer-readable recording medium that records a database used in one embodiment of the present disclosure.

[0024] According to the present disclosure, by focusing on exploring only promising scenarios centered on pivot schedules, it is possible to rapidly generate schedules of excellent quality while drastically reducing computational resources compared to the conventional exhaustive search method.

[0025] In addition, since the present disclosure enables multi-agents specialized for each process step to operate asynchronously but are coordinated through centralized evaluation, it is possible to achieve global optimization that maximizes the overall profit of the factory without falling into partial optimization.

[0026] In addition, the present disclosure can provide a highly reliable schedule that is immediately applicable in the field by incorporating unstructured data, such as safety regulations, past accident cases, and market news, into the scheduling process, going beyond simple numerical calculations through Agentic RAG and self-reflection functions.

[0027] Furthermore, the present disclosure enables autonomous control even in aging facilities without APIs or edge environments with unstable networks by applying LAM and SLM technologies, and allows operators to safely operate the system by trusting the AI's judgment through explainable AI (XAI) and a safe redundancy structure.

[0028] FIG. 1 is a diagram illustrating training buffers in MacDec-POMDP according to one embodiment of the present disclosure.

[0029] FIG. 2 is a diagram showing a centralized critique using joint history encoders according to one embodiment of the present disclosure.

[0030] FIG. 3 is a diagram illustrating an Agent-Oriented Centralized Critic (AOCC) method according to one embodiment of the present disclosure.

[0031] FIG. 4 is a drawing showing a reinforcement learning system according to one embodiment of the present disclosure.

[0032] FIG. 5 is a diagram showing an agent-centric encoder of an agent-centric centralized critic according to one embodiment of the present disclosure.

[0033] FIG. 6 is a diagram illustrating a method for calculating a value function by integrating agent-centered history according to one embodiment of the present disclosure.

[0034] FIGS. 7a to 7c are drawings illustrating an Agent-Centric Actor-Critic (ACAC) method according to one embodiment of the present disclosure.

[0035] FIG. 8 is a diagram illustrating an Agent-Centric Actor-Critic (ACAC) method according to one embodiment of the present disclosure.

[0036] FIG. 9 is a schematic diagram of a system according to one embodiment of the present disclosure.

[0037] FIG. 10 is a block diagram illustrating the internal structure of an agent-centered history encoding module according to one embodiment of the present disclosure.

[0038] FIG. 11 is a block diagram illustrating the internal structure of a centralized critique module according to one embodiment of the present disclosure.

[0039] FIG. 12 is a schematic diagram of a product production process of a naphtha cracking center according to one embodiment of the present disclosure.

[0040] FIG. 13 is a diagram illustrating a reinforcement learning method using multiple agents according to one embodiment of the present disclosure.

[0041] FIGS. 14a and FIGS. 14b are drawings illustrating a reinforcement learning method using multiple agents according to one embodiment of the present disclosure.

[0042] FIG. 15 is a drawing showing the schedule of a naphtha cracking center according to one embodiment of the present disclosure.

[0043] FIG. 16 is a diagram illustrating a method for determining information about a process consisting of a plurality of steps according to one embodiment of the present disclosure.

[0044] FIG. 17 is a flowchart illustrating a method for determining information about a process consisting of a plurality of steps according to one embodiment of the present disclosure.

[0045] FIG. 18 is a block diagram of a system for determining information about a process consisting of a plurality of steps according to one embodiment of the present disclosure.

[0046] FIG. 19 is a drawing showing a sequential multi-agent according to one embodiment of the present disclosure.

[0047] FIG. 20 is a drawing showing a supervisory agent according to one embodiment of the present disclosure.

[0048] FIG. 21 is a drawing showing a hierarchical agent system according to one embodiment of the present disclosure.

[0049] FIG. 22 is a drawing showing a discussion-type multi-agent system according to one embodiment of the present disclosure.

[0050] FIG. 23 is a drawing showing a mixed agent system according to one embodiment of the present disclosure.

[0051] FIG. 24 is a drawing showing a ReAct agent system according to one embodiment of the present disclosure.

[0052] FIG. 25 is a drawing showing a CodeAct agent system according to one embodiment of the present disclosure.

[0053] FIG. 26 is a drawing showing a modern tool-using agent system according to one embodiment of the present disclosure.

[0054] FIG. 27 is a drawing showing a self-reflective agent system according to one embodiment of the present disclosure.

[0055] FIG. 28 is a drawing illustrating a multi-agent workflow system according to one embodiment of the present disclosure.

[0056] FIG. 29 is a drawing showing an Agentic RAG (Retrieval-Augmented Generation) system according to one embodiment of the present disclosure.

[0057] FIG. 30 is a drawing showing a Multi-Agent Debate (MAD) system according to one embodiment of the present disclosure.

[0058] FIG. 31 is a diagram showing an A2A (Agent2Agent) protocol system according to one embodiment of the present disclosure.

[0059] FIG. 32 is a drawing showing an Agentic RAG (Retrieval-Augmented Generation) system according to one embodiment of the present disclosure.

[0060] FIG. 33 is a schematic diagram of an AI agent system according to one embodiment of the present disclosure.

[0061] FIG. 34 is a schematic diagram of a Large Language Model (LM) chatbot system according to one embodiment of the present disclosure.

[0062] FIG. 35 is a schematic diagram of a Robotic Process Automation (RPA) system according to one embodiment of the present disclosure.

[0063] FIG. 36 is a schematic diagram of a RAG (Retrieval-Augmented Generation) system according to one embodiment of the present disclosure.

[0064] FIG. 37 is a schematic diagram of a Learning-Augmented Mechanism (LAM) system according to one embodiment of the present disclosure.

[0065] FIG. 38 is a diagram showing the memory structure of an AI agent according to one embodiment of the present disclosure and a question-and-answer process utilizing the same.

[0066] FIG. 39 is a diagram illustrating a text generation process of a first language model according to one embodiment of the present disclosure.

[0067] FIG. 40 is a diagram illustrating the operation process of a second language model according to one embodiment of the present disclosure.

[0068] FIG. 41 is a diagram illustrating a method of operation of a third language model according to one embodiment of the present disclosure.

[0069] FIG. 42 is a diagram illustrating a method of operation of a fourth language model according to one embodiment of the present disclosure.

[0070] FIG. 43 is a diagram illustrating a method of operation of a fifth language model according to one embodiment of the present disclosure.

[0071] FIG. 44 is a diagram illustrating the operation method of a Large Action Model (LAM) according to one embodiment of the present disclosure.

[0072] FIG. 45 is a diagram illustrating the operation method of a Hierarchical Reasoning Model (HRM) according to one embodiment of the present disclosure.

[0073] FIG. 46 is a drawing showing a ToolFormer (Tools-trained Model) according to one embodiment of the present disclosure.

[0074] FIGS. 47 to 52 are drawings illustrating vulnerabilities of an MCP according to one embodiment of the present disclosure.

[0075] FIG. 53 is a diagram illustrating a context engineering structure in an AI agent system according to one embodiment of the present disclosure.

[0076] FIG. 54 is a block diagram of a system according to one embodiment of the present disclosure.

[0077] To clarify the technical concept of the present disclosure, embodiments of the present disclosure will be described in detail with reference to the attached drawings. In describing the present disclosure, detailed descriptions of related known functions or components will be omitted if it is determined that such detailed descriptions would unnecessarily obscure the essence of the present disclosure. Components having substantially the same functional configuration among the drawings have been assigned the same reference numerals and symbols as much as possible, even if they are shown in different drawings. For convenience of explanation, devices and methods will be described together where necessary. Each operation of the present disclosure does not necessarily have to be performed in the order described and may be performed in parallel, selectively, or individually.

[0078] The terms used in the embodiments of this disclosure have been selected to be as widely used and general as possible, taking into account the functions of this disclosure; however, these terms may vary depending on the intent of those skilled in the art, case law, the emergence of new technologies, etc. Additionally, in specific cases, terms have been selected at the applicant's discretion, and in such cases, their meanings will be described in detail in the description of the relevant embodiments. Therefore, terms used in this specification should be defined not merely by their names, but based on their meanings and the overall content of this disclosure.

[0079] Throughout this disclosure, singular expressions may include plural expressions unless the context clearly indicates otherwise. Terms such as “comprising” or “having” are intended to specify the presence of features, numbers, steps, actions, components, parts, or combinations thereof, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof. That is, throughout this disclosure, when a part is described as “comprising” a certain component, it means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0080] Expressions such as "at least one" modify the entire list of components and do not modify the components of the list individually. For example, "at least one of A, B, and C" and "at least one of A, B, or C" refer to only A, only B, only C, both A and B, both B and C, both A and C, all of A, B, and C, or any combination thereof.

[0081] Additionally, terms such as “...part,” “...module,” etc., as described in this disclosure refer to a unit that processes at least one function or operation, and may be implemented in hardware or software, or a combination of hardware and software.

[0082] Throughout the entire disclosure, when a part is described as being “connected” to another part, this includes not only cases where they are “directly connected” but also cases where they are “electrically connected” with other elements interposed between them. Furthermore, when a part is described as “comprising” a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0083] As used throughout this disclosure, the expression “configured to” may be replaced, depending on the context, with, for example, “suitable for,” “having the capacity to,” “designed to,” “adapted to,” “made to,” or “capable of.” The term “configured to” may not necessarily mean only “specifically designed to” in hardware. Instead, in some situations, the expression “system configured to” may mean that the system is “capable of” together with other devices or components. For example, the phrase “a processor configured (or set) to perform A, B, and C” may mean a dedicated processor for performing said operations (e.g., an embedded processor), or a generic-purpose processor (e.g., a CPU or an application processor) capable of performing said operations by executing one or more software programs stored in memory.

[0084] In one embodiment, an artificial intelligence (AI) agent refers to autonomous and intelligent software or system designed to perform specific tasks, and may mean a system equipped with the capability to make decisions and perform actions based on user instructions or changes in the environment. An AI agent is composed of five core elements: perception, knowledge base, reasoning and planning, action and actuation, and learning; each element interacts to enable the agent to perform given goals and continuously improve its performance. Perception may refer to the AI ​​agent detecting input data and interpreting it to recognize the environment. In the perception stage, sensors or external data

[0085] AI agents are computer system implementations of human decision-making and behavioral processes, and they have established themselves as a core component of current AI systems. The basic structure of an AI agent follows a cyclical pattern of 'Perception → Knowledge Base → Reasoning & Planning → Action & Actuation → Learning Loop.' Through this structure, AI agents perceive their environment, make decisions based on collected information, execute the results, and learn from those results to continuously improve.

[0086] AI agents can process various input data, make decisions based on it, and generate appropriate outputs or execute actions as a result. Furthermore, AI agents can possess the characteristic of continuously learning and improving by incorporating real-time feedback to achieve given goals. As part of a multi-agent system, AI agents can divide and optimize complex tasks through collaborative work and self-improvement.

[0087] [Table 1]

[0088]

[0089] In one embodiment, most of the structure may consist of a hierarchical structure of "Input → Plan / Evaluate / Branch → Output". Additionally, due to the agentic nature, complex problem solving may be possible through patterns such as iteration, parallelism, and collaboration rather than a single LLM call. Furthermore, since each structure is designed to suit a specific business purpose, selecting the optimal structure depending on the purpose may be important. For example, a workflow such as [Table 2] may be recommended depending on the business purpose.

[0090] [Table 2]

[0091]

[0092] Throughout this disclosure, reward shaping refers to a method of inducing learning by providing additional rewards when an agent performs a specific action or reaches a specific state. This may include manual reward shaping, where a person directly designs a reward function and grants additional rewards to the agent, and automatic reward shaping, where rewards are automatically formed according to specific rules or algorithms. A reinforcement learning system according to one embodiment is a learning method performed by an agent acquiring observations characterizing the current state of the environment and using those observations to select actions to be performed by the agent. In response to the agent performing a selected action, the environment transitions to a new state and the reinforcement learning system receives a reward. While interacting with the environment, the reinforcement learning system may attempt to maximize the long-term reward received by the agent in response to actions performed by the agent. A Macro-Action Decentralized Partially Observable Markov Decision Process (MacDec-POMDP) ​​may integrate an option framework into the Dec-POMDP by defining a set of macro-actions for each agent. In one embodiment, MacDec-POMDP is a tuple It can be expressed as. Here, is a set of identifiers for agents, S is a state space, and is a combined primitive behavior space, and is a combined macro behavior space, and is a combined primitive observation space, and is a combined macro-observation space, and is the probability of a state change, R is the reward shared by all agents, and is a combined observability, may be a combined macro-observability. Throughout this disclosure □ i represents an element or space of agent i, and is for all agents □ i It can imply the combination or multiplication of. Each macro action has a termination condition , initial set , and lower-level policies A tuple consisting of It could be. Here, (or ) is the observation history space of primitive (or macro) actions. In one embodiment, the goal of the MacDec-POMDP is a joint higher-level policy that maximizes the expected return from an initial state s0 for given lower-level policies (e.g., macro actions). It may be to find. For example, the joint upper-level policy can be determined by the following [Equation 1].

[0093] [Mathematical Formula 1]

[0094]

[0095] FIG. 1 is a diagram illustrating training buffers in MacDec-POMDP according to one embodiment of the present disclosure.

[0096] Referring to FIG. 1, the training buffers of Mac-CERTs (Macro-Action Concurrent Experience Replay Trajectories) (110) and Mac-JERTs (Macro-Action Joint Experience Replay Trajectories) (120) are illustrated. In FIG. 1, z i wa m i can mean the macro observation and macro action of agent i, respectively.

[0097] Referring to the example in FIG. 1, multiple agents may include Agent 1, Agent 2, and Agent 3. Agent 1 may act at time steps 4 and 6, Agent 2 at time steps 1 and 6, and Agent 3 at time step 5, so that a state transition may occur. In one embodiment, Mac-CERTs (110) may create a training buffer by collecting time steps where a state transition occurred for each agent. Accordingly, as in FIG. 1, a buffer containing information at time points t=0, 4, and 6 may be formed for Agent 1, a buffer containing information at time points t=0, 1, and 6 may be formed for Agent 2, and a buffer containing information at time points t=0 and 5 may be formed for Agent 3. Alternatively, Mac-JERTs (120) may create a training buffer by collecting time steps where a state transition occurred for any one of the agents. Accordingly, a buffer containing information for time points t=0, 1, 4, 5, and 6 can be formed. This will be explained in more detail below.

[0098] In Mac-CERTs according to one embodiment, the state transition experience of each agent i is a tuple It can be expressed as. Here, r i is the timestep t i Starting from macro behavior m that persists during a time step i It can mean accumulated rewards for. Accordingly, r i It can be defined as [Equation 2] below.

[0099] [Mathematical Formula 2]

[0100]

[0101] In Mac-JERTs according to one embodiment, the state transition experience is a tuple It can be expressed as. Here, joint macro action With the accumulated rewards, It is possible. Unlike Mac-CERTs, in Mac-JERTs, is the time step when an agent performs the corresponding macro action, and may refer to the end time step when the agent finishes the macro action. In the example of Fig. 1, Agent 2 starts the action at time step 1 and performs a joint macro action This is generated, and the next macro action is initiated by Agent 1 at timestep 4, and the next joint macro action is It could be.

[0102] Multi-Agent Reinforcement Learning is being actively developed and applied in various fields. Using Multi-Agent Reinforcement Learning, distributed agents can effectively collaborate to complete assigned tasks. Currently used Multi-Agent Reinforcement Learning adopts a method where all agents decide on their respective actions simultaneously due to identical durations between actions. However, in real-world scenarios, agents often decide and execute actions at different times, resulting in asynchronous decision-making and execution between agents. One embodiment of the present disclosure aims to provide a framework for modeling multi-agent decision-making where the decision-making and execution of actions between agents occur asynchronously over time using MacDec-POMDP. In one embodiment, in MacDec-POMDP, the start time of each agent's action and the duration of the action may differ.

[0103] FIG. 2 is a diagram showing a centralized critique using joint history encoders according to one embodiment of the present disclosure.

[0104] Referring to FIG. 2, a common history encoder (210) focused on the common history of all agents along the time axis is illustrated. In one embodiment, the training data (220) input to the common history encoder (210) is common macro observation data, which can be generated by collecting observations of time steps in which a state transition occurred for any one agent. For example, When represents the k-th macro observation of agent i, as agent 1 undergoes a state transition at time steps t=4 and 6, agent 2 undergoes a state transition at time steps t=2 and 6, and agent 3 undergoes a state transition at time step t=5, training data (220) can be generated to include time steps t=1, 4, 5, and 6 where a state transition occurs in any one of agents 1, 2, and 3. Accordingly, the training data (220) includes the observation of agent 1 at t=0 , Agent 2's observation , Agent 3's observation It may include, and since Agent 1 has no state transition at t=1, the same observation as at t=0. , Agent 2's observation , Agent 3 also has no state transition, so the same observation as at t=0. It may include, and at t=4, Agent 1's observation , Agent 2 has no state transition, so the same observation as at t=1 , Agent 3 also has no state transition, so the same observation as at t=0 or 1. It may include, and since Agent 1 has no state transition at t=5, the same observation as at t=4. , Agent 2 has no state transition, so the same observation as at t=1, 4. , Agent 3's observation It may include, and at t=6, the observation at Agent 1 , Agent 2's observation , Agent 3 has no state transition, so the same observation as at t=5 It may include. This training data (220) can be input into a joint history encoder (210).

[0105] In one embodiment, the joint history encoder (210) may include a first Multi-Layer Perceptron (MLP) (230) and a Gated Recurrent Unit (GRU) (240). The first MLP (230) may include an input layer, one or more hidden layers, and an output layer. The first MLP (230) uses training data (220) The output information is determined by taking the input, and the output information of the first MLP (230) can be input to the GRU (240). The GRU (240) is a model used to process sequential data and can be operated using a reset gate and an update gate. The reset gate determines how much of the previous state should be forgotten, and the update gate determines how much of the previous state information should be retrieved. The output information of the GRU (240) can be input to the second MLP (250).

[0106] A centralized critique focusing on the combination of the histories of all agents along this time axis can use a joint history encoder to abstract the combined macro observations. However, consecutive joint macro observations may contain duplicate observation data at time steps where one agent begins performing a macro action and other agents do not, and consequently, these duplicate observations may risk leading to inaccurate inferences. For example, at t=5 in Fig. 2, duplicate observation data for Agents 1 and 2 is included even though only Agent 3 undergoes a state transition, and such duplicate observation data can lead to inappropriate inference results.

[0107] In one embodiment, in the example of Figure 2 of the MacDec-POMDP, the combined macro observation history abstracted by the joint history encoder at time step t=6 can be expressed by the following [Equation 3].

[0108] [Mathematical Formula 3]

[0109]

[0110] As shown above, since the shared history encoder uses the same macro observation multiple times (for example, (used three times) can interfere with accurately determining the local history of all agents and reasoning between agents.

[0111] Therefore, because the common centralized critique structure for multi-agent reinforcement learning focuses on histories along a common time axis, redundant macro observations are introduced in the centralized critique for agents capable of non-macro actions at timesteps where one agent begins performing a macro action and other agents are unable to perform actions. However, such redundant macro observations can lead to inaccurate evaluations in the centralized critique. To address the limitations of this approach and enable efficient asynchronous learning, a special architecture is required that can effectively collect individual agent histories and facilitate inference between agents. To this end, one embodiment of the present disclosure aims to provide an agent-oriented centralized critique. This will be described in more detail later with reference to FIG. 3.

[0112] FIG. 3 is a diagram illustrating an Agent-Oriented Centralized Critic (AOCC) method according to one embodiment of the present disclosure.

[0113] Referring to FIG. 3, an Agent-Oriented Centralized Critic (AOCC) method for MacDec-POMDP is illustrated. In one embodiment, the agent-oriented reinforcement learning structure may include an agent-oriented history encoder (310, 320, 330) provided for each agent performing positional encoding, and an aggregation module (340) for integrating the histories of all agents. The agent-oriented history encoder (310, 320, 330) provided for each agent may receive the latest local macro observation of the corresponding agent as input and output a local macro observation history. Since this agent-oriented history encoder is not a centralized encoder like FIG. 2, it may not use the same observations redundantly. According to one embodiment, the agent-oriented history encoder may accurately capture local history by focusing only on determining the history of the corresponding agent.

[0114] In one embodiment, a GRU for an agent-oriented history encoder may be used without sharing parameters between agents. That is, a first agent-oriented history encoder (310) corresponding to a first agent may include a first GRU, a second agent-oriented history encoder (320) corresponding to a second agent may include a second GRU, and a third agent-oriented history encoder (330) corresponding to a third agent may include a third GRU.

[0115] In one embodiment, the history of agent i can be expressed as [Equation 4] as follows.

[0116] [Mathematical Formula 4]

[0117]

[0118] Here, Enc iis the agent-oriented history encoder of agent i, and can be the latest local macro observation of agent i.

[0119] In one embodiment, an agent-oriented history encoder may break the temporal alignment between agents because it encodes only each agent's local macro-observations rather than joint observations. In one embodiment of the present disclosure, sinusoidal position encoding may be performed to introduce time markers for each agent's macro-observations in order to account for temporal order. However, position encoding is not limited to the sinusoidal method, and various methods capable of representing time information, such as learnable embedding methods, may be applied. The time step of a local macro-observation is the position encoding vector p i It is encoded as, and the encoded result is observed by the local macro z i By connecting with it, the temporal order can also be considered. Accordingly, the history for agent i can be expressed as follows [Equation 5].

[0120] [Mathematical Formula 5]

[0121]

[0122] In one embodiment, location encoding may allow the chronological order and duration of macro actions across all agents to be taken into account.

[0123] In one embodiment, agent-oriented histories derived from an agent-oriented history encoder may be further processed by an aggregation module to approximate a value function. The aggregation module may include attention networks, an MLP following concatenation, a summing unit, etc. Alternatively, for simplification, only concatenation with MLP layers may be performed.

[0124] In one embodiment, the input of the aggregation module is a combination of the latest local histories, which can be expressed as [Equation 6] below.

[0125] [Mathematical Formula 6]

[0126]

[0127] The subsequent MLP layers use the shared history as their own values It can be switched to.

[0128] According to one embodiment of the present disclosure, the observation history and time information of each agent can be independently encoded through position encoding and then aggregated to enable efficient central critique learning in an asynchronous setting. That is, the observation history of each agent is encoded with the timestep information at the time when each agent begins to perform a macro action, and the time information is also encoded separately so that it can be explicitly associated for agent-oriented critique learning.

[0129] In addition, according to one embodiment of the present disclosure, reinforcement learning can be performed to derive more accurate results by deleting duplicate macro observations.

[0130] In one embodiment, the system inputs observation data of each of the multiple agents into an encoder module corresponding to each of the multiple agents to generate history information, and can generate output data based on the history information of each of the multiple agents. Here, the encoder module performs position encoding and may be provided for each of the multiple agents.

[0131] FIG. 4 is a drawing showing a reinforcement learning system according to one embodiment of the present disclosure.

[0132] Referring to FIG. 4, a reinforcement learning system performed by three agents, a first agent (410), a second agent (420), and a third agent (430), is illustrated as an example of a multi-agent reinforcement learning system. However, this is merely an example, and unlike FIG. 3, reinforcement learning may be performed by two agents, and it is obvious that reinforcement learning may be performed by more than three agents. For the convenience of explanation, the following will be described using the case of three agents as an example.

[0133] In one embodiment, the environment (405) may be changed by a first action (470) determined by a first agent (410), a second action (480) determined by a second agent (420), and a third action (490) determined by a third agent (430). Accordingly, the first agent (410) may obtain a first observation (440) based on the change in the environment, the second agent (420) may obtain a second observation (450) based on the change in the environment, and the third agent (430) may obtain a third observation (460) based on the change in the environment.

[0134] In one embodiment, a multi-agent reinforcement learning system may include a memory for storing one or more instructions and at least one processor for executing the one or more instructions stored in the memory. By executing one or more instructions, the at least one processor may input observation data corresponding to a first observation (440) of a first agent (410) into a first history encoder for reinforcement learning to generate first history information of the first agent (410). Additionally, the at least one processor may input observation data corresponding to a second observation (450) of a second agent (420) into a second history encoder to generate second history information of the second agent (420). Additionally, the at least one processor may input observation data corresponding to a third observation (460) of a third agent (430) into a third history encoder to generate third history information of the third agent (430). Such history encoders may be provided for each agent of the multi-agents. That is, if there are n agents, n history encoders can be provided.

[0135] In one embodiment, the first history encoder, the second history encoder, and the third history encoder may each perform encoding by including observation data of the corresponding agent and time information related to the observation. If time information is not included, a separate history encoder is provided for each agent, and inaccurate information may be generated because state transition times are not reflected; however, according to one embodiment of the present disclosure, accurate training can be enabled by encoding by matching time information to the observation data of each agent. In addition, the first history encoder, the second history encoder, and the third history encoder may each include a Multi-Layer Perceptron (MLP) and a Gated Recurrent Unit (GRU).

[0136] In one embodiment, a multi-agent reinforcement learning system may include an aggregation module that receives and processes one or more history information corresponding to the output of a history encoder provided for each agent of the multi-agents. That is, a single aggregation module that processes multiple history information may be provided. Additionally, the output value of the aggregation module may be processed by an MLP.

[0137] In one embodiment, the reward (400) may be determined by changing the environment (405) according to the first action (470) determined by the first agent (410), the second action (480) determined by the second agent (420), and the third action (490) determined by the third agent (430). Each agent may cooperate with one another to maximize the value of the reward.

[0138] Multi-agent reinforcement learning can face challenges in sparse reward environments where agents struggle to learn effective strategies. Macro-behaviors, which are a series of actions executed as a single decision, facilitate long-term planning; however, varying durations among agents introduce asynchronous behavior, which can complicate Centralized Training with Decentralized Execution (CTDE).

[0139] In the CTDE (Centralized Training with Decentralized Execution) framework, centralized critiques can generally be trained using the combined histories of all agents. The observation history of the i-th agent at each time step t can be expressed as follows.

[0140] [Mathematical Formula 7]

[0141]

[0142] Here, represents the sequence of all observations by Agent i up to time step t. The critique takes the set of all agents' histories as input and the value at time step t It can predict.

[0143] In an asynchronous setup, all agents observe the macro at every timestep t Since it is not possible to obtain, missing macro observations are the most recent valid macro observations when generating history It can be replaced with.

[0144] This mechanism critiques across time steps even when macro observations are missing It can be ensured that the input to is maintained consistently. By utilizing the latest valid macro observations, centralized critiques can effectively adapt to asynchronous environments while maintaining compatibility with the training process.

[0145] In a decentralized execution, each agent independently receives local macro observations and can select a macro action based on the current local macro observation and historical data. Unlike a critic, since each agent operates independently, there may be no structural difference between synchronous and asynchronous configurations. The macro action of the i-th agent at timestep t. Is It can be sampled as follows.

[0146] Currently studied multi-agent reinforcement learning methods utilize padding to handle asynchronous characteristics, but this padding process can misalign asynchronous experiences, leading to incorrect correlations.

[0147] According to one embodiment of the present disclosure, an Agent-Centric Actor-Critic (ACAC) method that resolves asynchronous behavior without relying on padding may be provided.

[0148] In one embodiment, a Proximal Policy Optimization (PPO) method incorporating a Generalized Advantage Estimate (GAE) method suitable for asynchronous settings may be used to train centralized critics and distributed actors. GAE is a technique that balances the variance of empirical returns and the bias of Temporal Difference (TD) estimation in estimating the advantage function using a hyperparameter λ. When λ=0, reliance is placed on a single-step TD, and when λ=1, the empirical returns over the entire interval excluding the baseline may be used. When Λ is a value between 0 and 1, a compromise balance between the two extremes may be provided. In asynchronous multi-agent reinforcement learning, the interval between consecutive macro observations can vary significantly depending on the agent, situation, etc. If the existing micro-timestep-based λ discount is applied as is, an excessive discount accumulates on the future reward of the decision as the duration of the macro action increases, which can lead to a problem where the relative importance of the macro decision is underestimated. Consequently, policy updates at the time of selecting a long-lasting macro action may be distorted, and learning stability and final performance may degrade.

[0149] According to one embodiment, a modified GAE method may be used to switch the criterion for the λ discount from a micro time step to a macro time step. A micro time step refers to a raw step of the simulator, and a macro time step may refer to an indicator that counts, in chronological order, a set of points in time when any agent acquires a new macro observation. Accordingly, according to one embodiment, the λ discount may be applied to the TD error based on the number of times a macro decision occurred, rather than the elapsed time (number of micro steps).

[0150] In one embodiment, using an actor-critic algorithm, each Distributed actors parameterized by r and centralized critics parameterized by ψ can be trained. The training procedure can be expressed as follows in [Equation 8].

[0151] [Mathematical Formula 8]

[0152]

[0153] Here, τ is the trajectory sampled using the current actors, and r t represents the reward at time step t, and the advantage function A t It can be defined as follows [Equation 9].

[0154] [Mathematical Formula 9]

[0155]

[0156] Here, t next is the next time step in which a new macro observation for an agent becomes possible.

[0157] In synchronous multi-agent reinforcement learning, it is common practice to achieve a shared history abstraction by collecting shared observations from all agents at every time step. However, in an asynchronous setup, only some agents may provide observations at a given time step, while observations are absent for the remaining agents. Conventionally, the shared history was constructed by filling in missing observations with padding data (e.g., copies of the previous observation); however, this padding causes redundant information and spurious correlations, which undermines the accuracy of the shared history representation.

[0158] FIG. 5 is a diagram showing an agent-centric encoder of an agent-centric centralized critic according to one embodiment of the present disclosure.

[0159] Throughout this disclosure, i represents the agent index and t represents the global time step of the environment. represents a macro-observation or equivalent state feature acquired by agent i at time step t, and represents the time information (time position) associated with the observation. is the hidden history representation of Agent i. The output V of the central critique t is the value function estimate at time step t.

[0160] The history of each agent can be abstracted from macro observations using recurrent networks. However, in asynchronous multi-agent reinforcement learning, it is essential to consider the elapsed time between consecutive macro observations. Without this duration information, the history of each agent may be learned inaccurately due to varying durations, as there is a lack of context regarding how long the interval between consecutive macro observations is—for example, whether it is 1 time step or 10 time steps.

[0161] Referring to FIG. 5, in order to solve these problems, one embodiment of the present disclosure involves macro observation and the corresponding time step An agent-centered encoder (500) integrating may be provided. The agent-centered encoder (500) is a time step It is embedded using sinusoidal position encoding, and this embedded time step can be associated with a macro observation. The combined information is processed through a multilayer perceptron (MLP) and a gated recurrence unit (GRU) to effectively abstract the history of each agent. The recurrence module can be implemented as an LSTM or a Transformer Encoder (including a time mask) instead of a GRU.

[0162] In one embodiment, the agent-centered encoder (500) may include an input unit (510), a time embedding unit (520), a combination node (530), a multi-layer perceptron (MLP) module (540), a GRU (550), etc.

[0163] In one embodiment, the input unit (510) and Can receive. may be a compressed vector of a vectorized observation or feature map. The time embedding unit (520) is It can be converted into sine / cosine-based position embeddings, learning-based embeddings, or log scale / clipping, etc. The resulting vector is TE( It can be expressed as ). In one embodiment, the joining node (530) is wa TE( ) can be connected. If necessary, additive combinations may be used. The MLP (multi-layer perceptron) module (540) can perform non-linear transformations by mapping the combination vectors to multiple perceptrons. Activation functions may include ReLU, GELU, etc. The GRU (Gated Recurrent Unit, 550) is the previous time step hidden state With the output of the MLP module (540) as input, the current hidden state It can be calculated. In FIG. 5, a GRU is shown as a recurrent module, but this is merely an example, and LTSM, RNN, etc. may be used. In one embodiment, the output unit (560) is the calculated It can be provided to Agent i's actors and centralized critique. The hidden state is at the start of the episode. Or initialized with a learning vector, optionally reset at macro-episode boundaries (e.g., completion of a specific process step), or damping coefficient Multiply by Memory reduction can be applied.

[0164] According to one embodiment, the combined information is processed by an MLP module (540) and a GRU (550) so that the history of each agent can be effectively abstracted. In addition, according to one embodiment, time step information macro observation By integrating with, the agent-centric encoder (500) can ensure that the history of each agent is learned accurately and consistently even in asynchronous multi-agent reinforcement learning.

[0165] According to one embodiment, the hidden state of the GRU (550) is updated only for agents that have received a new observation, and for agents that have not received an observation, the previous hidden state It can be maintained as is. Therefore, the history of each agent is preserved continuously without the injection of padding data, and redundancy and distortion of the history representation can be suppressed.

[0166] According to one embodiment, since the interval between observations is directly expressed without padding injection, history distortion caused by asynchronous duration inconsistency is reduced, the hidden state is maintained continuously and consistently even with imbalance in learning data between agents, and the bias in value estimation in time-sensitive rewards is lowered so that policy convergence can be stabilized.

[0167] In one embodiment, the parameters of the agent-centered encoder (500) are shared among all agents to increase data efficiency, while additionally connecting agent identification embeddings can be used to correct individual characteristics. In another embodiment, the parameters of the agent-centered encoder (500) may be partially shared only among sets of agents with different roles, or may not be shared at all.

[0168] FIG. 6 is a diagram illustrating a method for calculating a value function by integrating agent-centered history according to one embodiment of the present disclosure.

[0169] Referring to FIG. 6, the agent-centered centralized critique may include agent-centered encoders (500-1, 500-2, ..., 500-N) that abstract the history of each agent and an attention-based aggregation module (640) that combines these abstracted histories. Each of the agent-centered encoders (500-1, 500-2, ..., 500-N) may correspond to the agent-centered encoder (500) described above with reference to FIG. 5.

[0170] In one embodiment, an agent-centric encoder may be provided for each agent. The agent-centric encoder for agent i is and Current hidden state with as input It can output the last valid hidden state, even for agents with no observations at the last time step t. At time step t, the centralized critique can be calculated according to the following [Equation 10].

[0171] [Mathematical Formula 10]

[0172]

[0173] Here, refers to timestep information for agent i's macro observations up to timestep t.

[0174] In one embodiment, the self-attention module (610) is in a hidden state Using the input, a first aggregate expression can be produced by weighting the importance of interactions between agents. For example, the self-attention module (610) can convert input vectors into queries, keys, and values, and produce a first aggregate expression by weighting the importance of interactions between agents. The self-attention module (610) learnably reflects the relative importance of interactions between agents, enabling more precise co-inference compared to fixed-weight summation. Additionally, since the hidden state of unobserved agents is maintained without updating, duplicate inflow due to padding can be structurally blocked.

[0175] The average pooling module (620) can generate a global integrated representation by performing average pooling on the output of the self-attention module (610). In one embodiment, the output of the self-attention module (610) and the output of the average pooling module (620) may be combined or weighted summed. The MLP module (630) processes the integrated representation corresponding to the output of the average pooling module (620) using multiple perceptrons to perform a high-dimensional non-linear transformation, thereby generating a value function V t It can output. If necessary, layer normalization or scalar scale parameters may be added to stabilize the output scale.

[0176] That is, one embodiment of the present disclosure may organize the history of each agent on an agent-by-agent basis and then integrate these agent-by-agent histories into a common representation. Specifically, an agent-centric history encoder may be used to capture the macroscopic observation history of each agent. By aggregating this agent-centric history using an attention-based module, a more accurate common history representation for value estimation may be possible. By focusing on encoding the histories of each agent before combining them, according to one embodiment, cases where macroscopic observations are lacking can be effectively handled, and the limitations of padding-based methods can be overcome.

[0177] FIGS. 7a to 7c are drawings illustrating an Agent-Centric Actor-Critic (ACAC) method according to one embodiment of the present disclosure.

[0178] Referring to FIGS. 7a through 7c, at time step t=t1, the first agent (710) can acquire a new macro observation and, accordingly, select a new macro action. On the other hand, the second agent (720) and the third agent (730) cannot acquire a new macro observation because the previous macro action is still in progress. In this case, the first agent (710) that has acquired a new macro observation can input the macro observation into the encoder centered on the first agent to generate an updated history. Conversely, the second agent (720) and the third agent (730) that have not acquired a new macro observation can use the existing history as input to the aggregation module. Through this approach, central critics can accurately estimate the value function using the latest history information of each agent, even in an asynchronous multi-agent reinforcement learning setup.

[0179] For example, the first agent (710) obtains an observation at time step t=t1 and time step information indicating the time of acquisition of the observation or the location of the decision It can be input into the first agent-centered history encoder (500-1). The first agent-centered history encoder (500-1) is by the time embedding unit Converts into an embedding vector, and the embedding vector converted by the time embedding unit and Combines them, performs a non-linear transformation by the MLP module, and then the hidden state from all time steps by GRU Update to the current hidden state It can generate. The generated It can be provided to the policy output of the first actor and used to select a new macroscopic action, and at the same time, it can also be used in a centralized aggregation path.

[0180] On the other hand, since there is no observation for the second agent (720) and the third agent (730) at time step t=t1, each history encoder does not update the hidden state, and = , = The existing history can be maintained. The hidden representation maintained in this way can be input directly into the central aggregation path. In this case, padding may not be performed.

[0181] In one embodiment, at time step t=t1, the agent-specific history hiding representation{ , , } can be input to the aggregation module. The aggregation module can generate critique input features by weighting the relative importance of interactions between agents using a self-attention module, generating a global integrated representation using an average pooling module, and performing non-linear transformations using an aggregation MLP. Based on these features, the central critique has a value function V t1 It can generate.

[0182] In one embodiment, the update history of the first agent (710) that has acquired a new observation and the maintenance history of the second agent (720) and the third agent (730) that have no observation are considered simultaneously, so that the contribution of each history can be dynamically reflected in the aggregation module. Accordingly, even in an asynchronous multi-agent environment, the central critic can perform accurate value estimation by utilizing the latest and highly relevant history information.

[0183] FIG. 8 is a diagram illustrating an Agent-Centric Actor-Critic (ACAC) method according to one embodiment of the present disclosure.

[0184] Referring to FIG. 8, the ACAC system may include agent-centered distributed actors (810), agent-centered central critics (820), etc. Each agent may acquire macro observations at different times in an asynchronous environment.

[0185] In one embodiment, a system may be provided that includes an agent-specific history encoder and a central aggregation module to effectively process macroscopic observations acquired at different times by agent in an asynchronous multi-agent reinforcement learning environment. Even if some agents fail to acquire new observations at a specific time, previous history hiding is maintained without padding, so that a value function can be calculated precisely through aggregation based on self-attention and mean pooling.

[0186] In one embodiment, each agent may be equipped with an agent-centric history encoder. That is, a history encoder may be provided for each of the multiple agents. For example, if the multiple agents include a first agent and a second agent, the system may include a first history encoder corresponding to the first agent and a second history encoder corresponding to the second agent.

[0187] In one embodiment, an agent-oriented history encoder embeds temporal information, such as observation time and decision location, using positional encoding, combines the observation and temporal embeddings with an MLP, and updates the previous hidden state with a GRU to obtain the current history hidden representation h. i It can generate. The generated h i It is passed to the upper MLP policy head, and Agent i's policy π(h i A macro action based on ) can be selected. Only agents that have acquired a new observation update their GRU hiding, and the hiding of agents without observations is maintained, so that observation padding is not performed.

[0188] For example, when the first agent receives the first observation data at the first time point, the system can update the first agent's history information by inputting the first observation data and the corresponding time information into the first agent's history encoder. On the other hand, if the second agent's observation data is not acquired at the first time point, padding is not performed, and the previous hiding is maintained as is and used as the second agent's current history. That is, the second agent's history information can be maintained. Here, multiple agents may have different action start and / or duration times and may be asynchronous multi-agents that determine different information at different times.

[0189] In one embodiment, the history information of a plurality of agents is input into an aggregation module, and a value function can be calculated by performing self-attention and average pooling.

[0190] In one embodiment, h calculated by each agent encoder 1 , h 2 , h 3Hidden expressions such as can be input into the critique path. The critique may include another agent-centered history encoder and can generate the latest history by reflecting time information in a manner similar to the actor. In one embodiment, the actor and the critique may share a history encoder or have independent parameters. This set of histories can be input into the aggregation module (830).

[0191] In one embodiment, the aggregation module (830) is a module that aggregates the history information of each of a plurality of agents and may include a self-attention module and an average pooling module. The self-attention module may map input history into queries, keys, values, etc., to increase the relative importance of interactions between agents. The average pooling module may generate an aggregated representation by globally aggregating attention outputs. The MLP module may output a value function based on the aggregated representation.

[0192] In one embodiment, the history encoder of each agent may include an MLP module that processes observation data and time information corresponding to the observation data by combining them, and a GRU that processes the hidden state of the previous time point and the hidden state of the current time point in sequence.

[0193] In one embodiment, based on the output of the history encoder of the first agent, the policy of the first agent is determined, and the macro action of the first agent corresponding to the policy of the first agent can be determined.

[0194] In one embodiment, the system defines a set of macro decision time points l(0), l(1), ... and the advantage estimation at macro time point l(0) It can be calculated as. Here λ is the TD error, γ is the depreciation rate, and λ is a value between 0 and 1. The λ discount is accumulated based on the number of macro decision-making cycles, so that even if the duration of a macro action is long, the importance may not decrease excessively.

[0195] In one embodiment, policy updates can be performed using the clipping purpose of Proximal Policy Optimization (PPO). The value function is learned with a standard loss such as mean squared error, and if necessary, normalization (e.g., PopArt) can be applied to stabilize the value scale.

[0196] In one embodiment, an agent-centric actor-critic method can process the trajectory of each agent individually by using an agent-centric history encoder and perform attention-based centralized criticism to integrate this agent-specific information. According to one embodiment, by eliminating the redundancy inherent in padding-based methods, more accurate time abstraction, faster learning, and higher quality policies can be provided.

[0197] According to one embodiment, learning convergence is accelerated under sparse rewards, and higher accuracy can be achieved than with conventional padding-based approaches.

[0198] FIG. 9 is a schematic diagram of a system according to one embodiment of the present disclosure.

[0199] Referring to FIG. 9, the system may include a structure for performing multi-agent reinforcement learning. For example, the system may include a plurality of agents (910a to 910n), an agent-centric history encoder module (920), a centralized critique module (930), an actor-critic learning module (940), etc. The plurality of agents (910a to 910n) are entities that observe a state within an environment and perform actions, and each agent can select and execute a high-level action, i.e., a macro-action. Each macro-action is broken down into a plurality of low-level actions, i.e., micro-actions, and executed, and the execution time may vary from agent to agent. Accordingly, action selection between agents can be performed asynchronously. The system may collect history data including past state, action, and reward information of each agent.

[0200] In one embodiment, the agent-centric history encoder module (920) acquires history data including past state, behavior, and reward information of each agent, and can encode the collected history data including time point information. According to one embodiment, asynchronousness can be reflected by applying time point embeddings to each history data. To this end, the time point information may include behavior duration information, for example, the execution time of a macro behavior. Encoding can be performed using a Recurrent Neural Network (RNN), Long Short Term Memory (LSTM), Gated Recurrent Unit (GRU), or Transformer-based network. The agent-centric history encoder module (920) can naturally handle asynchronousness to enable learning without padding. Therefore, by processing the independent history of each agent without using padding, incorrect correlations occurring in the prior art can be prevented.

[0201] In one embodiment, the centralized critique module (930) can integrate history data encoded from multiple agents based on a self-attention mechanism. Through this, the centralized critique module (930) can reflect the importance of each agent as a learnable weight and estimate a co-value function. The self-attention may include multi-head attention, thereby allowing the interaction between agents to be effectively modeled. The centralized critique module (930) can estimate a co-value function by applying learnable weights that reflect the importance of each agent. Through this, the system can perform accurate value estimation even in an asynchronous environment.

[0202] In one embodiment, the actor-critic learning module (940) can update the policy of each agent based on the co-value function estimated by the centralized critique module (930). The policy update is performed based on the Proximal Policy Optimization (PPO) algorithm, and clipping and entropy regularization may be applied. Additionally, the actor-critic learning module (940) can strengthen long-term strategies using a modified Generalized Advantage Estimation (GAE) that applies a discount rate at the macro action level.

[0203] In one embodiment, the system can prevent spurious correlation and improve learning stability and efficiency by performing learning without padding while considering the asynchronous nature between agents.

[0204] In one embodiment, the system can identify whether the environment is a sparse reward environment and, if identified as such, apply reward shaping to improve learning efficiency. For example, the system can monitor the variance or frequency of rewards collected over a certain period and, if they fall below a threshold, automatically determine that it is a sparse reward environment and activate the reward shaping module. Thus, the system can achieve rapid convergence and high performance even in a sparse reward environment. Additionally, the system can dynamically change the reward shaping strategy according to the characteristics of the environment. For example, in a sparse reward environment, it can maximize learning efficiency by setting an intermediate goal to provide rewards, while in a dense reward environment, it can maintain the existing reward structure. Furthermore, by applying an adaptive algorithm that automatically adjusts reward shaping rules according to changes in the environment, the system can support stable learning in various simulation environments.

[0205] In one embodiment, the system can accelerate large-scale multi-agent learning through GPU-based parallel processing. The model can be lightweighted to enable execution on mobile devices or edge computing environments. To this end, the system can reduce model size and computational load by applying techniques such as network pruning, quantization, and knowledge distillation. This lightweighting can enable real-time learning and inference in environments with limited hardware resources, such as robot control, IoT devices, and autonomous vehicles.

[0206] In one embodiment, the system can operate not only on a single computing node but also in a distributed learning environment where multiple nodes are connected via a network. To this end, the system applies a parameter server structure to centrally aggregate and synchronize model parameters calculated at each node, thereby improving the learning speed in a large-scale multi-agent environment. This distributed processing method can enable efficient learning even in complex simulation environments containing thousands or more agents.

[0207] In one embodiment, the attention mechanism used in the centralized critique module may apply a hierarchical attention structure in addition to basic multi-head attention. Agent groups may be clustered based on physical distance, role, or functional similarity, and attention within the group may be performed in the first step, and attention between group representatives may be performed in the second step. For example, by first performing attention at the agent group level and then additionally applying attention between groups, the system can reduce computational complexity and effectively model interactions even in environments with a very large number of agents. This structure can be particularly useful in large-scale team-based simulations performing collaborative tasks.

[0208] FIG. 10 is a block diagram illustrating the internal structure of an agent-centered history encoding module according to one embodiment of the present disclosure.

[0209] Referring to FIG. 10, the agent-centered history encoder module (920) performs the function of encoding history data collected from multiple agents while reflecting asynchronousness, and may include a history data input unit (1010), a time point embedding processing unit (1020), and an encoding network (1030).

[0210] In one embodiment, the history data input unit (1010) may receive history data including past state, action, and reward information of each agent. The history data is collected independently for each agent and may be recorded in units of macro-actions. Since each macro-action is broken down into multiple micro-actions and executed, the history data may have asynchronous characteristics. In one embodiment, the input may be processed without data loss by taking this asynchronous nature into account.

[0211] In one embodiment, the time embedding processing unit (1020) may apply a time embedding to reflect time information in the input history data. The time embedding processing unit (1020) may model the difference in execution time between agents by adding an embedding that indicates the time at which each state and action occurred. In addition to time information, the time embedding processing unit (1020) may additionally include the duration of the action to reflect the execution length of the macro action. The embedding method may be implemented as a positional embedding or a learnable embedding.

[0212] In one embodiment, the encoding network (1030) can convert history data reflecting time information into a high-dimensional representation. The encoding network (1030) may use a Recurrent Neural Network (RNN), Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), or Transformer-based structure. When the encoding network (1030) applies a Transformer-based structure, it can model long-term dependencies through a self-attention mechanism. Additionally, the encoding network (1030) can effectively process asynchronous data sequences and learn agent-specific behavior patterns.

[0213] Here, the Self-Attention used when the encoding network (1030) is implemented based on a Transformer models the influence of a specific past point in time on current decision-making within the time-series data of a single agent, and can be distinguished from the attention that models the interaction between agents in the centralized critique module (930) described later.

[0214] In one embodiment, history data received from the history data input unit (1010) can be converted into a high-dimensional vector in the encoding network (1030) after asynchronous behavior is reflected through the time point embedding processing unit (1020). The result of this encoding is transmitted to a centralized critique module and can be used to model interactions between multiple agents.

[0215] FIG. 11 is a block diagram illustrating the internal structure of a centralized critique module according to one embodiment of the present disclosure.

[0216] Referring to FIG. 11, the centralized critique module (930) may include an encoding network (1110), a self-attention module (1120), a GAE computation unit (1130), a macro action processing unit (1140), etc. to reflect asynchronousness in multi-agent reinforcement learning and to efficiently perform policy updates.

[0217] In one embodiment, the encoding network (1110) receives a high-dimensional vector transmitted from an agent-centered history encoding module and can convert it into a form usable by a centralized critique module. The encoding network (1110) may apply a Transformer-based structure and may also apply recurrent structures such as RNN, LSTM, and GRU. The encoding network (1110) can normalize the input for modeling interactions between agents and prepare it to be combined with an attention mechanism.

[0218] In one embodiment, the self-attention module (1120) can model interactions between agents by integrating encoded history data. For example, when multi-head attention is used, relationships between agents can be learned from various perspectives using multiple attention heads. The self-attention module (1120) can estimate a co-value function by applying learnable weights that reflect the importance of each agent. Additionally, the self-attention module (1120) can enable efficient computation even in a large-scale agent environment by applying hierarchical attention.

[0219] In one embodiment, the GAE calculation unit (1130) can perform reward calculations suitable for an asynchronous environment based on Generalized Advantage Estimation (GAE). The GAE calculation unit (1130) can support stable learning by adjusting the discount rate according to the execution length of each macro action to strengthen long-term strategies and prevent reward distortion caused by asynchronousness.

[0220] In one embodiment, the macro action processing unit (1140) can identify the duration and end time during which a selected macro action is broken down into a plurality of micro actions and executed, and determine the policy update timing based thereon. Additionally, the macro action processing unit (1140) can adjust the learning schedule based on the execution time (duration) of the macro action.

[0221] In one embodiment, the system processes inputs in an encoding network (1110), models interactions between agents through a self-attention module (1120), calculates rewards in a GAE computer (1130), and performs policy updates in a macro action processor (1140). This process can be combined with a Proximal Policy Optimization (PPO) algorithm to apply clipping and entropy regularization.

[0222] In one embodiment, a new type of encoder combining a position encoding technique may be used in the process of encoding and aggregating observation history collected from multiple agents. This allows time-stamped information to be effectively reflected in the history data of each agent and prevents information loss caused by asynchronous action execution. Additionally, the encoder according to one embodiment is designed so that actors and critics can share the same type of encoding network, thereby maintaining consistency in the network structure and improving learning efficiency.

[0223] In addition, in one embodiment, a self-attention mechanism may be applied during the process of abstracting the joint history. This allows for encoding by considering the interrelationships between the histories of each agent, and enables more sophisticated interaction modeling compared to simple data merging methods. In particular, by applying multi-head attention, interactions between agents can be learned from various perspectives, and if necessary, the hierarchical attention structure can be extended to support efficient computation even in large-scale agent environments.

[0224] In addition, in one embodiment, a new reinforcement learning model and algorithm framework based on an Agent-Centric Actor-Critic structure may be utilized based on encoding and aggregation methods. This structure can independently encode the observation history of each agent and then train a Centralized Critic Network and a Decentralized Actor Network through an aggregation module. Compared to existing simple centralized learning methods, this structure is robust to asynchronous operations and can provide high learning stability in a cooperative multi-agent environment.

[0225] In addition, in one embodiment, a Proximal Policy Optimization (PPO)-based algorithm may be applied instead of the existing Naive Actor-Critic algorithm to learn the agent's policy function. PPO ensures the stability of policy updates through clipping and entropy regularization and can significantly improve sample efficiency. Furthermore, the system can strengthen long-term strategies and prevent reward distortion caused by asynchronous behavior by modifying Generalized Advantage Estimation (GAE) to apply a discount rate at the macro-action level.

[0226] In the following description of an embodiment of the present disclosure, for convenience of explanation, a method for determining the schedule of a naphtha cracking center (NCC) will be described as an example. However, the embodiments of the present disclosure are not limited to a method for determining the schedule of a naphtha cracking center, and can, of course, be applied to a method for determining the schedule of other processes or to a method for determining information other than a schedule. An embodiment of the present disclosure can be applied to a method for determining information about a process in multiple steps.

[0227] FIG. 12 is a schematic diagram of a product production process of a naphtha cracking center according to one embodiment of the present disclosure.

[0228] Referring to Fig. 12, at a Naphtha Cracking Center (NCC), naphtha, which is a gasoline fraction obtained from an atmospheric distillation unit of crude oil, is thermally cracked in a high-temperature cracking furnace, and then through processes such as rapid cooling, compression, and refining, ethylene, propylene, butylene, and BTX (Benzene, Toluene, Xylene), which are basic raw materials for petrochemical products, can be produced.

[0229] In other words, naphtha can be converted into substances with high industrial utility, such as ethylene, propylene, benzene, toluene, and xylene, through steam cracking or thermal cracking. For example, ethylene serves as a raw material for making polyethylene and polystyrene, propylene serves as a raw material for making polypropylene, and butane or butylene can be used to make synthetic rubber. These substances serve as raw materials for the plastics processing, textile, rubber, paint, and detergent industries, and can become final products such as daily necessities, adhesives, dyes, pesticides, pharmaceuticals, industrial products, and interior materials.

[0230] A naphtha cracking center is a core facility that produces petrochemical raw materials through a complex process and consists of a receiving stage for unloading naphtha, a mixing stage for blending naphtha, and a cracking stage for producing marketable products. More specifically, naphtha is initially transported from various geographically distributed refineries via vessels and unloaded into receiving tanks; various types of naphtha from these receiving tanks are then supplied to mixing tanks; and the naphtha blended in the mixing tanks is heated in a cracking furnace to produce marketable products of the desired quality. That is, the product production process of the naphtha cracking center may include a receiving process of storing naphtha supplied from one or more vessels (1210) or companies (e.g., other oil companies) in one or more receiving tanks (1220), a mixing process of transferring the naphtha from the receiving tanks (1220) to a mixing tank (1230) for a naphtha cracking process, and a cracking process of thermally cracking the naphtha supplied from the mixing tanks (1230) at high temperature in a furnace (1240). Here, the mixing tanks (1230) may also be referred to as blending tanks or feed tanks.

[0231] In one embodiment, the product production process of the naphtha cracking center may further include a process of measuring the paraffin content of naphtha supplied from a vessel (1210) or a company, and a process of measuring the paraffin content of naphtha stored in an incoming tank (1220), a mixing tank (1230), etc.

[0232] In one embodiment, the constraints may include a range for the paraffin content for each tank. For example, the paraffin content of the mixing tank (1230) may be limited to a range of about 80 to 83% based on the total weight of the naphtha. Since naphtha has different properties depending on the country of origin or company, the naphtha stored in the receiving tank also has different properties, and the receiving and mixing processes must be performed so that the paraffin content of the mixing tank (1230) satisfies the range of the constraints. However, the constraints are not limited to paraffin content and may include various physical / chemical constraints such as specific gravity, sulfur content, delivery date, and tank level limit.

[0233] Considering these constraints, determining the optimal schedule for the naphtha cracking center is crucial for profitability and efficiency. Generally, experts decide based on their experience and know-how which incoming tank to store naphtha, at what ratio to mix it into the mixing tank, and to what extent to heat it using which cracking furnace. However, relying on human experience to determine the schedule has limitations in predicting complex chemical reactions and actual results; results vary significantly depending on the level of the expert's experience and know-how; it is difficult to verify whether all constraints have been satisfied; and it is difficult to respond to sudden changes in circumstances.

[0234] Accordingly, the present disclosure aims to provide a method for determining optimal information (e.g., a schedule) using artificial intelligence. For example, multi-agent reinforcement learning may be utilized. A naphtha cracking center can be operated autonomously using multi-agent reinforcement learning, in which real-world constraints are overcome and each agent takes responsibility for a step and cooperates to achieve a common goal.

[0235] In the present disclosure, each furnace or each tank is modeled as an individual agent, capable of deciding its own actions and cooperating.

[0236] FIG. 13 is a diagram illustrating a reinforcement learning method using multiple agents according to one embodiment of the present disclosure.

[0237] Referring to FIG. 13, a naphtha cracking center can be operated according to an optimal schedule using scheduling information determined through reinforcement learning using multiple agents (1310, 1320, 1330). A simulator using reinforcement learning can take actions (1340, 1350, 1360) from agents (1310, 1320, 1330) and provide the next observation and reward (1380) based on the current action. In one embodiment, each agent is responsible for a specific process and can cooperate with each other to achieve goals such as profit maximization while complying with real-world constraints. For example, there may be realistic constraints such as performing the transfer process from the receiving tank to the mixing tank for at least 8 hours, within a range that does not exceed the minimum naphtha storage capacity of the receiving tank and the maximum naphtha storage capacity of the mixing tank.

[0238] In one embodiment, each agent can determine the information necessary to generate scheduling information for naphtha cracking centers for a predetermined future period based on current information and various constraints, such as the inventory status of each tank, ship arrival plans, naphtha supply plans from other companies, and prices of naphtha and marketable products.

[0239] In one embodiment, each of the multiple agents may produce different results (e.g., durations) at different times. For example, the first agent (1310) is an agent managing incoming goods and may decide actions such as selecting an incoming tank to store naphtha when a ship arrives irregularly and determining the amount to store in that incoming tank, and the second agent (1320) is an agent for mixing naphtha and may decide actions such as determining an incoming tank to bring naphtha to a mixing tank and determining the amount to bring from that incoming tank when the level of a certain incoming tank reaches a threshold (e.g., 90% of the tank capacity). Additionally, the third agent (1330) is an agent managing a cracking furnace and may decide actions such as receiving naphtha from a mixing tank and determining variables to operate the cracking furnace when the product inventory is below a predetermined amount. A virtual NAFTA operating environment (1370) can be created using actions (1340, 1350, 1360) determined at different times. The simulation device can determine expected profits in the virtual NAFTA operating environment (1370) and determine a reward (280) based thereon. This reward can be delivered to multiple agents (1310, 1320, 1330) and used by the agents to perform reinforcement learning. That is, multiple agents (1310, 1320, 1330) can be trained using the same reward during reinforcement learning. However, it is also possible for multiple agents to be trained using different rewards.

[0240] In one embodiment, the reward (1380) may be determined based on total revenue, facility operating costs, naphtha purchase costs, costs according to constraints, etc. For example, the reward may be determined by the following [Equation 11].

[0241] [Mathematical Formula 11]

[0242]

[0243] In [Equation 11], Constraints are constraint conditions, and w c is the weight per constraint, and Cost c can refer to the cost incurred per constraint. Accordingly, the more constraints are violated, the higher the Cost c The value can increase. For example, if the constraint includes the stability of the paraffin component—that is, the condition that the paraffin component must be maintained to a certain extent—the change in the paraffin component stored in the mixing tank can be used as variable c.

[0244] In addition, in [Equation 11], the profit can be calculated by subtracting the estimated production cost of marketable products from the estimated revenue generated from the sale of naphtha, taking into account facility operating costs (e.g., energy usage costs) and naphtha purchase costs. For example, the profit can be determined according to [Equation 12], which subtracts facility operating costs and naphtha purchase costs from total revenue as follows.

[0245] [Mathematical Formula 12]

[0246]

[0247] In one embodiment, Revenue can be calculated as, for example, "CH4 production volume * CH4 product price + PSA OFF GAS production volume * PSA OFF GAS price + RC2 production volume * ethane product price + C3 LPG production volume * propane product price + ethylene production volume * C2 product price + propylene production volume * C3 product price + H2 (99%) production volume * 99% H2 product price + HRPG production volume * HRPG product price + PFO production volume * PFO product price + Raw C5 production volume * Raw C5 product price + (Mixed C4 production volume * Mixed C4 product price) + (RPG production volume - Raw C5 production volume) * RPG product price".

[0248] In one embodiment, energy usage can be calculated as, for example, "[(Naphtha input amount + C3 LPG input amount + C4 LPG input amount) * A + Mixed C4 production amount * B + (RPG production amount Raw C5 production amount) * C] * C3 LPG price / C3 LPG calorific value / 1000", where A is the average energy unit value of a naphtha cracking center plant, B is the average energy unit value of a BD plant, and C is the average energy unit value of a BTX plant (for example, a plant that produces aromatic products using pyrolysis gasoline produced from an ethylene plant).

[0249] In one embodiment, hh can be calculated as "Total naphtha feed input * naphtha price + C3 LPG input * C3 LPG price + C4 LPG input * C4 LPG price + RC2 input * ethane product price".

[0250] In one embodiment, when an optimal scheduling is determined through such reinforcement learning, the naphtha cracking center can be operated according to the generated optimal scheduling. For example, if multiple schedules are formed and provided to a user, the user can operate the naphtha cracking center based on one of them.

[0251] FIGS. 14a and FIGS. 14b are drawings illustrating a reinforcement learning method using multiple agents according to one embodiment of the present disclosure.

[0252] Referring to FIG. 14a, an asynchronous multi-agent system is illustrated in which the start and duration times of the actions of each agent are different. For example, if the multi-agents consist of three first agents (1310), second agents (1320), and third agents (1330), each agent determines a different action at a different time, and the determined action vector The changed state vector is transmitted to the environment (1370) and the actions are applied to the environment (1370). It can be determined. Also, the action termination vector The reward generated as a result of applying actions to the environment (1370) can be provided to the agents. In addition, the reward generated as a result of applying actions to the environment (1370) can be provided. It can also be provided to each agent. The actions of each agent can be determined asynchronously as shown in Fig. 14b.

[0253] In another embodiment, the actions of each agent may be transmitted to their respective environment (1370) whenever an action is determined. For example, each action may be reflected in their respective environment (340) such that at the first time when the first agent (1310) determines the first action, the first action is reflected in the environment (1370), at the second time when the second action is determined, the second action is reflected in the environment (1370), and at the third time when the second agent (1320) determines the third action, the third action is reflected in the environment (1370). However, since the reward is determined by assuming that the product is ultimately produced and sold, the reward may be determined and provided to each agent after the actions of the first agent (1310), the second agent (1320), and the third agent (1330) are all reflected in the environment (1370).

[0254] In one embodiment, the state, behavior, and reward of each agent may include the following information.

[0255] 1. First Agent

[0256] - Status: Naphtha receiving schedule (e.g., vessel schedule or receiving plans from other companies), current status of receiving tanks (e.g., naphtha inventory and properties per tank), constraints related to receiving tanks

[0257] - Action: Identifier of the receiving tank to store naphtha upon receipt, the amount of naphtha to be stored in the corresponding receiving tank, or the pipeline connection schedule (e.g., pipeline connection with Vessel A to Receiving Tank No. 1 from 2:00 PM to 10:00 PM)

[0258] - Reward: Returns considering whether constraints are satisfied

[0259] 2. Second agent

[0260] - Status: Stock and properties of each incoming tank, stock and properties of the mixing tank

[0261] - Action: Identifier of the receiving tank to hold at least some naphtha in the mixing tank, the amount of naphtha to be brought from that receiving tank to the mixing tank, or the pipe connection schedule (e.g., pipe connection from Receiving Tank No. 1 to the mixing tank from 8:00 AM to 2:00 PM)

[0262] Compensation: Returns considering whether constraints are satisfied

[0263] 3. Third agent

[0264] Status: Inventory and properties of mixing tanks, operating status by disassembly furnace

[0265] Action: Feed rate, COT, DS ratio, etc. to feed naphtha from the mixing tank into each cracking furnace

[0266] Compensation: Returns considering whether constraints are satisfied

[0267] However, this is merely one example, and it goes without saying that the state, behavior, and rewards of each agent can be adjusted differently.

[0268] In one embodiment, the naphtha cracking center scheduling system can acquire input information. For example, the input information can be acquired through user input based on a User Interface (UI).

[0269] In one embodiment, the input information may include constraints, naphtha receiving schedule information, tank inventory information, naphtha property information within the tank, mixing tank operation information, cracking furnace operation plan information, target production volume information for each specific product, raw material unit price information, product unit price information, etc. In another embodiment, some of the information, such as constraints, may be pre-set information, and in that case, may not be included in the input information because it has been set in advance.

[0270] In one embodiment, constraints may include physical constraints such as tank storage capacity criteria to be satisfied and the number of pipes that can be connected at once, stability constraints regarding stability, and operational constraints for complying with a set target production volume during a specific period (e.g., weekly or monthly). Additionally, naphtha receiving schedule information may include a vessel receiving schedule, tank information of another company, a naphtha receiving schedule for a specific future period, scheduled receiving date and time, receiving rate, receiving quantity, naphtha property information, and identification information based on the naphtha receiving method (e.g., vessel identifier, tank identifier of another company, pipe identifier by company, etc.).

[0271] In one embodiment, at least one of the input information may include information for a specific period or information at a specific point in time. For example, the tank inventory information and the naphtha properties information within the tank, respectively, may each include the naphtha inventory information of the corresponding tank at the time of the scheduling start and the naphtha properties information of the corresponding tank at the time of the scheduling start.

[0272] In one embodiment, the blending tank operation information may include one or more blending schedules, such as a blending start time, a blending end time, the name of the receiving tank to be blended, and a blending speed per receiving tank. For example, the blending schedule may include a recent blending schedule.

[0273] In one embodiment, the decomposition furnace operation plan information may include schedule information for each decomposition furnace for a future specified period. For example, the decomposition furnace operation plan information may include schedule information determined for each decomposition furnace for the next 30 days. In one embodiment, the decomposition furnaces may exist in various types. For example, if there are a first decomposition furnace, a second decomposition furnace, and a third decomposition furnace of different types, the decomposition furnace operation plan information may include schedule information determined for each of the first decomposition furnace, the second decomposition furnace, and the third decomposition furnace for the next 30 days. The decomposition furnace operation plan information may include a decomposition start time, a decomposition end time, an operating mode (or feed mode), decoking schedule information, COT (coil outlet temperature), coil outlet pressure, a predetermined speed (e.g., feed rate), a DS (Dilution Steam) ratio, etc.

[0274] In one embodiment, the target production volume information for a specific product may include the target production volume or rate for a specific product during a specific period in the future. For example, the daily target production volume of ethylene for the next 30 days, the daily target production volume of propylene for the next 30 days, etc., may be included in the target production volume information for a specific product.

[0275] In one embodiment, the raw material unit price information may include raw material unit price information at the time of information input, raw material unit price information for a specific period prior to the time of information input, and expected raw material unit price information for a specific period after the time of information input. For example, the raw material unit price information may include the expected daily price of raw materials for the next 30 days.

[0276] In one embodiment, the product unit price information may include unit price information at the time of inputting information for each product, unit price information for a specific period prior to the time of inputting information for each product, and expected unit price information for a specific period after the time of inputting information for each product. For example, the product unit price information may include the expected daily price of naphtha products for the next 30 days.

[0277] However, the above input information is merely an example and is not limited thereto, and various input information for scheduling the naphtha cracking center may be included.

[0278] In one embodiment, a naphtha cracking center scheduling system may determine receiving tank information using a first agent based on input information. In one embodiment, the first agent may be an agent trained using reinforcement learning. Based on input information including a ship receiving schedule, a naphtha receiving schedule from another company, real-time receiving tank inventory, naphtha properties information within the tank, cracking furnace operation plan information, etc., the first agent may determine a receiving tank to receive naphtha from at least one of the tanks of a ship and another company, and determine the amount, ratio, or schedule information of naphtha to be stored in the corresponding receiving tank. For example, based on the input information, the first agent may determine an identifier for at least one receiving tank to store naphtha among a plurality of receiving tanks, and determine naphtha receiving ratio or amount information for each tank corresponding to each identifier. Additionally, based on the input information, the first agent may determine naphtha receiving schedule information and information on the period for storing naphtha in the corresponding receiving tank for each tank corresponding to each identifier. The receiving schedule information may include date or time information for connecting the receiving tank to a vessel or another company's device via a pipe (e.g., receiving naphtha from Vessel B to Receiving Tank A from 2:00 PM to 6:00 PM). In one embodiment, pipes may be connected as a method to transfer naphtha from a vessel or another company's device to a receiving tank; however, since connecting pipes and performing other tasks is inconvenient if the schedule changes frequently, constraints such as a minimum connection time of n hours per pipe may exist. The first agent may determine the receiving tank information by taking these constraints into account.

[0279] In one embodiment, the naphtha cracking center scheduling system can obtain naphtha property information corresponding to each receiving tank after a predetermined amount of naphtha has been distributed to the receiving tank.

[0280] In one embodiment, all receiving tank information may be determined using a single first agent, receiving tank information may be determined using a different first agent for each receiving tank, or receiving tank information may be determined using a different first agent for each receiving tank group. That is, there may be one or more first agents.

[0281] In one embodiment, the naphtha cracking center scheduling system can determine mixing tank combination information using a second agent. In one embodiment, the second agent may be an agent trained using reinforcement learning. The second agent can determine the mixing tank combination information based on the inventory of each receiving tank, the properties of the naphtha stored in each receiving tank, etc.

[0282] In one embodiment, the blending tank combination information may include an identifier of at least one receiving tank among a plurality of receiving tanks to transfer naphtha into the blending tank, naphtha ratio information (or amount information) to be transferred to the blending tank for each of the at least one receiving tank identifiers, blending schedule information with the blending tank for each of the at least one receiving tanks, naphtha blending ratio information for each of the at least one receiving tanks, blending execution date information, etc. The naphtha blending ratio information may include ratio information or amount information of naphtha to be taken from each receiving tank.

[0283] In one embodiment, all mixing tank combination information may be determined using a single second agent, receiving tank information may be determined using a different second agent for each mixing tank, or receiving tank information may be determined using a different second agent for each mixing tank group. That is, there may be one or more second agents.

[0284] In one embodiment, the naphtha cracking center scheduling system may determine cracking furnace operation information using a third agent. In one embodiment, the third agent may be an agent trained using reinforcement learning. The third agent may determine cracking furnace operation information based on the inventory information of the mixing tank, the properties of the mixing tank, cracking furnace status information, etc. In one embodiment, the cracking furnace operation information may include cracking furnace mode information, cracking furnace identifier, feed rate, COT (Coil Outlet Temperature), DSR (Dilution Steam Ratio), heating time, cracking furnace operation schedule information, and one or more variables for cracking furnace operation.

[0285] In one embodiment, there may be multiple third agents. For example, different agents may be used for each cracking furnace mode. For example, the third agents include a 3-1 agent reinforced learning for the cracking furnace of mode A, a 3-2 agent reinforced learning for the cracking furnace of mode B, and a 3-3 agent reinforced learning for the cracking furnace of mode C, and the naphtha cracking center scheduling system may determine the operation information for the cracking furnace of mode A using the 3-1 agent for the cracking furnace of mode A, determine the operation information for the cracking furnace of mode B using the 3-2 agent for the cracking furnace of mode B, and determine the operation information for the cracking furnace of mode C using the 3-3 agent for the cracking furnace of mode C. In another embodiment, a single third agent may determine all cracking furnace operation information.

[0286] In one embodiment, the naphtha cracking center scheduling system can determine one or more scheduling information of a naphtha cracking center based on receiving tank information generated by a first agent, mixing tank combination information generated by a second agent, and cracking furnace operation information generated by a third agent. In one embodiment, the scheduling information may include receiving scheduling information, mixing scheduling information, cracking furnace scheduling information, expected production volume information, expected profit information, expected naphtha inventory information, expected properties information, constraint satisfaction test result information, scheduling graph, etc.

[0287] In one embodiment, the receiving scheduling information may include receiving schedule information for a predetermined period in the future. For example, the receiving scheduling information may include tank identification information to be received for the next two weeks, the start time of receiving for the tank, the end time of receiving for the tank, etc.

[0288] In one embodiment, the mixing scheduling information may include mixing schedule information for a predetermined period in the future. For example, the mixing scheduling information may include mixing tank identification information to be used for the next two weeks, the mixing start time of the mixing tank, the mixing end time of the mixing tank, and mixing speed information of the tank (e.g., mixing speed of Tank A: about 100 Ton / hour).

[0289] In one embodiment, the decomposition furnace scheduling information may include schedule information for each decomposition furnace for a predetermined period in the future. For example, the decomposition furnace identification information to be used for the next two weeks, the start time of decomposition for the corresponding decomposition furnace, the end time of decomposition for the corresponding decomposition furnace, the decomposition speed (e.g., target control speed determined by artificial intelligence, feed rate, etc.), COT, DS ratio, etc. may be included in the decomposition furnace scheduling information.

[0290] In one embodiment, the expected production volume information, expected revenue information, expected naphtha inventory information, expected properties information, etc., may also be expected information for a predetermined period in the future. For example, the expected production volume information may include daily expected production volume provided by product for the next two weeks. Additionally, the expected naphtha inventory information may include naphtha inventory or naphtha change information provided by tank for the next two weeks, and the expected properties information may include properties change information provided by tank for the next two weeks.

[0291] In one embodiment, the constraint satisfaction test result information may include evaluation information on how well the generated schedule satisfies the predetermined constraints.

[0292] Furthermore, the naphtha cracking center scheduling system can provide one or more scheduling information to the user through a UI / UX. For example, an overview of each of the above one or more scheduling information can be displayed and provided in the form of a graph or figure through the UI / UX, and summary information such as cumulative profit and constraint satisfaction can also be provided.

[0293] According to one embodiment of the present disclosure, a naphtha cracking center scheduling system may determine scheduling information for a naphtha cracking center using an asynchronous multi-agent system comprising a first agent, a second agent, and a third agent. For example, each agent may determine different information at different times.

[0294] FIG. 15 is a drawing showing the schedule of a naphtha cracking center according to one embodiment of the present disclosure.

[0295] Referring to FIG. 15, scheduling at a naphtha cracking center may mean planning a continuous and multi-stage process that converts raw naphtha into high-value-added products such as ethylene. For example, the process at the naphtha cracking center may include the following three interdependent stages: 1) Unloading stage (1510): a stage of unloading naphtha transported from a vessel, etc., into an incoming tank; 2) Blending stage (1530): a stage of mixing the naphtha from the selected incoming tank in a blending tank to achieve a target composition ratio; 3) Cracking stage (1550): a stage of producing ethylene, etc., by high-temperature cracking of the mixed raw materials in a furnace.

[0296] These stages are so closely linked that the entire process can be halted if even one is out of schedule; therefore, a scheduling strategy that precisely coordinates the timing of each stage is essential.

[0297] To explain using the example in FIG. 15, when a new vessel arrives, an unloading step (1510) must be performed. When the first vessel arrives, unloading from the first vessel into the first tank must be performed, and when the second vessel arrives, unloading from the second vessel into the second tank must be performed. Accordingly, the processor must schedule the unloading step (1510) by anticipating the arrival time of the first vessel and the arrival time of the second vessel. In addition, based on the determined receiving tank information, the processor must determine the time to move from the receiving tank to the blending tank, the amount to be mixed, etc., and based on this blending information, determine the start time of operation of the disassembly furnace, the operating period, the temperature, etc.

[0298] That is, the processor must determine the timing and actions to be performed at each stage based on the fact that the unloading stage (1510), blending stage (1530), and cracking stage (1550) are interconnected. However, these stages are so closely linked that if even one of them is out of schedule, the entire process may be halted; therefore, a scheduling strategy that precisely coordinates the timing of each stage is essential. Furthermore, each stage of the naphtha cracking center process is also affected by external factors such as operational constraints (e.g., equipment tolerances, safety standards), demand fluctuations, and ship arrival schedules, requiring long-term and robust scheduling. In one embodiment, when determining the schedule, the plant operating status (e.g., current tank inventory, equipment utilization rate, maintenance schedule, etc.), the ship arrival schedule (e.g., arrival time and shipment volume based on prior shipment data (shipment plan)), tank capacity (e.g., remaining space in each receiving tank and blending tank), raw material quality objectives (e.g., composition ratio after blending, sulfur content, evaporation point, etc.), and market conditions (e.g., external economic variables such as oil price, ethylene price, contract delivery date, etc.) must be comprehensively considered.

[0299] In one embodiment, the processor determines which receiving tank will receive the naphtha unloaded from each vessel in the unloading step (1510), sets the tank combination and mixing ratio to satisfy the standard quality in the blending step (1530), determines the blending sequence, and determines operating conditions such as the feed rate and coil outlet temperature per cracking furnace in the cracking step (1550). The schedule determined in this way must be designed to maximize profitability while preventing conflicts between processes, and it is important that it possesses robust characteristics capable of responding to real-time fluctuations (vessel delays, equipment failures, etc.).

[0300] However, the mainstream approach has traditionally been to optimize by separating steps such as unloading, blending, and cracking. This step-by-step optimization method fails to adequately reflect the interactions between processes, making it difficult to fully resolve issues such as cascading delays and quality degradation that occur during actual operations. Furthermore, the NCC scheduling environment is extremely sensitive, where a single seemingly minor error can invalidate the entire schedule and ultimately halt operations. For instance, failing to start blending on time can cause the incoming tank to overflow, while an inappropriate blending ratio can generate off-spec feed, leading to a cascading impact on downstream processes. Due to these vulnerabilities, there is a problem in that reinforcement learning agents alone are insufficient to reliably generate a consistently valid and safe schedule.

[0301] In addition, reinforcement learning agents are generally trained based on a single scalar reward function, but in actual petrochemical processes, there may be a need to prioritize conflicting goals differently depending on the situation, such as profit maximization, process stability, and compliance with operating constraints. Since priorities fluctuate frequently depending on external factors such as market prices, delays in raw material arrival, and changes in equipment status, there is a problem in that a fixed reward function according to the method described with reference to FIGS. 12 to 14b is insufficient to adequately reflect these dynamic trade-offs.

[0302] One embodiment of the present disclosure aims to provide a method for determining information regarding a process consisting of multiple steps that solves these problems. Specifically, one embodiment of the present disclosure aims to provide a method for supporting an operator's decision-making in both long-term and short-term planning by generating, evaluating, and selecting multiple candidate schedules while simultaneously considering complex constraints.

[0303] That is, the present disclosure relates to a system / method for generating and updating a schedule to reduce the risk of overall failure due to interdependence between stages and to maximize process efficiency in a process consisting of multiple stages (e.g., manufacturing, logistics, IT pipeline, model learning pipeline, etc.). More specifically, the present disclosure relates to a technology in which artificial intelligence sequentially analyzes stage status, result information, etc., to branch and expand a pivot schedule, and combines reinforcement learning-based agent macro operations with a fitness estimator network (evaluator) to update the pivot schedule at each synchronization point and determine the final schedule. In the present disclosure, a pivot schedule refers to an intermediate schedule that is fixed or hypothetically set up to a specific point in time (branching point) during the entire process period, and various future schedules can be derived (branched) from this point. In one embodiment, the fitness estimator may correspond to the centralized critique module (930) described above.

[0304] FIG. 16 is a diagram illustrating a method for determining information about a process consisting of a plurality of steps according to one embodiment of the present disclosure.

[0305] Referring to FIG. 16, at the initialization point (1610), the processor can initialize the pivot schedule of each group into a blank macro operation sequence, reflecting the initial operating state of the NCC system.

[0306] In one embodiment, the processor may determine a planning period for creating a schedule. That is, the processor may determine a reference time for starting to create the schedule and a deadline for not extending the schedule further. For example, the processor may determine a planning period of three weeks from a specific point in time as the planning period for creating the schedule.

[0307] In one embodiment, the processor may create a plurality of groups. Referring to the example of FIG. 16, the processor may create group 1-1, group 1-2, group 2-1, group 3-1, etc.

[0308] In one embodiment, the processor may generate a plurality of groups corresponding to specific operating scenarios, including operating criteria, operating objectives, operating levels, constraints, etc. Operating objectives may include increased profitability, increased process stability, increased energy efficiency, compliance with quality specifications, savings in computing costs, reduced training time, reduced latency, reduced memory usage, and increased model accuracy. Operating criteria may include indicators representing process performance, such as profitability and process stability, and may be evaluated by a scalar goodness-of-fit function. Additionally, operating levels may include indicators representing the strictness of constraints and may be expressed as conservative, moderate, or stressed. Conservative may be selected when the range of constraints is narrow and safety is prioritized, while stressed may be selected under high-load operating requirements approaching equipment limits. Furthermore, constraints may include conditions regarding equipment operating limits, storage capacity, material property specifications, quality specifications, safety standards, memory capacity, the number of available server instances, network bandwidth, target inference latency, etc.

[0309] In one embodiment, the processor may configure multiple groups such that, as one moves toward lower groups, it satisfies higher (stricter) operating standards and operating levels. Since a schedule satisfying a higher level, i.e., a stricter level, will naturally also satisfy lower-level constraints, promising schedules can be transitioned between groups through these structured groups. For example, Group 1-2 may correspond to a group that must satisfy stricter operating standards and operating levels than Group 1-1, Group 2-1 than Group 1-2, and Group 3-1 than Group 2-1.

[0310] According to one embodiment of the present disclosure, such a hierarchical structure directly reflects real-world conditions in the model and can improve search efficiency through propagation from upper levels to lower levels. For example, in actual NCC operations, the intensity of constraints changes frequently depending on the state of the plant, market demand, etc., and the processor can select a stress level during a surge in demand and a maintenance level during stable operation.

[0311] - Pivot-based Branching operation

[0312] In one embodiment, the processor may duplicate the pivot schedule of each of the plurality of groups into a plurality of branch schedules for each of the plurality of groups. For example, the processor may duplicate the pivot schedule (1620) of the first-1 group in parallel to determine that all branch schedules of the first-1 group are identical to the pivot schedule (1620) of the first-1 group. Additionally, the processor may duplicate the pivot schedule of the first-2 group in parallel to determine that all branch schedules of the first-2 group are identical to the pivot schedule of the first-2 group.

[0313] - Scenario-based Rollout Action

[0314] In one embodiment, the processor may perform a rollout process for each quarter schedule, taking into account the operational scenario of the corresponding group. A rollout process may refer to a stage of creating a complete schedule by deploying and simulating candidate schedules (or policies) for a predetermined planning period. This rollout process may be performed by an agent trained based on an artificial intelligence model.

[0315] That is, an agent trained based on an artificial intelligence model can extend a schedule for each of multiple quarter schedules by reflecting operational scenarios corresponding to each of multiple groups. For example, the processor can have the agent trained based on an artificial intelligence model continue to extend the schedule until the schedule for a predetermined planning period is fully completed. The processor can have the agent trained based on an artificial intelligence model construct a complete schedule by sequentially attaching macro actions to quarter schedules that are still empty or only partially defined. For example, the processor can complete a three-week production plan by having the agent trained based on an artificial intelligence model simulate actions in the unloading, blending, and cracking phases.

[0316] In one embodiment, the processor can generate a complete schedule in which all timesteps up to a predetermined planning period are filled, by taking as input the branch schedule up to the current point in time, the operating scenario of the group (objective, constraint, etc.), and macro action candidates (pre-learned policy, etc.).

[0317] In one embodiment, the processor may calculate a reward or penalty score using a fitness function of the group during or at the time of completion of the rollout process. Such reward or penalty score may be used when selecting a schedule to replicate during the synchronization evaluation phase.

[0318] - Synchronized Evaluation & Update operation

[0319] In one embodiment, the processor may update the group's pivot schedule by performing an evaluation at a predefined synchronization point (1630, 1640, 1650). The synchronization points (1630, 1640, 1650) may include a time when a common event occurs. For example, the processor may determine the time of a ship's arrival at the NCC as the synchronization point (1630, 1640, 1650). Based on the fact that the candidate schedules have progressed to the same operational time, the processor may select the superior schedule up to that time as the pivot schedule.

[0320] In one embodiment, the processor may evaluate the branch schedule of a first group and the branch schedule of another group with the same or higher constraint level as the first group at a first synchronization time using a fitness estimator network, and update the pivot schedule of the first group according to the evaluation result. In one embodiment, the fitness estimator network (e.g., a multilayer perceptron or a transformer) may receive various information as input and output a scalar fitness. For example, the input information may include a summary of the state of each branch schedule (e.g., inventory, capacity utilization, quality margin, safety margin, etc.), resource allocations, energy estimates, cost estimates, constraint violation penalties, scenario levels (e.g., maintenance, medium, stress, etc.), and operational purpose weight vectors. Learning is performed by regression on labels (e.g., realized revenue, total penalty, etc.) generated from a simulator or historical operating data, and may be fine-tuned by mini-batch based on the results of recent executions in the online phase. If the processor retains the pivot schedule of the first group at the first synchronization time when the score of the pivot schedule of the first group is the highest, and if the score of the pivot schedule of the first group is not the highest at the first synchronization time, it may replace the pivot schedule of the first group with the schedule having the highest score.

[0321] In one embodiment, at predefined synchronization points (1630, 1640, 1650), each group may evaluate (i) its own group's branch schedule and (ii) the branch schedule of other groups having the same or higher operating level using its own fitness function. As a result of the evaluation, the best schedule may be updated as the new pivot schedule for that group.

[0322] - Multilayer recovery mechanism

[0323] In one embodiment, a recovery mechanism may be used. If a failed schedule exists, the processor may immediately replace it with a superior schedule within the group or a higher-level group and continue the search. That is, if any schedule is determined to have failed because it can no longer proceed throughout the rollout process, the recovery mechanism may be executed.

[0324] In one embodiment, if a specific branch schedule fails, the processor may replace the schedule with the schedule of highest fitness within the same group and continue the rollout process (Intra-group recovery). For example, the pivot schedule of Group 1-1 is replicated to become the first branch schedule and the second branch, and the schedule is expanded by reflecting the operational scenario of Group 1-1 for each of the first and second branch schedules. If, at some point, the first branch schedule fails, the processor evaluates the branch schedule of Group 1-1 and, if it determines that the second branch schedule has the highest fitness among the multiple branch schedules of Group 1-1, can replace the first branch schedule with the second branch schedule.

[0325] If all schedules belonging to a group fail, the processor may replace the entire group with a copy of the schedule with the highest fitness among other groups having the same or a higher level (inter-group recovery). For example, if the processor replicates the pivot schedule of Group 1-1 into the first and second branch schedules, and replicates the pivot schedule of Group 1-2 into the third and fourth branch schedules, and all branch schedules of Group 1-1 fail, the processor may determine the schedule with the highest fitness among the branch schedules of Group 1-2 as the pivot schedule of Group 1-1. In this case, Group 1-2 may be a group having requirements of a higher level (stricter level) than Group 1-1.

[0326] In one embodiment, if the schedule of all groups fails, the processor may perform a re-search starting from the pivot schedule of each group at the previous synchronization point.

[0327] This hierarchical recovery mechanism can increase the robustness of the plan by preventing premature search termination.

[0328] In one embodiment, after a predetermined planning period has ended, the processor may apply post-processing evaluation criteria to the completed pivot schedules to select one or more final schedules (1660).

[0329] For example, when the schedule is completed by a predetermined planning period, the processor may re-evaluate the completed pivot schedule based on final evaluation criteria. Unlike the fitness function used in the rollout process, the final evaluation criteria may include elements that can be precisely calculated only after the entire schedule is completed (e.g., cumulative profit, overall safety indicator, etc.). Based on the results of the re-evaluation, the processor may determine the final schedule. This final schedule (1660) may include a schedule consisting of multiple stages and may be multiple. For example, the top three schedules may be provided to the user.

[0330] For example, when one embodiment of the present disclosure is used to determine the schedule of an NCC, the final schedule includes the operational schedule of the NCC, and the synchronization point may include the time of vessel arrival, the time of completion of a process step, the time of simulation time elapsed, etc. Additionally, the processor may perform a rollout process by extending the schedule based on a pre-learned reinforcement learning policy, by having at least one agent perform at least one of a naphtha receiving operation, a blending operation, and a cracking operation.

[0331] A candidate-based multi-scenario planning method according to one embodiment may structure candidates into multiple groups to overcome the limitations of existing single-population-based methods. According to one embodiment of the present disclosure, by maintaining various candidates, multiple pieces of information optimized under different objective functions (profit, stability, etc.) and constraint levels can be searched and preserved in parallel. Furthermore, according to one embodiment of the present disclosure, by utilizing a scenario hierarchy, a schedule satisfying high (strict) constraint levels automatically satisfies low constraint levels as well, thereby facilitating transitions between schedules.

[0332] In one embodiment, a structured candidate population can enable targeted search for various operational scenarios while maintaining the advantages of candidate-based techniques such as parallel exploration and escaping local optima. As a result, a set of information with superior quality and diversity compared to a single population explored uniformly can be generated. For example, in NCC scheduling, a schedule can be provided that simultaneously ensures comprehensiveness and robustness of production planning.

[0333] According to one embodiment of the present disclosure, by always maintaining a set of schedules capable of responding quickly to changes in priority or unexpected situations, robustness and adaptability that are difficult to achieve with an agent alone can be secured.

[0334] In addition, according to one embodiment of the present disclosure, the driver can immediately select the option most suitable for the current situation among a plurality of scenario-based schedules, thereby reducing the burden of decision-making.

[0335] In addition, according to one embodiment of the present disclosure, the risk of a single selection error disrupting the entire process can be significantly reduced.

[0336] In addition, according to one embodiment of the present disclosure, the schedule can be flexibly reconfigured even when abnormal events occur, such as market price fluctuations or equipment malfunctions.

[0337] In addition, according to one embodiment of the present disclosure, the limitations of a fixed reward structure can be compensated for by multi-objective, multi-constraint search according to one embodiment of the present disclosure while effectively utilizing behavior candidates generated by a reinforcement learning agent.

[0338] In other words, one embodiment of the present invention serves as a key element that bridges the potential of reinforcement learning-based scheduling with real-world plant requirements, and by simultaneously improving usability, safety, and efficiency, it can enable the introduction of various multi-stage processes into industrial sites.

[0339] Although an NCC scheduling method has been described as an example to explain one embodiment of the present disclosure, the method according to one embodiment of the present disclosure is not applicable only to NCC scheduling. For example, one embodiment of the present disclosure may also be used for data preprocessing operations, distributed model training operations, fine-tuning operations, inference operations, etc.

[0340] FIG. 17 is a flowchart illustrating a method for determining information about a process consisting of a plurality of steps according to one embodiment of the present disclosure.

[0341] In operation 1710, the processor may create a plurality of groups including a first group and a second group. In one embodiment, the plurality of groups may be created hierarchically. For example, the second group may be formed as a group having a higher level of operational scenario than the first group.

[0342] In one embodiment, each group can be mapped to an operational scenario defined by an objective function (e.g., profit, stability, latency minimization) and a constraint level (conservative, moderate, stress, etc.). This allows the processor to simultaneously explore different operational goals and constraints even within the same system.

[0343] In operation 1730, the processor may duplicate pivot information for each of the multiple groups into multiple branch information for each of the multiple groups. In one embodiment, for each group, the processor may duplicate multiple branch information in parallel based on the pivot information of the group—a macro operation sequence that initially represents a blank or system initial state. For example, the processor may duplicate the pivot information of the first group into first branch information and second branch information, and duplicate the pivot information of the second group into third branch information and fourth branch information. This provides a basis for extensively exploring various alternative decision-making options at the same time.

[0344] In this case, the information may include schedules, logistics delivery routes, generator load curves, LLM training pipelines, etc. Additionally, macro actions may include unloading actions, blending actions, cracking actions, warehouse management actions, GPU training actions, validation actions, container deployment actions, etc.

[0345] In operation 1750, the processor can expand information for each of multiple branch information by reflecting the operational scenario corresponding to each of multiple groups. That is, the processor can roll out information by reflecting the operational scenario of the corresponding group for each branch information. For example, in a Large Language Model (LM) pipeline, the processor can expand the steps in the order of data preprocessing, distributed learning, checkpoint saving, and inference deployment.

[0346] In one embodiment, if the processor cannot proceed with the scenario using the branch information of the first group, the first branch information may be replaced with the second branch information having the highest suitability among the multiple branch information of the first group.

[0347] If there is no branch information to replace the multiple branch information in the first group, that is, if scenario progression is impossible with all the branch information in the first group, the processor may replace the first branch information with the third branch information having the highest suitability among the multiple branch information in the second group. In this case, the second group may be a group having a higher level of constraint than the first group.

[0348] If scenario progression is impossible with any of the information from all groups, including the first and second groups, the processor may perform a re-search based on the pivot information of each of the multiple groups at the previous synchronization point.

[0349] In one embodiment, the operation scenario may include operation objectives, constraints, etc. Operation objectives include increased profitability, increased process stability, increased energy efficiency, compliance with quality standards, reduced computing costs, reduced training time, reduced latency, reduced memory usage, increased model accuracy, etc., and constraints may include conditions regarding at least one of equipment operating limits, storage capacity, material property specifications, quality specifications, safety standards, memory capacity, the number of available server instances, network bandwidth, and target inference latency.

[0350] In operation 1770, the processor may update the pivot information of the first group at the first synchronization point based on the expanded information. In one embodiment, when the expanded branch information reaches the first synchronization point, the processor may evaluate all branch information of the first group and branch information of other groups having the same or higher constraint level using a fitness function. Additionally, the processor may update the pivot information of the first group to the best branch information based on the result evaluated by the fitness function. The synchronization point may include, for example, common system-wide events such as ship arrival, facility turnaround, or completion of a learning epoch.

[0351] In operation 1790, the processor may determine final information based on the updated pivot information of the first group. In one embodiment, after a predetermined planning period has ended, the processor may select one or more final information by applying post-processing evaluation criteria to the completed pivot information.

[0352] In one embodiment, when the time of plan completion is reached, the processor may re-evaluate the completed pivot information, including the updated pivot information of the first group, using a separate final evaluation criterion. Unlike the fitness function used during rollout, the final criterion may include elements that can be precisely calculated only at completion, such as cumulative revenue, total delay, and energy consumption. Information with the highest performance results from the evaluation may be selected as Final Info and transmitted to an operator UI (User Interface) or an automated execution module. In one embodiment, the Final Info may include information consisting of multiple steps.

[0353] FIG. 18 is a block diagram of a system for determining information about a process consisting of a plurality of steps according to one embodiment of the present disclosure.

[0354] Referring to FIG. 18, a system (1800) (the system may be referred to as a server or device) may include a transceiver (1810), memory (1820), a database (1830), and a processor (1840). However, not all components shown in FIG. 18 are essential components of the system (1800). The system (1800) may be implemented by more components than those shown in FIG. 18, or by fewer components than those shown in FIG. 18. Furthermore, the transceiver (1810), memory (1820), and processor (1840) may be implemented in the form of a single chip.

[0355] In one embodiment, the transceiver (1810) may communicate with a terminal or other electronic device connected to the system (1800) via wired or wireless connection. For example, the transceiver (1810) may receive input information from a user terminal. In one embodiment, the input information may include naphtha receiving plan information, production target quantity for each product (e.g., ethylene production target quantity), constraints, status information, scheduling start time, etc. The naphtha receiving plan information may include the scheduled time of receiving, receiving speed, receiving quantity, naphtha properties information, receiving type information (e.g., whether it is a ship or a tank of another company), etc. The status information may include a predetermined mixing schedule, a predetermined cracking furnace schedule, etc. The mixing schedule or cracking furnace schedule may include the start time, end time, tank name, and mixing or cracking speed for each tank, and the cracking furnace schedule may further include various variable information such as pressure and temperature information. Additionally, the status information may include naphtha inventory quantity and properties information for each tank at the time of scheduling start.

[0356] Various types of data, such as programs and files, such as applications, can be installed and stored in the memory (1820). The processor (1840) may access and use the data stored in the memory (1820) or store new data in the memory (1820). Additionally, one or more instructions may be stored in the memory (1820). The processor (1840) may execute one or more instructions stored in the memory.

[0357] The processor (1840) controls the overall operation of the system (1800) and may include at least one processor, such as a CPU, GPU, etc. The processor (1840) may control other components included in the system (1800) to perform operations for operating the system (1800). For example, the processor (1840) may determine incoming tank information using a first agent based on input information, determine mixing tank combination information using a second agent, and determine disassembly furnace operation information using a third agent.

[0358] However, according to one embodiment of the present disclosure, not all process steps (receiving, mixing, cracking) of a naphtha cracking center must necessarily be determined by multiple agents. In one embodiment, the processor (1840) may receive a fixed schedule for some process steps from a user and determine scheduling information using an agent only for the remaining process steps.

[0359] For example, if a user directly determines and inputs incoming tank information (such as a ship unloading schedule) based on their own experience or external circumstances, the processor (1840) may set the incoming tank information as a hard constraint or an environmental state. In this case, the second agent (responsible for mixing) and the third agent (responsible for disassembly) recognize the incoming tank information determined by the user as an observation and determine the mixing tank combination information and disassembly operation information optimized for it.

[0360] Conversely, if the user predetermines and inputs operation information for the disassembly furnace based on the final product production goal, the first and second agents may determine the optimal receiving and mixing schedules by working backward to match the operation schedule for the disassembly furnace.

[0361] That is, the system (1800) of the present disclosure can support not only a fully automated mode but also a hybrid scheduling mode that combines user decision-making and artificial intelligence recommendations. Through this, it can flexibly reflect unexpected variables at the site or the operator's intuition, while receiving assistance from artificial intelligence in complex computational areas that are difficult for humans to calculate (e.g., calculation of optimal mixing ratios).

[0362] In one embodiment, the processor (1840) may generate one or more scheduling information for the naphtha cracking center based on the receiving tank information, the mixing tank combination information, and the cracking furnace operation information. Additionally, the transceiver (1810) may transmit the scheduling information to the user terminal so that the scheduling information is displayed on the display of the user terminal. In one embodiment, the output scheduling information may include a receiving schedule for a future predetermined period (e.g., receiving start and end times, receiving tank identifier, etc.), a mixing schedule for a future predetermined period (e.g., mixing start and end times, mixing tank identifier, mixing speed, etc.), a cracking furnace schedule for a future predetermined period (e.g., cracking start and end times, target speed (feed rate) determined by an algorithm, COT, DS ratio, etc.), daily production volume and expected profit information by product for a future predetermined period, naphtha inventory quantity and change in properties for a future predetermined period, constraint check result information for the generated schedule, and a plot visualizing the generated schedule.

[0363] The process of generating and outputting scheduling information in this manner can be performed through the UI / UX of a user terminal. For example, when the processor (1840) obtains input information entered by a user, it verifies whether there is sufficient data in the input information to generate output data, and if it is determined that the input information is valid, it can generate one or more scheduling information using an artificial intelligence scheduler based on the scheduling start date entered by the user. Additionally, the processor (1840) can graph one or more scheduling information to provide information, and the user terminal can display this information in the form of a UI / UX.

[0364] Additionally, the processor (1840) can execute logic to generate a plurality of group-specific pivot schedules according to program instructions loaded into memory (1820) and to update the pivot schedules through a fitness estimator at each synchronization point.

[0365] The database (1830) can store various training data for training a learning model. Additionally, the database (1830) may store material information, phase information, simulation result information, etc., and in various embodiments, output data produced by the learning model may be stored. Although FIG. 18 shows the system (1800) including the database (1830), the database (1830) may be provided outside the device. In this case, the database (1830) may be connected to the system (1800) via wired or wireless connection.

[0366] Additionally, the learning model may be implemented outside the system (1800) (e.g., cloud-based) or included inside the system (1800).

[0367] FIG. 19 is a drawing showing a sequential multi-agent according to one embodiment of the present disclosure.

[0368] Referring to FIG. 19, a sequential multi-agent may include a user agent, a writing agent, a style agent, etc. In one embodiment, the sequential agents communicate sequentially and can perform a single task in order. Additionally, each agent can receive the result of the previous step and perform the next task. For example, the user agent may obtain user input from the user to generate a first processing result, the writing agent may generate a second processing result including the written text by generating the first processing result from the user agent, and the style agent may generate a third processing result by receiving the second processing result from the writing agent and applying a style. Accordingly, the third processing result may finally be output. Such a sequential multi-agent has a linear flow and may be suitable for processing a single task. For example, the sequential multi-agent may be used in the field of creative writing.

[0369] According to one embodiment of the present disclosure, the aforementioned first agent (responsible for receiving), second agent (responsible for mixing), and third agent (responsible for disassembly) can operate in a sequential multi-agent structure as illustrated in FIG. 19.

[0370] As explained with reference to FIG. 19, a sequential agent can perform a task by using the result of the previous step as the input for the next step. When this is applied to an NCC scheduling system, a first agent (corresponding to a user agent) can first determine and generate an entry schedule (first processing result) based on ship entry information, etc.

[0371] Next, the second agent (corresponding to the creation agent) receives the above-determined receiving schedule as an unchangeable environmental state and can generate a mixing tank combination and blending schedule optimized thereto (second processing result). Finally, the third agent (corresponding to the style agent) can generate a cracking furnace operation schedule (third processing result) based on the above-determined blending schedule and output a final NCC operation schedule.

[0372] This sequential structure can be effectively utilized when dependencies between stages are clear (linear flow) and when a top-down scheduling method is required where decisions at higher stages have an absolute influence on lower stages (e.g., situations where receiving plans must be prioritized due to unstable raw material supply).

[0373] FIG. 20 is a drawing showing a supervisory agent according to one embodiment of the present disclosure.

[0374] Referring to FIG. 20, a supervised agent may refer to an agent in which a centrally located Supervisor Language Model (LM) coordinates the entire process. In one embodiment, the supervised agent may direct the appropriate agent to perform necessary tasks and synthesize the results according to the user's request. That is, the supervised agent may manage communication between agents. Accordingly, the supervised agent may enable flexible task distribution. For example, the supervised agent may request research from a research agent and request calculations from a mathematics agent. Such a supervised agent may be used in fields such as deep research.

[0375] According to another embodiment of the present disclosure, a naphtha cracking center scheduling system may adopt a supervisor agent structure as illustrated in FIG. 20.

[0376] Referring to Fig. 20, a supervising agent (or orchestrator) coordinates the entire process and directs tasks to subordinate agents. When applied to an NCC system, the supervising agent can be the entity performing pivot schedule management and synchronization evaluation as described with reference to Fig. 16.

[0377] Specifically, the supervisory agent monitors the current process status (e.g., occurrence of urgent orders, equipment failure, etc.) and, accordingly, can dynamically distribute tasks, such as requesting the first agent (corresponding to the research agent) to establish a receiving plan or requesting the second agent (corresponding to the math agent) to calculate the optimization of blending ratios.

[0378] In particular, when a supervisory agent performs the multi-layer recovery mechanism of Fig. 16, if a specific subordinate agent reports that it has failed to generate a schedule, the agent can perform a central control role by immediately instructing an agent in another group to generate an alternative schedule or by relaxing higher-level constraints to re-instruct. This structure enables flexible work distribution and error recovery in complex and variable NCC processes.

[0379] FIG. 21 is a drawing showing a hierarchical agent system according to one embodiment of the present disclosure.

[0380] Referring to FIG. 21, a hierarchical agent system can refer to a system in which a meta-agent controls and coordinates subordinate agents. For example, the meta-agent can obtain user input from a user, request tasks from subordinate agents such as a research agent or a data analysis agent, and generate outputs by receiving task results from each agent. This hierarchical agent system has a hierarchical control structure, allowing tasks to be divided and managed in a more complex manner. In other words, a meta-agent acting as an intermediate manager can be utilized. This hierarchical agent system is suitable for complex systems or coding, and can be used as a coding agent, etc.

[0381] According to one embodiment of the present disclosure, a naphtha cracking center scheduling system may adopt a hierarchical agent system structure as shown in FIG. 21 to efficiently manage large-scale facilities.

[0382] Referring to Fig. 21, the meta-agent distributes tasks to subordinate agents and aggregates the results. When applied to an NCC system, the meta-agent can perform the role of a plant manager that determines the target production volume for the entire plant (e.g., 3,000 tons of ethylene / day) and the overall energy budget.

[0383] Furthermore, sub-agents can be subdivided by function or facility group. For example, a first sub-agent group corresponding to a research agent performs a precise analysis of the characteristics of each incoming tank and reports it, while a second sub-agent group corresponding to a data analysis agent is assigned to each individual furnace (Furnace #1 to #N) to optimize detailed operating variables (COT, Feed Rate) to achieve the quota assigned by the meta-agent.

[0384] This hierarchical structure reduces the computational load on the central meta-agent, which must handle even complex detailed controls in large-scale NCC processes involving dozens of tanks and cracking furnaces, and enables precise control tailored to the characteristics of each facility.

[0385] FIG. 22 is a drawing showing a discussion-type multi-agent system according to one embodiment of the present disclosure.

[0386] Referring to FIG. 22, a discussion-based multi-agent system is a system in which multiple agents present different opinions and select the most appropriate result by voting or evaluating it, thereby deriving the optimal solution based on discussion. That is, multiple agents perform discussion and evaluation based on user input and can output the optimal answer among them. Such a multi-agent discussion-based system can make the best choice by comparing various perspectives in a competitive structure and can be used in fields such as world simulation.

[0387] According to another embodiment of the present disclosure, a naphtha cracking center scheduling system may adopt a Debate-based Multi-Agent System structure as illustrated in FIG. 22 to coordinate conflicting goals.

[0388] Referring to Fig. 22, multiple agents present and evaluate different opinions to derive the optimal solution. This can be applied to the synchronization evaluation and pivot schedule update steps of the NCC system (see Fig. 16).

[0389] Specifically, the system may include multiple agents with different personas. For example, a profit-seeking agent may propose an aggressive schedule to maximize production, a safety management agent may propose a conservative schedule to minimize equipment overcrowding, and a maintenance agent may propose a schedule that considers equipment lifespan.

[0390] They undergo a virtual debate process as shown in Fig. 22. Each agent points out weaknesses in the opponent's schedule (e.g., "Your schedule is profitable, but the risk of pipe blockage increases by 80%), and the system (or a separate referee agent) aggregates these debate results and votes or assigns weights to finally select the optimal pivot schedule that balances profitability and safety. This can maximize the reliability of the schedule by mutually compensating for blind spots that an agent trained with a single objective function might miss.

[0391] FIG. 23 is a drawing showing a mixed agent system according to one embodiment of the present disclosure.

[0392] Referring to FIG. 23, in a hybrid agent system, multiple agents perform parallel processing layer by layer, and an aggregator can integrate the results at the end. For example, a first agent and a second agent may perform a process in parallel at the first layer, and then the first agent and the second agent may perform a process in parallel at the second layer, and an aggregator may synthesize these to generate an output. In one embodiment, the hybrid agent system may use a multi-stage approach for complex problems. The hybrid agent system has a hierarchical and parallel structure and is characterized by the distribution and combination of expertise, so it can be used for medical research, etc.

[0393] According to one embodiment of the present disclosure, a naphtha cracking center scheduling system may adopt a hybrid agent system structure as shown in FIG. 23 to manage the complexity of a large-scale process.

[0394] Referring to Fig. 23, agents can perform parallel processing layer by layer, and an aggregator can integrate them. When applied to an NCC system, multiple receiving agents in the first layer can generate unloading schedules in parallel for different vessels and receiving tanks. At this time, the first aggregator in the first layer can aggregate the receiving schedules generated in parallel, verify whether there is a pipeline conflict, and produce an integrated receiving plan.

[0395] Next, in the second layer, based on the integrated receiving plan, multiple mixing agents optimize the blending ratio for each mixing tank in parallel, and the second aggregator can comprehensively evaluate the uniformity of the properties of the entire raw material.

[0396] This hybrid structure can solve the NCC scheduling problem, which involves thousands of variables, by decomposing it into facility units (parallel) and process stages (hierarchy), thereby dramatically improving computational speed compared to a single agent method and enabling specialized optimization.

[0397] FIG. 24 is a drawing showing a ReAct agent system according to one embodiment of the present disclosure.

[0398] Referring to FIG. 24, ReAct is a compound word of "Reason + Act," and a ReAct agent can refer to an agent that solves problems by repeating reasoning (Reason) and action (Act). For example, if a user asks, "What is the weather like in New York these days?", the LLM can interpret the meaning of the question, search for the current weather in New York through a search engine's search tool, summarize the results, and deliver them back to the user. Such a ReAct agent can be used in AI chatbots, etc. According to one embodiment, the ReAct agent has excellent tool usage capabilities and can generate more accurate responses through the repetition of reasoning and action.

[0399] According to another embodiment of the present disclosure, each agent of the naphtha cracking center scheduling system (in particular, a third agent directly related to safety) may adopt a ReAct (Reason+Act) agent system structure as illustrated in FIG. 24.

[0400] Referring to Fig. 24, the ReAct agent solves problems by repeating reasoning and action. When applied to an NCC environment, the agent can make decisions by going through a reasoning stage and a tool use stage, rather than simply outputting an action regarding the current state. For example, if an abnormal sign is detected where the temperature of the cracking furnace rises rapidly, the ReAct-based third agent can send a reduction command to the actual cracking furnace control system through a reasoning stage where "it is necessary to verify whether the cause of the temperature rise is a change in the raw material properties or an over-fuel injection," an action stage where the "NCC simulator tool" is called to simulate the temperature change after 10 minutes if the current fuel injection amount is maintained, an observation stage where the simulation results predict that the critical temperature will be exceeded after 10 minutes, and a reasoning stage where "the fuel injection amount must be immediately reduced by 5%."

[0401] By introducing the ReAct structure in this way, the system can go beyond simple numerical optimization and respond to unexpected situations through logical thinking processes, utilizing virtual simulations as a tool just like a skilled engineer.

[0402] FIG. 25 is a drawing showing a CodeAct agent system according to one embodiment of the present disclosure.

[0403] Referring to Fig. 25, the CodeAct agent system can perform complex data processing and logic by executing Python code based on a query received from a user. Unlike existing JSON-based data processing systems, the CodeAct agent has the advantage of being able to handle more flexible and complex calculations and analysis tasks by using Python. JSON is primarily used as a data exchange format and can be advantageous for clearly defining the structure of data. However, JSON itself does not provide functions for logic execution or data processing. Therefore, to perform complex data analysis or calculations, it is necessary to define the data structure using JSON and then write additional code. On the other hand, since Python code is executable code capable of data processing and logic implementation, it can directly perform more complex and diverse tasks. By executing this Python, the CodeAct agent enables immediate data processing and calculation, and can process tasks dynamically and in real time.

[0404] For example, when the LLM receives input from a user such as "Analyze sales data for the last 3 months," it analyzes the request, and a Python structured data preprocessing library (e.g., Pandas) can be utilized. Pandas is a Python-based data analysis library that provides tools for easily manipulating and analyzing large datasets. For instance, if a user requests "Analyze sales data for the last 3 months," the CodeAct agent analyzes this query, loads the relevant data in CSV file format, and can efficiently process the data using the Pandas library. Pandas allows for easy handling of tabular data using DataFrame structures and enables simple execution of various data processing tasks, such as filtering, grouping, and aggregation.

[0405] For example, using Pandas, you can group "sales data for the last three months" and analyze sales trends through statistical calculations. In addition, complex tasks such as loading CSV files, handling missing values, filtering, and generating summary reports can also be easily executed using Python code and Pandas.

[0406] According to one embodiment, the CodeAct agent can process complex data analysis and calculation tasks in real time using Python. This can provide much more powerful and flexible capabilities than systems that use only data formats such as JSON.

[0407] In addition, according to one embodiment, CodeAct implements an automated data processing process to analyze user requests and automatically execute corresponding analysis tasks using Python code. For example, it can automatically perform data loading, filtering, and statistical calculations using Pandas, and provide the results in a form that the user can understand.

[0408] Furthermore, Pandas is a tool capable of efficiently handling advanced data analysis within a Python environment, and the CodeAct agent system leverages it to simplify the analysis of complex datasets and statistical processing. This enhances the technical efficiency required for large-scale data analysis. For example, if a user requests, "Analyze sales data for the past three months," the CodeAct agent first loads sales data from a CSV file using the Pandas library. It then analyzes the data using various Pandas functions, performing tasks such as calculating average sales volume, identifying sales trends, and generating graphs. All of these processes are executed using Python code, allowing users to obtain analysis results in real time.

[0409] In this way, the CodeAct agent system is highly useful for handling real-time data analysis and complex calculations, and can provide powerful capabilities utilizing Python and Pandas. Through this, users can experience more efficient and flexible task execution.

[0410] According to one embodiment of the present disclosure, a naphtha cracking center scheduling system may include a CodeAct (Code-Action) agent system as illustrated in FIG. 25 to process unstructured queries from an operator.

[0411] Referring to Fig. 25, the CodeAct agent converts a user's query into code and executes it. When this is applied to an NCC system, the system can be equipped with a data analysis agent. For example, if an operator queries in natural language, "Analyze the correlation between the paraffin content of naphtha unloaded from Vessel A and the ethylene production yield over the past month," it may be impossible to process this with an existing JSON-based rule engine.

[0412] However, the CodeAct agent according to the present embodiment can analyze the query using LLM and immediately generate Python code. The generated code may include (1) code to load the last month's incoming data and production data from the NCC database in the form of CSV or DataFrame (using Pandas), (2) code to calculate the correlation coefficient between the two data (using SciPy or NumPy), and (3) code to visualize the results and output them as a graph (using Matplotlib).

[0413] In one embodiment, the system can execute this code in a sandbox environment and output the results to an operator terminal. This allows the operator to perform a deep dive into complex process data in real time without the assistance of a developer.

[0414] According to another embodiment of the present disclosure, the compensation calculation unit or the constraint verification unit of the naphtha cracking center scheduling system may be implemented based on CodeAct.

[0415] As explained in Fig. 25, JSON does not provide logic execution capabilities, whereas Python can perform complex calculations. In the NCC process, the calculation logic for the expected reward may change frequently depending on fluctuations in crude oil prices, exchange rates, complex chemical reaction yield curves, etc.

[0416] Therefore, instead of using fixed formulas, the system can dynamically generate or update the profit calculation logic itself in the form of Python functions through the CodeAct agent. For example, if a command such as "The carbon tax emission factor has changed starting this month, so recalculate the profit to reflect this" is entered, the CodeAct agent can automatically modify and execute the carbon tax calculation function within the existing profit calculation Python script.

[0417] This goes beyond simple parameter changes and ensures flexibility in the algorithm itself, thereby providing a scheduling system that can respond agilely to rapidly changing market environments and environmental regulations.

[0418] FIG. 26 is a drawing showing a modern tool-using agent system according to one embodiment of the present disclosure.

[0419] Referring to FIG. 26, a modern tool usage agent may be an agent system that interprets a natural language-based request received from a user, calls various external SaaS tools, APIs, etc. through a Multi-Channel Processing (MCP) server, and provides the execution results to the user. In one embodiment, the agent receives a request in the form of text, voice, or other forms from a user, analyzes the intent of the request, and determines whether an external tool call is necessary. The agent may establish a plan to call multiple tools sequentially or in parallel, in addition to a single tool call.

[0420] In one embodiment, the agent may communicate with one or more MCP servers. An MCP server may be a component that acts as middleware between the agent and external tools or services. In one embodiment, the MCP server may perform functions such as providing a standardized interface to external SaaS tools or APIs, managing authentication information (API keys, tokens, etc.), converting agent requests to match the call specifications of each tool, and normalizing tool execution results into a form understandable by the agent. For example, as illustrated in FIG. 8, a first MCP server may be integrated with a search service or web search API (e.g., Brave Search), and a second MCP server may be integrated with a cloud service (API, e.g., AWS, other cloud resource management APIs). By separating the MCP servers by function in this way, the agent can use various tools in a consistent manner without directly being aware of the detailed implementation of each external tool.

[0421] For example, upon receiving a text request from a user such as "Stop my AWS EC2 instance," the agent analyzes the user input to determine that the request is an operation related to controlling cloud resources, decides that an AWS API call is required, and selects the corresponding MCP server. Additionally, it forwards the request regarding the EC2 instance stop to the MCP server, which can convert it into an AWS API call format and execute it. The agent receives the execution result from the MCP server (e.g., whether the instance stop was successful) and can provide it to the user in the form of a natural language message. Agents utilizing modern tools can be used in developer IDEs, AI integrated with them, and offer the advantages of enabling tool control with minimal code and facilitating easy integration with various cloud or web functions.

[0422] According to one embodiment of the present disclosure, a naphtha cracking center scheduling system may include a Modern Tool Use Agent and a Model Context Protocol (MCP) server structure as illustrated in FIG. 26 to reflect external environmental variables in real time.

[0423] Referring to Fig. 26, the agent can communicate with external SaaS tools through the MCP server. When applied to the NCC system, the market analysis agent or logistics management agent can be linked with a global crude oil price information API (e.g., Bloomberg API) or a maritime logistics tracking API (e.g., MarineTraffic API) through the first MCP server.

[0424] For example, if a scheduling agent requests, "Tell me the estimated delay time for naphtha transport vessels due to the current typhoon," the MCP server can convert this into calls to the marine weather information API and vessel location API, execute them, normalize the results, and return them to the agent.

[0425] Through this, the agent can call external data as if it were an internal function via a standardized interface provided by the MCP server, without the need to directly handle authentication or data formats (JSON, XML, etc.) of complex external APIs, and immediately reflect it in the scheduling.

[0426] According to another embodiment of the present disclosure, an agent of the system can utilize an MCP server as middleware to integrate and control heterogeneous legacy systems within a factory.

[0427] Within the NCC plant, ERP systems for material management, DCS (Distributed Control System) for equipment control, and MES for process monitoring are mixed together, and these systems use different communication protocols. The MCP server according to the present embodiment can serve as a standardized interface for these legacy systems.

[0428] For example, an NCC operator can issue a maintenance ticket for an emergency inspection of Unit 3's disassembly furnace and request the release of spare parts from the materials room using natural language. In this case, the agent can interpret the natural language request and establish a parallel planning process to (1) call the parts release API from the first MCP server (for ERP integration) and (2) call the work ticket creation API from the second MCP server (for maintenance system integration). This means that factory operations can be automated using natural language through an AI agent without having to learn the usage of complex internal systems one by one.

[0429] FIG. 27 is a drawing showing a self-reflective agent system according to one embodiment of the present disclosure.

[0430] Referring to FIG. 27, the self-reflective agent system may be an agent system that includes a metacognition-based iterative processing structure that evaluates and modifies results generated by a large language model (LLM). In one embodiment, the self-reflective agent system may include a main language model that interprets user input and generates a draft or intermediate result, a generator that generates a response based on the draft generated by the main language model, a critic that evaluates the generated draft or response and identifies areas requiring improvement, a memory that stores previous generation results, evaluation results, modification history, etc., and a tool that performs auxiliary functions such as grammar checking, keyword consistency analysis, and style rule application.

[0431] In one embodiment, the self-reflective agent system may acquire user input in the form of natural language. The main language model may analyze the user input and generate a corresponding draft response. The result generated at this stage may not be a completed final output, but an intermediate output subject to evaluation. The generated draft may be passed to a Critique for evaluation. The Critique may evaluate the draft by considering logical consistency, contextual appropriateness, consistency with user requests, clarity of expression or sentence structure, and suitability for the purpose (e.g., resume, report, explanatory text, etc.). Based on the Critique results, it may be determined whether the draft has been sufficiently improved. In one embodiment, if the Critique results do not meet the criteria, the system may determine that modifications are necessary. If modifications are determined, the Critique results are passed back to the main language model or the generator, and an improved draft may be regenerated. This process may be repeated until a preset number of iterations or quality criteria are satisfied. If the Critique results are determined to meet the criteria, the result may be finalized as the final output and provided to the user.

[0432] In one embodiment, the self-reflective agent system may utilize memory to store previous drafts and revision history, critique results and evaluation criteria, user preferences, or past request patterns. Additionally, tools may be implemented as grammar checkers, keyword analyzers, style rule engines, etc., and may be utilized as auxiliary tools during the critique or revision phases.

[0433] According to one embodiment of the present disclosure, a naphtha cracking center scheduling system may include a self-reflection agent system as illustrated in FIG. 27 to ensure the safety and feasibility of the generated schedule.

[0434] Referring to Fig. 27, the system generates a draft and modifies it by critiquing it. When this is applied to the NCC system, the main language model (generator) can generate a primary draft schedule as the reinforcement learning agent described earlier.

[0435] Next, the Critique module can perform physical and logical verification on the draft schedule. The Critique can identify not only simple constraint violations but also deep logical flaws, such as, "It was stated that the water would be transferred from Tank A to Tank B at 2:00 PM, but isn't there a risk that the water level in Tank A at that time will be lower than the minimum pump suction height?"

[0436] If a defect is identified, the critic can provide specific feedback to the generator, such as "delay the start time of transfer for Tank A by 30 minutes or reduce the transfer speed by 10%." The generator regenerates a modified schedule reflecting this feedback, and this process can be repeated until the critic makes a pass judgment.

[0437] Through this, the system can prevent hallucinations or safety regulation violations that may occur during the numerical optimization process, thereby generating a high-reliability schedule that can be immediately applied to the field.

[0438] According to another embodiment of the present disclosure, the system may utilize a self-reflection function to explain the basis of a determined schedule or to generate an operation report. Complex NCC scheduling results consist of thousands of data points, making them difficult for humans to understand intuitively. Accordingly, the reporting agent of the system may utilize the structure of FIG. 27 to generate an explanation of "why this schedule was determined."

[0439] For example, if a reporting agent drafts a statement saying, "Unit 3 was operated at full capacity for profitability," a critic may point out that "an explanation regarding compliance with safety regulations is missing." In response, the agent may modify the wording to, "Unit 3 was operated at full capacity for profitability, but safety was ensured by controlling the Coil Outlet Temperature (COT) to be maintained below the safety threshold of 850 degrees."

[0440] This self-reflective reporting capability can transform the decision-making process of AI, which can be regarded as a black box, into an explainable AI (XAI) form, thereby providing a high level of trust to factory operators.

[0441] FIG. 28 is a drawing illustrating a multi-agent workflow system according to one embodiment of the present disclosure.

[0442] Referring to FIG. 28, a multi-agent workflow system may be a system in which multiple specialized agents share roles to collaboratively perform a single task and aggregate the results to generate a final output. In one embodiment, the multi-agent workflow system may include a central agent that receives user input, analyzes the entire task to break it down into sub-tasks, and assigns each sub-task to an appropriate sub-agent; a sub-agent that performs the sub-task assigned by the central agent as an agent specialized in a specific function or domain; an aggregator that receives results generated by multiple sub-agents and generates a final output by synthesizing, organizing, and structuring them; a memory that stores intermediate results generated during the task execution process, sub-agent outputs, previous task history, etc.; a search engine; a database; an analysis tool; an external API; and tools that the sub-agent or central agent can call to perform the task.

[0443] In one embodiment, a user inputs a task request in the form of natural language, and a central agent analyzes the user input to break down the task into multiple sub-tasks. For example, the central agent may separate the task into sub-tasks such as market trend research, competitor analysis, and investment trend analysis. The central agent assigns the decomposed sub-tasks to each sub-agent, and the sub-agents may perform their respective tasks in parallel or sequentially. For example, a first sub-agent may collect the latest market trends, a second sub-agent may analyze competitor information, and a third sub-agent may summarize investment trends. The results of each sub-agent's work may be stored in memory or managed as intermediate outputs before being transmitted to an aggregator. An aggregator (Aggregator LLM) may synthesize the results received from multiple sub-agents to generate a final result having a logical structure. During this process, the removal of duplicate information, standardization of representation, and correction of contextual consistency may be performed. The aggregated result may be output to the user in the form of a report, summary, analysis results, etc.

[0444] According to one embodiment of the present disclosure, a naphtha cracking center scheduling system may include a multi-agent workflow system as illustrated in FIG. 28 at an upper layer to establish an optimal operation goal.

[0445] Referring to Fig. 28, the central agent breaks down tasks into sub-tasks and assigns them to sub-agents. When this is applied to the NCC system, when a user requests, "Please formulate the optimal production strategy for next month," the central agent (strategy planning agent) can break this down into the following sub-tasks.

[0446] 1. 1st Sub-Agent (Market Analyst): Analyzes international oil and naphtha price trends and spread (margin) trends of major products (ethylene, propylene, BD, etc.).

[0447] 2. Second Sub-Agent (Competitor Analyst): Collects news regarding the turnaround schedule and shutdowns of nearby competitor NCCs to predict the possibility of supply shortages.

[0448] 3. Third Sub-Agent (Inventory Analyst): Analyzes the current tank inventory levels in the factory and the planned volume of raw materials to be introduced.

[0449] The analysis reports generated by each sub-agent can be transmitted to the Aggregator LLM. The Aggregator can synthesize these reports to generate a final strategy report stating, "Since ethylene margins are declining but a surge in propylene demand is expected, adopt a Mode-B operation strategy to increase propylene yield by 5% and stockpile inventory at 80%." This final strategy is not limited to a simple text report but can be automatically converted and applied as weights or constraints of the reward function transmitted to the lower-level reinforcement learning scheduling agents (agents in FIGS. 1 to 18).

[0450] According to another embodiment of the present disclosure, the system may be equipped with an automated reporting system utilizing a multi-agent workflow to synthesize complex process data and provide insights to the user. In an NCC process, a vast amount of data may be generated from different domains, such as receiving, mixing, and disassembly. By applying the structure of FIG. 28, a central agent may direct analysis for each part to generate periodic reports. For example, a first sub-agent may summarize the number of vessel delays and the status of demurrage incurred for the receiving part, a second sub-agent may analyze the standard deviation and tank operation efficiency of the mixing part, and a third sub-agent may analyze the energy intensity and production achievement rate of the disassembly part. An aggregator may combine these individual reports, unify the context, remove duplicates, and generate a Daily / Weekly Plant Operation Report. Beyond simple summation, the aggregator can provide the operator with a final result featuring a logical structure that explains the causal relationships between processes, such as, "Although the margin rate of the mixing process decreased due to incoming delays, the overall production target was achieved through the adjustment of the operating rate of the disassembly process."

[0451] FIG. 29 is a drawing showing an Agentic RAG (Retrieval-Augmented Generation) system according to one embodiment of the present disclosure.

[0452] Referring to FIG. 29, the Agentic RAG system may be an agent-based search augmentation generation system in which an agent actively searches for external information sources and contextually integrates the searched information to generate a response. The Agentic RAG system may include an agent that receives a user query, determines whether external information search is required for the query, and controls the search and generation process overall; one or more search tools for searching for information from external knowledge sources such as a vector-based search system, a document database, and a web-based information source; a vector database in which documents collected and processed in advance or information collected in real time is stored in an embedding form and which can perform a search based on similarity with the query; a generator that combines the searched information and the user query to generate a natural language response; and a result output unit that provides the generated response to the user.

[0453] In one embodiment, a user may input a question in the form of natural language. The agent analyzes the user query to determine whether a response can be provided using only internal knowledge or if external information retrieval is required. If the agent determines that a search is necessary, it may invoke one or more search tools to retrieve relevant information from external sources. In one embodiment, the search may include vector embedding-based similarity search, document database search, real-time web content search, etc. The retrieved information is not used as is, but may be configured into a context suitable for response generation through summarization, sorting, duplicate removal, or confidence-based selection processes. The generator receives the user query and the configured context as input and can generate a description or answer in the form of natural language. At this time, the agent controls the entire generation process and may perform additional searches or reconstruction as necessary. The generated response may be provided to the user as the final result. Unlike a structure that simply attaches search results to a prompt, the Agentic RAG system according to this embodiment allows the agent to actively control the search and generation processes.

[0454] In one embodiment, the agent can generate a response with high accuracy and up-to-dateness depending on the nature of the query by determining the necessity of a search, selecting a suitable source among multiple sources, iteratively supplementing search results, and adjusting the contextual construction method.

[0455] According to one embodiment of the present disclosure, a naphtha cracking center scheduling system may include an Agentic RAG (Retrieval-Augmented Generation) system as illustrated in FIG. 29 to utilize text-based knowledge information as well as numerical optimization.

[0456] Referring to Fig. 29, the agent can retrieve and integrate information from external knowledge sources, such as the Vector database. When applied to an NCC system, the system can be equipped with a facility technical support agent. This agent can utilize the vast amount of disassembly operation manuals, accident / failure reports from the past 10 years, and maintenance history data stored in the Vector database. For example, if the third agent (responsible for disassembly) detects a condition where "the coil temperature of Unit 3 is rising abnormally," the Agentic RAG system can be automatically triggered. The RAG agent can search the Vector DB by converting the context of "coil temperature rise and vibration occurrence" into a query.

[0457] At this time, the agent can present to the operator a specific natural language guide by combining (1) searched past similar cases (similar accident in May 2023) and (2) emergency measures from the manufacturer's manual, stating, "The current situation is presumed to be due to intensified fouling, so immediately reduce the feed input by 10% and increase the steam ratio."

[0458] According to another embodiment of the present disclosure, an Agentic RAG system can be utilized to update scheduling constraints by collecting external unstructured information in real time.

[0459] NCC scheduling is sensitive to environmental regulations or market conditions. By applying the structure of Fig. 29, the regulatory monitoring agent can actively search for changes in government environmental regulations (e.g., carbon tax increase, stricter emission standards for specific chemicals) or news from the global petrochemical market (e.g., news of major port strikes) through a web-based information source (Web Search Tool).

[0460] If an agent searches for news stating that "the allowable benzene emission limit will be reduced by 5% starting next month" and determines it to be reliable information, it can interpret this and automatically modify the constraint parameters of the NCC scheduling system.

[0461] This ensures that the AI ​​can generate safe schedules free from compliance risk by autonomously learning and reflecting the latest external information, without the operator having to manually check news and change system settings.

[0462] FIG. 30 is a drawing showing a Multi-Agent Debate (MAD) system according to one embodiment of the present disclosure.

[0463] Referring to FIG. 30, the MAD system may be a system that derives a final response through a discussion-based structure in which a plurality of small language models (SLMs) present opinions from different perspectives on the same user query and perform mutual refutation and verification. In one embodiment, the MAD system may include an aggregator that receives a user query, controls the entire discussion, and determines a final response (Verdict) by synthesizing the discussion results generated by a plurality of small language models; a plurality of small language models participating in the discussion process, which are language models configured to present different inference paths, perspectives, or hypotheses for the same query or intermediate judgment; a tool including a search tool, a vector database, a semantic-based database, etc., which some small language models or aggregators can call for fact verification or evidence reinforcement during the discussion process; and a memory that stores statements at each discussion stage, intermediate judgments (Verdict), previous discussion history, etc.

[0464] In one embodiment, a user inputs a question or request in the form of natural language, and the query may be transmitted to an aggregator. The aggregator may analyze the user query to generate an initial response or hypothesis to initiate a discussion. This initial response may serve as a starting point for the discussion, rather than being a final answer. Multiple small language models may present their respective answers to the initial response or user query. For example, a first small language model may directly present a specific conclusion, a second small language model may present a different conclusion while providing supporting evidence, and a third small language model may present supplementary or rebuttal opinions regarding existing answers. The aggregator may compare and analyze the multiple opinions generated in the first discussion stage to derive an interim judgment. This interim judgment may be used as input for the next discussion stage. Based on the interim judgment, the small language models may participate in the discussion once again. In this stage, some small language models may call a tool to perform fact verification or present additional supporting data. The discussion and judgment process may be repeated until a preset number of times or quality criteria are met. Through the iterative process, the focus of the discussion can gradually narrow, and uncertainty can be reduced. The aggregator can synthesize the results of the repeated discussions, determine the response deemed most appropriate as the final response, and provide it to the user.

[0465] A MAD system according to one embodiment may have a discussion structure that includes mutual refutation and verification between models, unlike a method of simply averaging or selecting multiple model outputs. That is, each small language model operates as an independent decision-making entity, and the aggregator can dynamically adjust the flow of the discussion and the judgment criteria.

[0466] According to one embodiment of the present disclosure, a naphtha cracking center scheduling system may adopt a Multi-Agent Debate (MAD) system as illustrated in FIG. 30 to ensure data security and computational efficiency. Since NCC process data is a core corporate secret, transmission to an external cloud may be restricted. Accordingly, the system may utilize multiple Small Language Models (SLMs) that can be run on an internal factory server (on-premise).

[0467] Referring to Fig. 30, each SLM presents opinions from different perspectives. When applied to an NCC system, the first SLM (production optimization model) can propose an aggressive schedule for profit maximization, while the second SLM (equipment maintenance model) can propose a conservative schedule considering equipment lifespan. For example, if the first SLM suggests raising the reactor temperature to 860 degrees to increase production, the second SLM can rebuttal by stating, "In that case, the coke generation rate will accelerate, requiring operation to be stopped within 3 days, which is a loss in the long run." The aggregator synthesizes this discussion process to derive a balance point between profit and stability (operation at 855 degrees and decoking after 5 days) as the final response (Verdict), which is difficult to derive with a single SLM. This enables high-level decision-making in a closed network environment through the collective intelligence of lightweight models, without the need for high-cost, large models.

[0468] According to another embodiment of the present disclosure, the system may utilize the MAD system as an intelligent alarm system to improve the accuracy of judgment regarding signs of process anomalies. A single model may misidentify noise from a specific sensor as an actual hazardous situation. Accordingly, when an anomaly is detected, a plurality of SLMs may initiate a discussion as follows.

[0469] 1. 1st SLM: "The pressure at Sensor A has surged, posing a risk of explosion. Requesting an emergency shutdown." (Initial Hypothesis)

[0470] 2. 2nd SLM: "I refute this. The readings for flow meter B and thermometer C, which are linked to sensor A, are within the normal range. This suggests that sensor A is likely the only failure." (Rebuttal and presentation of evidence)

[0471] 3. 3rd SLM: "After searching the recent maintenance history using a tool, it was found that Sensor A was replaced yesterday and there is a possibility of a calibration error." (Fact check)

[0472] Based on their discussion, the aggregator can make a final decision to "require inspection of Sensor A (maintain process operation)" instead of halting the factory. Through this cross-validation process, the system can prevent unnecessary factory downtime and maximize operational efficiency.

[0473] FIG. 31 is a diagram showing an A2A (Agent2Agent) protocol system according to one embodiment of the present disclosure.

[0474] Referring to Fig. 31, the A2A protocol system may have a structure in which multiple AI agents (AI Agent 1, AI Agent 2, etc.) cooperate organically with each other through the A2A protocol (Agent2Agent Protocol). First, the central A2A protocol can provide communication protocols to support efficient and secure collaboration between agents. Specifically, the A2A protocol may have the following characteristics. First, the system can provide a function that allows communication by exchanging only necessary information without directly sharing each other's source data. This enables collaboration while protecting the data privacy of each agent. Second, the system can support the distribution of complex tasks among multiple agents and the performance of negotiations to derive optimal results. Third, through state maintenance characteristics, the system can manage agents so that they can maintain and update shared context and state information in real time.

[0475] In one embodiment, the first AI agent (AI Agent 1) may primarily be responsible for local-based data processing and search tasks in the system. The first AI agent may have an independent technology stack and, for example, may run based on a language model (LLM) on the ADK framework and may use a database for data management. The first AI agent may be connected to various external function servers via the Model Context Protocol (MCP). In the example of FIG. 31, the first AI agent may access local files (documents, images, videos, etc.) or perform search tasks through a specific platform via the MCP server.

[0476] In one embodiment, the second AI agent (AI Agent 2) may be configured to primarily handle cloud infrastructure utilization and external communication tasks. The second AI agent may also have a technology stack independent of the first agent. The second AI agent may also connect to various MCP servers using the MCP protocol. Through this, the second AI agent can control resources of a cloud service provider (AWS, Azure, etc.) or perform communication tasks through collaboration and messaging tools (Slack, Whatsapp, etc.).

[0477] In a system according to one embodiment, the Model Context Protocol (MCP) can maximize the scalability and flexibility of the system by separating each detailed function (file access, search, cloud control, communication, etc.) into an independent server form to communicate with the agent. As a result, the A2A protocol system illustrated in FIG. 31 can ensure that each agent operates independently while maintaining its own language model, framework, and database. At the same time, the system can provide the advantage of simultaneously securing strong security and high-level task processing capabilities by enabling agents to coordinate necessary tasks and collaborate without direct data exposure between them.

[0478] According to one embodiment of the present disclosure, a naphtha cracking center scheduling system may adopt an Agent-to-Agent (A2A) protocol system as illustrated in FIG. 31 for secure collaboration with external partners (e.g., shipping companies, raw material suppliers). In the NCC process, the supply of raw materials from external sources is a key element of scheduling, but each company is reluctant to share internal inventory levels or detailed vessel operation logs for security reasons. Referring to FIG. 31, the A2A protocol allows for the exchange of only necessary information without sharing source data.

[0479] The first AI agent (the receiving agent on the NCC side) and the second AI agent (the logistics agent on the shipping company side) can conduct negotiations through the A2A protocol. For example, based on the internal context that "receiving is required within 10 days due to a current inventory shortage," the first agent can query the second agent solely regarding "the possibility of arrival on X." The second agent can internally analyze vessel location data in the cloud without directly exposing it, and instead propose a negotiation proposal stating, "Day X is impossible, but Day X+1 is possible, and in this case, a freight rate discount is available." The first agent can receive this proposal, run internal simulations, and then decide whether to accept it. By introducing the A2A protocol in this manner, the system can achieve cooperative scheduling that optimizes the efficiency of the entire supply chain while perfectly protecting the privacy of sensitive data between companies.

[0480] FIG. 32 is a drawing showing an Agentic RAG (Retrieval-Augmented Generation) system according to one embodiment of the present disclosure.

[0481] Referring to FIG. 32, the Agentic RAG system may include a system that extracts data from external sources, such as websites, stores it in a Vector Database (Vector DB), searches for optimal information regarding user queries, and generates intelligent responses through a Large Language Model (LLM). In particular, the Agentic RAG system may support advanced question-response by including agentic elements such as memory, tools, and planning, going beyond simple search and generation functions. The pipeline of the Agentic RAG system according to one embodiment may be broadly divided into a data extraction stage, a search stage, and a generation stage.

[0482] In one embodiment, the data extraction step is a step of building a knowledge base from an external source, and the system may collect data from a designated website or various web content. The content collected may include not only text but also multimodal data in various formats, such as images, audio, and video. The system extracts text and metadata from the collected content and may chunk it into small units for efficient processing. Each chunked piece is vectorized through an embedding model and then stored in a vector database (Vector DB) to be utilized for future searching.

[0483] In one embodiment, when a user's query is input during the search phase, the system may perform a search optimization process. For example, the system may perform query rewriting to clarify the user's intent, followed by a query rewriting and embedding step that vectorizes the query; a similarity search step that searches for information fragments most similar to the user query within a vector database; and a context configuration and ranking step that selects and configures the most suitable context for response generation by performing relevance-based ranking based on the search results.

[0484] In one embodiment, during the generation phase, the system can derive a final response by combining the extracted context and agent functions. For example, an input prompt for the LLM can be generated by sophisticatedly combining the user query and the retrieved context. Furthermore, the Large Language Model (LLM) can generate an optimal response by performing functions such as memory functions (maintaining past conversational context), tool calling (utilizing external APIs), and planning (designing complex query resolution procedures), rather than simply generating text. The finally generated intelligent response can be delivered through a user interface.

[0485] The Agentic RAG system according to the present disclosure enables the generation of contextually accurate answers through architecture-based search accuracy enhancement, and can be utilized for automating complex tasks that are difficult to solve through simple search by utilizing planning and external tool calling functions. Furthermore, it allows for the processing of various forms of content, such as images and videos, in addition to text, thereby enabling broad information utilization. Additionally, since the Agentic RAG system can collect and utilize real-time information from websites, it can be widely applied to technical support requiring up-to-date information, search engine enhancement, personal assistant services, and the like.

[0486] According to another embodiment of the present disclosure, the system may include an Agentic RAG (Retrieval-Augmented Generation) system as illustrated in FIG. 32 to incorporate web-based real-time market information into scheduling.

[0487] Referring to Fig. 32, the system builds a knowledge base by collecting data from external sources such as websites. When applied to an NCC environment, the system can collect "global petrochemical news," "status of major port strikes," "reports on competitor shutdowns," etc., in real time and embed them into a vector database (Vector DB).

[0488] The Agentic RAG pipeline can be activated when a central agent responsible for scheduling performs the task of "establishing an ethylene production strategy for the next month." The agent queries "recent ethylene supply issues in the East Asia region" to search for relevant news in the Vector DB (Search stage), obtains information from the search results stating that "Competitor A's scheduled maintenance has been extended by one week" (Planning stage), and based on this, can generate a strategic decision such as "Generate an aggressive schedule to increase the cracking furnace operating rate from 95% to 100%, as a temporary price increase is predicted due to the competitor's supply disruption" (Generation stage). This enables the implementation of a Market-Adaptive AI system that goes beyond simple numerical optimization to interpret vast amounts of unstructured information (text, news videos, etc.) on the web in real time and reflect it in scheduling.

[0489] FIG. 33 is a schematic diagram of an AI agent system according to one embodiment of the present disclosure.

[0490] Referring to Fig. 33, the AI ​​agent system may include an advanced architecture that goes beyond a simple question-and-answer structure to autonomously interpret and execute the user's intent.

[0491] In one embodiment, a user can transmit commands to an agent system through system prompts and user prompts. Here, the system prompt defines the agent's role, persona, behavioral guidelines, and constraints, while the user prompt may include specific queries or requests for performing specific tasks. Based on the input prompts, the AI ​​agent system can generate an optimal response by organically utilizing complex internal functions.

[0492] In one embodiment, the AI ​​agent system may support a planning function that identifies sub-tasks necessary to achieve an input complex goal and autonomously establishes execution priorities and sequences; a reasoning function that performs logical judgments based on the established plan and given data and derives the optimal solution for problem solving; a tool calling function that handles actual tasks by dynamically selecting and executing various tools such as external API integration, software execution, and search engine utilization; a memory storage function that ensures information consistency and supports intelligent judgment by storing and maintaining the results of previous steps or contextual information of conversations; a function that queries necessary information from large-scale data sources or systematically stores and manages generated data; and a feedback collection function that analyzes results generated during the execution process and forms a feedback loop to improve performance through continuous learning and behavior correction.

[0493] An agent system according to one embodiment of the present disclosure can generate an advanced final response by repeating the actions of planning, calling tools, storing in memory, and collecting feedback. Since such a system can dynamically select tools through an autonomous planning and execution structure and enables continuous improvement through a feedback loop, it can be widely used for complex multi-step tasks, business process automation, professional research, etc. Consequently, the system can function as an intelligent autonomous agent that goes beyond a simple input-output model.

[0494] According to one embodiment of the present disclosure, a naphtha cracking center scheduling system may adopt an Autonomous AI Agent System structure as shown in FIG. 33 to autonomously perform complex multi-stage process control.

[0495] Referring to Fig. 33, the system generates an advanced response by repeating planning, tool calling, and feedback collection. When applied to an NCC environment, the user can assign a persona to "maximize profit while complying with safety constraint Grade A" via a system prompt and request "create next week's schedule" via a user prompt.

[0496] Accordingly, the AI ​​agent system breaks down the entire scheduling task into sub-tasks such as raw material supply analysis, blending optimization, and load distribution for decomposition, establishes a sequence (planning), dynamically selects and executes external tools such as an NCC process simulator, a profit calculator, and a constraint verifier according to the plan (tool calling), stores simulation results (intermediate outputs) in memory, logically determines that "profit is high but an inventory shortage is expected on the third day" (inference and storage), modifies the plan itself to solve the expected problem, and can call the tool again with measures such as increasing the incoming quantity on the third day (feedback). In this way, the present embodiment can implement an autonomous scheduling system that corrects errors and finds the optimal solution through a feedback loop, rather than a simple input / output model.

[0497] FIG. 34 is a schematic diagram of a Large Language Model (LM) chatbot system according to one embodiment of the present disclosure.

[0498] Referring to Fig. 34, an LLM chatbot system may have a basic structure that receives user input and generates an output through Large Language Models. A query entered by a user can be transmitted to a Large Language Model. At this time, the model can improve the accuracy of the response by referencing Enterprise Data in various formats, such as documents, images, and videos. The system may be composed of a single LLM call process without complex agent functions (planning, tool calling, etc.). In other words, the flow from input to output is unidirectional, and the system may not include separate external tool integration or complex work pipelines. A system with such a simple structure can provide efficiency suitable for simple question-and-answer, FAQ systems, and basic customer service, where fast and accurate information delivery is key rather than complex logical reasoning. However, since the LLM chatbot system lacks a process for collecting feedback or iterative plan modification, it may lack the ability to maintain detailed context during long conversations or may have limitations in autonomously performing complex tasks that require multiple steps.

[0499] According to another embodiment of the present disclosure, the system may provide an LLM chatbot system as illustrated in FIG. 34 as a separate interface to provide rapid and accurate process information to the operator.

[0500] Referring to Fig. 34, the LLM chatbot system can provide immediate answers to queries by referencing corporate data without a complex planning process. When applied to an NCC system, the system can include a process monitoring chatbot.

[0501] When an operator inputs, "Tell me the current coil exit temperature (COT) and trend of Unit 3 cracking furnace," the chatbot skips complex reasoning processes, immediately accesses the NCC real-time sensor log to extract the corresponding value, and can generate and output an accurate answer such as, "The current COT of Unit 3 is 845 degrees, and it has been trending upward by 2 degrees over the past hour."

[0502] Unlike the autonomous agent in Fig. 33, this structure has very low latency and fast response speed, so it can be specialized for FAQ systems, daily briefings, and field inspection support that require urgent situation identification or simple information retrieval, thereby increasing operational efficiency.

[0503] FIG. 35 is a schematic diagram of a Robotic Process Automation (RPA) system according to one embodiment of the present disclosure.

[0504] Referring to FIG. 35, the RPA system may include a structure that performs structured tasks according to predefined rules using user input as a trigger. In one embodiment, the RPA system can execute rule-based triggers. When user input occurs, the RPA system may initiate an operation according to a pre-set logical structure (such as If-Then). At this stage, rather than flexible judgment, a fixed tool section may be invoked to perform a defined procedure. In one embodiment, the RPA system may use LLA secondarily. Data that has passed through the rule-based triggers may be passed to a Large Language Model (LLM). Here, the LLM may be utilized in a limited, secondary role of refining results or performing simple processing by referencing a fixed tool section or a database. In one embodiment, since the RPA system operates consistently according to defined rules, it can be optimized for handling repetitive automation tasks or simple back-office tasks. The RPA system can demonstrate high efficiency in workflows where data input and output are relatively clear.

[0505] In other words, the RPA system can implement an automation architecture that ensures business stability and repeatability by leveraging the language processing capabilities of LLM while retaining overall control over fixed rules.

[0506] According to one embodiment of the present disclosure, a naphtha cracking center scheduling system may include a Robotic Process Automation (RPA) system, as illustrated in FIG. 35, as a post-processing module to automate the execution procedure of a confirmed schedule. Referring to FIG. 35, the RPA system can perform standardized tasks according to predefined rules. When applied to an NCC system, the RPA system can be triggered at the moment an AI agent generates and confirms an optimal receiving / production schedule. Specifically, the RPA system can execute tool sections such as ERP input, which parses the generated schedule data and automatically enters it into the production planning module of the in-house ERP system; sending purchase orders, which drafts and sends an 'Order Email' to be sent to a supplier according to the determined raw material receiving plan; and creating daily reports, which converts a summary of today's production plan into a PDF and sends it via messenger to the plant manager and relevant department heads. At this time, an auxiliary LLM within the RPA system can perform an auxiliary role, such as politely refining the email wording or generating ERP input comments. By combining RPA in this way, the system can maximize work efficiency by entrusting scheduling requiring high-level judgment to AI and repetitive administrative processing to RPA.

[0507] FIG. 36 is a schematic diagram of a Retrieval-Augmented Generation (RAG) system according to one embodiment of the present disclosure.

[0508] Referring to Fig. 36, the RAG system is broadly composed of three stages—Retrieval, Augmentation, and Generation—to derive accurate responses based on external knowledge. When user input occurs during the retrieval stage, the RAG system can embed and vectorize it. Subsequently, the RAG system can search for data highly relevant to the query among the numerous information rules stored within the Vector DB and Database. The Augmentation stage is a step in which the input is reinforced based on the retrieved data; the RAG system can utilize external tools and combine the retrieved knowledge data with existing System Prompts and User Prompts to generate an augmented prompt in a form most suitable for processing by the LLM. The augmented prompt can be passed to the Large Language Model (LLM) during the Generation stage. The LLM can generate a final response (Output) by adding the retrieved external context to the learned internal knowledge and provide it to the user.

[0509] In one embodiment, the RAG system generates correct answers by referencing external knowledge (documents, etc.) in real time, so it can be utilized to improve the accuracy of simple LLM or for professional QA systems, document search, and summarization services. Additionally, the RAG system can reflect the latest information without retraining the model and can suppress hallucination because it finds the basis for the answer in external data.

[0510] The system may include a Retrieval-Augmented Generation (RAG) system as illustrated in FIG. 36 to provide the basis for scheduling results and to accurately answer operator queries. Referring to FIG. 36, the RAG system retrieves relevant information from a vector database to augment the response. When applied to an NCC environment, the system can vectorize and store 'Standard Operating Procedures (SOPs)', 'Material Safety Data Sheets (MSDS)', 'Safety and Environmental Regulations', etc.

[0511] If an operator asks, "Does increasing the pressure of Unit 3 by 5% according to the currently determined schedule violate safety regulations?", the RAG system does not answer solely based on the LLM's internal knowledge (prevention of hallucinations), but can search for 'Chapter 3, Section 4 of the High Pressure Operation Safety Guide' in the Vector DB. The RAG system then configures the retrieved document into an augmented prompt, enabling the LLM to generate an accurate, evidence-based answer such as, "According to Chapter 3, Section 4 of the Safety Guide, the safety valve setpoint must be readjusted upon pressure increase; therefore, failure to take such action constitutes a potential violation of regulations." This suppresses the hallucination phenomenon where AI provides plausible answers based on false information, thereby ensuring reliability in chemical processes where safety is paramount.

[0512] FIG. 37 is a schematic diagram of a Learning-Augmented Mechanism (LAM) system according to one embodiment of the present disclosure.

[0513] Referring to FIG. 37, the LAM system may include an intelligent automation architecture that combines user intent with learned behavioral patterns to lead to the actual execution of a task. In the training data stage, when user input occurs, the LAM system may refer to pre-trained data. The training data utilized at this time includes screenshots of specific task execution or tool usage data, which can serve as a basis for the agent to learn how to operate in a real user environment. In one embodiment, a large language model may process the task based on the training data. The LLM may call external tools or access a database to refine the task if necessary, and can derive executable commands beyond simply generating language. The optimal path determined by the model leads to actual action, and finally, a response indicating the completion of the task can be output to the user.

[0514] In one embodiment, the LAM system uses a model trained by directly utilizing tool usage data, thereby enabling behavior-based learning and execution in an actual interface environment. Additionally, because the LAM system learns user operation patterns, it can demonstrate high efficiency in the field of repetitive GUI (Graphical User Interface) task automation.

[0515] According to one embodiment of the present disclosure, a naphtha cracking center scheduling system may include a Learning-Augmented Mechanism (LAM) system as illustrated in FIG. 37 as an execution module to automatically control legacy control systems that are difficult to integrate with APIs. Referring to FIG. 37, the LAM system performs tasks by learning user input (screenshots, tool usage). When applied to an NCC environment, the system can visually learn action patterns for a skilled operator to adjust valves or set temperatures on the GUI (Graphical User Interface) screen of the control room. For example, if a scheduling agent issues a command to "set the temperature of Unit 3 to 850 degrees," but the equipment does not support APIs, the LAM system can operate. The LAM agent can automatically perform a series of GUI operations, such as capturing and analyzing the current control screen, moving the mouse cursor to the 'temperature setting field' according to the learned pattern, entering 850, and clicking a confirmation button. This allows aging process equipment to be perfectly integrated with the latest AI scheduling system through visual learning alone, without the need for separate hardware replacement or high-cost system integration projects.

[0516] A summary of the AI ​​system mentioned above is as follows [Table 3].

[0517] [Table 3]

[0518]

[0519] FIG. 38 is a diagram illustrating the memory structure of an AI agent and a question-and-answer process utilizing the same according to an embodiment of the present disclosure. Referring to FIG. 38, the AI ​​agent can utilize a memory unit to perform advanced decision-making on an input query and generate an optimal response. The memory is divided into short-term and long-term memory and operates hierarchically, supporting complex decision-making and task execution through various types of memory. Short-term memory is a temporary memory space for focusing on the currently ongoing workflow and may include a working memory that tracks and manages the inference process and real-time work flow for each workflow, and a cache memory that supports immediate access to data or result values ​​frequently used in the current workflow. Long-term memory is a knowledge and experience-based memory that is continuously preserved and may include episodic memory, which manually or automatically stores specific events or incidents that occur during workflow execution; semantic memory, which stores conceptual and factual knowledge such as "Paris is the capital of France"; and procedural memory, which stores methodologies and procedural knowledge for performing specific actions or tasks, such as "How to reset a server," which stores methods for performing tasks or procedural knowledge (e.g., "How to reset a server").

[0520] In one embodiment, the central memory controller can operate in conjunction with a Language Model Framework. When a query is input, the Language Models within the framework refer to memory to analyze the query, and based on the analyzed information, search for additional relevant information in memory if necessary, and generate a final response through a decision loop. Both the input query and the output response can support various data formats, such as text, documents, images, and videos.

[0521] In one embodiment, the MCP server is responsible for interfacing with external knowledge and tools and may include a Vector DB that stores search-based embedding information, a Semantic DB that is a conceptual knowledge base, and the like. Additionally, the MCP server can perform functions by exchanging data in real time with external services such as Stripe (payment), Slack (communication), and Brave search (web search) through a Third Party API integration unit. Consequently, an AI agent equipped with the memory structure of the present disclosure can autonomously perform complex multi-step tasks through the harmonious utilization of short-term and long-term memory, and can achieve practical business automation through integration with external tools.

[0522] In one embodiment, the process can be handled as follows.

[0523]

[0524] One embodiment of the present disclosure can systematically integrate design elements essential for implementing an agent system that can continuously learn and make decisions according to the situation, as shown in [Table 4] below.

[0525] [Table 4]

[0526]

[0527] Next, we would like to explain the types of language models utilized by the AI ​​agent system. Each model has a specific processing method and role, and a suitable model can be selected and utilized depending on the nature of the task. FIG. 39 is a diagram illustrating the text generation process of a first language model according to an embodiment of the present disclosure.

[0528] Referring to FIG. 39, the first language model is a general-purpose language model pre-trained based on large-scale text data, and can generate text through the following step-by-step process.

[0529] (1) Pretraining stage (Pretrain): The first language model can acquire the structure and knowledge of the language by training the model in advance using a large text corpus.

[0530] (2) Prompt encoding step: The first language model can convert the input prompt into a vector form that the model can process through tokenization and embedding processes.

[0531] (3) Transformer layer processing step: The first language model can process the encoded input through a transformer layer to generate a hidden state containing context information.

[0532] (4) Logit and probability calculation step: The first language model can calculate the logit and occurrence probability for the next token based on the generated hidden state.

[0533] (5) Token sampling and selection step: The first language model may sequentially sample or select one of the top K (Top-k) tokens based on the calculated probability.

[0534] (6) Decoding and output step: The first language model can decode the selected tokens into text form and output the final response text.

[0535] According to one embodiment of the present disclosure, a naphtha cracking center scheduling system may include a first language model (Pre-trained LLM) as illustrated in FIG. 39 for natural language communication with an operator and for providing general engineering knowledge. Referring to FIG. 39, the first language model pre-trains a large-scale text corpus. When applied to an NCC system, the corpus may include chemical engineering textbooks, NCC facility operation manuals, petrochemical-related patents and papers, etc. By utilizing the first language model pre-trained with such domain-specific data, the system can understand the context through prompt encoding and transformer processing, even if an operator inputs an ambiguous query such as, "What are the expected problems when the specific gravity of naphtha is 0.75?" Subsequently, through a token selection process, the model can generate an answer with clear chemical causal relationships, such as, "If the specific gravity is high, there is a possibility that the aromatic content is high, which may accelerate coking." The model can explain and guide general phenomena of the process based on pre-trained knowledge without the need for separate rule input.

[0536] FIG. 40 is a diagram illustrating the operation process of a second language model according to one embodiment of the present disclosure.

[0537] Referring to FIG. 40, the second language model may include a Mixture of Experts (MoE) model. The Mixture of Experts (MoE) model may include a distributed model structure that selectively activates only some experts (sub-networks) of the entire network depending on the input data. The second language model can perform a network training step in which multiple sub-networks and a gating or routing network for assigning each token to an appropriate expert are trained together; an input encoding step in which the input is tokenized and embedding to convert it into a vector form that the model can process; an expert selection step in which the gating network selects the top-k most suitable expert sub-models for each token in real time; a conditional execution step in which input data is forwarded only to the expert sub-models selected by the gating network to perform operations; a result merging step in which the individual output values ​​derived from the selected experts are merged into one using a weighted average or aggregation method; and a final decoding step in which decoding is performed based on the merged results to generate final text data. This MoE model has the advantage of very high computational efficiency relative to the number of parameters, as it maintains a large overall parameter scale while activating only some expert networks during actual inference.

[0538] To efficiently process complex and diverse NCC process data, the system may adopt a second language model based on Mixture of Experts (MoE), as illustrated in Fig. 40, as a core inference engine. The expertise required for the NCC process varies depending on the receiving, mixing, and decomposition stages, and the optimal operating conditions differ according to the characteristics of the feed. Referring to Fig. 40, the MoE model can select appropriate experts through a gating network. When applied to NCC scheduling, the system's gating network can identify what data needs to be processed at the moment.

[0539] For example, if the input is a ship unloading schedule, the system can activate the logistics expert sub-network (Expert A); if the input is the thermal cracking of paraffinic naphtha, the system can activate the paraffin reaction expert sub-network (Expert B); and if the input is the thermal cracking of naphthenic naphtha, the system can activate the naphthenic reaction expert sub-network (Expert C).

[0540] In this way, instead of using all parameters, the system can perform computations by activating only the top K experts that are perfectly suited to the current situation through conditional computation. This allows for the handling of large-scale NCC scheduling problems with tens of thousands of variables while drastically reducing inference latency and obtaining precise prediction results optimized for each process characteristic.

[0541] FIG. 41 is a diagram illustrating a method of operation of a third language model according to one embodiment of the present disclosure.

[0542] Referring to Fig. 41, the third language model may include a Large Reasoning Model (LRM). Beyond simply generating results, the LRM can autonomously generate and verify a chain of thought to solve complex problems. The LRM recognizes the input query and related context by dividing them into tokens that the model can process. In the initial stage, it internally generates chain of thought steps, which are logical steps for problem solving. Through a reflection layer, it evaluates and prioritizes various possible reasoning paths generated. If the evaluation results are insufficient or logical errors are found, it can selectively regenerate the reasoning steps to enhance logical completeness. Based on the verified optimal reasoning path, the LRM determines the final logical answer and outputs the final correct answer to the user, which is sufficiently supported by internal reasoning processes. Through this deep thinking process, the LRM can provide high reliability in fields such as solving high-difficulty mathematical and coding problems and logic-based question answering, which were difficult for existing language models to solve.

[0543] According to one embodiment of the present disclosure, a naphtha cracking center scheduling system may include a third language model based on a Large Reasoning Model (LRM), as illustrated in FIG. 41, to analyze the root cause of complex process anomalies. Referring to FIG. 41, the LRM generates a chain of thought and verifies it through a reflection layer. This can be applied to a fault diagnosis agent of an NCC system. For example, in a situation where the yield of the cracking furnace drops sharply, a conventional model may simply issue a one-dimensional prescription of raising the temperature. The LRM can provide a highly reliable solution in high-difficulty process troubleshooting situations by not only performing numerical optimization but also inferring engineering causal relationships step-by-step and re-verifying its own judgment.

[0544] FIG. 42 is a diagram illustrating a method of operation of a fourth language model according to one embodiment of the present disclosure.

[0545] Referring to FIG. 42, the fourth language model may include a Vision Language Model (VLM). The VLM is a multimodal model capable of integrally understanding and processing data of different formats, such as images and text. It performs image encoding to encode input image data and extract visual feature information that the model can process, performs text tokenization to recognize text inputs such as user queries or context by dividing them into tokens, combines the encoded visual information and the tokenized text information to generate a single integrated embedding, and the generated integrated embedding passes through a joint transformer layer to learn deep correlations between the two modalities. The VLM can generate complex inference by cross-referencing between visual information and language information. Additionally, the VLM can decode the final answer based on the integrated representation and output it to the user.

[0546] A VLM according to one embodiment can demonstrate superior performance compared to existing language models in the field of advanced question answering and image description generation including visual information through a multimodal integration mechanism.

[0547] According to another embodiment of the present disclosure, the system may include a fourth language model based on a Vision Language Model (VLM) as illustrated in FIG. 42 to utilize visual information of the site that sensor data cannot capture. Referring to FIG. 42, the VLM combines images and text to generate an integrated embedding. This can be applied to a visual monitoring agent of the NCC system. The condition inside the NCC furnace is difficult to fully determine using only temperature sensors, and the color or shape of the burner flame can serve as an important indicator.

[0548] The VLM can receive visual images captured internally through disassembly and real-time temperature sensor data (text input). The VLM performs image encoding by extracting visual features such as the flame being close to red and fluctuating irregularly. It learns through the Joint Transformer layer that while the temperature appears normal in text, there are clear visual signs of incomplete combustion. Consequently, in situations where the system would be determined to be normal based solely on sensor data, it can generate a sophisticated diagnosis based on visual information, stating, "Inspection required due to suspected burner tip blockage."

[0549] FIG. 43 is a diagram illustrating a method of operation of a fifth language model according to one embodiment of the present disclosure.

[0550] Referring to FIG. 43, the fifth language model may include a Small Language Model (SLM). As a language model that operates efficiently even in environments with limited computational resources through a lightweight structure, it performs input tokenization by dividing input data into token units that the model can process, performs low-dimensional embedding by projecting the divided tokens into a low-dimensional space rather than a high-dimensional space to reduce computational complexity, processes a lightweight transformer that improves computational speed by processing context through a relatively small number of transformer layers, calculates the probability of occurrence for the next token, samples or selects the optimal token based on the calculated probability, and decodes and outputs the final text. Through this optimized structure, such an SLM can enable low-power, high-speed inference in edge devices or on-device AI environments.

[0551] According to one embodiment of the present disclosure, a naphtha cracking center scheduling system may install a fifth language model based on a Small Language Model (SLM) as shown in FIG. 43 on an edge device to ensure real-time control without communication delay and safety against network failures.

[0552] Referring to Fig. 43, the SLM can perform low-power, high-speed inference with a lightweight structure. When applied to an NCC site, while the central server's massive model (LLM) establishes the overall 24-hour schedule, specialized SLMs can be installed in each decomposition controller or smart valve at the site to operate independently. For example, in an emergency situation where communication with the central server is lost and the pressure in Unit 3 rises rapidly, the SLM installed in Unit 3 can make a decision on its own without waiting for instructions from the central server. The SLM can immediately infer, "Although communication is lost, the current pressure data pattern is 99% similar to signs of an explosion," and generate and execute a local command to "open the emergency pressure relief valve." This overcomes the communication dependency that is a weakness of centralized AI and enables the establishment of a distributed AI control ecosystem where thousands of facilities possess independent intelligence.

[0553] FIG. 44 is a diagram illustrating the operation method of a Large Action Model (LAM) according to one embodiment of the present disclosure.

[0554]

[0555] Referring to FIG. 44, LAM is a model designed to perform specific actions in a real or virtual environment beyond simple text generation. It receives inputs of task descriptions and current environmental state information, tokenizes the input descriptions and environmental information, and performs context embedding. Additionally, LAM autonomously designs a sequence of actions to achieve a goal based on the Chain-of-Thought technique, generates API calls, command calls, or control signals necessary to execute planned actions, and executes actions within the real environment based on the generated signals. LAM monitors execution results and performs modification and iteration procedures as needed to derive optimal results by modifying the plan or revising the task as necessary. In one embodiment, LAM can be used in robot control, game agents, automation systems, etc.

[0556] FIG. 45 is a diagram illustrating the operation method of a Hierarchical Reasoning Model (HRM) according to one embodiment of the present disclosure.

[0557] Referring to Fig. 45, the HRM is a model that maximizes problem-solving ability and efficiency by hierarchically separating and processing complex inference tasks. It encodes input data and task contexts to convert them into internal representations, establishes an abstract and macroscopic plan for overall task resolution in the upper layer, the H-module, and performs specific and detailed operations based on the high-level plan in the lower layer, the L-module. The HRM can repeat a feedback loop between the H-module and the L-module until the task is completely completed, undergoing an iterative coordination process to ensure that the high-level plan and the results of the low-level operations match and converge (Hierarchical convergence loop). Based on the converged final state, the HRM can decode and output a logical final answer. Through this hierarchical structure, the HRM can demonstrate outstanding performance in complex multi-step planning and high-difficulty inference tasks that are difficult for a single-layer model to solve.

[0558] According to one embodiment of the present disclosure, a naphtha cracking center scheduling system may include a Hierarchical Reasoning Model (HRM) as illustrated in FIG. 45 as a core reasoning engine to convert abstract management goals into specific facility operation variables. For example, the H-module (strategy layer) may establish a macroscopic and abstract plan such as "the goal for this quarter is to increase ethylene production by 10% and reduce energy costs by 5%," and the L-module (execution layer) may receive the plan from the H-module and perform specific operations such as "raising the temperature of cracking furnace No. 3 from 840 degrees to 850 degrees and increasing the quenching pump speed by 3%." If the L-module calculates that "raising the temperature to 850 degrees increases energy costs by 7%, violating the H-module's goal (5% reduction)," this feedback is passed back to the H-module, and the H-module can modify the goal (relaxing the energy cost reduction goal) or guide the L-module to find other alternatives (utilizing other decomposition), thereby ultimately establishing a final schedule in which management goals and physical control align without contradiction.

[0559] FIG. 46 is a drawing showing a ToolFormer (Tools-trained Model) according to one embodiment of the present disclosure.

[0560] Referring to Fig. 46, ToolFormer is a language model trained to use various external tools. It starts based on a pre-trained LLM, samples examples of tool calls, determines valid tool usage through test and evaluation, and performs fine-tuning with filtered data. ToolFormer has advantages in integration with calculators, searches, API calls, etc.

[0561] Table 5 summarizes the four major types of artificial intelligence systems—basic LLM workflow, RAG (Retrieval-Augmented Generation), single AI agent, and multi-agent based Agentic AI—in terms of structure, function, characteristics, and use cases.

[0562] [Table 5]

[0563]

[0564] FIGS. 47 to 52 illustrate vulnerabilities of an MCP according to an embodiment of the present disclosure. Referring to FIG. 47, a command injection problem may occur in an MCP system. An attacker may insert a hidden command that an agent can interpret into a prompt entered by a user. The agent mistakes this for a legitimate request and accesses external resources through the MCP server. In this process, unauthorized access to databases and external servers may occur. Referring to FIG. 48, a tool poisoning vulnerability may exist in an MCP environment. An attacker may register a malicious tool containing malicious code on the MCP server or contaminate an existing tool. When the tool is executed, the agent performs actions intended by the attacker. For example, access rights to a trusted service such as Slack may be granted to the attacker, thereby allowing API keys or personal information to be leaked externally.

[0565] Referring to Fig. 49, structural problems related to Server-Sent Events (SSE), the communication method of MCP, may occur. Since the SSE method transmits data by dividing it into multiple segments, the communication connection must remain open for a long time. When interoperating with external services, system delays may occur as the connection is maintained excessively, and there is a risk of exposure to additional security risks, such as interception through the open connection channel.

[0566] Referring to Fig. 50, a privilege escalation issue caused by a malicious tool may occur. A malicious server installed within the MCP server can intercept calls directed to other trusted tools or override their functions. Even if a user calls a legitimate tool such as Slack, a malicious tool overridden by an attacker may actually be executed, thereby stealing the user's trust privileges.

[0567] Referring to Fig. 51, there may be security threats related to persistent context maintained between sessions. MCP records and maintains conversational context and context across sessions to improve user experience, but if an attacker tamperes with this recorded context, session information of integrated tools such as AWS, Kagi, and Notion may be corrupted, causing the agent to make incorrect decisions or rendering security settings ineffective.

[0568] Referring to Fig. 52, a problem of server data theft may occur on the client side. There is a risk that an attacker who has infiltrated the client may use a malicious MCP tool to gain control of connected tool servers, thereby transmitting or stealing sensitive credential information, such as user data or chat passwords stored in external services, to another malicious server.

[0569] FIG. 53 is a diagram illustrating a context engineering structure in an AI agent system according to one embodiment of the present disclosure.

[0570] Referring to Fig. 53, the AI ​​agent system may have context engineering and a memory hierarchy to generate situation-optimized responses by sophisticatedly configuring and utilizing various contextual information generated during the interaction process. When a user inputs a query or request in various forms, such as text, voice, or images, the agent can analyze the input information and the current context to formulate a plan and determine the direction of the task. The RAG module collects relevant information by searching for similar documents in a vector database when necessary, and the system calls action tools according to the requested task to obtain specific execution results. It can also dynamically configure or update the final prompt by integrating the collected search results, tool execution values, and the agent's inference content. Additionally, the system can generate a final response based on the configured prompt and deliver it to the user. All contextual elements within the session (input, tool usage history, inference, etc.) are recorded in 'Chat History,' and specific contexts or results with high reusability can be added to long-term memory based on an MCP server or database to be utilized for future interactions.

[0571] In one embodiment, the prompt may be composed of user input corresponding to a query or command, tool usage results including API call results, calculation results, etc., search-based context including documents retrieved from the RAG, agent reasoning including internal reasoning and planning, user information including preferences, ID, status, etc., conversation history including the previous conversation context, etc.

[0572] In one embodiment, the prompt may be deleted based on priority when the maximum prompt length is exceeded. For example, the priority may be configured in the order of agent inference > search context > tool results > user information > past conversation history. In one embodiment, result values ​​after a tool call may be inserted into the prompt in the form of a "contextual tag." Additionally, the search context may be inserted along with a summary and confidence score, rather than the original text.

[0573] In one embodiment, short-term memory is intended to maintain the entire context within a session and may store user input, prompt components, reasoning processes, tool usage results, etc. Short-term memory is deleted upon session termination, but important information may be transferred to long-term memory.

[0574] In one embodiment, the long-term memory may be composed of an MCP server (Agent Metadata) and a domain knowledge DB (Structured Knowledge), etc. The long-term memory may be updated when the Add to memory command is executed or when automatic saving conditions are satisfied. For example, information such as "User A prefers tools related to 'data visualization'" and "On August 7, 2025, the 'RAG + Tools' path was used in the 'Context Engineering' flow" may be stored in the long-term memory.

[0575] In one embodiment, the agent can determine whether to call a tool, the necessity of searching, and the possibility of repeated calls. Additionally, the agent can perform priority-based reasoning. For example, priorities may proceed in the order of user goal → environment state → available resources → execution strategy. In one embodiment, the agent can support parallel calls to multiple tools and support feedback-based iterative execution after execution.

[0576] For example, in the case of multimodal question and answer, when a user image is uploaded, a corresponding description is generated, and a date corresponding to the user image can be calculated via a tool call.

[0577] As another example, in the case of report generation, the search and summarization process proceeds based on user instructions, and templates can be inserted and edited.

[0578] As another example, in the case of automated schedule coordination, the calendar API is invoked based on natural language requests, and schedule recommendations can be provided.

[0579] According to one embodiment, a multi-agent-based Agentic AI extension structure may be supported. Additionally, according to one embodiment, a prompt dynamic optimization (auto-slimming) algorithm may be implemented. Furthermore, a memory vectorization-based summary storage module may be constructed, and user-specific customized context weighting profiling may be performed.

[0580] In particular, AI agents are characterized by autonomy and independence. Autonomous agents act according to their own rules without external intervention; a representative example is Automated Guided Vehicles (AGVs) in the robotics field, which recalculate their own paths based on real-time order data. Furthermore, they possess decentralized characteristics, allowing agents to make independent decisions without central control. This is implemented in the robotics field by multi-robot systems establishing collaborative path planning using only local sensor data.

[0581] FIG. 54 is a block diagram of a system according to one embodiment of the present disclosure.

[0582] Referring to Fig. 54, the AI ​​agent acts from a local perspective. Local perspective is an important concept in AI agent technology, referring to the characteristic where the agent perceives only the part related to its mission rather than the entire system. This is a key element that enables efficient decision-making and resource utilization in distributed systems; the agent perceives only the part related to its mission rather than the entire system, and in cyber-physical systems (CPS), this can be implemented in a way where each agent monitors only a specific section of the power grid. This local perspective enables efficient decision-making and resource utilization in distributed systems.

[0583] A distributed reinforcement learning-based AI agent system according to one embodiment can implement autonomy through a cyclical structure of 'Perception → Knowledge Base → Reasoning & Planning → Action & Actuation → Learning Loop'. Instead of simultaneously grasping the state of the entire system, each agent in the system can adopt a local perspective, focusing only on the specific area or task under its charge. This can provide the effect of saving computational resources and reducing the cognitive load of individual agents in large-scale, complex systems, such as power grid monitoring in cyber-physical systems (CPS) or path planning in multi-drone systems. The local perspective can serve as a core principle for enhancing the decentralization, autonomy, and scalability of the system.

[0584] In one embodiment, the system may adopt a distributed reinforcement learning framework so that each agent can learn independently while achieving a global goal. The sensing unit (5420) can generate local state information by integrating and processing multimodal sensor data such as cameras, lidar, and radar. The memory unit (5450) includes short-term and long-term memory structures and can manage persistent knowledge such as perceived environmental information, traffic regulations, and road structures. The inference unit (5430) can plan an optimal path and perform decision-making by utilizing a Transformer-based neural network architecture. In particular, computational efficiency can be enhanced through a multi-token prediction technique. The execution unit (5440) can reflect the determined action in an external environment, such as robot arm, drone movement, or vehicle engine control. The learning module (5460) can update the policy by receiving feedback on the reward value based on the execution result.

[0585] In one embodiment, game-theoretic methods may be utilized to address the complexity of cooperative reward design arising from each agent learning independently. By using a reward scheduler that considers Nash Equilibrium or Pareto optimization, conflicts of interest between agents can be prevented, and overall system optimization can be achieved. Communication between agents is conducted via a standardized Agent-to-Agent (A2A) protocol, which can coordinate model information sharing and collaborative learning. A safety monitoring module may be included to address the issue of unpredictability associated with autonomy. The system may have a redundancy structure in which control is automatically transferred to a manual control mode or transferred to a central control unit when risk indicators exceed a threshold. Additionally, the reliability of the system can be enhanced by clearly presenting the basis for the agents' decisions to the user through an explainable AI module.

[0586] In one embodiment, a distributed reinforcement learning-based AI agent system that autonomously determines a path and performs cooperative actions based on multi-modality sensor data input from an environment may include a plurality of agents that perform decisions independently without central control. Each agent may include a sensing unit (5420) that collects sensor data within the agent's assigned area to generate local state information, a memory unit (5450) that stores and manages local state information, an inference unit (5430) that performs decision-making and path planning for achieving a goal based on local state information and includes a transformer-based neural network architecture, an execution unit (5440) that reflects the determined path or action to an external environment, and a learning module (5460) that receives a reward value from the execution result and updates a policy. The plurality of agents may be implemented to achieve a global goal for the entire system by cooperating with each other through a game theory-based cooperative reward structure.

[0587] A system according to one embodiment may include a cyclic structure for autonomously optimizing policies and improving performance through interaction with an external environment. A sensing unit (5420) may collect state information from an environment (5410). The sensing unit (5420) may receive multi-modality sensor data, such as a camera, lidar, or radar, and convert it into local state information in a form that can be processed by an agent. An inference unit (5430) may determine the optimal action for achieving a goal and plan a path based on the state information transmitted from the sensing unit (5420). At this time, the inference unit (5430) may improve the accuracy of decision-making by referring to existing knowledge, experience, and learned policy models stored in a memory unit (5450). An execution unit (5440) may reflect the determined action command to the environment (5410). The execution unit (5440) may perform the role of actually executing the determined action, such as the movement of a robot arm, the movement of a drone, or the control of a vehicle. The learning module (5460) can receive feedback on a reward value from the environment (5410) as a result of executing a reward-based policy update action. The learning module (5460) can update the policy parameters of the neural network within the inference unit by analyzing the correlation between the reward value and the state change. The learning module (5460) can increase learning efficiency by utilizing multi-token prediction techniques, and can perform game theory-based reward scheduling so that the performance of individual agents contributes to the reward of the entire system. The memory unit (5450) can store the learning results updated by the learning module (5460). The stored data can be referenced again during the inference process of the next cycle to continuously enhance the intelligence of the agent.

[0588] In one embodiment, each agent can reduce the computational cost of the entire system by maintaining a local perspective through processing only sensor information in its assigned area without accessing information outside its mission scope.

[0589] In one embodiment, the game theory-based cooperative reward structure may include a cooperative reward scheduler that adjusts rewards to approach a Nash Equilibrium state based on the individual utility function of each agent.

[0590] In one embodiment, each agent can use an A2A communication protocol to send and receive messages for sharing model information, collaborative learning, and resource exchange between agents.

[0591] In one embodiment, the AI ​​agent system may further include a federated learning-based parameter aggregator that receives parameters individually learned from a plurality of agents in an encrypted form and updates the policy of the entire system.

[0592] In one embodiment, the AI ​​agent system may further include an explainable AI module that analyzes the output and internal state of the inference unit to generate explanatory information regarding the reason for a specific action choice.

[0593] In one embodiment, the system may include a redundant control structure comprising a Safety Supervisor module that monitors the system's risk indicators and switches to a manual control mode or transfers control to a central unit when a threshold is exceeded.

[0594] In one embodiment, the inference unit (5430) can improve computational efficiency by predicting multiple tokens simultaneously using a multi-token prediction technique.

[0595] In one embodiment, the learning module (5460) may support a dual learning mode capable of switching between an on-device learning mode and a federated learning mode. In one embodiment, each agent may be implemented as at least one of a power grid management agent, an autonomous vehicle, a logistics robot, and a drone.

[0596] According to one embodiment of the present disclosure, a naphtha cracking center scheduling system may adopt a distributed reinforcement learning-based AI agent system structure as illustrated in FIG. 54 for efficient control of a large-scale process. Referring to FIG. 54, each agent has a local perspective that recognizes only its assigned area, rather than the entire system. When applied to an NCC environment, this is as follows. The cracking agent can collect temperature / pressure sensor data only for the 'No. 3 cracking furnace' under its charge, rather than the entire plant. This can reduce unnecessary data communication load. The agent can immediately decide and execute actions, such as adjusting the fuel injection amount, based solely on the collected local information. If No. 3 operates excessively to increase its own efficiency and overloads the downstream process (compression process), a game theory-based reward scheduler can detect this and reduce the reward for No. 3. Through this, each agent autonomously does their best within their own area, but as a result, can reach a Nash Equilibrium state that achieves the global goal of the entire factory: 'total profit maximization and zero accidents.'

[0597] According to another embodiment of the present disclosure, the system may be equipped with a redundant control structure including a Safety Supervisor to prevent accidents caused by the unpredictability of the autonomous agent. The system transfers control when a risk index exceeds a threshold. For example, if an autonomous scheduling agent issues an aggressive command to "raise reactor pressure to 99% of the design limit to maximize profits," a separate, independent Safety Supervisor monitors this, and if it is determined that the Risk Index exceeds the Safety Threshold, it can immediately interlock the agent's control signal. Then, the control mode can be automatically switched to Manual Mode, or a predefined Emergency Shutdown logic can be executed to ensure the safety of the plant as a top priority.

[0598] In addition, the explainable AI (XAI) module reports to the operator the reasoning behind why the agent made such a dangerous judgment (e.g., "It was safe for 1 hour even at 99% pressure in past data"), which can be used as training data to prevent future recurrence.

[0599] One embodiment of the present disclosure may also be implemented in the form of a recording medium comprising computer-executable instructions, such as program modules executed by a computer. A computer-readable medium may be any available medium accessible by a computer and includes both volatile and non-volatile media, and both removable and non-removable media. Additionally, a computer-readable medium may include both computer storage media and communication media. A computer storage medium includes both volatile and non-volatile, removable and non-removable media implemented by any method or technique for storing information, such as computer-readable instructions, data structures, program modules, or other data. A communication medium typically includes computer-readable instructions, data structures, or program modules and includes any information transmission medium.

[0600] The foregoing description of the present disclosure is for illustrative purposes only, and those skilled in the art will understand that modifications can be easily made to other specific forms without altering the technical spirit or essential features of the present invention. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, each component described as a single unit may be implemented in a distributed manner, and components described as distributed may likewise be implemented in a combined form.

[0601] The scope of the present disclosure is defined by the claims set forth below rather than by the detailed description above, and all modifications or variations derived from the meaning and scope of the claims and equivalent concepts thereof should be interpreted as being included within the scope of the present disclosure.

Claims

As a system performed by a computer, At least one processor; It includes at least one memory that stores instructions that cause the system to perform operations when executed by the above-mentioned at least one processor, and The above operations are, An operation to generate a pivot schedule including an initial macro operation sequence based on the initial state of the process; The operation of generating multiple branch schedules by duplicating the above pivot schedule; An operation to expand a schedule by sequentially adding macro operations to each of the plurality of branch schedules using an artificial intelligence agent, reflecting different operational scenarios or probabilistic behavior policies; When a predefined synchronization point is reached, an operation to evaluate the suitability for each of the extended plurality of branch schedules; and A system comprising an operation to determine final scheduling information by repeatedly performing a process of updating the pivot schedule by selecting at least one of the plurality of branch schedules based on the above evaluation result. In paragraph 1, the above process is a product production process of a naphtha cracking center, and The above artificial intelligence agent is, A first agent determining the schedule for unloading naphtha from a vessel or external supply chain and storing it in a receiving tank; A second agent determining the ratio and schedule for transferring naphtha stored in the above-mentioned receiving tank to a mixing tank for blending; and It includes at least one of a third agent (decomposition agent) that determines operating conditions for pyrolyzing the raw material of the above-mentioned mixing tank by introducing it into the decomposition furnace, and The above operations are, A system comprising an operation to update the pivot schedule by integrating macro operations asynchronously generated by at least one of the first agent, the second agent, and the third agent. In paragraph 2, the above operations are, The operation of collecting observation information and operation history of at least one of the first agent, the second agent, and the third agent, respectively, and storing them in an agent-centered history buffer; An operation to convert the data in the history buffer into a high-dimensional vector by applying time point embeddings using a transformer-based encoding network; The operation of inputting the transformed high-dimensional vector into a centralized critique network to estimate a value function reflecting the interaction between agents; and A system comprising an operation to learn a policy of at least one of the first agent, the second agent, and the third agent based on the above value function. In paragraph 2, At least one of the first agent, the second agent, and the third agent is, It includes a Mixture of Experts (MoE) model for performing specialized inference based on the properties of input raw materials or operating modes, and The above MoE model is, A system comprising: a gating network that analyzes the characteristics of input raw material data and selects a suitable expert among a plurality of expert sub-networks; and a decoding layer that integrates the computational results of the selected expert sub-networks to predict the yield or coking speed to decomposition. In paragraph 1, the above at least one processor is, Includes a RAG (Retrieval-Augmented Generation) module, The above RAG module is, A vector database that stores at least one of standard operating procedures, past accident case reports, and external market news data related to the above process by vectorizing them; and A system comprising a search agent that, when a constraint violation or anomaly is detected during the process of determining the above scheduling information, searches the vector database to extract a corresponding solution guide or constraint update information and provides it as an input context to the agent. In paragraph 1, the above at least one processor is, It includes a non-criteria module that verifies the physical feasibility and safety compliance of the schedule draft generated by the above artificial intelligence agent, The above operations are, A system comprising an operation to perform a self-reflection loop that, when the above-mentioned critical module identifies a logical error or risk factor in the above-mentioned schedule draft, transmits feedback to the above-mentioned artificial intelligence agent to control the modification of the above-mentioned schedule draft. In paragraph 1, the artificial intelligence agent is, A system that, prior to determining a macro action in the current state, determines the necessity of the action through an inference step, calls a process simulator as a tool to simulate the expected result of the action, and then executes the final action based on the result. In paragraph 1, the above at least one processor is, When the above final scheduling information is determined, it further includes an action module that converts it into a physical control signal or administrative processing command and executes it. The above action module is, A Large Action Model (LAM) that recognizes the control screen of legacy equipment not supported by API and generates mouse or keyboard input to control the equipment; and A system comprising at least one of a Robotic Process Automation (RPA) system that inputs the above-determined scheduling information into an enterprise resource management system or sends an email to an external partner. In paragraph 2, The above-mentioned first agent is, It supports an A2A (Agent-to-Agent) protocol for communicating with agents of external raw material suppliers, and A system that optimizes the receiving schedule by exchanging negotiation messages regarding the timing of receiving and price conditions with an agent of the external raw material supplier while concealing accurate inventory data of the receiving tank. In paragraph 1, the above operations are, A system comprising the operation to perform intra-group recovery, which replaces a specific branch schedule with another valid branch schedule within the same group when a failure occurs due to a constraint violation in that specific branch schedule; and inter-group recovery, which replaces a valid branch schedule from another group having a higher constraint level by replicating it when all branch schedules within the same group fail. In paragraph 2, It further includes an interface that receives from a user a fixed schedule for process steps handled by some of the agents among the first agent, the second agent, or the third agent, and The above operations are, A system comprising the operation in which, when the above fixed schedule is input, the remaining agents recognize the above fixed schedule as an unchangeable environmental constraint and determine scheduling information for the remaining process steps based thereon. In paragraph 1, the above system is, A small language model that is mounted on the equipment or edge device of the above process and operates independently; and further includes a safety monitoring module that monitors in real time whether the schedule generated by the artificial intelligence agent exceeds a risk threshold. The above safety monitoring module is, A system that blocks control of the artificial intelligence agent and transfers control to the small language model or predefined emergency logic when a risk threshold exceedance is detected. As a method performed by a computer, An operation to generate a pivot schedule including an initial macro operation sequence based on the initial state of the process; The operation of generating multiple branch schedules by duplicating the above pivot schedule; An operation to expand a schedule by sequentially adding macro operations to each of the plurality of branch schedules using an artificial intelligence agent, reflecting different operational scenarios or probabilistic behavior policies; When a predefined synchronization point is reached, an operation to evaluate the suitability for each of the extended plurality of branch schedules; and A method comprising determining final scheduling information by repeatedly performing a process of updating the pivot schedule by selecting at least one of the plurality of branch schedules based on the above evaluation results. A computer-readable recording medium having a program recorded thereon for performing the method of paragraph 13 on a computer.

Images