Method and system for determining action of artificial intelligence agent in observation-limited environment

Abstract

An embodiment provides a method comprising the steps of: acquiring, through a data input interface, a user input indicating a predetermined task; acquiring information on an environment required for performing the task; generating, by at least one processor using at least one artificial intelligence model, environment reflection data on the basis of the information on the environment; establishing, by the at least one processor using at least one artificial intelligence model, a primary action plan for performing the task, on the basis of the environment reflection data; calculating, by the at least one processor using at least one artificial intelligence model, first action data for executing the primary action plan; and inputting the first action data into at least one subsequent processing component.

Description

Method and system for determining the behavior of an artificial intelligence agent in an environment with limited observation.

[0001] The present disclosure relates to a method and system for determining the behavior of an artificial intelligence agent in an environment where observation is limited, and more specifically, to an artificial intelligence agent technology that generates an environment-reflecting sentence and determines a behavior based thereon by utilizing a first artificial intelligence model responsible for general reasoning and a second artificial intelligence model responsible for executing task-specific actions.

[0002] With the recent advancement of artificial intelligence technology, agent systems based on Large Language Models (LLMs) are being utilized for performing various tasks. LLM-based agents are demonstrating the potential to execute complex tasks by understanding instructions in natural language and formulating corresponding action plans.

[0003] However, when an agent operates in a real-world environment, it faces a partially observable environment where it is difficult to obtain complete information about the entire environment. Due to limitations in the agent's sensor range, obstructions to the line of sight caused by physical obstacles, or dynamic changes in the environment, the observational information acquired by the agent is inevitably incomplete. In such an environment, if the agent makes decisions based solely on incomplete observational information, problems arise where task execution fails or is delayed due to incorrect judgments.

[0004] There are two main conventional approaches to addressing these problems. First, there is a method of determining actions by directly controlling a closed-source LLM based on prompts. While this method can leverage the powerful generalization capabilities of the LLM, it is difficult to fine-tune internal parameters, and there are limitations in generalization performance when the environment changes. Second, there is a method of performing tasks by fine-tuning an open-source language model using success trajectory data from a specific environment. Although this method demonstrates high performance in specific environments, it has limitations in that it lacks adaptability to new environments due to overfitting to the training environment.

[0005] Furthermore, existing Retrieval-Augmented Generation (RAG)-based methods adopt an approach that assists agent decision-making by retrieving relevant information from external knowledge bases. However, since these methods rely on predetermined fixed search rules, they have limitations in actively searching for necessary information in new situations that the agent has not previously experienced and in adaptively updating search rules.

[0006] For example, when a robot agent performs a task of assembling parts in a manufacturing process or transporting specific cargo in a logistics environment, the existing action plan may fail due to objects located outside the agent's observation range or unexpected obstacles. In such cases, conventional methods struggle with adaptive responses that involve independently identifying the cause of failure and actively seeking out necessary additional information to modify the action plan; consequently, problems arise such as the repetition of the same failure or delays in task completion.

[0007] Therefore, in real-world environments with limited observation, there is a need for new methods and systems that enable an agent to infer environmental states based on incomplete environmental observation information, actively seek additional information to update action plans upon failure of action execution, and adaptively improve information search rules based on search experience.

[0008] One embodiment of the present disclosure aims to provide a method and system for generating environment-reflecting data based on information about the environment using an artificial intelligence model responsible for general reasoning in an environment with limited observation and an artificial intelligence model responsible for executing task-specific actions, and for establishing an action plan based thereon to determine an action.

[0009] In addition, one embodiment of the present disclosure aims to provide a method and system for determining behavior by enabling an agent to self-evolutionarily adapt even in a new environment with limited observation, by performing active retrieval including querying external information sources, inferring additional information, and re-exploring the environment upon failure of action execution, updating environment-reflective data based on the retrieved knowledge, and adaptively updating information retrieval rules.

[0010] Furthermore, one embodiment of the present disclosure aims to provide a method and system capable of effective action decision and task execution even in complex and highly freedom-limited observation environments by dynamically generating new actions based on a code or a decision tree structured neural network to expand the action set when task execution is impossible with an existing action set.

[0011] However, the technical problems that the various embodiments of the present disclosure aim to solve are not limited to those described above, and there may be other technical problems that can be achieved through the technical means described in this specification.

[0012] One embodiment is,

[0013] A method executed by a computer comprises: obtaining user input directing a predetermined task through a data input interface; obtaining information about an environment required for performing the task; at least one processor generating environment reflection data based on the information about the environment using at least one artificial intelligence model; at least one processor establishing a primary action plan for performing the task based on the environment reflection data using at least one artificial intelligence model; at least one processor calculating first action data for executing the primary action plan using at least one artificial intelligence model; and inputting the first action data into at least one subsequent processing component.

[0014] In another aspect, the method may further include the steps of: executing the task based on the first action data; receiving negative feedback regarding the result of executing the task; performing active information search to actively obtain additional information necessary for performing the task based on at least one of the negative feedback, the environment reflection data, and the previous action history; and updating the environment reflection data based on the additional information obtained through the active information search.

[0015] In another aspect, the method may further include the step of establishing a modified secondary action plan based on the updated environment-reflecting sentence; and the step of at least one processor generating second action data for executing the secondary action plan using the second artificial intelligence model.

[0016] In another aspect, the step of performing the active information search may include inputting at least one of the negative feedback, the environment reflection data, and the previous behavior history into the at least one artificial intelligence model to infer additional information necessary for performing the task.

[0017] In another aspect, the step of performing the active information search may include: generating a search query for searching the inferred additional information; and acquiring knowledge regarding the additional information from an external information source based on the search query and storing it in at least one memory.

[0018] In another aspect, the step of performing the active information search may include: a step of re-searching the surrounding environment based on the inferred additional information; and a step of storing the new observation information obtained through the re-search in the at least one memory.

[0019] In another aspect, the method may further include the step of updating information search rules, including the search scope and search query generation method for active information search to be performed in similar situations in the future, based on additional information obtained through the active information search and the execution result of the task.

[0020] In another aspect, the step of updating the information search rule may include: adding a new search target to an existing search range or modifying an existing search query based on additional information obtained through the active information search; and storing the updated information search rule in at least one memory.

[0021] In another aspect, the negative feedback may include at least one of the following: when there is no response from the environment from the execution result of the task, an uncertain state in which the output token probability of the at least one artificial intelligence model is below a preset threshold, and when the goal of the task is not achieved.

[0022] In another aspect, the method may further include the step of the at least one processor determining, using the at least one artificial intelligence model, whether an action code corresponding to an action required for performing the task exists in an existing action set; the step of, if the result of the determination is that the action code does not exist, generating code that performs an action required for performing the task using the at least one artificial intelligence model; and the step of executing the generated code to perform an action required for performing the task.

[0023] In another aspect, the step of generating the code may include: a step of verifying the operation of the code by executing the generated code in a simulation environment; a step of modifying the code if the operation of the code differs from the intended operation as a result of the verification; and a step of refining the code by repeating the verification and modification on the modified code.

[0024] In another aspect, the step of verifying the operation of the code may include: obtaining a state-action pair from the operation of the code executed in the simulation environment; and inputting the obtained state-action pair into a vision language model (VLM) to evaluate the operation of the code.

[0025] In another aspect, the method may further include the step of the at least one processor generating parameters of an artificial neural network model for calculating control information necessary for performing an operation based on the first action data using the at least one artificial intelligence model; the step of generating control information necessary for performing the operation using the artificial neural network model composed of the generated parameters; and the step of performing the task by executing an action based on the generated control information.

[0026] In another aspect, the step of generating parameters of the artificial neural network model may include: generating parameters of the artificial neural network model of a decision tree structure in text format using at least one artificial intelligence model; and configuring the artificial neural network model of the decision tree structure based on the parameters generated in text format.

[0027] In another aspect, the step of acquiring information about the environment may include: a step in which the at least one processor controls a physical device equipped with at least one sensor to scan the surrounding environment with the sensor and acquire raw data; and a step of acquiring information about the environment by identifying at least one of the type, location, and state of an object from the acquired raw data.

[0028] In another aspect, the at least one artificial intelligence model used to generate the environment reflection data may include an unfine-tuned closed-source large language model, and the at least one artificial intelligence model used to generate the first action data may include an open-source small language model fine-tuned with success trajectory data.

[0029] In another aspect, the step of executing the task based on the first action data may include: the step of the at least one processor generating a control signal that controls the physical movement of a physical device equipped with a driving unit based on the first action data; and the step of the physical device receiving the control signal and driving the driving unit.

[0030] One embodiment is,

[0031] The system provides a method comprising at least one memory; and at least one processor that reads at least one instruction stored in the at least one memory and performs a method for controlling the behavior of an artificial intelligence agent in an environment where observation is restricted, wherein the at least one instruction comprises: a step of obtaining user input instructing a predetermined task through a data input interface; a step of obtaining information about an environment required for performing the task; a step in which the at least one processor uses at least one artificial intelligence model to generate environment reflection data based on the information about the environment; a step in which the at least one processor uses at least one artificial intelligence model to establish a primary action plan for performing the task based on the environment reflection data; a step in which the at least one processor uses at least one artificial intelligence model to calculate first action data for executing the primary action plan; and a step of inputting the first action data into at least one subsequent processing component.

[0032] In another aspect, the system comprises: a plurality of neurons configured in an array including at least one register, at least one programmable logic, and at least one input interface; a plurality of synapse circuits storing synapse weights that regulate the connection strength between the plurality of neurons; and at least one routing network that controls the data flow between the plurality of neurons; wherein each of the plurality of neurons is connected to at least one other neuron through the routing network to establish a transmission path for the weights, and may further comprise a Field Programmable Gate Array (FPGA) implementation for a predetermined artificial neural network.

[0033] In another aspect, the system comprises: a plurality of neurons organized into an array including at least one register, at least one microprocessor, and at least one input; and a plurality of synapse circuits storing synapse weights that regulate the connection strength between the plurality of neurons; wherein each of the plurality of neurons may further comprise an Application Specific Integrated Circuit (ASIC) for a predetermined artificial neural network that is connected to at least one other neuron through any one of the plurality of synapse circuits.

[0034] A method and system for determining the behavior of an artificial intelligence agent in an environment with limited observation according to one embodiment of the present disclosure can significantly improve the task success rate of the agent by combining the establishment of a behavior plan based on environment-reflecting data and the execution of precise actions, even in a real environment where observation information of at least one artificial intelligence model is incomplete.

[0035] In addition, one embodiment of the present disclosure performs active information search, including querying external information sources, inferring additional information, and re-searching the environment, without relying on a fixed knowledge base when an action execution fails, and adaptively updates information search rules, thereby having the effect of improving generalization performance in a new environment (Unseen) compared to conventional methods.

[0036] In addition, one embodiment of the present disclosure has the effect of being effectively applicable to real physical environments with high degrees of freedom, such as robotics and autonomous driving, by dynamically generating new actions based on a code or a decision tree structure of a neural network to expand the action set even in complex situations that are difficult to process with an existing action set.

[0037] In addition, one embodiment of the present disclosure has the effect of simultaneously securing task-specific performance and generalization performance through a role-sharing structure in which a closed-source large language model is responsible for environment-reflective sentence generation and action plan formulation, and an open-source small language model fine-tuned with success trajectory data is responsible for precise action execution.

[0038] However, the effects obtainable from the present disclosure are not limited to those mentioned above, and there may be other effects that can be clearly understood by a person skilled in the art to which the various embodiments of the present disclosure belong through the configurations described in this specification.

[0039] FIG. 1 illustrates an example of a block diagram of a computing system implementing a method for determining the behavior of an artificial intelligence agent in an environment where observation is limited, according to one embodiment of the present disclosure.

[0040] FIG. 2 briefly illustrates the structure of a neuromorphic circuit that may be included in a processor according to one embodiment.

[0041] FIG. 3 illustrates an example of a block diagram of a computing device that implements a method for determining the behavior of an artificial intelligence agent in an environment where observation is limited, according to one embodiment of the present disclosure.

[0042] FIG. 4 is a block diagram illustrating the internal architecture and data processing pipeline of an artificial intelligence model according to one embodiment of the present disclosure.

[0043] FIG. 5 illustrates an example of a block diagram in another aspect of a computing device that implements a method for determining the behavior of an artificial intelligence agent in an environment where observation is limited, according to one embodiment of the present disclosure.

[0044] FIG. 6 is a block diagram illustrating the data flow and system interaction of a process for applying an artificial intelligence agent's behavior decision service in an environment where observation is limited, according to one embodiment of the present disclosure.

[0045] FIG. 7 illustrates the architecture of a Universal Dynamic Multi-Agent System according to one embodiment of the present disclosure.

[0046] FIG. 8 is a flowchart showing the overall flow of a method for determining the behavior of an artificial intelligence agent according to one embodiment of the present disclosure.

[0047] FIG. 9 is a conceptual diagram showing an operation scenario of an artificial intelligence agent according to one embodiment of the present disclosure.

[0048] FIG. 10 is a block diagram showing the module configuration of a server computing device according to one embodiment of the present disclosure.

[0049] FIG. 11 is a flowchart illustrating the flow of an active information search and environment-reflecting data update method according to one embodiment of the present disclosure.

[0050] FIG. 12 is a conceptual diagram illustrating the process of updating environment-reflecting sentences and information search rules through active information search according to one embodiment of the present disclosure.

[0051] FIG. 13 is a flowchart illustrating the flow of a code-based action generation pipeline according to one embodiment of the present disclosure.

[0052] FIG. 14 is a flowchart showing the flow of an action decision pipeline using an artificial neural network model of a decision tree structure according to one embodiment of the present disclosure.

[0053] As the embodiments of the present disclosure are subject to various modifications and may have various embodiments, specific embodiments are illustrated in the drawings and described in detail in the detailed description. The effects and features of the present disclosure and the methods for achieving them will become clear by referring to the embodiments described below in detail together with the drawings. However, the present disclosure is not limited to the embodiments disclosed below but may be implemented in various forms. In the following embodiments, terms such as "first," "second," etc., are used not in a limiting sense but for the purpose of distinguishing one component from another. Also, singular expressions include plural expressions unless the context clearly indicates otherwise. Furthermore, terms such as "include" or "have" mean that the features or components described in the specification exist, and do not preclude the possibility that one or more other features or components may be added. Additionally, in the drawings, the size of components may be exaggerated or reduced for convenience of explanation. For example, the size and thickness of each component shown in the drawings are arbitrarily depicted for convenience of explanation, so the embodiments of the present disclosure are not necessarily limited to those depicted.

[0054] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the attached drawings. When describing with reference to the drawings, identical or corresponding components are given the same reference numerals, and redundant descriptions thereof will be omitted.

[0055]

[0056] - A system for controlling the behavior of an artificial intelligence agent in an environment where observation is limited (1000)

[0057] A system (1000) according to one embodiment of the present disclosure can effectively perform various tasks by an artificial intelligence agent operating in an environment where observation is limited, generating environment-reflecting data using at least one artificial intelligence model responsible for general reasoning and at least one artificial intelligence model responsible for task-specific action execution, and determining actions based thereon.

[0058] In particular, the present system (1000) can perform real-world tasks by determining actions based on real environment observation information obtained through sensors in a Physical AI environment equipped with physical devices such as robot manipulation, autonomous mobile robots, and drone control, and by directly controlling the driving part of the physical device according to the determined actions.

[0059] The system (1000) can achieve a high task performance success rate even in a partially observable environment by not relying simply on a pre-learned policy, but by inferring the current state of the environment based on observation information obtained from the actual environment by the agent and reflecting this in the decision of action.

[0060] When negative feedback is received during the task execution process, the system (1000) can establish a modified action plan by autonomously obtaining additional information necessary for task execution through active information search and updating environment-reflecting data. Through this, the agent can perform an adaptive response by identifying the cause and modifying its behavior on its own, even in failure situations caused by unexpected environmental changes or incomplete observation information.

[0061] In addition, the system (1000) can continuously improve search efficiency in similar situations by dynamically updating information search rules based on new knowledge acquired during the active information search process. The user can assign tasks to the agent using only instructions in the form of natural language, and the agent provides an environment for completing complex tasks step by step through the establishment and updating of environment-reflecting sentences and action plans.

[0062] In addition, the present system (1000) can flexibly expand the action space of an agent by generating a new action code using a first artificial intelligence model when an action not present in the existing action set is required, and then executing it after verifying it through simulation and visual language model (VLM)-based verification. Furthermore, by enabling the first artificial intelligence model to directly generate and modify artificial neural network model parameters of a decision tree structure in text format, a highly scalable agent system that encompasses even complex physical control actions that are difficult to express in code can be implemented.

[0063] FIG. 1 illustrates an example of a block diagram of a computing system (1000) that implements a method for determining the behavior of an artificial intelligence agent in an environment where observation is limited, according to one embodiment of the present disclosure.

[0064] Referring to FIG. 1, a computing system (1000) for implementing a method for determining the behavior of an artificial intelligence agent in an environment where observation is limited according to one embodiment includes a user computing device (110), a server computing system (130), and a training computing system (150), and each device can communicate through a network (170).

[0065] A method for determining the behavior of an artificial intelligence agent according to one embodiment of the present disclosure may be implemented and provided locally by a user computing device (110), implemented and provided in the form of a web service by a server computing system (130) communicating with the user computing device (110), or implemented and provided by the user computing device (110) and the server computing system (130) in conjunction with each other.

[0066] In this embodiment, the user computing device (110) and / or server computing system (130) can train a machine learning model (120 and / or 140) through interaction with a training computing system (150) that is communicatedly connected via a network (170). In the embodiment of the present disclosure, the machine learning model (120, 140) may include an agentic architecture that includes a first artificial intelligence model and a second artificial intelligence model, going beyond a simple prediction model. The first artificial intelligence model is responsible for general inference that generates environment-reflecting sentences from environment observation information and establishes action plans, while the second artificial intelligence model is responsible for a task-specific role that calculates action values ​​to execute the established action plans.

[0067] In one embodiment, the machine learning model may be an AI agent having a structure that receives user input and environment observation information, generates environment-reflecting sentences through core components such as planning, memory, reasoning, and tools, autonomously establishes an action plan, and updates the action plan through feedback on the task execution results.

[0068] In another embodiment, the machine learning model may include a Search Augmentation Generative (RAG) architecture that actively retrieves relevant information from an external database and updates environment-reflective sentences based on it to generate a more accurate action plan. In this case, the search target is not limited to a fixed query, and an adaptive search method may be adopted that dynamically expands and updates the search query through active information retrieval in the event of a task execution failure.

[0069] In another embodiment, the machine learning model may be a multi-agent system in which multiple AI agents cooperate to accomplish a specific task. The multi-agent system may have a supervisory pattern in which a central supervisor agent distributes tasks to subordinate specialist agents and aggregates the results. Alternatively, it may have a hierarchical pattern in which a meta-agent acts as an intermediary manager to control and coordinate subordinate agents. It is also possible to include a multi-agent debate pattern in which multiple agents present different action plans and select the optimal action through evaluation.

[0070] In another embodiment, the machine learning model may include a structure combining a Large Language Model (LLM) and a Diffusion Model. The Large Language Model is a model that learns vast amounts of text data to demonstrate excellent capabilities in context understanding, natural language generation, question answering, etc., and may include GPT family, BERT family, T5, PaLM, etc. The Diffusion Model is a probabilistic generative model composed of a forward process that gradually adds noise to the data and a reverse process that gradually removes noise to restore the original data or generate new data, and can generate sophisticated, high-quality samples for various forms of data such as images, videos, voice, and motion trajectories.

[0071] When combining a large language model and a diffusion model, the large language model analyzes user input and contextual information to generate condition embeddings in a format understandable by the diffusion model, while the diffusion model generates target data by incorporating these condition embeddings and performing a noise removal process. Specifically, during the training phase, the diffusion model learns how to remove noise from data with varying levels of added noise, and the large language model learns how to extract meaningful condition information from text into embedding vectors. In the execution (inference) phase, the large language model analyzes user input to generate condition embeddings and passes them to the diffusion model, which then generates high-quality output data that meets the conditions through the noise removal process.

[0072] In one embodiment, the combined structure of a large language model and a diffusion model can be applied to various types of tasks, such as text-to-image generation, text-to-video generation, and multimodal generation. For example, in a text-to-image generation task, the large language model extracts key keywords and detailed conditions from a user's natural language request and converts them into condition embeddings, and the diffusion model generates high-quality images through a noise removal process based on the condition embeddings. Additionally, in a context-aware refinement task that modifies parts of already generated images or videos, the large language model can operate by interpreting the user's modification request to identify the modification target and changes, and the diffusion model can perform partial noise removal and inpainting on the corresponding area. Furthermore, it can be utilized for agent planning-based content generation in large-scale content generation projects, where the large language model establishes the overall scenario and key scene plans, calls the diffusion model for each scene to generate image or video sequences, and if the generation result is unsatisfactory, the large language model directs repainting with modified conditions.

[0073] In this way, the combined structure of a large language model and a diffusion model fuses text-based flexible prompt control with the high-quality generative capabilities of the diffusion model. This allows the large language model to interpret and reinforce linguistic contexts and conditions, thereby inducing the diffusion model to generate detailed and diverse samples, and can be applied to various fields such as art, design, education, and entertainment.

[0074] The server computing system (130) can host AI agents such as those mentioned above, particularly multi-agent systems requiring complex computations, or large-scale long-term memory that preserves the behavior history and environmental observation information of agents for a long period. In addition, the server computing system (130) includes a Model Context Protocol (MCP) server for managing integration with various external tools, and can efficiently obtain external information necessary for active information search by relaying communication with external knowledge bases, cloud APIs, search engines, etc.

[0075] The training computing system (150) can fine-tune a second artificial intelligence model by utilizing the agent's success trajectory data and failure trajectory data, or perform iterative learning to gradually improve the agent's performance through a self-reflection mechanism in which the first artificial intelligence model evaluates and modifies the results of task execution. Additionally, the training computing system (150) can continuously improve the agent so that it can perform tasks more effectively in similar situations by dynamically updating information search rules based on new knowledge obtained through active information search.

[0076] The training computing system (150) may be separate from the server computing system (130) or part of the server computing system (130). Additionally, in some embodiments, the training computing system (150) may be separate from the user computing device (110) or part of the user computing device (110).

[0077] The artificial intelligence model can be 1) trained directly locally by a user computing device (110), 2) trained by the server computing system (130) and the user computing device (110) interacting with each other through a network (170), and 3) trained by a separate training computing system (150) using various training and learning techniques. The artificial intelligence model trained by the training computing system (150) can be transmitted to the user computing device (110) and / or the server computing system (130) through the network (170) to be provided and updated.

[0078]

[0079] - User Computing Device (110: User Computing Device)

[0080] The user computing device (110) may include all other types of computing devices such as a smartphone, mobile phone, digital broadcasting device, PDA (personal digital assistants), PMP (portable multimedia player), desktop, wearable device, embedded computing device and / or tablet PC.

[0081] In one embodiment, the user computing device (110) may include a computing device that is mounted on or communicates with a physical device equipped with a driving unit, such as a robot, an autonomous mobile device, or a drone, and may support operation in a Physical AI environment that directly controls the driving unit of the physical device based on an action value determined by an agent.

[0082] In another embodiment, the user computing device (110) may be a user terminal device such as a smartphone, and may present an action plan and action values ​​generated by the server computing system (130) to the user through a user interface. In this case, the user may review and approve or modify the presented action plan and then direct its execution, and may control the execution of the corresponding action only when the user's approval is confirmed. In this way, by combining the user's confirmation step with the agent's autonomous action decision, a Human-Agent Collaboration environment can be provided that prevents the agent from performing unexpected actions and allows the user to directly supervise the progress of the task.

[0083] The user computing device (110) may include at least one processor (111) and memory (112). Here, the processor (111) may be composed of at least one of a central processing unit (CPU), a graphics processing unit (GPU), ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays), controllers, microcontrollers, microprocessors, and / or electrical units for performing other functions, or a plurality of electrically connected processors.

[0084] In particular, according to the embodiment, the processor (111) may be configured based on a Field Programmable Gate Array (FPGA) implementation and / or an Application Specific Integrated Circuit (ASIC), which is a hardware technology for implementing a predetermined digital circuit.

[0085] Here, a Field Programmable Gate Array (FPGA) can refer to a flexible digital circuit that is programmable according to user needs. A Field Programmable Gate Array implementation may include registers that support the synchronized operation of the FPGA by temporarily storing data and controlling signal flow and timing to maintain intermediate results of operations or state information; programmable logic, which consists of logic circuits configurable according to user needs to program operations within the FPGA to perform specific functions or operations; and input interfaces that serve as channels for receiving data from outside the FPGA, receiving signals from external devices or sensors and transmitting them to internal circuits. Through the combination of these components, a Field Programmable Gate Array implementation can provide flexible and diverse forms of digital circuits.

[0086] An Application-Specific Integrated Circuit (ASIC) may include registers, which are small memory devices that temporarily store and manage data and support rapid processing of ASIC operations by storing intermediate calculation results or state information; a microprocessor, which acts as a central processing unit performing control and computations within the ASIC to coordinate the operation of the entire system by performing various operations or generating control signals when necessary; and an input block, which serves as an interface for receiving data from the outside, receiving data to be processed by the ASIC, transmitting it internally, and receiving various input data through connections with sensors or external devices. Through the combination of these components, the Application-Specific Integrated Circuit can perform specific tasks in an optimized manner. For example, the ASIC may have a structure of a neuromorphic circuit in the form of an array containing multiple neuron circuits, thereby enabling the execution of artificial intelligence computations required for an agent's behavioral decisions with low power consumption and high efficiency.

[0087] FIG. 2 briefly illustrates the structure of a neuromorphic circuit (600) that may be included in a processor (111, 131, 151) according to one embodiment.

[0088] Referring to FIG. 2, for example, a neuromorphic circuit (600) may include a plurality of presynaptic neuron circuits (610), a plurality of presynaptic lines (611) extending laterally from the plurality of presynaptic neuron circuits (610), a plurality of postsynaptic neuron circuits (620), a plurality of postsynaptic lines (621) extending longitudinally from the plurality of postsynaptic neuron circuits (620), and a plurality of synaptic circuits (630) provided at the intersection of the plurality of presynaptic lines (611) and the plurality of postsynaptic lines (621).

[0089] A plurality of free synaptic neuron circuits (610) can transmit signals input from the outside in the form of electrical signals to a plurality of synaptic circuits (630) through a plurality of free synaptic lines (611).

[0090] Additionally, a plurality of post-synaptic neuron circuits (620) can receive electrical signals from a plurality of synaptic circuits (630) through a plurality of post-synaptic lines (621).

[0091] Furthermore, multiple post-synaptic neuron circuits (620) may transmit electrical signals to multiple synaptic circuits (630) through multiple post-synaptic lines (621).

[0092] A plurality of synapse circuits (630) can store weights included in layers constituting a neural network system implemented by a neuromorphic circuit (600) and perform a predetermined operation based on the weights and input data.

[0093] For example, each of the plurality of synaptic circuits (630) may include a resistive memory cell having a variable resistance. In this case, the resistance value of the plurality of synaptic circuits (630) changes by a voltage applied through the plurality of presynaptic neuron circuits (610) or the plurality of postsynaptic neuron circuits (620), and can store weight data according to this resistance change.

[0094] The neuromorphic circuit (600) is formed by mimicking the structure of neurons and synapses, which are essential elements of the human brain. When a deep neural network (DNN) is realized using the neuromorphic circuit (600), the data processing speed can be improved and power consumption can be reduced compared to when the existing von Neumann structure is utilized.

[0095] The memory (112) of the user computing device (110) may include one or more non-transient / transient computer-readable storage media such as RAM, ROM, EEPROM, EPROM, flash memory device, magnetic disk, etc., and combinations thereof, and may include web storage of a server that performs memory storage functions on the internet. This memory (112) may store data (113) and instructions (114) necessary for the at least one processor (111) to perform functional operations such as training an artificial intelligence model or performing data filtering through an artificial intelligence model.

[0096] In one embodiment, the user computing device (110) may store at least one machine learning model (120). For example, the user computing device (110) may be various machine learning models, such as a plurality of neural networks (e.g., deep neural networks) that perform a method for determining the behavior of an artificial intelligence agent in a limited environment, or other types of machine learning models including non-linear models and / or linear models, and may be composed of a combination thereof.

[0097] For example, the machine learning model (120) may store a model for determining the behavior of an artificial intelligence agent in a limited environment based on linear regression, decision tree, random forest, gradient boosting or / and deep learning. And the neural network may include at least one of feed-forward neural networks, recurrent neural networks (e.g., long short-term memory recurrent neural networks), convolutional neural networks, Transformer or / and other forms of neural networks.

[0098] Additionally, according to an embodiment, the user computing device (110) may store a model to be used in each process and a prompt template that serves as the basis for input to the model in order to perform at least part of the process for determining the behavior of an artificial intelligence agent in a limited environment through a large language model (LLM).

[0099] In one embodiment, a user computing device (110) receives at least one machine learning model (120) from a server computing system (130) through a network (170), stores it in memory (112), and then executes the stored machine learning model (120) by a processor (111) to perform a method for determining the behavior of an artificial intelligence agent in a limited environment.

[0100] In another embodiment, the user computing device (110) can provide an action decision service for an artificial intelligence agent to the user in a restricted environment by performing an operation through a machine learning model (140) including at least one machine learning model (140) in conjunction with a server computing system (130) and communicating related data to the outside.

[0101] For example, a user computing device (110) can provide an artificial intelligence agent action decision service in a limited environment by having a server computing system (130) provide an output for the user's input using a machine learning model (140) via the web.

[0102] Additionally, the artificial intelligence model can be implemented in such a way that at least some of the machine learning models (120 and / or 140) are executed on a user computing device (110) and the rest are executed on a server computing system (130).

[0103] Additionally, the user computing device (110) may include at least one input component (121) that detects user input.

[0104] For example, the user input component (121) may include a touch sensor (e.g., a touch screen and / or a touch pad, etc.) that detects a touch of the user's input medium (e.g., a finger or a stylus), an image sensor that detects the user's motion input, a microphone that detects the user's voice input, a button, a mouse and / or a keyboard, etc.

[0105] Here, the image sensor may include an image processing module. Specifically, the image sensor may process still images or video obtained by an image sensor device (e.g., CMOS or CCD).

[0106] In addition, the image sensor can process a still image or video acquired through the image sensor device using an image recognition process (e.g., OCR, etc.) and / or an image processing module to extract necessary information and transmit the extracted information to a processor.

[0107] Additionally, the input component (121) can receive input from an external controller (e.g., mouse, keyboard, etc.) based on an interface module, and in this case, may include an external output device (e.g., speaker).

[0108] At this time, the interface module may be configured to include at least one of a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, a port for connecting a device equipped with an identification module, an audio I / O (Input / Output) port, a video I / O (Input / Output) port, an earphone port, a power amplifier, an RF circuit, a transceiver, and other communication circuits.

[0109] In addition, the external output device may include a display system that outputs various information related to the action decision service of an artificial intelligence agent in a limited environment as a graphic image.

[0110] Such a display system may be implemented by including at least one of a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), a flexible display, a 3D display, and an e-ink display.

[0111] Meanwhile, the user computing device (110) including the above-described components may further perform at least some of the functional operations performed by the server computing system (130) described later.

[0112]

[0113] -Server Computing System (130: Server Computing System)

[0114] The server computing system (130) can perform a series of processes to provide an action decision service for an artificial intelligence agent in an environment where observation is limited.

[0115] Specifically, in an embodiment, the server computing system (130) can provide an artificial intelligence agent's action decision service in an environment where observation is limited by exchanging data necessary to drive the action decision service process of the artificial intelligence agent with the external device, such as the user computing device (110).

[0116] More specifically, in an embodiment, the server computing system (130) may provide an environment in which an application for providing an action decision service for an artificial intelligence agent on a user computing device (110) can operate. To this end, the server computing system (130) may include an application, data and / or commands, etc., for performing functions such as generating environment-reflective sentences, establishing action plans, calculating action values, actively searching for information, and updating information search rules, and may transmit and receive various data based thereon with the external device.

[0117] A server computing system (130) may include at least one processor (131) and memory (132). Here, the processor (131) of the server computing system (130) may be composed of at least one of a central processing unit (CPU), a graphics processing unit (GPU), ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays), controllers, microcontrollers, microprocessors, and / or electrical units for performing other functions, or a plurality of electrically connected processors.

[0118] For example, ASICs may have a structure of a neuromorphic circuit in the form of an array containing multiple neuron circuits (see Fig. 2).

[0119] The memory (132) may include one or more non-transient / transient computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. The memory (132) may store data (133) and instructions (134) necessary for the processor (131) to perform functional operations, such as training a first artificial intelligence model and a second artificial intelligence model, or executing a method for determining the behavior of an artificial intelligence agent in an environment where observation is restricted.

[0120] In one embodiment, the server computing system (130) may be implemented to include at least one computing device. For example, the server computing system (130) may be implemented to operate a plurality of computing devices according to a sequential computing architecture, a parallel computing architecture, or a combination thereof. Additionally, the server computing system (130) may include a plurality of computing devices connected to a network (170).

[0121] Additionally, the server computing system (130) may store at least one machine learning model (140). For example, the server computing system (130) may include a first artificial intelligence model and a second artificial intelligence model as machine learning models (140), and may include neural networks and / or other multi-layer non-linear models. Exemplary neural networks may include feed-forward neural networks, deep neural networks, recurrent neural networks, and convolutional neural networks.

[0122] In an embodiment, the server computing system (130) may further include a data store computing system (hereinafter, data store) which is a repository for continuously storing and managing raw data such as environmental observation information, success trajectory data, failure trajectory data, and information search rules that form the basis of an action decision service.

[0123] These data stores may include various forms of data storage, ranging from file systems to cloud storage. For example, a data store may include a relational database that uses a structured query language (SQL) to define and manipulate data, a NoSQL database designed for flexibility and scalability to process unstructured and semi-structured data, a data warehouse optimized for querying and analysis by centralizing large volumes of data from multiple sources, a data lake that stores structured data, semi-structured data, and unstructured data, and at least one database among local storage devices or Network Attached Storage (NAS).

[0124]

[0125] - Training Computing System (150: Training Computing System)

[0126] The training computing system (150) may include at least one processor (151) and memory (152). Here, the processor (151) of the training computing system (150) may be composed of at least one of a central processing unit (CPU), a graphics processing unit (GPU), application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, and / or electrical units for performing other functions, or a plurality of electrically connected processors. For example, ASICs may have the structure of a neuromorphic circuit in the form of an array including a plurality of neuron circuits (see FIG. 2).

[0127] The memory (152) may include one or more non-transient / transient computer-readable storage media such as RAM, ROM, EEPROM, EPROM, flash memory device, magnetic disk, etc. and combinations thereof. This memory (152) may store data (153) and instructions (154) necessary for the processor (151) to perform learning of the first artificial intelligence model and the second artificial intelligence model, updating of information search rules, etc.

[0128] For example, a training computing system (150) may include a model trainer (160) that trains a machine learning model (120 and / or 140) stored in a user computing device (110) and / or a server computing system (130) using various training or learning techniques, such as backpropagation of error. The model trainer (160) may perform backpropagation updates to one or more parameters of the machine learning model (120 and / or 140) for determining the behavior of an artificial intelligence agent in an environment with limited observation based on a defined loss function. In some embodiments, performing backpropagation of error may include performing truncated backpropagation through time. The model trainer (160) may perform a number of generalization techniques (e.g., weight devaluation, dropout, and / or knowledge distillation, etc.) to improve the generalization ability of the machine learning model (120 and / or 140) being trained.

[0129] Additionally, the model trainer (160) can train a machine learning model (120 and / or 140) based on a series of training data (161). Here, the training data (161) may include agent success trajectory data, failure trajectory data, environmental observation information, knowledge obtained through active information exploration, etc., and may include data of different forms such as images, sensor data, text, etc. Such training data (161) may be provided by a user computing device (110) and / or a server computing system (130). When the training computing system (150) trains the machine learning model (120 and / or 140) on specific environmental data of the user computing device (110), the machine learning model (120 and / or 140) may be characterized as a personalized model specialized for that environment.

[0130] The model trainer (160) includes computer logic utilized to provide desired functions and may be implemented as hardware, firmware and / or software controlling a general-purpose processor. In one embodiment, the model trainer (160) includes a program file stored in a storage device, loaded into memory (152), and may be executed by one or more processors (151). In another embodiment, the model trainer (160) includes one or more sets of computer-executable data (153) and instructions (154) stored in a tangible computer-readable storage medium such as RAM, a hard disk, or an optical or magnetic medium.

[0131] In this system (1000), a user computing device (110) and a server computing system (130) may be connected via a wired / wireless network (170) for communication. The network (170) includes, but is not limited to, a 3GPP (3rd Generation Partnership Project) network, an LTE (Long Term Evolution) network, a WIMAX (World Interoperability for Microwave Access) network, the Internet, a LAN (Local Area Network), a Wireless LAN (Wireless Local Area Network), a WAN (Wide Area Network), a PAN (Personal Area Network), a Bluetooth network, a satellite broadcasting network, an analog broadcasting network and / or a DMB (Digital Multimedia Broadcasting) network. Generally, communication through the network (170) can be performed using any type of wired and / or wireless connection through various communication protocols (e.g., TCP / IP, HTTP, SMTP and / or FTP, etc.), encodings or formats (e.g., HTML and / or XML, etc.), and / or protection schemes (e.g., VPN, Secure HTTP and / or SSL, etc.).

[0132] The server computing system (130) may further include a plurality of logically and physically separated specialized engines and repositories to perform a method for determining the behavior of an artificial intelligence agent in an environment where observation is limited. In one embodiment, the server computing system (130) may include at least one engine among a reasoning engine that processes user input and environment observation information to generate environment-reflective sentences and establish a behavior plan, a search engine that performs active information search, and a rule management engine that manages information search rules. Here, the term "engine" may include not only a set of instructions executed by a processor (131) to perform specific logic, but also dedicated hardware circuits to accelerate said logic.

[0133] These engines may run on hardware accelerators optimized to handle the computational load of the first artificial intelligence model and the second artificial intelligence model. The hardware accelerator is a processor specialized in matrix operations and vector processing and may include at least one of a Tensor Processing Unit (TPU), a Graphics Processing Unit (GPU), a Field-Programmable Gate Array (FPGA), or an Application-Specific Integrated Circuit (ASIC). These hardware accelerators can provide technical improvements that distribute the computational load of the first artificial intelligence model and the second artificial intelligence model and enable real-time action decisions.

[0134] Additionally, the data (133) may include a structured embedding repository to support active information search rather than a simple set of data. The embedding repository stores environmental observation information, success trajectory data, failure trajectory data, and high-dimensional vector representations of information search rules, thereby enabling the action decision engine to perform high-speed search based on semantic similarity. This can serve as a technical means to suppress the hallucination phenomenon of the second artificial intelligence model and increase the accuracy of action value calculation.

[0135] Specifically, the memory (132) of the server computing system (130) may include a structured Knowledge Base Layer to physically support active information retrieval. The Knowledge Base Layer may include an Embedding Repository that stores high-dimensional vector representations of environmental observation information and knowledge related to task execution, and a Policy Document Repository that stores information retrieval rules and action execution constraints. In this case, the action decision engine vectorizes observation information or negative feedback received from the environment and queries the Embedding Repository to search for context information with high semantic similarity in real time, thereby allowing the agent to technically suppress hallucination phenomena by referring to external verified knowledge rather than relying solely on intrinsic parameters.

[0136] Additionally, the user computing device (110) may include a trigger event detector that provides an interface for interaction with the agent and initiates the operation of the agent. The trigger event detector can detect not only user input but also environmental changes obtained from sensors, negative feedback received regarding the result of task execution, etc., and transmit a processing request to the server computing system (130).

[0137] Additionally, the model trainer (160) of the training computing system (150) may include a Supervision Signal Engine. The Supervision Signal Engine may calculate a difference value by comparing the execution result of an action performed by an agent with verified knowledge obtained through active information search, and execute a reinforcement learning process to update a reward model or fine-tune a second artificial intelligence model based on this.

[0138] As such, in one embodiment, the system (1000) of the present disclosure may be implemented not as a simple set of software algorithms, but as a technical system in which specialized hardware accelerators, vectorized data storage, and physical engines controlling them are organically combined.

[0139] FIG. 3 illustrates a block diagram example of a computing device (100) that implements a method for determining the behavior of an artificial intelligence agent in an environment where observation is limited, according to one embodiment of the present disclosure.

[0140] Referring to FIG. 3, the computing device (100) included in the user computing device (110), server computing system (130), and training computing system (150) may include a plurality of applications (e.g., Application 1 to Application N). Each application may include a machine learning library and one or more machine learning models. For example, the applications may include an image processing application (e.g., Detection, Classification and / or Segmentation, etc.), a sensor data processing application, an environmental observation information collection application, a natural language processing application, a robot control application and / or an agent action decision application, etc.

[0141] Additionally, for example, the computing device (100) may include a model that performs a method for determining the behavior of an artificial intelligence agent in an environment where observation is limited, and may include an application that provides related services to the user, namely, an application specialized for generating environment-reflective sentences, establishing action plans, calculating action values, and active information retrieval.

[0142] In an embodiment, the computing device (100) may include a model trainer (160) for training a first artificial intelligence model and a second artificial intelligence model, and by storing and operating the trained artificial intelligence model, it may provide an action plan and action value based on user input and environment observation information as output data.

[0143] Each application of the computing device (100) may communicate with a number of other components of the computing device (100), such as, for example, at least one sensor, a context manager, a device state component, and / or additional components. In one embodiment, each application may communicate with each device component using an API (e.g., a public API). For example, an environment observation information collection application may communicate with a sensor component to obtain raw data, and an action decision application may communicate with the inference engine of the server computing system (130) via an API to receive environment-reflective sentences and action plans. In one embodiment, the API used by each application may be specific to that application.

[0144] FIG. 4 is a block diagram illustrating the internal architecture and data processing pipeline of an artificial intelligence model according to one embodiment of the present disclosure.

[0145] Referring to FIG. 4, the computing device may have a pipeline structure that receives input data (202) and generates output data (212) through a series of transformation processes to perform a method for determining the behavior of an artificial intelligence agent in a restricted environment. This process is performed through a preprocessing module (204), an encoder / embedding model (206), a neural network layer (208), and a decoder / generation head (210).

[0146] First, the preprocessing module (204) receives input data (202) (e.g., text prompt, image, or multimodal signal) from a user or system. The preprocessing module (204) performs tokenization and normalization on the input data to generate a sequence of tokens, which are the smallest units that the model can process.

[0147] Next, the encoder / embedding model (206) receives the generated token as input and converts it into a vector / embedding mapped to a number in a high-dimensional vector space. At this stage, the discrete information of the input data is converted into a continuous numeric matrix, making it a form that can be computed by the machine learning model.

[0148] Next, the neural network layer (208) receives the vector / embedding and performs deep computation. The neural network layer (208) may have a structure in which a plurality of sub-layers (e.g., Layer 1 to Layer N) are stacked. Each layer abstracts and refines input features through an attention mechanism or convolution operation, etc.

[0149] In particular, the final output of the neural network layer (208) is defined as a latent representation. This latent representation has a structure different from the original input data (202) and may correspond to an intermediate representation in which the semantic features of the data are highly compressed and abstracted. This implies that it is not a simple transmission of data, but a technical data structure that is valid only within the system.

[0150] Finally, the decoder / generation head (210) receives the potential representation as a conditioning input. The decoder / generation head (210) interprets the compressed potential representation and reconstructs or generates output data (212) in a form recognizable by the user (e.g., natural language text, image pixels, control codes, etc.).

[0151] This stepwise data transformation structure (token → vector → latent representation → output) can clearly demonstrate that it functions not as a simple sequence of operations, but as a concrete device that technically processes input data to generate useful information.

[0152] FIG. 5 illustrates an example of a block diagram in another aspect of a computing device (200) that implements a method for determining the behavior of an artificial intelligence agent in an environment where observation is limited, according to one embodiment of the present disclosure.

[0153] Referring to FIG. 5, the computing device (200) includes a plurality of applications (e.g., Application 1 to Application N). Each application can communicate with a central intelligence layer. For example, applications may include an image processing application, a text messaging application, an email application, a dictation application, a virtual keyboard application and / or a browser application. In one embodiment, each application can communicate with the central intelligence layer (and a model stored therein) using an API (e.g., a common API across all applications).

[0154] Additionally, in one embodiment of the present disclosure, the application may include an application for determining the behavior of an artificial intelligence agent in an environment where observation is limited, an energy management application, a logging and analysis application, etc.

[0155] The central intelligence layer may include a number of machine learning models. For example, as illustrated in FIG. 5, at least some of the machine learning models may be provided for each application and managed by the central intelligence layer. In another embodiment, two or more applications may share a single machine learning model. For example, in some embodiment, the central intelligence layer may provide a single model for all applications. In some embodiment, the central intelligence layer may be included within the operating system of the computing device (200) or otherwise implemented.

[0156] In one embodiment, the central intelligence layer may be integrated as part of the operating system or implemented as a separate logical layer, and may perform the role of transmitting input time series data to the corresponding model to return a prediction result.

[0157] The central intelligence layer can communicate with the central device data layer. The central device data layer may be a centralized data store for the computing device (200).

[0158] For example, the central device data layer can integrate and store sensor data, device state information, external environment information, etc., stored within the computing device (200), and provide this as input data required for the action decision service of an artificial intelligence agent in an environment where observation is limited. Each device component (e.g., sensor, state manager, etc.) can communicate with the corresponding data layer through a private API, etc.

[0159] As illustrated in FIG. 5, the central device data layer can communicate with a number of other components of the computing device (200), such as, for example, one or more sensors, a context manager, a device state component, and / or additional components. In some embodiments, the central device data layer can communicate with each device component using an API (e.g., a private API).

[0160] The technology described herein may refer to servers, databases, software applications, and other computer-based systems, as well as actions taken and information transmitted to or from said systems. It will be recognized that the inherent flexibility of computer-based systems allows for a wide range of possible configurations, combinations, division of tasks, and functionality between and from components. For example, the processes described herein may be implemented using a single device or component or multiple devices or components operating in combination. Databases and applications may be implemented in a single system or in a distributed system across multiple systems. Distributed components may operate sequentially or in parallel.

[0161] FIG. 6 is a block diagram illustrating the data flow and system interaction of a process for applying an artificial intelligence agent's behavior decision service in an environment where observation is limited, according to one embodiment of the present disclosure.

[0162] Referring to FIG. 6, the computing system may be configured as an organic data pipeline between a trigger event detector (310), an inference engine (330), and a mobile device screen (350).

[0163] First, the trigger event detector (310) is configured to monitor and detect a signal initiating the operation of the system. The trigger event detector (310) receives at least one of i) a time event indicating the arrival of a specific point in time, ii) a user action resulting from a user's physical input, or iii) a system state indicating a change in internal system data, and transmits an activation signal to the inference engine (330). This means that the service can be actively initiated depending on the situation without an explicit request from the user.

[0164] The reasoning engine (330) performs a multi-stage operation process that transforms raw data into a final result in response to the activation signal. This process is implemented as a series of logically connected prompt chains.

[0165] For example, the first prompt, contextualization (332), allows the inference engine (330) to receive unstructured raw data (e.g., user logs, channel metadata), analyze and refine it, and generate compressed summary information.

[0166] And the second prompt, Content Generation (334), can generate multiple candidate results that match the user's intent or situation using a generative AI model based on the summary information generated above.

[0167] In addition, the third prompt, Verification (336), performs filtering and verification by comparing the generated candidate results with a predefined policy (e.g., safety guidelines, format rules) to derive a reliable final result.

[0168] Finally, the final result generated by the inference engine (330) can be transmitted to and implemented on the mobile device screen (350) through an auto pre-filling (354) operation. Specifically, the system changes the state of the interface by directly writing the final result to a memory address of a specific target field (352) (e.g., text input field, setting value) within the mobile device screen (350). Here, the mobile device screen (350) may be an example of a user computing device (110).

[0169] Such a configuration can go beyond the simple display of information and provide specific technical means for data generated by an external trigger to physically control and complete the input interface of the user terminal.

[0170] FIG. 7 illustrates the architecture of a multi-agent system (Universal Dynamic Multi-Agent System, 400) according to one embodiment of the present disclosure.

[0171] Referring to FIG. 7, the system (400) may be configured around an orchestration engine (410) that determines and executes an optimal agent collaboration structure in real time according to the nature of the user's request or task. The system (400) may operate in an organically combined manner, including a complexity analyzer (405), an agent pool (420), a shared memory fabric (430), and a tool execution interface (440).

[0172] 1. Input Analysis and Dynamic Topology

[0173] The complexity analyzer (405) evaluates the complexity of the user query and the required domain expertise. Based on this evaluation result, the orchestration engine (410) dynamically configures an optimal agent collaboration topology for task resolution.

[0174] For example, in the case of a simple question, the engine (410) activates a single agent mode.

[0175] If a complex plan is required, the engine (410) configures a Supervisor pattern and instantiates one supervisor agent to control subordinate agents.

[0176] When high accuracy is required, the engine (410) configures a debate pattern and sets a control path so that multiple agents perform mutual criticism.

[0177] 2. Agent Pool and Instantiation (Agent Pool)

[0178] The agent pool (420) is a repository of template agents that have prompts and tool sets specialized for specific functions (e.g., web search, code generation, data analysis). The orchestration engine (410) can select and activate the necessary agents at runtime according to the determined topology.

[0179] 3. Shared Memory Fabric

[0180] The shared memory fabric (430) is a data pipeline that synchronizes the state and context between multiple collaborating agents. By mediating the short-term memory of individual agents and the knowledge base of the entire system, it can ensure that the output of Agent A is transferred to the input of Agent B without loss.

[0181] 4. Tool Execution & Feedback Loop

[0182] Each agent communicates with an external API (search engine, calculator, AWS, etc.) through a tool execution interface (440). The execution result generated at this time is fed back to the orchestration engine (410), and the engine can determine the consistency of the result and perform self-correction logic to instruct the agent to rework or proceed to the next step.

[0183] This structure enables the implementation of an adaptive artificial intelligence system that flexibly changes the system's processing structure according to the nature of the input problem, rather than a fixed (static) algorithm.

[0184] Meanwhile, the orchestration engine (410) analyzes the characteristics of the task received from the complexity analyzer (405) (e.g., creativity, logic, whether coding is required) and selects specialized agents waiting in the agent pool (420) to form a dynamic collaboration topology. The topology can be reconfigured into various operation modes as follows depending on the type of task.

[0185] First Operation Mode: Sequential Pattern

[0186] For tasks such as a single-flow creation or report writing, the orchestration engine (410) connects multiple agents in series. For example, it forms a pipeline in the order of user agent -> writer agent -> style agent to control the output of the previous stage to be passed as input to the next stage.

[0187] Second Operation Mode: Supervisor Pattern

[0188] In cases where complex sub-tasks are mixed, the engine (410) forms a centralized star topology in which one agent is designated as a supervisor and the remaining agents (e.g., research agent, math agent) are assigned as workers. The supervisor agent distributes tasks to sub-agents and aggregates the results.

[0189] Third Operation Mode: Hierarchical Pattern

[0190] In cases of high complexity, such as large-scale project management, the engine (410) establishes a command system by forming a tree structure topology in which a meta-agent is placed as the upper manager and multiple specialized agent groups are placed below it.

[0191] 4th Operation Mode: Debate Pattern

[0192] In cases where the correct answer is unclear or high reliability is required (e.g., social issue analysis), the engine (410) constructs a competitive topology in which multiple agents present different perspectives on the same topic and perform mutual criticism and voting to derive the optimal conclusion.

[0193] Fifth Operation Mode: Mixture-of-Agents Pattern

[0194] When parallel processing is required, the engine (410) forms a parallel processing structure that places multiple agents in multiple layers to perform tasks simultaneously and finally integrates the results through an aggregator agent.

[0195] Additionally, the agent pool (420) includes various template agents that can be deployed into the topology. Each agent is instantiated to perform the following core methodologies under the control of the orchestration engine (410).

[0196] For example, at least one agent can execute a loop that repeats reasoning and action to perform reasoning and action (ReAct Paradigm) that refines answers based on results from external tools (e.g., Google Search).

[0197] In addition, at least one agent can perform code-based behavior (CodeAct Paradigm) that involves complex calculations or data analysis by generating and executing executable code (e.g., Python) instead of natural language.

[0198] In addition, at least one agent can perform self-reflection to improve the quality of the output by carrying out a metacognitive process of self-evaluating (Critique) and modifying the generated output.

[0199] Furthermore, the system provides a shared memory fabric (430) to enable individual agents to cooperate organically without disconnection. This prevents context loss by managing the conversation history (Short-Term Memory) between agents and an external knowledge base in an integrated manner. Additionally, the tool execution interface (440) securely connects the agents with external APIs, cloud services, and databases through protocols such as Multi-Channel Processing (MCP).

[0200] In this way, the system (400) of the present disclosure implements an adaptive artificial intelligence platform that is not a fixed single model, but rather dynamically changes the system's processing structure and behavior according to the nature of the input problem.

[0201] Specifically, one embodiment of the present disclosure may be implemented as an Agentic RAG (Retrieval-Augmented Generation) system that supports advanced question-answering by including agent functions. This system is a search-based generation system that retrieves information from external data sources and generates answers based thereon.

[0202] The system's data processing pipeline can consist of a data extraction phase and an Agentic RAG pipeline phase. In the data extraction phase, content in various formats, such as text and images, is collected from designated websites. Subsequently, text and metadata are extracted from the collected content, the text is chunked into small units, and each chunk is vectorized using an embedding model and stored in a vector database.

[0203] The Agentic RAG pipeline can be divided into search and generation phases. In the search phase, when a user query is input, query rewriting and embedding are performed, and highly relevant documents are identified through similarity searches in a vector database. Subsequently, the retrieved results can be ranked based on relevance to construct context. In the generation phase, the user query and the retrieved context are combined to construct a prompt, which is then input into a large language model to generate the final response. During this process, the agent can respond to complex queries by utilizing agentic elements such as memory storage, calling external tools, and planning.

[0204] An AI agent according to one embodiment of the present disclosure can support complex decision-making and task execution through a hierarchical memory structure similar to human memory. The agent's memory can be broadly divided into short-term memory and long-term memory.

[0205] Short-term memory is a temporary memory space focused on the currently ongoing workflow and may include working memory, which manages workflow-specific reasoning and task flows, and cache memory, which provides quick access to frequently used data or results.

[0206] Long-term memory is a memory space based on knowledge and experience that is continuously preserved, and may include episodic memory, which records events or incidents manually saved in a specific workflow; semantic memory, which stores conceptual or factual knowledge; and procedural memory, which stores knowledge of how to perform specific tasks or procedural knowledge.

[0207] This memory structure can be integrated with the language model framework through a central memory controller. Additionally, it can leverage external knowledge or support real-time integration by connecting with external vector databases, semantic databases, or third-party APIs via the MCP server. Through this, the agent can generate user-customized responses that comprehensively reflect past experiences and current context.

[0208] One embodiment of the present disclosure may include various agentic workflows to solve complex problems. These workflows may be designed to suit specific business purposes.

[0209] For example, the system may include a Plan and Execute workflow. In this workflow, a Planner breaks down a single top-level task into multiple sub-tasks, specialized agents process each sub-task, and then integrates the results. This can be utilized for business process automation or data pipeline orchestration.

[0210] As another example, the system may include an Orchestrator-Worker workflow. In this structure, a central orchestrator language model breaks down tasks, distributes them to multiple worker language models for processing, and then integrates the results. This can be used in the implementation of Agentic RAGs or coding agents.

[0211] As another example, the system may include a routing workflow. This workflow is structured to analyze input tasks, classify them into the most suitable ones among several predefined subtasks, and forward them to a specialized language model or path capable of handling the task. This can be applied to customer support agents or multi-agent discussion systems.

[0212] One embodiment of the present disclosure may include a protocol for efficient and secure communication between a plurality of agents or between an agent and an external tool.

[0213] In one embodiment, an Agent2Agent (A2A) protocol may be used for communication between agents. The A2A protocol can enhance security by enabling each agent to communicate without directly sharing their internal data. Through this protocol, multiple agents can share tasks and negotiate, and each agent can operate independently using its own language model, framework, and database.

[0214] In another embodiment, the Multi-Channel Processing (MCP) protocol may be used for communication between an agent and an external function server. MCP has a structure that separates each external function, such as file access, search, and cloud API calls, into a separate server for communication. For example, one agent may communicate with a local file system or a search engine via the MCP protocol, while another agent may communicate with a cloud provider such as AWS or a communication tool such as Slack via the same protocol.

[0215]

[0216] Hereinafter, a method for determining the behavior of an artificial intelligence agent in an environment where observation is limited, using at least one artificial intelligence model, by a computing system (1000) according to the present disclosure, will be described in detail with reference to FIGS. 8 to 14.

[0217] In one embodiment, the method described below may be performed by at least one processor included in the server computing device (500). However, it is not limited thereto, and at least a part of the method may be performed by a processor provided in an agent, and another part may be performed by a processor of the server computing device (500).

[0218] FIG. 8 is a flowchart showing the overall flow of a method for determining the behavior of an artificial intelligence agent according to one embodiment of the present disclosure.

[0219] First, at least one processor can obtain user input that directs a predetermined task through a data input interface (S101).

[0220] For example, the user can input tasks that the agent needs to perform in natural language, such as "boil the lead." The data input interface can support various forms of input, including text, voice, and images, and the received user input can be utilized for generating environment-reflective data and establishing action plans.

[0221] In one embodiment, the acquired user input may be stored in at least one memory, but is not limited thereto, and may be processed by streaming processing or pipeline processing, or converted into other forms such as vector embeddings and utilized. The acquired user input may subsequently be utilized for the generation of hierarchical belief states and candidate behaviors.

[0222] Next, at least one processor can obtain information about the environment required for task execution (S103).

[0223] Information about the environment can be obtained in various ways. In one embodiment, the environment information can be extracted from raw data obtained through at least one sensor, such as a visual sensor or a distance measuring sensor, equipped in an agent. In this case, the server computing device (500) can control a physical device equipped with a sensor to scan the surrounding environment and identify at least one of the type, location, and state of an object from the obtained raw data to use as environment information.

[0224] For example, when an agent is mounted on a physical device such as a robot or an autonomous mobile device, it can acquire environmental observation information by directly scanning the surrounding environment through sensors, such as visual sensors or distance measuring sensors equipped on the device. In this case, since the agent directly collects information within its observation range that changes in real time according to the physical device's current location and direction of movement, the updating of environmental observation information is synchronized with the agent's physical movement.

[0225] In another embodiment, when an agent operates on a server computing system (130) and remotely controls a physical device, raw data acquired by a sensor equipped on the physical device can be received via a network (170) and used as environmental observation information. In this case, since the environmental observation information may be temporally out of sync with the actual environmental state due to network delay or communication failure, it is desirable to reflect this uncertainty when generating an environment-reflecting sentence.

[0226] In other embodiments, information about the environment is not limited to sensor data and can be obtained from various sources, such as environment maps or 3D models stored in a database (DB), state information from a physical simulation environment, real-time environment data obtained through an external API, or a pre-built knowledge graph. For example, in an autonomous driving environment, environment information can be constructed by combining a High Definition Map (HD Map) and GPS data, and in a smart factory environment, the current state information of the process line can be obtained from the DB of a process control system as information about the environment.

[0227] In any case, since only information recognizable within the agent's current observation range is acquired, for example, when the agent observes the living room, information about the environment containing only a portion of the overall environment may be obtained, such as "there is a banana in the living room, the location of the lead is unconfirmed." As such, since the information about the environment includes only what is recognizable within the agent's current observation range, it reflects the characteristics of a partially observable environment.

[0228] Next, at least one processor can generate environment-reflecting data based on information about the environment using at least one artificial intelligence model (S105).

[0229] Environment-reflective data may be data in which an artificial intelligence model interprets and infers environmental information acquired by an agent, transforming it into an abstracted form suitable for task execution.

[0230] This environment-reflective data functions as a key basis for judgment for the agent to make logical decisions in subsequent stages. In one embodiment, the environment-reflective data is not limited to a specific format and can be generated in various forms, such as sentences in natural language, structured JSON data, graphs representing relationships between objects, high-dimensional feature vectors, or token sequences, depending on the characteristics of the artificial intelligence model or the type of task.

[0231] For example, an environment-reflecting sentence (E) in the form of natural language is a sentence that summarizes the environmental state currently perceived by the agent in the form of natural language, and can serve as a basis for establishing a subsequent action plan. Referring to FIG. 10, when the agent (robot) is located in the living room and recognizes only bananas within the observation range and does not identify the locations of lead and tongs, the first artificial intelligence model can generate an environment-reflecting sentence, "Lead will be in the workshop, tongs are needed," from the environmental observation information, "Bananas are in the living room, lead location unconfirmed."

[0232] In this way, the environment reflection sentence can include not only facts directly observed by the agent, but also information inferred from environmental states outside the observation range by utilizing the prior knowledge of the first artificial intelligence model.

[0233] For example, environment-reflecting sentences may include the following information. First, they may include information regarding the types and locations of objects recognized within the agent's current location and observation range. For instance, directly observed facts may be described, such as "The agent is currently located in the living room, and there is a banana in the living room." Second, they may include inference information regarding objects or situations outside the observation range. For instance, inferences based on the prior knowledge of the first AI model may be included, such as "Since lead is a metallic material, it is presumed to be near the workshop or the furnace." Third, they may include information regarding tools or conditions required to perform the task. For instance, requirements associated with the task may be described, such as "Tools like tongs may be needed to pick up the lead."

[0234] In one embodiment, the environment-reflecting sentence may be generated in a structured format by the first artificial intelligence model. For example, an environment-reflecting sentence in a structured format organized by item may be generated, such as "Current location: Living room / Observed object: Banana / Estimated location of task-related object: Lead (estimated workroom), tongs (estimated workroom) / Requirement for task execution: Handle lead after securing tongs." This structured format enables the first artificial intelligence model to utilize key information more efficiently when establishing an action plan from the environment-reflecting sentence.

[0235] In another embodiment, the environment reflection sentence may be updated upon receiving negative feedback. For example, if the initial environment reflection sentence was "Lead is presumed to be in the workshop," but an attempt to pick up the lead with bare hands fails, additional information obtained through active information search, such as "Tongs tool needed," may be reflected to generate an updated environment reflection sentence like "Lead is in the workshop, tongs are needed to pick up the lead, and tongs are located in the workshop toolbox." As such, the environment reflection sentence has the characteristic of a dynamic sentence that is refined by continuously reflecting new information obtained during the task execution process.

[0236] At least one artificial intelligence model used to generate environment-reflective data may include an unfine-tuned closed-source large language model, and may generate environment-reflective data by extracting relevant information about task-related objects from information about the environment and converting it into a summary in a preset natural language format. At least one artificial intelligence model used to generate environment-reflective data may be referred to as the first artificial intelligence model.

[0237] The first AI model is a general-purpose, massive language model pre-trained with vast text data, capable of demonstrating high generalization capabilities for new environments and tasks without separate fine-tuning. Due to these characteristics, the first AI model can appropriately interpret environmental observation information and generate environment-reflective sentences even in new environments that were not included in the training data.

[0238] In one embodiment, the first artificial intelligence model can generate an environment-reflecting sentence by utilizing various reasoning methods such as Chain-of-Thought, ReAct, and Reflection to analyze environmental observation information step by step and extract key information necessary for task execution. For example, when applying the ReAct method, the first artificial intelligence model can progressively refine the environment-reflecting sentence by repeatedly performing a reasoning step that analyzes current environmental observation information and an action step that searches for necessary additional information.

[0239] In another embodiment, the first artificial intelligence model may include a Vision Language Model (VLM) capable of processing multimodal input, and may receive not only environmental observation information in text form but also visual information such as images and videos acquired from sensors to generate richer environment-reflecting sentences. In this case, the first artificial intelligence model may directly recognize the type, location, state, etc. of an object from the visual information and convert it into an environment-reflecting sentence in natural language format.

[0240] Next, at least one processor can establish a primary action plan for task execution based on environment-reflecting data using at least one artificial intelligence model (S107).

[0241] A primary action plan may include a sequence of actions to achieve a task. Referring to FIG. 9, in a situation where input "Boil lead" is received from a user and an environment reflection sentence "Lead is in the workshop, tongs needed" is generated, at least one AI model may establish a primary action plan in the sequence of 1) moving to the workshop, 2) securing tongs, 3) picking up lead, 4) moving to the furnace, and 5) boiling lead. At least one AI model derives an optimal sequence of actions by combining current environmental state information contained in environment reflection data with the goal of the task instructed by the user. Here, at least one AI model establishing the primary action plan may include a first AI model used to generate environment reflection data.

[0242] For example, the establishment of a first action plan may include the following process. At least one artificial intelligence model identifies the final target state of the task from user input and analyzes the current environmental state and the conditions necessary for task execution from environmental reflection data. Subsequently, an action plan is constructed by arranging intermediate steps to reach the target state from the current state in a logical order.

[0243] As illustrated in FIG. 9, a five-step action plan can be established by comprehensively considering information included in environment-reflecting sentences, such as the location of lead, the need for tools to handle lead, and the location of the furnace, in order to achieve the final goal of "boiling lead." In this process, at least one artificial intelligence model can determine the sequence of actions by considering the preconditions and postconditions between each action, for example, so that the "picking up lead" action must be performed after the "securing tongs" is completed.

[0244] In one embodiment, a primary action plan may be established in a structured format that includes execution conditions, expected results, and alternative actions for each action. For example, for the "solder picking" action in FIG. 9, each action may be specified by including "finish securing tongs" as an execution condition, "finish acquisition" as an expected result, and "search for a tool if tongs are not secured" as an alternative action. Such a structured action plan provides clear guidelines necessary for at least one artificial intelligence model to generate action data to be executed at each stage.

[0245] In another embodiment, at least one artificial intelligence model may generate multiple action plan candidates by utilizing a search method based on Tree-of-Thought or Monte Carlo Tree Search (MCTS), and select an optimal first action plan by evaluating the feasibility and efficiency of each candidate.

[0246] For example, in Fig. 9, if the location of lead is uncertain, a plan to "search the workshop first, then search the furnace" and a plan to "search the furnace first, then search the workshop" are generated respectively, and the plan with a higher probability of success can be selected as the primary action plan based on probabilistic estimation information included in the environment reflection data.

[0247] In another embodiment, the primary action plan can be established in a hierarchical structure according to the complexity of the task. The higher-level action plan includes key steps for achieving the task, and each key step can be broken down into lower-level detailed actions. For example, the higher-level action "pick up tongs" illustrated in the action execution sequence of FIG. 9 is broken down into lower-level actions such as "locate toolbox in the workshop," "move to toolbox," and "pick up tongs," allowing the second artificial intelligence model to calculate action values ​​more precisely.

[0248] Next, at least one processor can generate first action data for executing a first action plan using at least one artificial intelligence model (S109).

[0249] Action data refers to control information generated to enable an agent to actually execute an established action plan in a physical or digital environment. Such action data is not limited to a specific format and may include text data in natural language form that instructs specific actions the agent must perform, such as 'moving to the workroom' or 'securing the tongs'.

[0250] Furthermore, the above action data may be defined as a comprehensive concept including at least one of a command for directly controlling the agent's hardware, a token which is the output unit of a language model, a feature vector which is the result of a policy network's judgment, or a script in the form of code that can be directly executed within the environment.

[0251] At least one artificial intelligence model used to generate the first action data may include an open-source Small Language Model (SLM) fine-tuned with success trajectory data, and may convert each action directed in the first action plan into an action value in the form of a specific instruction in an execution environment. At least one artificial intelligence model used to generate the first action data may be referred to as the second artificial intelligence model.

[0252] Referring to Fig. 9, for the action plan "move to workshop," first action data such as "go to workshop" is generated, and then action data corresponding to each action is generated sequentially in the order of 1) go to workshop → 2) pick up tongs → 3) pick up lead → 4) go to foundry → 5) heat lead, so that the task can be executed.

[0253] In addition, the first AI model may have a division of roles where it is responsible for establishing the overall strategy and understanding the context of the task, while the second AI model is responsible for executing precise actions in a specific environment.

[0254] For example, the second artificial intelligence model is an open-source Small Language Model (SLM) that has learned the format of action commands in a specific execution environment, the control method of physical devices, and the interaction patterns with the environment through fine-tuning using success trajectory data, so that each step of the action plan can be converted into a specific action value that can be immediately executed in that environment.

[0255] Open-source SLMs have the advantage of faster inference speeds and lower operating costs compared to closed-source LLMs due to their smaller number of parameters. Furthermore, because their open-source nature allows for direct access to internal parameters for fine-tuning, they can generate action values ​​optimized for tasks in specific environments. This division of roles can simultaneously resolve the issues of high costs and inference latency associated with using only closed-source LLMs, as well as the performance degradation in generalization for new environments that occurs when fine-tuning only open-source SLMs.

[0256] In one embodiment, the second AI model may operate by processing each action of the action plan sequentially, while reflecting the execution result of the current action in the calculation of action data for the next action. For example, information about the new environment acquired after the "go to workshop" action is executed and the agent arrives at the workshop may be reflected in the calculation of action data for the next action, "pick up tongs." Through this, the second AI model can calculate action data by reflecting the environmental state that changes in real time, rather than simply executing a pre-established action plan in sequence.

[0257] In another embodiment, the second AI model can evaluate the uncertainty regarding the current action data output based on the probability distribution of the output token. If the probability of the output token is below a preset threshold, this can be treated as a type of negative feedback to control the re-examination of action data after performing active information search and updating the environment-reflecting sentence. For example, if the output token probability of the second AI model is low for the action of attempting to pick up a tongs, this may indicate that there is insufficient accurate location information of the tongs in the current environment-reflecting data; accordingly, the action data can be re-examined to confirm the location of the tongs.

[0258] In another embodiment, the second artificial intelligence model can generate low-level control signals as action data for directly controlling the actuator of a physical device. For example, specific control parameters required for the operation of the physical device, such as the joint angle of a robot arm, the pressure value of a gripper, and the movement speed, can be generated as action data and transmitted to the actuator of the physical device. In this case, the second artificial intelligence model can generate precise control signals by learning the kinematic characteristics of the physical device and the physical interaction patterns with the environment through task-specific fine-tuning.

[0259] Next, at least one processor can input the generated first action data into a subsequent processing component (S111).

[0260] For example, at least one follow-up processing component is a component that receives first action data and executes an actual task, and can be implemented in various forms depending on the type of environment in which the agent operates.

[0261] In one embodiment, at least one subsequent processing component may include a control module of a physical device equipped with a drive unit, such as a robot arm, an autonomous mobile robot, or a drone. In this case, the server computing device (500) generates a control signal that controls the physical movement of the physical device based on the first action data, and the physical device can execute a task by receiving the control signal and driving the drive unit. For example, when the first action data for "go to workshop" is input, the driving module of the autonomous mobile robot can generate a movement path toward the workshop and drive the motor to move to that location.

[0262] In another embodiment, at least one subsequent processing component may include an execution module of a simulation environment. In this case, the first action data is input into a physics engine-based simulator to execute a task in a virtual environment, and state information obtained from the simulation results can be used as information about the environment of the next action. For example, the execution module of the simulation environment can be utilized as a subsequent processing component in scenarios such as verification of a robot control algorithm or pre-testing in a dangerous physical environment.

[0263] In another embodiment, at least one subsequent processing component may include an industrial control system, such as a manufacturing process control system of a smart factory, a conveyor control module of a logistics automation system, or an IoT device control interface of a smart home. For example, when a first action value, such as "start part assembly," is input in a smart factory environment, the manufacturing process control system may sequentially control robot devices on the corresponding process line to perform an assembly task.

[0264] In another embodiment, at least one subsequent processing component may include a user interface module of a user computing device (110). In this case, the first action data may be presented to the user through the user interface, and the actual execution may be performed only when the user approves the action, in a Human-Agent Collaboration manner. For example, an action "Move to furnace" proposed by an agent may be displayed on the screen via a smartphone application, and the action may be controlled to be executed when the user presses a confirmation button.

[0265] As such, the subsequent processing component can be implemented in various forms depending on the characteristics of the environment in which the agent operates, and the action determination method according to the present disclosure has high versatility in that it is applicable regardless of the type of the subsequent processing component.

[0266] The action decision method described above can be implemented by a plurality of modules included in the server computing device (500).

[0267] Referring to FIG. 10, the server computing device (500) may include a user input acquisition module (510), an environment information acquisition module (520), an environment reflection data generation module (530), an action plan establishment module (540), an action data output module (550), a negative feedback reception module (560), an active information search module (570), an environment reflection data update module (580), and an information search rule update module (590).

[0268] The user input acquisition module (510) can acquire input instructing a user to perform a task through a data input interface and store it in memory. For example, it acquires and stores user input such as "boil lead" and transmits it to the environment-reflecting data generation module (530) and the action plan establishment module (540). In one embodiment, the user input acquisition module (510) can receive various forms of user input, such as text input, voice recognition, and image input, and can convert the received input into a natural language format and store it.

[0269] The environment information acquisition module (520) can receive environment observation information acquired from the surrounding environment through sensors equipped in the agent and store it in memory. For example, it receives and stores environment information such as "banana in the living room, lead location unconfirmed" acquired within the agent's observation range. In one embodiment, the environment observation information acquisition module (520) can preprocess raw data acquired from a plurality of sensors, such as a visual sensor and a distance measuring sensor, to identify the type, location, and state information of an object, and convert it into a structured format for storage.

[0270] The environment reflection data generation module (530) receives user input from the user input acquisition module (510) and environment observation information from the environment information acquisition module (520), respectively, and can generate environment reflection data using at least one artificial intelligence model. For example, it generates environment reflection data such as "Lead will be in the workshop, tongs are needed" and transmits this to the action plan establishment module (540). The environment reflection data generation module (530) can supplement incomplete environment information by including inference results utilizing prior knowledge of at least one artificial intelligence model, as well as information directly observed by the agent, in the environment reflection data.

[0271] The action plan establishment module (540) receives environment reflection data from the environment reflection data generation module (530) and can establish an action plan for task execution using at least one artificial intelligence model. For example, based on the environment reflection data, a primary action plan can be established for task execution in the order of 1) moving to the workroom, 2) securing tongs, 3) picking up lead, 4) moving to the furnace, and 5) boiling lead. As illustrated in FIG. 10, when the environment reflection data update module (580) updates the environment reflection data, the action plan establishment module (540) can re-establish a modified action plan based on the updated environment reflection data.

[0272] The action data output module (550) receives an action plan including a plurality of unit actions from the action plan establishment module (540) and can output action data for executing the action plan using at least one artificial intelligence model.

[0273] For example, if the established action plan includes a series of processes such as "go to the workshop and secure the tongs," the action data output module (550) can output a first action data called "go to workshop" as the first unit action for executing the action plan. Subsequently, the action data output module (550) can sequentially output action data corresponding to each step according to the flow of the action plan and transmit it to a subsequent processing component.

[0274] Referring to FIG. 11, a task is executed based on the first action data calculated by the action data calculation module (550) (S121), and if negative feedback is received from the result of the task execution (S123), the following steps may be performed.

[0275] The negative feedback receiving module (560) can receive negative feedback regarding the execution result of the task. Corresponding to step (S123) of FIG. 11, the negative feedback may include at least one of the following: no response from the environment after the action execution (None Feedback), the probability of the output token of the artificial intelligence model being below a preset threshold, or the goal of the task not being achieved.

[0276] First, "None Feedback" occurs when there is no response from the environment after an action is executed, or when the action performed by the agent is impossible or invalid in the current environmental state. For example, if an agent attempts to pick up lead with bare hands but the lead is a high-temperature object or the agent's gripper is unable to grasp it, the environment may return None without any state change. As such, None Feedback serves as a signal indicating a discrepancy between the action the agent intends to perform and the actual environmental state, suggesting an error in the current environmental data or the action plan.

[0277] Second, if the output token probability of at least one AI model is below a preset threshold, it indicates that the AI ​​model exhibits high uncertainty when generating action data based on current environment reflection data and action plans. For example, in a situation where an agent needs to determine the next action to pick up lead, if the current environment reflection data does not include location information of the tongs and at least one AI model generates an output with a low probability among multiple action candidates such as "pick up tongs" and "search for tongs," this can be treated as negative feedback indicating that the current environment information is insufficient. In this case, since executing an action without additional information search is highly likely to lead to an incorrect action, the negative feedback receiving module (560) can transmit this to the active information search module (570) to induce an update of the environment reflection data.

[0278] Third, the case where the task goal is not achieved is when the action performed by the agent is executed but the expected result does not appear, or when the final goal of the task is not achieved. For example, this applies when the agent moves to a furnace and performs the action of boiling lead, but the lead does not boil because the temperature of the furnace is insufficient, or when the final goal state of the task is not achieved despite the agent having completed all action plans. In this case, the negative feedback receiving module (560) can determine whether the goal was not achieved by comparing the task execution result with the goal state, process this as negative feedback, and transmit it to the active information search module (570).

[0279] The active information search module (570) can perform information search to actively obtain additional information necessary for task execution based on at least one of negative feedback, environment reflection data and previous action history, in response to step (S125) of FIG. 11.

[0280] The active information search module (570) is distinguished from the conventional passive information search method that relies on fixed search rules by analyzing negative feedback generated during the task execution process, determining for itself the information needed in the current situation, and actively searching for it. Referring to FIG. 10, the active information search module (570) may include an additional information inference module (571), an external information source search module (572), and an environment re-search control module (573).

[0281] The additional information inference module (571) can infer additional information necessary for task performance by inputting at least one of negative feedback, environmental reflection data, and previous action history into at least one artificial intelligence model.

[0282] Specifically, the additional information inference module (571) comprehensively analyzes the type of negative feedback, environmental state information included in the environmental reflection data up to that point, and the history of actions performed up to that point to identify the cause of the task failure and infer what additional information is needed to resolve it. For example, if None Feedback is returned for an attempt to pick up lead with bare hands, the additional information inference module (571) can identify the cause that "a separate tool is needed to pick up the lead" and infer "the type and location of a tool suitable for handling lead" as additional information.

[0283] As another example, if the output token probability of at least one artificial intelligence model is below a threshold, the additional information inference module (571) can identify that location information of a specific object is missing from the current environment reflection data and infer the location of the object as additional information. In one embodiment, the additional information inference module (571) can analyze the root cause of negative feedback through stepwise inference using a chain-of-thought method and determine the type and method of obtaining additional information to resolve it.

[0284] The external information source search module (572) can generate a search query to obtain additional information inferred by the additional information inference module (571), and based on the search query, obtain knowledge about additional information from the external information source and store it in memory.

[0285] Specifically, the external information source search module (572) can convert the content of the inferred additional information into a search query suitable for searching and send the search query to an external information source such as an external knowledge base, search engine, API, etc., to collect relevant knowledge.

[0286] For example, a search query such as "lead handling tool" can be generated and an external knowledge base searched to acquire the knowledge that "tongs must be used when handling high-temperature metals such as lead, and tongs are generally stored in a toolbox in a workshop." In one embodiment, the search query generated by the external information source search module (572) may be determined based on information search rules updated by the information search rule update module (590). For example, what initially used only "lead location" as a search query may be expanded to use "lead location + lead handling tool" as a search query based on information search rules updated from previous task performance experience. By adaptively updating the information search rules in this way, the agent can more efficiently obtain the necessary information when similar situations recur.

[0287] The environment re-exploration control module (573) re-explores the surrounding environment by controlling a physical device based on additional information inferred by the additional information inference module (571), and stores the new observation information obtained through re-exploration in memory.

[0288] Specifically, the environment re-exploration control module (573) identifies a location or object that needs to be explored from the inferred additional information and re-explores the environment by controlling the agent's physical device (e.g., camera, moving drive unit, etc.) to move to that location or intensively observe that object.

[0289] For example, based on the inference result that "the tongs will be in the workshop toolbox," the agent can be moved toward the workshop toolbox and scanned to confirm the location of the tongs. In one embodiment, the environment re-search control module (573) can increase the efficiency of re-search by linking with external knowledge obtained from the external information source search module (572) and prioritizing the search for location information presented in the external knowledge. For example, if information that "the tongs are stored in the workshop toolbox" is obtained from an external knowledge base, the environment re-search control module (573) can confirm the location of the tongs more quickly by prioritizing the search for the toolbox instead of randomly searching the entire workshop.

[0290] In this way, the additional information inference module (571), the external information source search module (572), and the environment re-search control module (573) operate complementarily to enable the agent to obtain additional information necessary in a negative feedback situation from various angles. In one embodiment, the three modules may operate sequentially, and either or both of the external information source search module (572) and the environment re-search control module (573) may be selectively activated according to the result inferred by the additional information inference module (571).

[0291] For example, if the inferred additional information is general knowledge searchable in an external knowledge base, the external information source search module (572) is activated, and if direct observation of a specific location in the current environment is required, the environment re-search control module (573) is activated to obtain additional information.

[0292] The environment reflection data update module (580) can update existing environment reflection data based on additional information obtained by the active information search module (570) in response to step (S127) of FIG. 11, and transmit the updated environment reflection data to the action plan establishment module (540).

[0293] Specifically, the environment reflection data update module (580) can update the environment reflection data by integrating additional information obtained from the additional information inference module (571), the external information source search module (572), and the environment re-search control module (573) to correct errors in the existing environment reflection data or supplement missing information.

[0294] At this time, the environment reflection data update module (580) can perform the update by reflecting newly acquired additional information while maintaining valid information included in the existing environment reflection data, rather than simply replacing the existing environment reflection data.

[0295] For example, in the existing environment reflection data "Lead is presumed to be in the workshop," the estimated information regarding the location of the lead is maintained, but new information acquired through active information search—"Tongs are needed to pick up the lead, and the tongs are located in the workshop toolbox"—is added, so the environment reflection data can be updated to "Lead is in the workshop, tongs are needed to pick up the lead, and the tongs are located in the workshop toolbox."

[0296] In one embodiment, the environment reflection data update module (580) can determine the weight to be reflected in the update by evaluating the reliability of additional information. For example, additional information obtained by direct observation through the environment re-exploration control module (573) is reflected in the environment reflection data with high reliability, whereas additional information based on external knowledge obtained through the external information source search module (572) can be reflected in the environment reflection data by being marked as estimated information. In this way, the updated environment reflection data includes direct observed facts and inferred information separately, so that the action plan establishment module (540) can contribute to establishing an action plan more accurately.

[0297] In another embodiment, the environment reflection data update module (580) may operate by gradually updating the environment reflection data according to the progress stage of the task. That is, at the beginning of the task execution, environment reflection data is generated based on limited information obtained within the agent's observation range, and whenever negative feedback is received during the task execution process, additional information obtained through active information search is cumulatively reflected, so that the environment reflection data gradually becomes richer.

[0298] For example, the environment reflection data can be continuously refined throughout the task execution in such a way that the initial environment reflection sentence "Lead is presumed to be in the workshop" is updated to "Lead is in the workshop, tongs are needed" through a first update, and then updated to "Lead is on the workshop shelf, tongs are in the toolbox, furnace temperature is sufficient" through a second update.

[0299] The environment reflection data updated by the environment reflection data update module (580) is transmitted to the action plan establishment module (540) and used to establish a modified action plan. As illustrated in FIG. 10, the action plan establishment module (540) re-establishes the modified action plan based on the updated environment reflection data, and the action data output module (550) can output second action data based on the modified action plan.

[0300] For example, based on updated environment-reflecting data "Lead is in the workshop, tongs are needed and tongs are located in the toolbox," a modified action plan is established in which the "pick up tongs" action is inserted before the existing "pick up lead" action, and accordingly, a second action data called "pick up tongs" is generated so that the task can be re-executed. In this way, the environment-reflecting data update module (580) functions as a key component that enables the agent to correct errors on its own and adaptively improve the action plan in a negative feedback situation.

[0301] The information search rule update module (590) can update information search rules including the search scope and search query generation method for active information search to be performed in similar situations in the future, based on additional information obtained through active information search and the execution results of tasks.

[0302] Specifically, the information search rule update module (590) updates the information search rules by comprehensively analyzing the cause of negative feedback that occurred during the current task execution process, the content of additional information obtained through active information search, and the execution results of a modified action plan based on updated environment reflection sentences. The information search rules include rules regarding what type of information to search for in a specific situation, what search query to generate, which external information source to refer to first, and which environment location to re-search.

[0303] In one embodiment, the information search rule update module (590) can update the information search rule by adding a new search target to the existing search range or modifying the existing search query based on additional information obtained through active information search. For example, if knowledge that "tongs are needed for handling lead" is acquired through an experience of failing to pick up lead with bare hands while performing a lead handling task, the information search rule can be updated by adding "lead handling tool" as a new search target to the existing search targets and modifying the existing "lead location" search query to "lead location + lead handling tool." Through this, if a similar lead handling task occurs in the future, the agent can perform the task more efficiently without repeating the same failure by searching for the lead location and the lead handling tool together from the beginning.

[0304] In another embodiment, the information search rule update module (590) can update the information search rules by adjusting the search priority. For example, if a pattern of failure occurring at every tool acquisition step in a task handling high-temperature metal is repeated, the information search rule update module (590) can adjust the search priority to search for tool-related information with the highest priority in the high-temperature metal handling task. Through this, the agent can prevent unnecessary failures and improve task completion efficiency by preemptively searching for information related to tool acquisition from the initial stage of task execution.

[0305] In another embodiment, the information search rule update module (590) may reflect the results of the reliability and usefulness evaluation of external information sources in the information search rules. For example, if a specific external information source has a history of providing accurate information regarding lead handling tools, the information search rules may be updated to prioritize referencing that external information source, and conversely, if there is a history of providing inaccurate information that led to task failure, the information search rules may be updated by lowering the priority of that external information source or excluding it from the search target.

[0306] The information search rule update module (590) can store the information search rules updated as described above in at least one memory. The stored information search rules can be referenced by the external information source search module (572) and the environment re-search control module (573) when a situation requiring active information search occurs in a task performed thereafter, and can be utilized for generating a search query and determining the search range.

[0307] In one embodiment, information search rules stored in memory can be structured and stored according to the type of task, characteristics of the environment, and the type of negative feedback generated, and provide a self-evolving characteristic in which the agent's information search ability is gradually improved by being continuously updated as new task performance experience accumulates.

[0308] The process of updating environment-reflecting sentences and information search rules through active information search described above is conceptually illustrated in Fig. 12.

[0309] Referring to FIG. 12, when at least one artificial intelligence model generates action data and executes a task based on an initial environment reflection sentence E, but negative feedback is received, active retrieval is performed to acquire new knowledge, and the environment reflection data is updated based on this. The updated environment reflection data is input into at least one artificial intelligence model to generate modified action data, and the task is successfully completed by executing the modified action.

[0310] In addition, new knowledge obtained through active information search is not only utilized for updating environment-reflecting data, but is also reflected in information search rules by adding new search targets to existing search ranges or modifying existing search queries through the information search rule update module (590), and the updated information search rules are stored in memory and utilized for active information search in similar situations in the future.

[0311] As such, the mechanism illustrated in FIG. 12 is a key mechanism that implements self-evolving characteristics, in which an agent analyzes task execution failures in an environment with limited observation and adaptively modifies its behavior, while reflecting that experience in information search rules to continuously improve its ability to perform tasks in similar situations in the future.

[0312] For example, referring to the flow on the left in Fig. 12, with "Lead is in the workshop" generated as the initial environment reflection sentence E, the second AI model calculates action data based on the environment reflection sentence and executes the "Pick up lead with bare hands" action. However, since lead is a high-temperature metal material and cannot be picked up with bare hands, negative feedback (Failure - None) is received from the result of the task execution. Such None Feedback, where no response is returned from the environment, indicates that the current environment reflection sentence is missing information necessary for performing the task, namely, information that a separate tool is required to pick up the lead.

[0313] When negative feedback is received, active retrieval can be performed. First, in step 1) additional information inference, the additional information inference module (571) inputs the negative feedback "failure to pick up lead with bare hands" and the previous action history into the first artificial intelligence model to analyze the cause of the task failure. At least one artificial intelligence model recognizes that lead is a high-temperature metal material and infers additional information that "a separate tool such as tongs is needed to safely pick up high-temperature metals such as lead."

[0314] Next, 2) in the external source query step, the external information source search module (572) creates a search query for "lead handling tool" to search an external knowledge base and obtains the knowledge that "tongs must be used when handling high-temperature metal materials such as lead, and tongs are generally kept in a toolbox in a workshop."

[0315] Next, in the 3) environment re-search stage, the environment re-search control module (573) first searches the toolbox in the workshop based on location information presented from external knowledge and directly observes and confirms that there is a tongs inside the toolbox. By performing these three methods of active information search in a mutually complementary manner, new, specific, and highly reliable knowledge is obtained, such as "tongs are needed when handling lead, and the tongs are located in the toolbox in the workshop."

[0316] The newly acquired knowledge can be utilized in two ways, for example. Referring to the flow on the right side of FIG. 12, first, the environment reflection sentence E is updated by the environment reflection data update module (580) to “Lead is in the workshop, tongs are needed when handling lead, tongs are located in the workshop toolbox.” The updated environment reflection sentence is transmitted to the action plan establishment module (540) to re-establish a modified action plan, and at least one artificial intelligence model produces modified action data based on the updated environment reflection sentence. Accordingly, the agent successfully completes the task by executing the modified action in the order of first picking up tongs from the toolbox and then picking up lead using the tongs. In this way, through the update of the environment reflection sentence, the agent can perform an adaptive response that corrects the failure of the initial action execution and switches to the correct action.

[0317] Secondly, as illustrated at the bottom of FIG. 12, the search rule update can be performed by the information search rule update module (590). The information search rule update module (590) updates the information search rule by adding a "lead handling tool" as a new search target to the existing search range and modifying the existing search query based on new knowledge obtained through active information search. Specifically, the information search rule is updated in such a way that the search query, which was a search for "lead location" before, is expanded to a search for "lead location + lead handling tool" after, and the updated information search rule is stored in at least one memory. Through this, when a similar task related to lead handling is performed in the future, the agent can complete the task more efficiently without repeating the same failures, such as the attempt to pick up lead with bare hands, by searching for the location of the lead handling tool along with the location of the lead from the initial stage of task execution.

[0318] Meanwhile, an agent according to one embodiment of the present disclosure provides a code-based action generation pipeline that can flexibly handle actions that are not present in the existing action set, as illustrated in FIG. 13.

[0319] Referring to FIG. 13, the code-based action generation pipeline may include a step for determining whether an action code exists (S201), a step for generating an action code (S203), a step for executing a simulation environment (S205), a step for evaluating a VLM-based action (S207), and a step for executing with a new action (S209).

[0320] First, at least one processor can determine whether an action code corresponding to an action required for task execution exists in an existing action set using at least one artificial intelligence model (S201).

[0321] The existing action set is a set of actions that the agent is already registered to perform, and may include action commands predefined in the execution environment, such as "go to workshop", "pick up lead", "go to foundry", etc.

[0322] At least one AI model determines whether an action instructed in the action plan is included in an existing action set, and if it is included (YES), the task can be performed by executing the existing action as is. In this case, processing efficiency is high because the commands of the existing action set are directly utilized without a separate code generation process.

[0323] If the corresponding action code does not exist in the existing action set (NO), at least one processor can generate code that performs the necessary actions for task execution using at least one artificial intelligence model (S203).

[0324] At least one artificial intelligence model understands the meaning and purpose of the action directed in the action plan and generates code that conforms to the physical laws and interface specifications of the corresponding execution environment. For example, if there is no action code corresponding to "picking up lead with tongs" in the existing action set, at least one artificial intelligence model can generate code that implements the action of picking up lead using tongs by considering the current position of the tongs, the position of the lead, the range of motion of the gripper, etc. In one embodiment, the generated code can be written in a form that is executable in both a simulation environment and an actual physical device, and can be generated in accordance with the API specifications of the execution environment in which the agent operates.

[0325] Next, at least one processor can execute the generated code in a simulation environment to obtain a State-Action Pair (S205).

[0326] A simulation environment is a virtual execution environment that mimics the actual physical environment, providing a space to safely verify generated code before executing it directly on a real physical device. Through code execution in the simulation environment, pairs of an agent's state and the action performed by the code in that state are obtained; these are utilized as foundational data to evaluate whether the code's behavior is performed as intended. For example, by executing code to pick up lead with tongs in the simulation environment, state-action pairs can be obtained that include changes in the tongs' position, whether the lead is grasped, and the object's movement path.

[0327] Next, state-action pairs obtained from the operation of the code executed in the simulation environment can be input into a Vision Language Model (VLM) to evaluate the operation of the code (S207). The VLM functions as an Action Code Critic that evaluates whether the generated code correctly performs the intended operation by visually analyzing the simulation results.

[0328] Specifically, the VLM determines the appropriateness of the code's operation by evaluating whether the tongs stably gripped the lead in the simulation execution results, whether the lead did not move or fall in an unintended direction, and whether the agent's actions conformed to the laws of physics. In one embodiment, the VLM may receive a simulation result image or video sequence as input and output an evaluation result in a natural language format; the evaluation result may include an explanation of whether the code's operation is appropriate, as well as the cause and direction for correction if it is inappropriate.

[0329] If the evaluation result of the VLM indicates that the operation is inappropriate (NO), at least one processor returns to step (S203) and repeats the process of modifying and re-verifying the code. At this time, at least one artificial intelligence model refers to the evaluation result and modification direction presented by the VLM to identify problems in the existing code and regenerate the modified code. For example, if the VLM outputs an evaluation stating that "there is a risk that the lead will deform because the gripping pressure of the clamp is excessive," at least one artificial intelligence model can regenerate the code by modifying it to adjust the gripping pressure of the clamp. In this way, the quality of the code is gradually improved through the repeated performance of code generation, simulation execution, and VLM-based evaluation.

[0330] If the evaluation result of the VLM determines that the operation is appropriate (YES), at least one processor executes the verified code as a new action to perform the task (S209). The verified code is passed to a subsequent processing component and executed on an actual physical device or execution environment, and in one embodiment, the code may be added to the existing action set as a new action and stored so that it can be reused in similar situations in the future. Through this, the self-evolving characteristic of the agent's action set, which gradually expands through task performance experience, is implemented.

[0331] In this way, the code-based action generation pipeline illustrated in FIG. 13 enables the agent to flexibly perform tasks in diverse and complex real-world environments by allowing the first artificial intelligence model to directly generate code and verify it through simulation and VLM-based evaluation, even in situations where new actions are required beyond the limitations of the existing action set.

[0332] Furthermore, an agent according to one embodiment of the present disclosure provides a decision tree-based artificial neural network generation pipeline in which, when complex physical control operations that are difficult to express in code are required, at least one artificial intelligence model directly generates parameters of an artificial neural network model and uses them to calculate control information, as illustrated in FIG. 14.

[0333] Referring to FIG. 14, the decision tree-based artificial neural network generation pipeline may include an action data output step (S301), an LLM NN parameter generation step (S303), an artificial neural network model configuration step of a decision tree structure (S305), a current state information input and control information output step (S307), and a control information-based action execution step (S309).

[0334] First, at least one processor can generate action data for executing a behavior plan using at least one artificial intelligence model (S301). The generated action data represents the goal of the action to be performed by the agent and can subsequently be used to generate parameters for an artificial neural network model for physically executing the action.

[0335] For example, if action data such as "heat lead" is generated, executing it requires complex physical controls, such as controlling the furnace temperature and the timing and method of lead input; this corresponds to a continuous control operation that is difficult to express with simple code.

[0336] Next, at least one processor can generate parameters of an artificial neural network model for calculating control information required for performing actions based on action data using at least one artificial intelligence model (S303).

[0337] In this step (S303), at least one artificial intelligence model can analyze the goal of the action directed in the action plan and the characteristics of the execution environment to directly generate parameters of an artificial neural network model for calculating control information required for performing the action.

[0338] This is differentiated from conventional methods, namely neural networks learning parameters through backpropagation or diffusion model-based neural network weight generation methods (e.g., NeRN), in that at least one artificial intelligence model directly generates the parameters of the artificial neural network model by utilizing language generation capabilities.

[0339] In one embodiment, at least one artificial intelligence model constrains the structure of an artificial neural network model into a decision tree form and can generate parameters of the artificial neural network model of the decision tree structure in text format. When the artificial neural network model is constrained into a decision tree structure, parameters for calculating control information required for operation execution can be expressed as text, making it possible for at least one artificial intelligence model to directly generate parameters and, if necessary, read and modify them.

[0340] In this case, the text representation of the parameter may be a description of the judgment criteria for conditional branching, control values ​​at each branch, threshold values ​​of branching conditions, etc., in natural language or structured text format.

[0341] In another embodiment, at least one artificial intelligence model can generate parameters for artificial neural network models of various structures capable of expressing and interpreting parameters in text format, and is not limited to a specific model structure.

[0342] Next, at least one processor can construct an artificial neural network model of a decision tree structure based on parameters generated in text format (S305). As shown in FIG. 14, the artificial neural network model of a decision tree structure has a tree structure that performs a judgment on condition 1 at the root node and outputs a control value A if condition 1 is satisfied (YES), and if condition 1 is not satisfied (NO), performs an additional judgment on condition 2 and outputs a control value B if condition 2 is satisfied (Y) and a control value C if condition 2 is not satisfied (N).

[0343] For example, in a lead boiling task, if condition 1 is "Is the temperature of the lead 327°C or higher?", if it is YES, "turn off the power to the furnace (control value A)" is performed, and if it is NO, additionally determine condition 2, "Is the furnace operating time 10 minutes or longer?", and if it is YES, output "maintain temperature mode (control value B)", and if it is NO, output "continue furnace operation (control value C)". An artificial neural network model with a decision tree structure can be configured in this way.

[0344] By constraining the artificial neural network model with a decision tree structure in this way, the process of generating control information for complex physical control operations is expressed in an interpretable form, so that at least one artificial intelligence model can understand and modify it.

[0345] As illustrated on the left side of Fig. 14, if task execution fails, the method of calculating control information of the artificial neural network model can be improved by returning to step (S303) through the “parameter modification on failure” path and reviewing and modifying at least one artificial intelligence model generated parameter in text format.

[0346] For example, if the power to the furnace is cut off before the lead boils sufficiently because the lead temperature threshold is incorrectly set, at least one artificial intelligence model can modify the parameters by analyzing text-based parameters and adjusting the temperature threshold, and then reconstruct the artificial neural network model with the modified parameters to re-execute the task. The fact that direct parameter interpretation and modification by at least one artificial intelligence model is possible in this way distinguishes it from conventional black-box neural network methods.

[0347] Next, at least one processor can generate control information using an artificial neural network model composed of generated parameters (S307).

[0348] Specifically, current state information is input into a configured artificial neural network model to output control information necessary for performing operations. The current state information includes real-time environmental observation data acquired from the agent's sensors and may include, for example, the current temperature of the lead, the operating status of the furnace, and the current position and angle of the tongs. The artificial neural network model receives the current state information, sequentially determines each conditional branch, and outputs a final control value; the output control value is utilized as a specific control parameter to control the actuator of the physical device. For example, by receiving state information such as the current temperature of the lead and the operating time of the furnace, it can output control information such as the furnace temperature setpoint and whether to continue heating.

[0349] Finally, at least one processor can perform a task by executing an action based on the generated control information (S309).

[0350] Specifically, the server computing device (500) generates a control signal to control the driving unit of the physical device based on control information, and the physical device can perform a task by receiving the control signal and driving the driving unit. If the task execution is successfully completed, the pipeline is terminated, and if it fails, it can return to step (S303) via the parameter modification path as described above to modify the parameters and re-execute the task.

[0351] In this way, the pipeline illustrated in FIG. 14 provides a highly scalable action decision system that enables an agent to flexibly perform physical tasks of various complexity by directly generating parameters of an artificial neural network model for calculating control information necessary for performing operations, using an artificial neural network model composed of generated parameters for complex physical control operations that are difficult to express in code, and executing actions based on the generated control information.

[0352] The method and system for determining the behavior of an artificial intelligence agent in an environment where observation is limited, according to the embodiments of the present disclosure described above, can be applied to various industrial fields.

[0353] In one embodiment, the present disclosure may be applied to the fields of manufacturing and logistics. In a smart factory environment, a robot agent operates in a partially observable environment where it is difficult to fully grasp the status of the entire process line due to limitations in the observation range of sensors, blockage of view by physical obstacles, etc.

[0354] When applying the method according to the present disclosure, a robot agent can generate an environment-reflecting statement from the currently observable state of the process line and perform tasks such as component assembly, welding, and painting based thereon.

[0355] For example, if an agent performs an assembly task without knowing the location of a specific part, it can successfully complete the assembly task by identifying the part's location through active information search and updating environment-reflecting statements. In a logistics environment, autonomous mobile robots can actively search for the location of cargo within a warehouse, generate obstacle avoidance paths in real time, and flexibly respond through code-based action generation when a new loading method not in the existing action set is required.

[0356] In other embodiments, the present disclosure may be applied to the medical and healthcare fields. A surgical assistance robot operates in an environment where it is difficult to fully grasp the overall state of the surgical site due to limitations in the camera field of view, obstruction of view by tissues and instruments, etc. When the method according to the present disclosure is applied, the surgical assistance robot can generate an environment-reflecting sentence from the currently observable state of the surgical site and perform precise surgical assistance movements based on control information generated through an artificial neural network model with a decision tree structure. Additionally, a drug delivery robot within a hospital can perform the task of safely transporting drugs while actively responding to unpredictable environmental changes, such as obstacles in hallways and the open / closed status of doors.

[0357] In another embodiment, the present disclosure may be applied to the fields of autonomous driving and mobility. An autonomous vehicle operates in a partially observable environment, such as road conditions outside the sensor range, limited visibility due to weather and lighting conditions, and obscuration by other vehicles and pedestrians. When applying the method according to the present disclosure, the autonomous driving agent can generate an environment-reflecting sentence based on environmental observation information obtained from current sensors, and establish action plans such as driving path planning, lane changes, and obstacle avoidance based thereon. For example, if driving along the existing path is impossible due to the detection of an obstacle ahead, additional information regarding an alternative route can be obtained through active information search, and the environment-reflecting sentence can be updated to perform a safe detour. In drone delivery services as well, delivery tasks can be completed while actively adapting to unpredictable environmental changes, such as obstacles on the flight path and changes in weather.

[0358] In another embodiment, the present disclosure may be applied to the fields of agriculture and environmental monitoring. An agricultural robotic agent operates in an environment where the growth status of crops, the occurrence of pests and diseases, and soil moisture conditions can be observed only partially through sensors in vast farmlands. When applying the method according to the present disclosure, the agricultural agent can generate an environment-reflecting sentence from the currently observable crop status and infer the condition of the farmland outside the observation range to perform tasks such as sowing, weeding, and harvesting. For example, if a pest or disease is detected in a specific area, the agent can further explore the condition of surrounding areas through active information search, establish a modified action plan to prevent the spread of the pest or disease, and perform pest control operations. An environmental monitoring drone can infer the overall environmental condition based on observation information within a limited sensor range and actively adjust the search range in tasks such as wildfire monitoring, marine pollution detection, and wildlife observation.

[0359] In another embodiment, the present disclosure may be applied to the field of construction and infrastructure inspection. At a construction site, a robot agent operates in an environment where it is difficult to fully grasp the entire site due to complex structures, the movement of workers and equipment, etc. When the method according to the present disclosure is applied, the construction robot agent can generate environment-reflecting sentences from the currently observable state of the construction site and perform tasks such as material handling, welding, and bolt fastening. In particular, based on control information generated through an artificial neural network model with a decision tree structure, it can perform precise grasping and handling operations according to the weight, size, and shape of various materials. Inspection drones for infrastructure facilities such as bridges, tunnels, and transmission towers can expand the inspection scope and effectively detect defects such as cracks and corrosion through active information retrieval in an environment where it is difficult to grasp the condition of the entire structure at once due to the complex shape of the structure and limitations in sensor range.

[0360] In another embodiment, the present disclosure may be applied to the field of home service robots. Home service robots operate in a dynamic, partially observable environment where environmental conditions change frequently due to the arrangement of various objects within the home, lighting conditions, and the movement of residents. When the method according to the present disclosure is applied, the home service robot can understand user instructions in the form of natural language, generate environment-reflecting sentences from the currently observable environmental conditions within the home, and perform tasks such as cleaning, organizing objects, and preparing food. In particular, by displaying an action plan established by an agent on the user's smartphone screen through a Human-Agent Collaboration method and executing it after user approval, a safe service environment can be realized in which the user directly supervises and controls the agent's actions.

[0361] Meanwhile, the embodiments according to the present disclosure described above may be implemented in the form of program instructions that can be executed through various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc., either alone or in combination. The program instructions recorded on the computer-readable recording medium may be those specifically designed and configured for the present disclosure or those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. Hardware devices may be modified into one or more software modules to perform processing according to the present disclosure, and vice versa.

[0362] The specific embodiments described in this disclosure are examples and do not limit the scope of this disclosure in any way. For the sake of brevity of the specification, descriptions of prior electronic configurations, control systems, software, and other functional aspects of said systems may be omitted. Additionally, the connections of lines or connecting members between components shown in the drawings are illustrative of functional connections and / or physical or circuit connections, and may be replaced or additionally represented as various functional connections, physical connections, or circuit connections in actual devices. Furthermore, unless specifically stated as “essential,” “importantly,” etc., a component may not be strictly necessary for the application of the embodiments of this disclosure.

[0363] Furthermore, although the detailed description of the present disclosure has been explained with reference to preferred embodiments of the present disclosure, those skilled in the art or those with ordinary knowledge in the art will understand that various modifications and changes can be made to the embodiments of the present disclosure without departing from the spirit and technical scope of the present disclosure as set forth in the claims below. Accordingly, the technical scope of the present disclosure should not be limited to the contents described in the detailed description of the specification but should be determined by the claims.

[0364] Various embodiments of the present disclosure utilize at least one artificial intelligence model to perform inference based on environment-reflecting data and to generate action data for execution, thereby maximizing task execution efficiency in various industrial settings such as service robots, autonomous driving, and smart factories. In particular, since it provides a mechanism to generate and sequentially execute actionable actions based on data derived from environmental information, it has industrial applicability in that it can be widely utilized in the field of intelligent agent solutions requiring sophisticated action decisions even in complex environments with limited observation.

Claims

1. As a method executed by a computer, A step of obtaining user input that directs a predetermined task through a data input interface; A step of obtaining information about the environment required for performing the above task; A step in which at least one processor uses at least one artificial intelligence model to generate environment-reflecting data based on information about the environment; A step in which at least one processor uses at least one artificial intelligence model to establish a primary action plan for the execution of the task based on the environment-reflecting data; The step of the above at least one processor calculating first action data for executing the above first action plan using at least one artificial intelligence model; and A method comprising the step of inputting the above first action data into at least one subsequent processing component.

2. In Paragraph 1, A step of executing the task based on the first action data; A step of receiving negative feedback on the execution result of the above task; A step of performing active information search to actively acquire additional information necessary for performing the task based on at least one of the above negative feedback, the above environment reflection data, and the previous action history; and A method further comprising the step of updating environment-reflecting data based on additional information obtained through the active information search above.

3. In Paragraph 2, A step of establishing a modified secondary action plan based on the above-mentioned updated environment-reflecting sentence; and A method further comprising the step of the at least one processor generating second action data for executing the second action plan using the second artificial intelligence model.

4. In Paragraph 2, The step of performing the above active information search is, A method comprising the step of inputting at least one of the above negative feedback, the above environment reflection data, and the previous behavior history into the above at least one artificial intelligence model to infer additional information necessary for performing the above task.

5. In Paragraph 4, The step of performing the above active information search is, A step of generating a search query to search for the additional information inferred above; and A method comprising the step of acquiring knowledge regarding the additional information from an external information source based on the above search query and storing it in at least one memory.

6. In Paragraph 4, The step of performing the above active information search is, A step of re-exploring the surrounding environment based on the additional information inferred above; and A method comprising the step of storing new observation information obtained through the above re-search in the at least one memory.

7. In Paragraph 2, A method further comprising the step of updating information search rules, including the search scope and search query generation method for active information search to be performed in similar situations in the future, based on additional information obtained through the active information search and the execution result of the task.

8. In Paragraph 7, The step of updating the above information search rules is, A step of adding a new search target to an existing search range or modifying an existing search query based on additional information obtained through the above active information search; and A method comprising the step of storing the above-mentioned updated information retrieval rule in at least one memory.

9. In Paragraph 2, The above negative feedback is, A method comprising at least one of the following: when there is no response from the environment from the execution result of the above task, an uncertain state in which the output token probability of the at least one artificial intelligence model is below a preset threshold, and when the goal of the above task is not achieved.

10. In Paragraph 2, A step in which the at least one processor determines, using the at least one artificial intelligence model, whether an action code corresponding to an action required for performing the task exists in an existing action set; If the above judgment result indicates that the above action code does not exist, the step of generating code that performs an action necessary for the execution of the above task using at least one artificial intelligence model; and A method further comprising the step of executing the generated code to perform an action necessary for performing the task.

11. In Paragraph 10, The step of generating the above code is, A step of verifying the operation of the code by executing the generated code in a simulation environment; If the above verification result indicates that the operation of the above code differs from the intended operation, a step of modifying the above code; and A method comprising the step of refining the code by repeatedly performing the verification and modification on the modified code.

12. In Paragraph 11, The step of verifying the operation of the above code is, A step of obtaining a State-Action Pair from the operation of code executed in the above simulation environment; and A method comprising the step of inputting the acquired state-action pairs into a Vision Language Model (VLM) to evaluate the operation of the code.

13. In Paragraph 1, The step of the above at least one processor generating parameters of an artificial neural network model for calculating control information necessary for performing an operation based on the first action data using the above at least one artificial intelligence model; A step of generating control information necessary for performing the operation using an artificial neural network model composed of the parameters generated above; and A method further comprising the step of performing the task by executing an action based on the generated control information.

14. In Paragraph 13, The step of generating parameters for the above artificial neural network model is, A step of generating parameters of an artificial neural network model of a decision tree structure in text format using at least one artificial intelligence model; and A method comprising the step of constructing an artificial neural network model of the decision tree structure based on parameters generated in the text format above.

15. In Paragraph 1, The step of obtaining information about the above environment is, The step of the above at least one processor controlling a physical device equipped with at least one sensor to scan the surrounding environment with the sensor and acquire raw data; and A method comprising the step of identifying at least one of the type, location, and state of an object from the acquired raw data to obtain information about the environment.

16. In Paragraph 1, The at least one artificial intelligence model used to generate the above environment-reflecting data includes an unfine-tuned closed-source large language model, and A method in which at least one artificial intelligence model used to generate the first action data comprises an open-source small language model fine-tuned with success trajectory data.

17. In Paragraph 2, The step of executing the task based on the first action data is The step of the at least one processor generating a control signal that controls the physical movement of a physical device equipped with a driving unit based on the first action data; and A method comprising the step of the physical device receiving the control signal and driving the driving unit.

18. At least one memory; and At least one processor that reads at least one instruction stored in the above-mentioned at least one memory and performs a method of controlling the behavior of an artificial intelligence agent in an environment where observation is restricted; comprising The above at least one instruction is, A step of obtaining user input that directs a predetermined task through a data input interface; A step of obtaining information about the environment required for performing the above task; A step in which at least one processor uses at least one artificial intelligence model to generate environment-reflecting data based on information about the environment; A step in which at least one processor uses at least one artificial intelligence model to establish a primary action plan for the execution of the task based on the environment-reflecting data; The step of the above at least one processor calculating first action data for executing the above first action plan using at least one artificial intelligence model; and A system comprising a command to perform the step of inputting the above-mentioned first action data into at least one subsequent processing component.

19. In Paragraph 18, A plurality of neurons comprising an array including at least one register, at least one programmable logic, and at least one input interface; a plurality of synapse circuits storing synapse weights that regulate the connection strength between the plurality of neurons; and at least one routing network that controls the data flow between the plurality of neurons; comprising A system further comprising a Field Programmable Gate Array (FPGA) implementation for a predetermined artificial neural network, wherein each of the plurality of neurons is connected to at least one other neuron through the routing network to establish a transmission path for the weights.

20. In Paragraph 18, A plurality of neurons organized into an array comprising at least one register, at least one microprocessor, and at least one input; and a plurality of synapse circuits storing synapse weights that regulate the connection strength between the plurality of neurons; comprising A system comprising an Application Specific Integrated Circuit (ASIC) for a predetermined artificial neural network, wherein each of the plurality of neurons is connected to at least one other neuron through any one of the plurality of synaptic circuits.

Images