Reinforcement Learning-based Multi-agent General-purpose AI System for Domain Intelligence Collaboration and Operation Method Thereof

Abstract

The present invention discloses a reinforcement learning-based multi-agent general-purpose AI system for domain intelligence collaboration and a method of operation thereof, which is designed so that domain-specific agents solve complex problems through cooperation, explanation, and discussion, thereby providing users with in-depth analysis and diverse perspectives rather than simple response types. The reinforcement learning-based multi-agent general-purpose AI system for domain intelligence collaboration according to the present invention comprises: a user input unit into which user text data, voice data, or image data is input; a modality interpretation unit in which data input to the user input unit is preprocessed, modalities are recognized, and tasks are classified; a domain agent routing unit in which a domain is determined for the tasks classified by the modality interpretation unit and tasks are assigned according to the domain; a multi-agent unit in which interaction and collaboration are performed by a plurality of AI agents on the tasks assigned by the domain agent routing unit; a result output unit in which the results produced by the interaction and collaboration performed by the multi-agent unit are integrated and a response is provided; and a learning unit in which self-improvement learning is performed for a plurality of AI agents based on feedback regarding the tasks.

Description

Technology Field

[0001] The present invention relates to a reinforcement learning-based multi-agent general-purpose AI system for domain intelligence collaboration and a method of operation thereof. More specifically, it relates to a reinforcement learning-based multi-agent general-purpose AI system for domain intelligence collaboration and a method of operation thereof, which is designed so that domain-specific agents solve complex problems through cooperation, explanation, and discussion, thereby providing users with in-depth analysis and diverse perspectives rather than simple response types. Background Technology

[0002] Artificial General Intelligence (AGI) refers to artificial intelligence possessing human-level intelligence. Unlike current AI, AGI can demonstrate capabilities equal to or greater than humans in various fields; that is, it mimics human cognitive abilities such as learning, reasoning, problem-solving, and planning, and can flexibly adapt to new situations.

[0003] A five-step roadmap to achieve this AGI has been announced. Step 1 is the Conversational AI stage, where AI can converse with humans using natural language. Step 2 is the Reasoning AI stage, where AI can solve problems at a level similar to humans with Ph.D.-level education. Step 3 is the Autonomous AI stage, where AI can perform tasks independently for several days without user intervention. Step 4 is the Innovation AI stage, where AI can go beyond simply performing given tasks to independently develop and improve new methods. Step 5 is the Organizational AI stage, where AI can replace the roles of all employees and perform all organizational functions.

[0004] With the proliferation of AI agents that have the authority to interact with the environment, collect relevant data, and perform tasks to achieve user goals, problem-solving through collaboration among AI agents capable of encompassing the aforementioned 3rd, 4th, and 5th stages is becoming important, and further research on this is needed. Prior art literature

[0005] Republic of Korea Published Patent Application No. 10-2024-0154354 Republic of Korea Published Patent Application No. 10-2024-0158651 Republic of Korea Published Patent Application No. 10-2025-0071493 Republic of Korea Registered Patent Application No. 10-2688166 The problem to be solved

[0006] The present invention aims to solve the aforementioned problems by providing a reinforcement learning-based multi-agent general-purpose AI system for domain intelligence collaboration and a method of operation thereof, which is designed so that domain-specific agents solve complex problems through cooperation, explanation, and discussion, thereby providing users with in-depth analysis and diverse perspectives rather than simple response-based responses. means of solving the problem

[0007] An AI system for domain intelligence collaboration according to an embodiment of the present invention for achieving the above-mentioned purpose comprises: a user input unit into which one or more data selected from user text data, voice data, and image data are input; a modality interpretation unit in which the data input to the user input unit is preprocessed, modalities are recognized, and tasks are classified; a domain agent routing unit in which a domain is determined for the tasks classified by the modality interpretation unit and tasks are assigned according to the domain; a multi-agent unit in which interaction and collaboration are performed by a plurality of AI agents on the tasks assigned by the domain agent routing unit; a result output unit in which the results produced by the interaction and collaboration performed by the multi-agent unit are integrated and responded to; and a learning unit in which self-improvement learning is performed on the plurality of AI agents based on feedback from the user regarding the results responded to the result output unit.

[0008] The above user input unit includes a text input section for inputting the user's text data, a voice input section for inputting the user's voice data, and an image input section for inputting the user's image data.

[0009] The modality interpretation unit includes a preprocessing unit in which data input to the user input unit is preprocessed, and a modality recognition unit in which modality is recognized and tasks are classified for the data preprocessed by the preprocessing unit.

[0010] The multi-agent unit includes an AI agent section in which work is performed by a plurality of AI agents for a task assigned by the domain agent routing unit.

[0011] Alternatively, the multi-agent unit includes an AI agent section in which work is performed by a plurality of AI agents on a task assigned by the domain agent routing unit, and a supervisory AI agent section that enables interaction and collaboration among a plurality of AI agents based on an A2A (Agent-to-Agent) protocol.

[0012] The above multi-agent unit performs at least one of translation, summarization, and sentiment analysis of data related to finance and education in an assigned task, and generates a graph of a workflow composed of nodes and edges for the data related to finance and education,

[0013] Alternatively, the multi-agent unit uses LangChain and LangGraph as the multi-agent framework, where LangChain performs at least one of translation, summarization, and sentiment analysis of data related to finance and education in the assigned task, and LangGraph generates a graph of a workflow composed of nodes and edges for the data related to finance and education.

[0014] The above learning unit is reinforced based on RLHF (Reinforcement Learning from Human Feedback), and PPO (Proximal Policy Optimization) is used to optimize the policy of the above reinforcement learning.

[0015] The above education refers to education related to finance.

[0016] The above multi-agent unit further comprises a retraining library unit, wherein the retraining library unit assigns scores to results generated by the plurality of AI agents and reflects only the results having scores greater than or equal to a reference score in the learning unit.

[0017] The above-mentioned retraining library unit includes at least one of a data quantity reflection unit and a lower limit value reflection unit that apply weights to the above-mentioned assigned score.

[0018] The above data quantity reflection unit generates a weight that controls the reference score in proportion to the data quantity of results having a score greater than or equal to the reference score, and reflects it in the learning unit. Here, the weight is generated by setting a certain quantity of data of results having a score greater than or equal to the reference score. Specifically, when reflecting results with a threshold score of 80 or higher into a learning unit, if there is a large amount of data for results with an assigned score of 80 or higher, a weight value is selected to raise the threshold score higher than 80 to reflect the result data with a high threshold score. For example, if there are 1,000 result data points with a score of 80 or higher and a set quantity of 400 results with a score of 85, a weight is created to set the threshold score to 85. If there is a small amount of data for results with an assigned score of 80 or higher, a weight value is selected to lower the threshold score below 80 to reflect the result data with a low threshold score. For example, if there are 300 result data points with a score of 80 or higher and a set quantity of 400 results with a score of 78, a weight is created to set the threshold score to 78. This allows for the reflection of a certain volume of result data.

[0019] The lower limit reflection unit applies weights to the data quantity reflection unit to result data with a score greater than or equal to the reference score obtained by applying weights to the data quantity reflection unit, and limits this to a certain score value or higher to reflect in the learning unit. For example, if the score is 60 for 100 result data points with a set quantity of 400 points or more, and the lower limit is set to a certain score value of 75, result data with a score too low can be excluded from reflection.

[0020] The above AI agent department includes financial AI agents and education AI agents.

[0021] The above financial AI agent includes one or more agents selected from: a financial product comparison agent that compares financial conditions between deposits, installment savings, and loans of different banks; a credit rating-based product recommendation agent that recommends financial products based on the user's credit rating; an investment propensity analysis agent that analyzes risk preference based on a survey of financial investment propensity; a customized asset portfolio agent that suggests the user's asset weight based on financial risk; an exchange rate forecasting and analysis agent that provides exchange rate trends and forecasts; a loan limit calculation agent that calculates the loanable amount based on the user's input information; a financial fraud detection agent that recognizes and warns of abnormal transaction patterns in financial transactions; an abnormal transaction monitoring agent that monitors suspected money laundering patterns in real time; an insurance product recommendation agent that calculates the user's expected insurance coverage scope and premiums; a voice phishing response agent that detects and warns of fraudulent conversations based on voice; and an automatic tax calculation agent that calculates tax rates incurred during financial transactions.

[0022] The above-mentioned educational AI agent includes one or more agents selected from: a personalized learning content recommendation agent that provides materials based on the user's learning level and learning history; a textbook summary agent that summarizes papers within a set range; an assignment evaluation agent that automatically grades the user's assignments and provides feedback; a learning plan establishment agent that generates a study timetable based on the user's exam schedule; an error analysis report generation agent that identifies areas where errors occur based on the user's problem-solving results; a learning motivation analysis agent that motivates the user based on the user's study time and test results; and a learning type recommendation agent that recommends learning difficulty and amount based on the user's grades.

[0023] In addition, a method of operation of a reinforcement learning-based multi-agent general-purpose AI system for domain intelligence collaboration according to an embodiment of the present invention for achieving the above-mentioned purpose comprises: a first step in which one or more of the user’s text data, voice data, and image data selected from the user’s question content are input by a user; a second step in which the data input in the first step is preprocessed, modalities are recognized, and tasks are classified; a third step in which a domain is determined for the tasks classified in the second step and tasks are assigned according to the domain; a fourth step in which interaction and collaboration are performed by a plurality of AI agents for the tasks assigned in the third step; and a fifth step in which the results produced by the interaction and collaboration performed by the plurality of AI agents in the fourth step are integrated and a response is provided. Effects of the invention

[0024] As described above, the present invention has the advantage of supporting AI-based investment research and portfolio decision-making, enabling collaboration in data research such as portfolio construction, corporate research, and comprehensive market analysis.

[0025] In addition, the present invention supports the automation of loans and risk assessment, and has the advantage of cost reduction through reduced complexity of financial operations and increased operational efficiency, such as loan document classification, credit standard evaluation, and workflow adjustment.

[0026] In addition, the present invention has the advantage of increasing the efficiency of teachers' work through the automation of administrative tasks such as test grading, learning analysis, and learning path recommendation.

[0027] In addition, the present invention has the advantage of providing personalized services by linking counseling, educational programs, and welfare systems to address complex problems that are insufficient with individual domain AI.

[0028] In addition, the present invention has the advantage of providing a wide spectrum of information through multi-faceted background RAG-based agent collaboration when responding to customers. Brief explanation of the drawing

[0029] Figure 1 is a configuration diagram of a reinforcement learning-based multi-agent general-purpose AI system for domain intelligence collaboration according to the present invention. FIG. 2 is a conceptual diagram of a reinforcement learning-based multi-agent general-purpose AI system for domain intelligence collaboration according to the present invention. Figure 3 is a configuration diagram of the user input unit of the present invention. Figure 4 is a configuration diagram of the modality analysis unit of the present invention. Figure 5 is a configuration diagram of the multi-agent unit of the present invention. Figure 6 shows a system performance improvement design through a reinforcement learning policy in a learning unit in the present invention. FIG. 7 is a flowchart of the operation method of a reinforcement learning-based multi-agent general-purpose AI system for domain intelligence collaboration according to the present invention. Figure 8 is an example diagram illustrating the content generated for an expected task in the financial AI agent of the present invention. Figure 9 is an example diagram illustrating the content produced for an expected task in the educational AI agent of the present invention. Specific details for implementing the invention

[0030] Expressions such as “comprising” or “may comprise” that may be used in various embodiments of the present disclosure indicate the presence of the disclosed corresponding function, operation, or component, etc., and do not limit one or more additional functions, operations, or components, etc. Furthermore, in various embodiments of the present disclosure, terms such as “comprising” or “having” are intended to specify the presence of the features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0031] In various embodiments of the present disclosure, expressions such as “or” include any and all combinations of the words listed together. For example, “A or B” may include A, may include B, or may include both A and B.

[0032] Expressions such as "first," "second," "first," or "second" used in various embodiments of the present disclosure may modify various components of the various embodiments, but do not limit such components. For example, such expressions do not limit the order and / or importance of such components. Such expressions may be used to distinguish one component from another. For example, the first user device and the second user device are both user devices and represent different user devices. For example, without departing from the scope of the various embodiments of the present disclosure, the first component may be named the second component, and similarly, the second component may be named the first component.

[0033] When it is stated that a component is "connected" or "connected" to another component, it should be understood that the component may be directly connected or connected to the other component, but that a new component may exist between the component and the other component. On the other hand, when it is stated that a component is "directly connected" or "directly connected" to another component, it should be understood that no new component exists between the component and the other component.

[0034] In embodiments of the present disclosure, terms such as "module," "unit," "part," etc. are used to refer to a component that performs at least one function or operation, and such component may be implemented in hardware or software, or a combination of hardware and software. Additionally, a plurality of "modules," "units," "parts," etc. may be integrated into at least one module or chip and implemented as at least one processor, except where each needs to be implemented in specific individual hardware.

[0035] The terms used in the various embodiments of this disclosure are used merely to describe specific embodiments and are not intended to limit the various embodiments of this disclosure. The singular expression includes the plural expression unless the context clearly indicates otherwise.

[0036] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art to which the various embodiments of this disclosure pertain.

[0037] Terms such as those defined in commonly used dictionaries should be interpreted as having meanings consistent with their meanings in the context of the relevant technology, and should not be interpreted in an ideal or overly formal sense unless explicitly defined in the various embodiments of the present disclosure.

[0039] Hereinafter, a reinforcement learning-based multi-agent general-purpose AI system (10) for domain intelligence collaboration according to an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a configuration diagram of the reinforcement learning-based multi-agent general-purpose AI system (10) for domain intelligence collaboration according to the present invention, and FIG. 2 is a conceptual diagram of the reinforcement learning-based multi-agent general-purpose AI system (10) for domain intelligence collaboration according to the present invention. Referring to FIG. 1, the reinforcement learning-based multi-agent general-purpose AI system (10) for domain intelligence collaboration according to the present invention may be configured to include a user input unit (100), a modality interpretation unit (200), a domain agent routing unit (300), a multi-agent unit (400), a result output unit (500), and a learning unit (600). The reinforcement learning-based multi-agent general-purpose AI system (10) for domain intelligence collaboration according to the present invention is particularly specialized for the financial domain and the education domain, but is not limited thereto and can be expanded and used in various domains.

[0040] Text data, voice data, or image data of the user is input into the user input unit (100). A user who wishes to use the reinforcement learning-based multi-agent general-purpose AI system (10) for domain intelligence collaboration of the present invention may input questions in the form of text, voice, or image. FIG. 3 is a configuration diagram of the user input unit (100) of the present invention. Referring to FIG. 3, the user input unit (100) of the present invention may be configured to include a text input unit (110), a voice input unit (120), and an image input unit (130). Text data of the user is input into the text input unit (110). The user may input content to ask the artificial intelligence through an input device such as a keyboard, mouse, or touchscreen into the text input unit (110). The user may also input content to ask into the text input unit (110) in the form of a document. Next, voice data of the user is input into the voice input unit (120). The voice input unit (120) is equipped with a voice input device such as a microphone to recognize the user's voice as voice data. Additionally, the user may directly input voice data into the voice input unit (120). Next, the user's image data is input into the image input unit (130). The user may input the content of the question into the image input unit (130) in the form of an image file.

[0041] The modality analysis unit (200) preprocesses data input into the user input unit (100) and classifies tasks by recognizing modalities. Since the LLM used in the present invention processes data in text form, all input data is preprocessed into text and then the modality is recognized. FIG. 4 is a configuration diagram of the modality analysis unit (200) of the present invention. Referring to FIG. 4, the modality analysis unit (200) of the present invention may be configured to include a preprocessing unit (210) and a modality recognition unit (220).

[0042] The preprocessing unit (210) preprocesses the data input into the user input unit (100). The data input into the user input unit (100) undergoes a preprocessing process to facilitate recognition by artificial intelligence. Voice data input into the voice input unit (120) is converted into text data and undergoes a text preprocessing process in the same manner as the text data input into the text input unit (110). The preprocessing unit (210) preprocesses the text data through text normalization, removal of special characters, text tokenization, removal of stop words, and extraction of stems or headwords. The preprocessing unit (210) of the present invention may use ASR (Automatic Speech, Recognition), STT (Speech-to-Text), and OCR (Optical Character Recognition) technologies for data preprocessing. Additionally, the preprocessing unit (210) extracts text from image data using OCR technology and performs preprocessing on the extracted text.

[0043] The modality recognition unit (220) recognizes modalities for data preprocessed by the preprocessing unit (210) and classifies tasks. The modality recognition unit (220) of the present invention simultaneously processes and integrates various data modalities with different characteristics, such as text, voice, and images. Text recognition is performed on the preprocessed data by the modality recognition unit (220) according to the modality of the input data, and the data is classified into respective tasks according to the text recognition results. The modality recognition unit (220) classifies tasks into the financial domain if text is recognized in the financial domain, and classifies tasks into the education domain if text is recognized in the education domain.

[0044] The domain agent routing unit (300) determines the domain for the tasks classified by the modality interpretation unit (200) and assigns the tasks according to the domain. Although the present invention may use various domains, in one embodiment of the present invention, the domain may be limited to a financial domain and an education domain. The domain agent routing unit (300) may determine whether to assign each task classified by the modality interpretation unit (200) to the financial domain or to the education domain.

[0045] The multi-agent unit (400) performs interaction and collaboration by multiple AI agents on tasks assigned by the domain agent routing unit (300). FIG. 5 is a configuration diagram of the multi-agent unit (400) of the present invention. Referring to FIG. 5, the multi-agent unit (400) of the present invention may be configured to include an AI agent unit (410) and a supervisory AI agent unit (420).

[0046] The AI ​​agent unit (410) performs tasks assigned by the domain agent routing unit (300) using a plurality of AI agents. The AI ​​agent unit (410) is equipped with a plurality of AI agents, and the AI ​​agent unit (410) according to one embodiment of the present invention may include a financial AI agent and an education AI agent.

[0047] Financial AI agents are AI specialized in financial operations that are classified within the financial domain and process assigned tasks, performing various financial functions such as recommending customized financial products to customers and analyzing risks. Financial AI agents are based on the GPT model, undergo fine-tuning to specialize in the financial domain, and utilize a financial domain-specific RAG (Retrieval-Augmented Generation) database. Furthermore, financial AI agents are characterized by their ability to integrate with external tool servers via the Model Context Protocol (MCP) and facilitate data exchange through JSON-RPC, a standardized communication protocol. MCP is a standardized protocol that enables AI agents to communicate using clear, predictable, and structured data when sending requests to external services, rather than natural language-based communication which can be interpreted differently by each agent. While fast and accurate message delivery is crucial for multi-agent collaboration, existing JSON-RPC is suitable for simple message exchange but has limitations in complex collaboration. In this invention, collaboration among AI agents is enhanced by adding context, role, and reliability information to JSON-RPC, and communication stability can be ensured through MCP-style JSON-RPC. Additionally, the financial AI agent may include multiple financial agents. Examples include a policy search agent that searches for relevant policy data from the Financial Services Commission and the Korea Federation of Banks; a case analysis agent that collects and compares instances of bias in existing loan screening algorithms; an ethics evaluation agent that analyzes the potential for discrimination in algorithms and derives proposals for fairness improvement; a legal review agent that examines the legal compliance of the system according to the standards of the Financial Consumer Protection Act; and a consumer response agent that generates explanations that are easy to understand from a user's perspective and presents alternatives. The multiple financial agents of this invention can function as hierarchical decision-making agents that mimic the thought processes of human expert groups.

[0048] The Education AI Agent is an AI specialized in educational tasks that processes assigned tasks classified within the education domain, performing various educational operations such as analyzing learning achievements and generating customized content. The Education AI Agent is based on the GPT model, undergoes fine-tuning to be specialized for the education domain, and utilizes an education domain-specific RAG database. Furthermore, the Education AI Agent is characterized by its ability to integrate with external tool servers via the Model Context Protocol (MCP) and to easily exchange data using JSON-RPC, a standardized communication protocol. The Education AI Agent utilizes MCP, a standardized protocol that enables communication using clear, predictable, and structured data instead of natural language-based communication, which can be interpreted differently by each AI agent, when an AI agent sends requests to external services. Fast and accurate message delivery is crucial for multi-agent collaboration; however, while existing JSON-RPC is suitable for simple message exchange, it has limitations in complex collaboration. In this invention, collaboration between AI agents is enhanced by adding context, role, and reliability information to JSON-RPC, and communication stability can be ensured through MCP-style JSON-RPC. In addition, the educational AI agent may include multiple educational agents.

[0049] The supervising AI agent unit (420) enables interaction and collaboration among multiple AI agents based on the A2A (Agent-to-Agent) protocol. The supervising AI agent unit (420) handles the interaction and collaboration among multiple AI agents based on the A2A (Agent-to-Agent) protocol. A2A is a standard communication method that enables direct interaction between different AI agents. The characteristics of A2A are as follows: First, as a standardized communication protocol, it establishes a uniform method for AI agents to request and receive data, thereby ensuring compatibility between various systems. Second, through dynamic context management, it allows AI agents to maintain context when interacting with multiple data sources, thereby improving the relevance and consistency of the output. Third, through modular integration, it reduces development overhead by providing a plug-and-play function that allows new data sources to be added with minimal configuration. Additionally, while A2A focuses on communication between AI agents, it complements existing protocols such as MCP, which standardizes how AI agents interact with tools and APIs. A2A and MCP provide a comprehensive framework that enables AI agents to access external resources and collaborate effectively.

[0050] The multi-agent unit (400) of the present invention uses a LangChain-based multi-agent framework. The advantages of LangChain are as follows. LangChain is a framework that connects LLM APIs and supports prompt templates, chain configuration, tool invocation, document search, etc. In addition, LangChain provides advanced features such as knowledge-based question answering (RAG), multi-turn conversation, tool-based reasoning, and workflow automation. Furthermore, LangChain enables LLM-based agents with a Supervisor-Worker structure to perform autonomous collaboration. Additionally, LangChain implements high-dimensional interactions that go beyond simple Q&A, including explainability, emotional empathy, and reliability. LangChain has the following architectural configuration. First is the Supervisor Agent, which analyzes user requests to plan the entire workflow and distributes each task to the appropriate worker agents. In the present invention, the role of the Supervisor Agent is performed by the Supervisor AI Agent Unit (420). Second, there are Worker Agents that perform specialized roles such as translation, summarization, sentiment analysis, legal interpretation, and medical information organization. Third, there is a Memory & Planner Module that stores conversation history and reasoning processes and can establish plans considering long-term context. In the present invention, the Worker Agents and the Memory & Planner Module are performed by the AI ​​Agent Unit (410).

[0051] In addition, the multi-agent unit (400) of the present invention may use a LangGraph-based framework. Here, LangGraph is a library that defines and executes the flow of LLM applications as a graph as part of the LangChain ecosystem. LangGraph adopts a graph-based structure to create a flexible workflow composed of nodes and edges. This helps to intuitively model complex logic and multi-stage processes. Using LangGraph allows for the easy implementation of multi-agent multi-stage decision-making processes or complex workflows. Using LangGraph enables the construction of a Multi-Agent Reinforcement Learning (MARL) environment that goes beyond simple multi-agent collaboration. That is, an agent holding the correct answer can provide direction, and a learning agent can achieve cooperative goals through exploration; agents can perform in parallel without conflict and optimize common goals, such as financial recommendations, through role sharing; and agents can compete based on user feedback and improve performance through reinforcement learning. The MARL environment can induce independent judgment and the sharing of strategies and information among agents.

[0052] Additionally, the multi-agent unit (400) is characterized by the AI ​​agent recording agent self-reflection and building and using a retraining library. In the present invention, to build a growing general-purpose artificial intelligence, “success / failure logs” are listed within the multi-agent collaboration and built into a retraining library, or scores are generated. This retraining library can be learned through the learning unit (600), and through the retraining of the retraining library, the multi-agent unit (400) has the advantage of being able to improve without repeating the same mistakes in tasks associated with the retraining library.

[0053] The multi-agent unit (400) further comprises a retraining library unit, wherein the retraining library unit assigns a score to the results generated by the plurality of AI agents and reflects only the results having a score greater than a certain score in the learning unit. Here, the assignment of the score may be based on user feedback regarding the responded results, or evaluated by creating a score evaluation model. For example, when Agent A generates 80 points, Agent B generates 70 points, and Agent C generates 60 points among the plurality of AI agents, if only results with a score of 65 points or higher are reflected in the learning unit, Agent A and Agent B are used, but Agent C is not used, so inaccurate data is not reflected in the retraining library.

[0054] The above retraining library section includes at least one of a weight reflection section, a data quantity application section, a period reflection section, and an average value reflection section.

[0055] The above-mentioned weighting unit reflects the results generated by the plurality of AI agents by assigning different weights to the obtained scores, and reflects them in the learning unit only when the score is greater than or equal to a specific score. For example, if the user sets the weights to Agent A 0.7, Agent B 1, and Agent C 1.5, and only results of 65 points or more are reflected in the learning unit, when Agent A 80 points, Agent B 70 points, and Agent C 60 points are generated, the weighted scores are Agent A 56 points, Agent B 70 points, and Agent C 90 points, so Agents B and C are used, but Agent A is not used.

[0056] The data quantity application unit generates a first weight inversely proportional to the quantity of result data generated by the plurality of AI agents with respect to the weight given by the weight reflection unit. For example, if the quantity of result data generated by the plurality of AI agents is 1,000 or more, the first weight is set to 0.9, and if the quantity of result data generated by the plurality of AI agents is less than 1,000, the first weight is set to 1.1. This allows as much data as possible to be used in the self-learning unit when the quantity of data is less than a certain amount, but as accurate data as possible is used in the self-learning unit when the quantity of data exceeds a certain amount, thereby increasing the accuracy of retraining.

[0057] The above data quantity application unit may be applied by a weight in which the first weight changes as a linear function with respect to the quantity of result data, for example, by applying the linear function y = - a * x (where a is a value that determines the slope of the first weight with respect to the quantity of data) such that when the quantity of data x increases, the linear function y corresponding to the weight continuously decreases, thereby allowing for more accurate selection of the quantity of data than when the same weight is applied to a predetermined interval by dividing it into certain intervals.

[0058] The above period reflection unit is linked to the quantity of result data generated by the plurality of AI agents within a certain period with respect to the weight given in the above weight reflection unit, for example, linked inversely proportionally, to generate a second weight. For example, if the quantity of result data generated within 6 months is 500 or more, the second weight is set to 0.9, and if the quantity of data within 6 months is less than 500, the second weight is set to 1.1. This allows the self-learning unit to use as much data as possible when the quantity of data is small within a certain period, but to use as accurate data as possible when the quantity of data is large within a certain period, thereby increasing the accuracy of re-learning.

[0059] The above average value reflection unit reflects the result generated by the plurality of AI agents in the learning unit only if the result is greater than or equal to the average value score reflected in the learning unit. For example, if the average score of the result reflected in the learning unit is 80 and the score of the result generated by the plurality of AI agents is 75, even if only the result of 65 points or higher is reflected in the learning unit, the result of the average value is not used in the self-learning unit, thereby allowing the accurate data of the case higher than the average value to be used in the self-learning unit, thereby increasing the accuracy of retraining.

[0060] The above-mentioned weight reflection section, data quantity reflection section, period reflection section, and average value reflection section may be applied individually or in combination.

[0061] The result output unit (500) integrates the results produced by the interaction and collaboration performed in the multi-agent unit (400) and responds to the user. The result output unit (500) can integrate the results produced by the interaction and collaboration performed between AI agents regarding questions related to the financial or education domain and output them to the user in a multimodal format such as text, voice, or image.

[0062] The learning unit (600) performs self-improvement learning on multiple AI agents based on feedback regarding tasks. The learning unit (600) performs self-evolutionary reinforcement learning based on RLHF (Reinforcement Learning from Human Feedback) and PPO (Proximal Policy Optimization). First, RLFH (Reinforcement Learning from Human Feedback) is a machine learning technique that learns a reward model through direct human feedback and then optimizes the performance of an AI agent through reinforcement learning. RLFH utilizes human feedback to allow the model to reflect human intuition and domain knowledge, which enables the AI ​​agent to be more human-friendly and effectively solve real-world problems. Additionally, RLFH can improve the performance of an AI agent by directly learning human preferences, which is particularly useful in problems where it is difficult to define an explicit reward function. Furthermore, RLFH enables the AI ​​agent to adapt better to new situations or environments, as the model can learn how to respond to various situations through human feedback.

[0063] Reinforcement learning of the learning unit (600) via RLFH may include the following five steps. The first step is the initial policy learning step. In the first step, the AI ​​agent selects an action in the environment using a basic reinforcement learning algorithm and begins learning according to an existing reward structure. In this step, the AI ​​agent establishes a basic course of action. The second step is the human feedback collection step. Humans provide feedback on the actions performed by the AI ​​agent in the environment. This feedback can be collected through absolute evaluation, which assigns a score to a specific action, or through relative evaluation, which selects the better of several actions. The third step is the reward model learning step. A reward model is learned using the collected human feedback. This model serves as a criterion for evaluating the AI ​​agent's actions. The reward model can be learned using a supervised learning algorithm and understands the relationship between the AI ​​agent's actions and rewards through human feedback. The output of the reward model predicts the reward the AI ​​agent will receive when performing a specific action. The fourth step is the reinforcement learning step. The AI ​​agent's actions are evaluated using the reward model, and the policy is optimized through a reinforcement learning algorithm. The AI ​​agent learns a better policy using the reward signals provided by the reward model. Here, Proximal Policy Optimization (PPO) can be used for policy optimization in Step 4. PPO is a new policy gradient method for reinforcement learning. This method alternately performs the process of sampling data through interaction with the environment and optimizing a surrogate objective function using stochastic gradient ascent. While conventional policy gradient methods perform a gradient update once per data sample, PPO can propose a new objective function capable of performing multiple mini-batch updates. Step 5 is the periodic feedback and iterative learning step.In this stage, as the AI ​​agent's behavior improves, new feedback is continuously collected and the reward model is updated. This is an iterative process that periodically updates the reward model and improves the AI ​​agent's policies. Through the periodic collection of human feedback and the updating of the reward model, the AI ​​agent progressively improves its performance.

[0064] FIG. 6 illustrates a design for improving system performance through a reinforcement learning policy in a learning unit (600) in the present invention. As shown in FIG. 6, the learning unit (600) can achieve the following effects through RLHF. Clear response quality target indicators can be set at the beginning of the project. In addition, system configuration strategies and evaluation criteria can be established to achieve the set values. Next, criteria for document collection, classification, and filtering optimized for the target domain can be presented. This enables the systematic preparation of a vectorized document structure that serves as the basis for the RAG system. Next, prompts can be optimized to improve the quality of the results generated by the RAG search and LLM. In addition, the retrieval structure of the RAG can be supplemented, such as by adjusting chunk size, overlap ratio, and the number of search results.

[0066] Hereinafter, a method of operation of a reinforcement learning-based multi-agent general-purpose AI system (10) for domain intelligence collaboration according to an embodiment of the present invention will be described. FIG. 7 is a flowchart of the method of operation of a reinforcement learning-based multi-agent general-purpose AI system (10) for domain intelligence collaboration according to the present invention. Referring to FIG. 7, the method of operation of a reinforcement learning-based multi-agent general-purpose AI system (10) for domain intelligence collaboration may be composed of the following five steps.

[0067] The first step (S10) is a step in which text data, voice data, or image data of the question content is input by the user. The user can input questions related to finance or education into the user input unit (100) in the form of text, voice, or image. The reinforcement learning-based multi-agent general-purpose AI system (10) for domain intelligence collaboration of the present invention has the advantage of being able to input in multimodal formats such as text, voice, and image.

[0068] The second step (S20) is a step in which the data input in the first step (S10) is preprocessed, modalities are recognized, and tasks are classified. The voice data input into the user input unit (100) in the first step (S10) is converted into text data and undergoes a text preprocessing process in the same manner as text data. Text preprocessing is performed through text normalization, removal of special characters, text tokenization, removal of stop words, and extraction of stems or lemmas. For image data, text is extracted from the image data using OCR technology, and preprocessing is performed on the extracted text. The preprocessed data is recognized for modalities and classified for tasks. In the present invention, various data modalities with different characteristics, such as text, voice, and images, are processed and integrated simultaneously. Text recognition is performed on the preprocessed data by the modality recognition unit (220) according to the modality of the input data, and the data is classified into respective tasks based on the text recognition results. The modality recognition unit (220) classifies tasks into finance when text is recognized into the finance domain, and classifies tasks into education when text is recognized into the education domain.

[0069] Step 3 (S30) is a step in which a domain is determined for a classified task and the task is assigned according to the domain. Although various domains may be used in the present invention, in one embodiment of the present invention, the domain may be limited to a financial domain and an education domain. In Step 3 (S30), it may be determined whether each task classified by the modality interpretation unit (200) is assigned to the financial domain or the education domain.

[0070] Step 4 (S40) is a step in which interaction and collaboration are performed by multiple AI agents on an assigned task. The present invention comprises multiple AI agents, and in one embodiment of the present invention, may include a financial AI agent and an education AI agent. The financial AI agent is an AI specialized in financial tasks classified under the financial domain and processing assigned tasks, performing various financial tasks such as recommending customized financial products to customers and risk analysis. Additionally, the education AI agent is an AI specialized in education tasks classified under the education domain and processing assigned tasks, performing various education tasks such as analyzing learning achievements and generating customized content. Furthermore, in Step 4, multiple AI agents interact and collaborate based on an A2A (Agent-to-Agent) protocol; for example, financial content can be converted into education-related materials through collaboration, and education-related content can be converted into financial-related materials through collaboration. A2A is a standard communication method that enables direct interaction between different AI agents.

[0071] Step 5 (S50) is a step in which the results produced by the interaction and collaboration performed by the plurality of AI agents are integrated and responded to. Step 5 (S50) can integrate the results produced by the interaction and collaboration performed between AI agents regarding questions related to the financial or education domain and output them to the user in a multimodal format such as text, voice, or image. FIG. 8 is an example diagram explaining the content produced for an expected task in the financial AI agent of the present invention, and FIG. 9 is an example diagram explaining the content produced for an expected task in the education AI agent of the present invention. By referring to FIG. 8 and FIG. 9, the results produced by the collaboration of the plurality of AI agents can be anticipated.

[0073] The system described above may be implemented as a hardware component, a software component, and / or a combination of a hardware component and a software component. For example, the system and components described in the embodiments may be implemented using one or more general-purpose or special-purpose computers, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing unit may execute an operating system (OS) and one or more software applications executed on said operating system. Additionally, the processing unit may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing unit may be described as being used as a single unit, but those skilled in the art will understand that the processing unit may include multiple processing elements and / or multiple types of processing elements. For example, the processing unit may include multiple processors or one processor and one controller. Additionally, other processing configurations, such as parallel processors, are also possible.

[0074] Software may include computer programs, code, instructions, or a combination of one or more of these, and may configure a processing unit to operate as desired or command the processing unit independently or collectively. Software and / or data may be permanently or temporarily embodied in any type of machine, component, physical device, virtual equipment, computer storage medium or device, or transmitted signal wave so as to be interpreted by the processing unit or to provide instructions or data to the processing unit. Software may be distributed over networked computer systems and may be stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

[0075] The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., either alone or in combination. The program instructions recorded on the medium may be those specifically designed and configured for the embodiment, or they may be those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate as one or more software modules to perform the operation of the embodiment, and vice versa.

[0076] Although the embodiments have been described above with reference to limited examples and drawings, those skilled in the art can make various modifications and variations from the description above. For example, suitable results can be achieved even if the described techniques are performed in a different order than described, and / or the components of the described system, structure, device, circuit, etc. are combined or assembled in a form different from described, or replaced or substituted by other components or equivalents.

[0077] Therefore, other implementations, other embodiments, and equivalents to the claims also fall within the scope of the claims set forth below. Explanation of the symbols

[0078] 10: Multi-Agent General-Purpose AI System 100: User input unit 110 : Text input section 120 : Voice input section 130 : Image input section 200: Modality Interpretation Unit 210 : Preprocessing section 220 : Modality recognition unit 300: Domain Agent Routing Unit 400 : Multi-agent unit 410 : AI Agent Department, 420 : Director AI Agent Department 500: Result output unit 600: Learning Units

Claims

Claim 1 A user input unit into which one or more selected data from user text data, voice data, and image data are input; a modality interpretation unit in which data input into the user input unit is preprocessed, modalities are recognized, and tasks are classified; a domain agent routing unit in which a domain is determined for the tasks classified by the modality interpretation unit and tasks are assigned according to the domain; a multi-agent unit in which interaction and collaboration are performed by a plurality of AI agents on the tasks assigned by the domain agent routing unit; a result output unit in which the results produced by the interaction and collaboration performed by the multi-agent unit are integrated and responded to; and a learning unit in which self-improvement learning is performed on the plurality of AI agents based on feedback from the user regarding the results responded to the result output unit.It includes, wherein the user input unit includes a text input section for inputting the user's text data, a voice input section for inputting the user's voice data, and an image input section for inputting the user's image data; the modality interpretation unit includes a preprocessing section for preprocessing the data input to the user input unit and a modality recognition section for recognizing modalities and classifying tasks regarding the data preprocessed by the preprocessing section; the multi-agent unit includes an AI agent section in which work is performed by a plurality of AI agents on a task assigned by the domain agent routing unit, and a supervisory AI agent section that enables interaction and collaboration among a plurality of AI agents based on an A2A (Agent-to-Agent) protocol; the multi-agent unit uses LangChain and LangGraph as the multi-agent framework; LangChain performs at least one of translation, summarization, and sentiment analysis of data related to finance and education on a task assigned by the supervisory AI agent section; LangGraph generates a graph of a workflow composed of nodes and edges for the data related to finance and education; and the learning A multi-agent general-purpose AI system for domain intelligence collaboration, characterized in that the unit is reinforced using RLHF (Reinforcement Learning from Human Feedback), and PPO (Proximal Policy Optimization) is used to optimize the policy of the said reinforcement learning. Claim 2 delete Claim 3 delete Claim 4 In claim 1, the AI ​​agent unit comprises a financial AI agent and an education AI agent, wherein the financial AI agent comprises a financial product comparison agent that compares financial conditions between deposits, installments, and loans of different banks; a credit rating-based product recommendation agent that recommends financial products based on the user's credit rating; an investment propensity analysis agent that analyzes risk preference based on a survey of financial investment propensity; a customized asset portfolio agent that proposes the user's asset weight based on financial risk; an exchange rate prediction and analysis agent that provides exchange rate trends and forecasts; a loan limit calculation agent that calculates the loanable amount based on the user's input information; a financial fraud detection agent that recognizes and warns of abnormal transaction patterns in financial transactions; an abnormal transaction monitoring agent that monitors suspected money laundering patterns in real time; an insurance product recommendation agent that calculates the user's expected insurance coverage scope and premium; a voice phishing response agent that detects and warns of fraudulent conversations based on voice; and an automatic tax calculation agent that calculates tax rates incurred during financial transactions, wherein the education AI agent comprises a customized learning content recommendation agent that provides materials based on the user's learning level and learning history; a textbook summary agent that summarizes papers within a set range; and the user's task to be performed A multi-agent general-purpose AI system for domain intelligence collaboration, characterized by including: an assignment evaluation agent that automatically grades and provides feedback; a learning plan establishment agent that generates a learning timetable based on the user's test schedule; an error analysis report generation agent that identifies areas where errors occur based on the user's problem-solving results; a learning motivation analysis agent that motivates the user's learning based on the user's study time and test results; and a learning type recommendation agent that recommends learning difficulty and amount based on the user's grades. Claim 5 delete

Images