Method and system for executing user requests on basis of artificial intelligence model

Abstract

The present invention relates to a method and a system for providing responses through an artificial intelligence model that constructs a tool chain in order to generate a response to a user request involving a complex task when the user request is received, and, more particularly, to a method and a system for executing user requests on the basis of an artificial intelligence model that constructs a tool chain by planning and determining an operation order using a plurality of tasks in order to derive accurate responses to complicated and various types of requests.

Description

Method and system for performing user requests based on artificial intelligence models

[0001] The present invention relates to a method and system for providing an answer through an artificial intelligence model that builds a tool chain to generate an answer to a user's request when a user's request requiring a complex task is obtained.

[0002] Artificial intelligence (AI) is a computer system that analyzes and learns data and patterns to make intelligent decisions in order to perform a given task. It is showing even more powerful performance when combined with big data, cloud computing, etc., and is being used in various fields such as medical diagnosis, financial forecasting, autonomous driving, natural language processing, and game development.

[0003] Among them, with the recent emergence of pre-trained language models (Large Language Pretraining Models, LLMs) for large-scale general domain data, various tasks that were previously handled manually are being replaced by artificial intelligence-based ones.

[0004] These large-scale language models perform simple tasks for everyday conversations, as well as complex tasks for specialized questions such as search, summarization, calculation, and prediction in a wide range of fields such as meteorology, economics, medicine, and resource management.

[0005] Among these, the field of natural language processing, which extracts or summarizes the core content of a given document and provides it in a concise and easy-to-understand form, is essential for performing the various tasks mentioned above and providing it to users.

[0006] However, when performing a task using artificial intelligence, the task must be performed based on the given data, but due to the hallucination phenomenon, there was a possibility that content that did not actually exist in the text would be created as a summary, or words or sentences that were not semantically consistent would be created as a summary, which resulted in a problem of lowering the quality of the summary.

[0007] Hallucination can render artificial intelligence unreliable, and is a serious problem, especially in applications where reliability is critical. Therefore, companies, universities, and other organizations are actively conducting research and development to address hallucination.

[0008] Accordingly, the present invention proposes a method for improving the quality of responses by eliminating hallucinations that may occur when obtaining a user request requiring a complex task and generating a response to the request.

[0009] The present invention proposes a method and system for performing a user's request based on an artificial intelligence model that constructs a tool chain by planning and determining an operation sequence using multiple tasks to derive accurate answers to complex and diverse types of requests.

[0010] In addition, the present invention can provide a method and system for performing a user's request based on an artificial intelligence model that learns a task order by generating at least one Q&A set based on the user's request.

[0011] In addition, the present invention can provide a method and system for performing a user's request based on an artificial intelligence model that performs a main task of generating an answer to a query and / or at least one sub-task constituting the main task.

[0012] In addition, the present invention can provide a method and system for performing a user's request based on an artificial intelligence model that determines the sub-task operation order according to a constructed tool chain when a user's request is obtained and generates a response.

[0013] However, the technical problems to be solved by the present invention and embodiments of the present invention are not limited to the technical problems described above, and other technical problems may exist.

[0014] A method for performing a user request based on an artificial intelligence model according to an embodiment of the present invention is a method for performing a user request based on an artificial intelligence model performed by a processor of a server computing system, comprising: a step of controlling a tool chain database to be built based on a training computing system; a step of obtaining a query included in a user request received from a user computing device; a step of detecting a tool chain pair corresponding to the query from the tool chain database; a step of determining a first tool chain for performing the user request based on the detected tool chain pair; a step of sequentially performing a first to n-th sub-tasks matching the determined first tool chain; and a step of generating and providing an answer to the query based on the performed first to n-th sub-tasks.

[0015] In addition, the step of controlling to build a tool chain database based on the training computing system includes the step of controlling to create a plurality of Q&A sets matching pre-questions and pre-answers, the step of requesting a sub-task operation sequence plan for the created Q&A sets, the step of controlling to create a tool chain pair by matching the planned tool chain with the Q&A sets according to the request, and the step of storing the created tool chain pair in the tool chain database.

[0016] In addition, the step of controlling to generate a plurality of Q&A sets includes the step of controlling to input a pre-question generated in the same form as the user request into the question-and-answer system, the step of obtaining at least one pre-answer output from the question-and-answer system, and the step of generating a first pre-question and a first pre-answer corresponding to the first pre-question as a first Q&A set.

[0017] Additionally, the pre-answer output from the question-and-answer system includes a hallucination answer.

[0018] In addition, the step of requesting a sub-task operation sequence plan for the Q&A set includes the step of requesting a large-scale language model (LLM) to plan the sub-task operation sequence and obtaining an output result, and the step of storing a tool chain in which the types and operation sequences of at least two or more sub-tasks are determined based on the obtained result.

[0019] Additionally, the step of detecting the tool chain pair from the tool chain database includes at least one of the steps of detecting a first tool chain pair in which a pre-question and a tool chain are vectorized, and the steps of detecting a second tool chain pair in which a pre-question, a pre-answer, and a tool chain are vectorized.

[0020] In addition, the step of detecting the second tool chain pair includes the steps of obtaining a plurality of dictionary answers corresponding to one dictionary question, performing a type analysis task on each of the plurality of dictionary answers, and determining an answer type for each vectorized tool chain pair according to the performance of the type analysis task.

[0021] Additionally, the step of detecting the tool chain pair from the tool chain database further includes the step of providing pre-answers with the determined answer types to a user computing device, and the step of obtaining an input from the user computing device for selecting one of the pre-answers of the different answer types provided.

[0022] In addition, a method for performing a user's request based on an artificial intelligence model according to an embodiment of the present invention further includes a step of updating reference information of a tool chain pair included in the tool chain database constructed based on a RAG (Retrieval Augmented Generation) model to the latest information after a predetermined point in time.

[0023] Meanwhile, a system for performing a user's request based on an artificial intelligence model according to an embodiment of the present invention is a server computing system that receives a user's request from a user computing device and performs the user's request based on an artificial intelligence model, the system comprising: a tool chain database storing a plurality of tool chain pairs; a memory storing commands and data for performing the user's request; and at least one processor for performing the user's request according to the commands and data of the memory, wherein the at least one processor controls to build a tool chain database based on a training computing system, obtains a query included in a user request received from a user computing device, detects a tool chain pair corresponding to the query from the tool chain database, determines a first tool chain for performing the user's request based on the detected tool chain pair, sequentially performs a first to n-th sub-tasks matching the determined first tool chain, and generates and provides an answer to the query according to the performed first to n-th sub-tasks.

[0024] The method and system for performing a user's request based on an artificial intelligence model according to the present invention has the effect of reducing the illusionary effect that may occur when generating answers to complex questions and increasing the accuracy of answers by planning and determining an operation sequence using multiple tasks and constructing a tool chain to derive accurate answers to complex and diverse types of requests.

[0025] In addition, the method and system for performing a user's request based on an artificial intelligence model according to the present invention generates at least one Q&A set based on the user's request and learns the task order, thereby performing learning based on the user's actual questions, thereby increasing practical usability.

[0026] In addition, the method and system for performing a user's request based on an artificial intelligence model according to the present invention has the effect of reducing errors that may occur while providing a response in natural language to a user by performing a main task of generating a response to a query and / or at least one sub-task constituting the main task.

[0027] In addition, the method and system for performing a user's request based on an artificial intelligence model according to the present invention generates a response by determining the sub-task operation order according to the constructed tool chain when obtaining a user's request, thereby generating a response according to a predetermined order, thereby increasing the response speed.

[0028] However, the effects that can be obtained from the present invention are not limited to the effects mentioned above, and other effects that are not mentioned can be clearly understood from the description below.

[0029] FIG. 1 is a block diagram of a computing system that performs a target prediction method according to an embodiment of the present invention.

[0030] FIG. 2 is a block diagram of a computing device, which is one of the components of a computing system that performs a target prediction method according to an embodiment of the present invention.

[0031] FIG. 3 is a block diagram of another aspect of a computing device, which is one of the components of a computing system that performs a target prediction method according to an embodiment of the present invention.

[0032] FIG. 4 is a flowchart illustrating a method for constructing a tool chain database according to an embodiment of the present invention.

[0033] FIG. 5 is a flowchart illustrating a method for providing a request execution service according to an embodiment of the present invention.

[0034] FIGS. 6 to 8 are drawings for explaining a method for detecting a tool chain pair according to various embodiments of the present invention.

[0035] The present invention is capable of various modifications and embodiments. Therefore, specific embodiments are illustrated in the drawings and described in detail in the detailed description. The effects and features of the present invention, as well as the methods for achieving them, will become clear with reference to the embodiments described in detail below together with the drawings. However, the present invention is not limited to the embodiments disclosed below and can be implemented in various forms. In the following embodiments, the terms "first," "second," etc. are not used in a limiting sense but are used for the purpose of distinguishing one component from another. Furthermore, the singular expression includes the plural expression unless the context clearly indicates otherwise. Furthermore, terms such as "include" or "have" indicate the presence of a feature or component described in the specification, and do not preemptively exclude the possibility that one or more other features or components may be added. Furthermore, in the drawings, the sizes of components may be exaggerated or reduced for convenience of explanation. For example, the size and thickness of each component shown in the drawings are arbitrarily shown for convenience of explanation, and thus the present invention is not necessarily limited to what is shown.

[0036] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings. When describing with reference to the drawings, identical or corresponding components are given the same drawing reference numerals, and redundant descriptions thereof will be omitted.

[0037]

[0038] FIG. 1 illustrates an example block diagram of a computing system that performs a request execution method according to an embodiment of the present invention.

[0039] Referring to FIG. 1, a computing system (1000) that performs a user request according to one embodiment of the present invention includes a user computing device (110), a training computing system (150), and a server computing system (130), and each device and system are communicatively connected via a network (170).

[0040] According to an embodiment of the present invention, 1) a user computing device (110) can perform a user's request using a local or / and external machine learning model (120) or a machine learning model (140) provided by a server.

[0041] In addition, according to another embodiment of the present invention, 2) a server computing system (130) communicating with a user computing device (110) can provide a request execution service to the user computing device (110) through an application or / and on the web in response to a user's request through the user computing device (110).

[0042] In addition, according to another embodiment of the present invention, 3) the user computing device (110) and the server computing system (130) can perform at least a part of the method of performing a user's request in conjunction with each other to provide a request performance service to the user.

[0043] Additionally, according to various embodiments of the present invention, a user computing device (110) and / or a server computing system (130) may learn a machine learning model (120 / 140) utilized for request execution through interaction with a training computing system (150) communicatively connected via a network (170). In this case, the training computing system (150) may be separate from the server computing system (130) or may be a part of the server computing system (130).

[0044] In some embodiments, the training computing system (150) may be part of a server computing system (130) or part of a user computing device (110).

[0045] In the following description, the user computing device (110) connects to a server computing system (130) to execute a request execution service, collects and analyzes data required for request execution using a language model of a separate server or directly from the server computing system (130), and plans a task sequence based on the collected and analyzed data to execute the user's request. However, it can be understood that a part of the process described as being performed in the server computing system (130) is naturally included in the description of the present invention when it is performed in the user computing device (110).

[0046] The user computing device (110) may include any type of computing device, such as a smart phone, a mobile phone, a digital broadcasting device, a personal digital assistant (PDA), a portable multimedia player (PMP), a desktop, a wearable device, an embedded computing device, and / or a tablet PC.

[0047] Such a user computing device (110) includes at least one processor (111) and memory (112). Here, the processor (111) may be composed of at least one or a plurality of electrically connected processors among a central processing unit (CPU), a graphics processing unit (GPU), application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors, and / or other electrical units for performing functions.

[0048] The memory (112) may include one or more non-transitory / transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, and combinations thereof, and may include web storage of a server that performs a memory storage function on the Internet. This memory (112) may store data and instructions necessary for the at least one processor (111) to perform the operation of an application to perform a user's request.

[0049] In one embodiment, the user computing device (110) may store at least one machine learning model (120). For example, the user computing device (110) may be configured with a combination of various machine learning models, such as a plurality of neural networks (e.g., a deep neural network) that derive task sequences for a user's request, or other types of machine learning models including non-linear models and / or linear models.

[0050] For example, the predictive model may store a linear regression, decision tree, random forest, gradient boosting, pre-trained language model, or / and a deep learning model. The neural network may include at least one of feed-forward neural networks, recurrent neural networks (e.g., long short-term memory recurrent neural networks), convolutional neural networks, or / and other types of neural networks.

[0051] Additionally, the user computing device (110) may store a model to be used in each process and a prompt template that serves as the basis for input to the model in order to perform at least part of the process for generating a response to a request through a large-scale language model (LLM).

[0052] For example, the user computing device (110) may store 1) a prompt for generating a query from a user's input, 2) a prompt for generating a Q&A set, 3) a prompt for deriving a task order used in the Q&A set, 4) a prompt for performing a sub-task (e.g., search, prediction, summary, calculation, etc.) for each sub-task, etc.

[0053] That is, in one embodiment, the user computing device (110) may request a language model of an external server to prompt some of the execution steps in performing the request, and may perform the user's request based on the received data.

[0054] In another embodiment, the server computing system (130) may perform the user's request through at least one machine learning model (140) and a machine learning model of another server in response to a user's request input through a user computing device (110) and provide response data to the user computing device (110).

[0055] Such a user computing device (110) may include at least one input component (121) that detects a user's input. For example, the user input component (121) may include a touch sensor (e.g., a touch screen or / and a touch pad, etc.) that detects a touch of a user's input medium (e.g., a finger or a stylus), an image sensor that detects a user's motion input, a microphone, a button, a mouse, and / or a keyboard that detects a user's voice input, etc. In addition, the user input component (121) may include an interface and an external controller when receiving an input to an external controller (e.g., a mouse, a keyboard, etc.) through an interface.

[0056] A server computing system (130) includes at least one processor (131) and memory (132). Here, the processor (131) may be composed of at least one or a plurality of electrically connected processors, such as a central processing unit (CPU), a graphics processing unit (GPU), application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors, and / or other electrical units for performing functions.

[0057] And the memory (132) may include one or more non-transitory / transitory computer-readable storage media such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. This memory (132) may store data and instructions necessary for the processor (131) to perform a task through the language model of the server computing system (130) or / and the language model of an external server, a prompt template, a machine learning model (140) for future casting, etc.

[0058] For example, the server computing system (130) may include an LLM model that obtains a user request written in natural language and provides a response to the request in natural language. Furthermore, the server computing system (130) may include a task execution model for performing various types of sub-tasks. For example, the server computing system (130) may include a neural network or / and other multi-layer non-linear models as a machine learning model (140) for futurecasting in a prediction task among the sub-tasks. The neural network may include a feedforward neural network, a deep neural network, a recurrent neural network, and a convolutional neural network.

[0059] In one embodiment, the server computing system (130) may be implemented to include at least one computing device. For example, the server computing system (130) may be implemented to operate multiple computing devices according to a sequential computing architecture, a parallel computing architecture, or a combination thereof. Additionally, the server computing system (130) may include multiple computing devices connected via a network.

[0060] In an embodiment, the server computing system (130) may further include a tool chain database, which is a repository for continuously storing and managing the tool chain that serves as the basis for providing responses to user requests. This tool chain database may include various forms of data storage, ranging from file systems to cloud storage.

[0061] For example, a tool chain database may include at least one of the following: a relational database that uses a structured query language (SQL) to define and manipulate data; a NoSQL database that is designed for flexibility and scalability to handle unstructured and semi-structured data; a data warehouse that centralizes large amounts of data from multiple sources and is optimized for querying and analysis, a data warehouse that stores large amounts of raw data in its native formats of structured, semi-structured, and unstructured data; and a local storage device or Network Attached Storage (NAS) that stores data in files in a format typically accessible by a computer operating system.

[0062] The training computing system (150) includes at least one processor (151) and memory (152). Here, the processor (151) may be composed of at least one or a plurality of processors electrically connected among central processing units (CPUs), graphics processing units (GPUs), application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, and / or other electrical units for performing functions. In addition, the memory (152) may include one or more non-transitory / transitory computer-readable storage media such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. The memory (152) may store data and instructions necessary for the processor (151) to train an artificial intelligence model.

[0063] For example, the training computing system (150) may include a model trainer (160) that trains a machine learning model stored in the user computing device (110) or / and the server computing system (130) using various training or learning techniques, such as backpropagation of errors.

[0064] For example, the model trainer (160) may perform backpropagation when updating one or more parameters of a machine learning model for performing a user's request based on a defined loss function.

[0065] In some implementations, performing backward propagation of errors may include performing truncated backpropagation through time. The model trainer (160) may perform a number of generalization techniques (e.g., weight reduction, dropout, knowledge distillation, etc.) to improve the generalization ability of the trained artificial intelligence model.

[0066] The model trainer (160) comprises computer logic utilized to provide the desired functionality. The model trainer (160) may be implemented as hardware, firmware, and / or software that controls a general-purpose processor. For example, in one embodiment, the model trainer (160) comprises a program file stored on a storage device, which may be loaded into memory and executed by one or more processors. In another implementation, the model trainer (160) comprises one or more sets of computer-executable instructions stored on a tangible computer-readable storage medium, such as a RAM hard disk or an optical or magnetic medium.

[0067] The network (170) includes, but is not limited to, a 3rd Generation Partnership Project (3GPP) network, a Long Term Evolution (LTE) network, a World Interoperability for Microwave Access (WIMAX) network, the Internet, a Local Area Network (LAN), a Wireless Local Area Network (Wireless LAN), a Wide Area Network (WAN), a Personal Area Network (PAN), a Bluetooth network, a satellite broadcasting network, an analog broadcasting network, and / or a Digital Multimedia Broadcasting (DMB) network.

[0068] In general, communication over a network (170) may be performed using any type of wired and / or wireless connection, using various communication protocols (e.g., TCP / IP, HTTP, SMTP, FTP), encodings or formats (e.g., HTML, XML), and / or protection schemes (e.g., VPN, Secure HTTP, SSL).

[0069]

[0070] FIG. 2 illustrates an example of a block diagram of a computing device, which is one of the components of a computing system (1000) that performs a user's request according to an embodiment of the present invention.

[0071] As illustrated in FIG. 2, the computing device (100) included in the user computing device (110), the server computing system (130), and the training computing system (150) includes a plurality of applications (e.g., Application 1 to Application N). Each application may include a machine learning library. For example, the applications may include a futurecasting application, a text messaging application, an email application, a dictation application, a virtual keyboard application, a browser application, a chat-bot application, a separate futurecasting application, and the like.

[0072] In an embodiment, the computing device (100) may include a model trainer (160) for training an artificial intelligence model, and may store and operate the artificial intelligence model to perform a main task and / or a sub-task on input data.

[0073] Each application of the computing device (100) may communicate with a number of other components of the computing device, such as, for example, one or more sensors, a context manager, a device state component, and / or additional components. In one embodiment, each application may communicate with each device component using an API (e.g., a public API). In one embodiment, the API used by each application may be specific to that application.

[0074]

[0075] FIG. 3 illustrates an example block diagram of another aspect of a computing device, which is one of the components of a computing system (1000) that performs a user's request according to an embodiment of the present invention.

[0076] Referring to FIG. 3, a computing device (200) includes a plurality of applications (e.g., Application 1 to Application N). Each application may communicate with a central intelligence layer. For example, the applications may include an image processing application, a text messaging application, an email application, a dictation application, a virtual keyboard application, a browser application, etc. In one embodiment, each application may communicate with the central intelligence layer (and the models stored therein) using an API (e.g., a common API across all applications).

[0077] The central intelligence layer may include prompts utilizing multiple machine learning models and / or language models. For example, as illustrated in FIG. 3, each machine learning model and at least some of the models may be provided to each application and managed by the central intelligence layer. In other implementations, two or more applications may share a single machine learning model. For example, in some implementations, the central intelligence layer may provide a single model to all applications. In some implementations, the central intelligence layer may be included within the operating system of the computing device (200) or implemented differently.

[0078] The central intelligence layer may communicate with a central device data layer. The central device data layer may be a centralized data repository for the computing device (200). As illustrated in FIG. 3, the central device data layer may communicate with a number of other components of the computing device, such as, for example, one or more sensors, a context manager, a device state component, and / or additional components. In some implementations, the central device data layer may communicate with each device component using an API (e.g., a private API).

[0079] The techniques described herein may refer to servers, databases, software applications, and other computer-based systems, as well as actions taken and information transmitted to or from such systems. It will be appreciated that the inherent flexibility of computer-based systems allows for a wide range of possible configurations, combinations, and division of labor and functionality between and among components. For example, the processes described herein may be implemented using a single device or component, or multiple devices or components operating in combination. Databases and applications may be implemented on a single system or in a distributed system across multiple systems. Distributed components may operate sequentially or in parallel.

[0080]

[0081] Hereinafter, a request execution method and system for constructing a tool chain to generate a response to a user's request when such a computing system (1000) obtains a user's request requiring a complex task using an artificial intelligence model, and providing a response to the user's request based on the constructed tool chain will be described with reference to FIGS. 4 to 8.

[0082] FIG. 4 is a flowchart illustrating a method for constructing a tool chain database according to an embodiment of the present invention.

[0083] First, the training computing system (150) can generate multiple Q&A sets (S101).

[0084] Here, the Q&A set according to the embodiment may mean a pair of dictionary questions generated in the same format as the user's request and dictionary answers corresponding to the dictionary questions.

[0085] In an embodiment, the training computing system (150) can generate a Q&A set based on a question-and-answer system that automatically responds to questions posed by humans in natural language. The question-and-answer system may be a system trained using various methods, depending on the method or domain.

[0086] For example, a question-answering system may be a system trained based on an extraction-based method that predicts the location of an answer within a context given a question, a context, and a correct answer, and / or a generation-based method that directly generates the correct answer given a question and a context. Furthermore, the question-answering system may be a closed-domain system that answers questions within a predefined domain, such as medicine or finance, and / or an open-domain system that answers questions whose domain is not predefined (e.g., an ontology).

[0087] When the above training computing system (150) generates a Q&A set based on a question-and-answer system, the generated pre-questions and / or pre-answers may include at least two or more sub-tasks.

[0088] Subtasks according to the embodiment are tasks specialized for a specific field, such as search tasks, prediction tasks, summary tasks, and / or calculation tasks. For each of these subtasks, prompts for performing the corresponding subtask may be preset.

[0089] For example, in the case of a prediction task, it can be a sub-task that sets a prompt (or process sequence) for prediction, such as target setting - deriving target influence variables - setting a period - predicting target outlook.

[0090] Meanwhile, the main task according to the embodiment is a task of generating an answer to a user's request (query). If it is a simple question, the answer can be derived using only one sub-task. However, in the embodiment of the present invention, the main task is described based on the fact that it is a task that necessarily combines at least two or more sub-tasks.

[0091] In addition, when the training computing system (150) generates a Q&A set based on a question-and-answer system, the generated pre-answer may include a hallucinatory answer.

[0092] Hallucination refers to a phenomenon in which an AI model presents information that is not actually present or true as fact. In other words, in this example, a hallucinatory response could refer to a prior response containing false information.

[0093] The reason why the Q&A set according to the embodiment includes a hallucination answer is that the Q&A set is not intended to determine whether a pre-answer is a correct answer to a pre-question, but rather to determine how the training computing system (150) planned the sequence of operations for a plurality of sub-tasks to derive an answer in order to derive a pre-answer to a pre-question.

[0094] Here, a tool chain according to an embodiment may mean a single process chunk that has planned and stored in advance the types and operation sequences of sub-tasks used to derive answers to questions.

[0095] These tool chains can be matched and stored for each Q&A set, and the training computing system (150) can derive an answer through an automated process according to the tool chain when a user's query is input.

[0096] That is, the reason for creating multiple Q&A sets in the embodiment is to construct a tool chain database through the Q&A sets, thereby easily determining the fastest and most accurate optimal logical flow through the constructed tool chain database in order to derive an answer when obtaining a user request through a user computing device (110) in the future.

[0097] Additionally, the training computing system (150) can convert the generated Q&A set into a vector and store it in memory (152).

[0098] Additionally, the training computing system (150) can request a sub-task operation sequence plan for the generated Q&A set (S103).

[0099] In detail, the training computing system (150) may be requested to plan the operation sequence of sub-tasks for generating a Q&A set based on a large-scale language model (LLM).

[0100] More specifically, the training computing system (150) can request a large-scale language model (LLM) to plan the sequence of actions of sub-tasks in order to derive a pre-answer from a pre-question included in a Q&A set.

[0101] Accordingly, the large-scale language model can plan the requested sub-task operation sequence and provide it to the training computing system (150).

[0102] For example, upon request, a large-scale language model can plan the sub-task operation order for the first Q&A set in the order of search - summary - calculation - prediction and provide it to the training computing system (150).

[0103] That is, the training computing system (150) can obtain a tool chain in which the sub-task operation order is determined for the first Q&A set.

[0104] In the above manner, the training computing system (150) can create a tool chain pair by matching the planned tool chain with the Q&A set according to the above request (S105).

[0105] In other words, a tool chain pair can mean a pair of a pre-question, a pre-answer corresponding to the pre-question, and a tool chain used to derive the pre-answer.

[0106] Additionally, the training computing system (150) can vectorize the generated tool chain pair and store it in the tool chain database. (S107)

[0107] In this case, in the first embodiment, the tool chain pair may include a pre-question and a tool chain. Furthermore, in the second embodiment, the tool chain pair may include a pre-question, a pre-answer, and a tool chain. This will be described in detail later.

[0108] Such a tool chain database can store Q&A sets that combine at least one to n sub-tasks for various fields, and tool chains matched to each set.

[0109] That is, the training computing system (150) can build a tool chain database that stores tool chain pairs by vectorizing them.

[0110] And the server computing system (130) can filter the main information to be searched when obtaining a query among the features of the tool chain pair, and integrate the filtered main information and store it in the tool chain database.

[0111] To this end, the server computing system (130) can list features available in the tool chain database by feature name. Furthermore, a description for each feature can be listed.

[0112] At this time, the server computing system (130) can refine the description using a large-scale language model (LLM) and perform embedding. This has the effect of better capturing important information in the description of features during embedding.

[0113] And the server computing system (130) can filter the main information to be searched among the features of the tool chain pair based on the semantic level.

[0114] For this purpose, the server computing system (130) can utilize a machine learning model or language model that extracts features of the tool chain pair.

[0115] In an embodiment, the server computing system (130) can map features to each tool chain pair by listing feature names and descriptions of tool chain pairs and inputting keywords of the tool chain pairs into a word embedding model to detect tool chain pairs based on feature relevance. Here, word embedding refers to a model trained to classify features relevant to a tool chain pair based on feature names and descriptions.

[0116] And the server computing system (130) can retrieve tabular data corresponding to the names of classified features from the tool chain database, organize and preprocess the retrieved tabular data, and arrange it in a structured format to make it suitable for searching when a request is obtained.

[0117] In this way, the server computing system (130) can vectorize tool chain pairs and build a tool chain database by matching features to the tool chain pairs.

[0118] That is, the tool chain database can be utilized to determine a tool chain when providing an actual question (hereinafter, query) included in a user request obtained later based on a user computing device (110) and an actual answer (hereinafter, answer) corresponding to the query.

[0119] Meanwhile, the training computing system (150) can update the tool chain database with queries and answers obtained based on the request execution service (S109).

[0120] The training computing system (150) can collect and store queries and answers based on user requests in the request execution service, as well as the tool chain used to derive the answers. Hereinafter, a query, answer, and tool chain used to derive the answers are referred to as a "query pair."

[0121] Additionally, the training computing system (150) can update the tool chain database by retraining the query pairs on a large-scale language model (LLM).

[0122] At this time, the training computing system (150) can use the RAG (Retrieval Augmented Generation) model when updating the tool chain database.

[0123] Here, the RAG model can operate as a kind of module that is operated to update the tool chain database with the latest contents by injecting the latest information into the large-scale language model (LLM) according to an embodiment of the present invention.

[0124] For example, a RAG model can be one of the following: naive RAG, advanced RAG, or modular RAG. For example, the advanced RAG model can refine data by removing unnecessary information and special characters through a pre-retrieval process, optimizing the index structure through chunk size adjustments, index path changes, and the like, and adding metadata such as date and purpose to each data chunk.

[0125] Additionally, the embedding model can be tuned to improve query-answer relevance through fine-tuning embedding or / and dynamic embedding.

[0126] In addition, data can be refined by combining important contexts from the retrieved content with queries through a post-retrieval process, entering them into LLM as query pairs, rearranging the retrieved content in order of relevance, and compressing prompts according to importance.

[0127] A RAG model such as this may be a model that combines a pre-trained parametric memory (e.g., a sequence-to-sequence (seq2seq) model) and a non-parametric memory (e.g., a dense vector index of Wikipedia). The parametric memory may perform searches conditioned on the same phrase across the entire sequence, while the non-parametric memory may perform searches conditioned on different phrases per token.

[0128] That is, the training computing system (150) can determine the collection time of information constituting tool chain pairs previously stored in the tool chain database based on the RAG model in order to update the tool chain database with the latest information.

[0129] Among them, the training computing system (150) may request the large-scale language model (LLM) to update the first tool chain pair if the identified collection time for the first tool chain pair is earlier than the preset first time point.

[0130] In other words, the training computing system (150) can remove unnecessary information included in the tool chain database constructed based on the RAG model and update the tool chain database to include more specific and diverse up-to-date information.

[0131] Accordingly, the training computing system (150) can construct a tool chain database based on the above-described steps S101 to S109, and provide a request execution service based on the constructed tool chain database.

[0132] FIG. 5 is a flowchart illustrating a method for providing a request execution service according to an embodiment of the present invention.

[0133] Here, the request execution service according to the embodiment may mean a service that receives a text-based request from a user through a chat interface provided to a user computing device (110), transmits a text including a query to a server computing system (130), and executes a request execution task of the server computing system (130).

[0134] In an embodiment, the server computing system (130) can obtain a query entered through the chat interface of the user computing device (110). (S201)

[0135] In detail, the server computing system (130) may receive text input from the user computing device (110) through the chat interface of the user computing device (110), detect pre-stored phrases for a query from the text, or analyze the text based on context to detect the context of the query, and execute a request execution task.

[0136] That is, in an embodiment, obtaining a query may mean obtaining a user's request.

[0137] And the server computing system (130) can recognize text including a query and determine search target elements for answer generation.

[0138] At this time, the server computing system (130) provides a separate interface for inputting search target elements for request execution when the search target elements are not specified or are abstract, and transmits the search target elements input through the provided interface to the server computing system (130) to execute the request execution task.

[0139] In other words, when the search target elements included in the query are classified from a superordinate concept to multiple subordinate concepts according to category, the server computing system (130) can list keywords mapped to the superordinate and subordinate concepts and provide them for the user to select.

[0140] For example, the server computing system (130) can obtain a query “Check the nickel price trend” input based on the user computing device (110).

[0141] Additionally, the server computing system (130) can detect a tool chain pair corresponding to the above query from the tool chain database (S203).

[0142] FIGS. 6 to 8 are drawings for explaining a method for detecting a tool chain pair according to various embodiments of the present invention.

[0143] Referring to FIG. 6, in the first embodiment, the server computing system (130) can detect a tool chain pair in which a pre-question and a tool chain are vectorized. For example, the tool chain database (TDB) can include a first vector (701) in which a first pre-question (501) and a first tool chain (601) are vectorized.

[0144] Thus, referring to FIG. 7, in the first embodiment, the server computing system (130) can detect a first pre-question (501) matching the query (Q) based on the tool chain database (TDB).

[0145] Additionally, in the first embodiment, the server computing system (130) can detect a first tool chain (601) matching the first pre-question (501).

[0146] According to the first embodiment, the server computing system (130) can detect the tool chain to be used for the user's request using only the query (Q), thereby simplifying the process of detecting an answer to the query, thereby providing an advantage in that an answer to the user's request can be provided more quickly.

[0147] Also, referring back to FIGS. 6 and 7 , in the second embodiment, the server computing system (130) can detect a tool chain pair in which a pre-question, a pre-answer, and a tool chain are vectorized. For example, the tool chain database (TDB) can include a first vector (711) in which a first pre-question (511), a first pre-answer (512), and a first tool chain (611) are vectorized.

[0148] In a second embodiment, the server computing system (130) can detect a first pre-question (511) matching a query (Q) and at least one first pre-answer (512a, 512b, 512c) matching the first pre-question based on a tool chain database (TDB).

[0149] Additionally, in the second embodiment, the server computing system (130) can determine one of the first pre-answers (512a, 512b, 512c) detected based on the user input.

[0150] For example, multiple first-predefined answers (512a, 512b, 512c) matching a first-predefined question (511) may all have different highlighted features, and thus, their matching tool chains may all be different. To achieve this, the user may input a selection from among the presented predefined answers based on the desired answer style (type).

[0151] That is, in the first and second embodiments, various dictionary answers and various tool chains can be determined for one dictionary question, and accordingly, multiple vectors corresponding to one dictionary question can be generated and stored in the tool chain database.

[0152] As shown in the example in Figure 8, the first pre-answer (512a) may focus on past data analysis, the second pre-answer (512b) may focus on future data prediction, and the third pre-answer (512c) may focus on visual data. Accordingly, each pre-answer may contain different content, and thus, the tool chains may differ.

[0153] To this end, in the second embodiment, the server computing system (130) can perform a type analysis task on a plurality of detected pre-answers. Accordingly, in the second embodiment, the server computing system (130) can control the transmission of the plurality of pre-answers, whose types have been analyzed, to the user computing device (110) and display them on the display so that an answer can be selected based on the user's input.

[0154] At this time, the type analysis task performed above may be, for example, a summary task. Such a type analysis task may provide a portion of a pre-answer and / or answer, as illustrated in Figure 8, or provide a summary of a portion with well-defined characteristics.

[0155] Next, in the second embodiment, the server computing system (130) can detect a first tool chain (611) matching the determined first pre-answer (512a, 512b, 512c).

[0156] According to the second embodiment, there is an advantage in that the server computing system (130) can provide a variety of types of pre-answers that can be derived from the query in advance, and can provide an answer in the style desired by the user, thereby providing a customized answer that suits the user's taste.

[0157] That is, the server computing system (130) can determine the first tool chain for performing the query based on the detected tool chain pair (S205).

[0158] In other words, the server computing system (130) can determine a first tool chain in which the types and operation sequences of sub-tasks are planned in advance to provide answers to user queries.

[0159] Additionally, the server computing system (130) can perform the first to nth sub-tasks included in the determined first tool chain. (S207)

[0160] In detail, the server computing system (130) can request the large-scale language model (LLM) to sequentially perform the first to nth sub-tasks included in the determined first toolchain.

[0161] Accordingly, the server computing system (130) can obtain an answer generated by sequentially performing the first to nth sub-tasks from a large-scale language model (LLM).

[0162] For example, if the server computing system (130) obtains a query such as “Check the nickel price trend,” the tool chain corresponding to the query may be configured in the following order: financial data search -> financial data forecasting -> fundamental analysis -> conclusion. Accordingly, the server computing system (130) may perform subtasks in the order of the tool chain configuration.

[0163] Additionally, the server computing system (130) can check whether the reference information of the sub-task performed above has been updated to the latest information after a predetermined point in time based on the RAG model.

[0164] To this end, the server computing system (130) can extract and confirm the information generation time of the information collected in the sub-task corresponding to the information collection step.

[0165] At this time, if the time of creation of the extracted information is before a predetermined time point, the subtask of the tool chain can be re-executed so that the time of creation of the information is after a predetermined time point.

[0166] Accordingly, the server computing system (130) can obtain an answer including the latest information as described above in step S109.

[0167] Additionally, the server computing system (130) can generate and provide an answer according to the performed subtask. (S209)

[0168] That is, the server computing system (130) can generate an answer, which is an actual answer corresponding to a user request, and provide it to the user computing device (110).

[0169] At this time, the server computing system (130) can provide the user with a visualization of the sub-task order of the tool chain used when generating the answer.

[0170] In detail, the server computing system (130) can control the user computing device (110) to output the task order of the tool chain used in generating the answer as visual data to the user based on the tool chain database in which the tool chain pairs listed by feature name are stored.

[0171] That is, the server computing system (130) can provide a basis for determining which characteristics of a tool chain pair influence the search and are thus determined as a tool chain.

[0172] Accordingly, the user can understand the logical flow by which the obtained answer was generated, which has the advantage of further improving user reliability.

[0173] Above, the method and system for performing a user's request based on an artificial intelligence model according to the present invention has the effect of reducing the illusionary effect that may occur when generating answers to complex questions and increasing the accuracy of answers by planning and determining an operation sequence using multiple tasks and constructing a tool chain to derive accurate answers to complex and diverse types of requests.

[0174] In addition, the method and system for performing a user's request based on an artificial intelligence model according to the present invention generates at least one Q&A set based on the user's request and learns the task order, thereby performing learning based on the user's actual questions, thereby increasing practical usability.

[0175] In addition, the method and system for performing a user's request based on an artificial intelligence model according to the present invention has the effect of reducing errors that may occur while providing a response in natural language to a user by performing a main task of generating a response to a query and / or at least one sub-task constituting the main task.

[0176] In addition, the method and system for performing a user's request based on an artificial intelligence model according to the present invention generates a response by determining the sub-task operation order according to the constructed tool chain when obtaining a user's request, thereby generating a response according to a predetermined order, thereby increasing the response speed.

[0177] Although the detailed description of the present invention has been described with reference to preferred embodiments of the present invention, it will be understood by those skilled in the art or having ordinary knowledge in the art that various modifications and changes can be made to the present invention without departing from the spirit and technical scope of the present invention as set forth in the claims below. Accordingly, the technical scope of the present invention should not be limited to the contents described in the detailed description of the specification, but should be defined by the claims.

[0178] The form for carrying out the invention is the same as the best form for carrying out the invention described above.

[0179] By building a tool chain that plans and determines the sequence of operations using multiple tasks to derive accurate answers to complex and diverse types of requests, it improves the usability of artificial intelligence models by reducing the illusionary effect that may occur when generating answers to complex questions and increasing the accuracy of answers, and thus has industrial applicability.

Claims

1. A method for performing a user's request based on an artificial intelligence model performed by a processor of a server computing system. A step of controlling the construction of a tool chain database based on a training computing system; A step of obtaining a query included in a user request received from a user computing device; A step of detecting a tool chain pair corresponding to the above query from the tool chain database; A step of determining a first tool chain for performing the user request based on the detected tool chain pair; A step of sequentially performing the first to nth subtasks matching the first tool chain determined above; and A step of generating and providing an answer to the query according to the first to nth sub-tasks performed above; A method for executing user requests based on an artificial intelligence model.

2. In paragraph 1, The step of controlling to build a tool chain database based on the above training computing system is: A step for controlling the generation of multiple sets of Q&A matching pre-questions and pre-answers, A step of requesting a sub-task operation sequence plan for the above-generated Q&A set, A step of controlling the planned tool chain according to the above request to create a tool chain pair by matching the above Q&A set, A step of storing the generated tool chain pair in the tool chain database. A method for executing user requests based on an artificial intelligence model.

3. In paragraph 2, The step of controlling the generation of multiple sets of the above Q&A is as follows: A step of controlling input of a pre-question generated in the same format as the above user request into the question-and-answer system, A step of obtaining at least one pre-answer output from the above question and answer system, A step of generating a first pre-question and a first pre-answer corresponding to the first pre-question as a first Q&A set. A method for executing user requests based on an artificial intelligence model.

4. In paragraph 3, The above pre-answer output from the above question-and-answer system includes a hallucination answer. A method for executing user requests based on an artificial intelligence model.

5. In paragraph 2, The step of requesting a plan for the sub-task operation sequence for the above Q&A set is: A step of requesting a large-scale language model (LLM) to plan the sub-task operation sequence and obtaining the output result; A step of storing a tool chain in which the types and operation sequences of at least two sub-tasks are determined based on the above-obtained results. A method for executing user requests based on an artificial intelligence model.

6. In paragraph 1, The step of detecting the above tool chain pair from the above tool chain database is: A step for detecting a first tool chain pair that is vectorized by a pre-question and tool chain, At least one step of detecting a second tool chain pair, which is a pre-question, a pre-answer, and a tool chain vectorized. A method for executing user requests based on an artificial intelligence model.

7. In paragraph 6, The step of detecting the second tool chain pair is: A step of obtaining multiple dictionary answers corresponding to one dictionary question, and A step of performing a type analysis task for each of the above multiple pre-answers, Including a step of determining the answer type for each vectorized tool chain pair according to the performance of the above type analysis task. A method for executing user requests based on an artificial intelligence model.

8. In paragraph 7, The step of detecting the above tool chain pair from the above tool chain database is: A step of providing pre-answers of the above-mentioned answer types to a user computing device, further comprising a step of obtaining an input for selecting one of the provided pre-answers of different answer types from the user computing device. A method for executing user requests based on an artificial intelligence model.

9. In paragraph 1, A step of updating the reference information of the tool chain pair included in the tool chain database constructed above based on the RAG (Retrieval Augmented Generation) model to the latest information after a predetermined point in time is further included. A method for executing user requests based on an artificial intelligence model.

10. A server computing system that receives a user request from a user computing device and performs the user request based on an artificial intelligence model. A tool chain database that stores multiple tool chain pairs; Memory that stores commands and data to perform user requests; and At least one processor that performs the user's request according to the instructions and data of the memory, At least one processor above, Controls the construction of a tool chain database based on a training computing system, Obtain a query included in a user request received from a user computing device, Detecting a tool chain pair corresponding to the above query from the tool chain database, Determine a first tool chain for performing the user request based on the detected tool chain pair, Perform the first to nth subtasks that match the first tool chain determined above in sequence, Generate and provide an answer to the query according to the first to nth sub-tasks performed above. A system that carries out user requests based on an artificial intelligence model.

Images