Interactive information delivery method, interactive information delivery system, computer device, and program

Abstract

To provide a method and system for providing information data while minimizing infrastructure resources.SOLUTION: A method according to one embodiment includes the steps of: searching for an answer to a user's question from a DB (database) that stores questions and answers; displaying the answer stored in the DB if the answer to the user's question is present in the DB; and displaying an answer generated by a LLM (large language model) if the answer to the user's question is not present in the DB.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The following description relates to a method and system for providing information data as an answer to a user's question.

Background Art

[0002] Large Language Models (hereinafter referred to as "LLMs") are a type of artificial intelligence trained with large amounts of text data to generate human-level responses to natural language inputs, and are language models composed of artificial neural networks having billions or more parameters.

[0003] Such LLMs are trained using large amounts of unlabeled text through self-supervised learning or semi-self-supervised learning.

[0004] As an example of a technology for providing an interactive service, Patent Document 1 (publication date: February 27, 2018) discloses a technology for providing a personal secretary service using an artificial intelligence-based chatbot.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] Using an LLM, it is possible to generate predicted questions for each theme and answers thereto.

[0007] To minimize infrastructure requirements, a hybrid service can be provided that offers both database-based and LLM-based solutions. [Means for solving the problem]

[0008] An interactive information provision method for a computer device including at least one processor is provided, comprising the steps of: using at least one processor to search for an answer to a user's question from a database (DB) where questions and answers are stored; using at least one processor to display the answer stored in the DB if an answer to the user's question exists in the DB; and using at least one processor to display an answer generated by a large language model (LLM) if an answer to the user's question does not exist in the DB.

[0009] In one respect, the interactive information provision method may further include a step in which at least one processor stores the user's questions and the answers generated by the LLM in a database for use in subsequent responses.

[0010] From another perspective, the interactive information delivery method may further include a step of reforming the user's question using at least one processor.

[0011] Furthermore, from another perspective, the renovation stage may include a stage of generating renovation questions in response to user inquiries, based on information registered by the information provider or previous conversations with the user.

[0012] Furthermore, from another perspective, the interactive information provision method may further include a step in which at least one processor determines whether or not it is possible to answer the user's question.

[0013] Furthermore, from another perspective, the stage of determining whether a user's question can be answered may include a stage of identifying questions that cannot be answered because they violate the established theme or safety standards.

[0014] Furthermore, from another perspective, the step of determining whether a user's question can be answered may include a step of displaying a message indicating that the user's question cannot be answered if it is determined that the question cannot be answered.

[0015] Furthermore, from another perspective, the step of displaying the answer generated by LLM may include a step of generating the LLM answer using documents that match the user's question.

[0016] Furthermore, from another perspective, the stage of generating an LLM answer may involve searching for documents that match the user's question from the documents provided by the information provider and then generating the LLM answer.

[0017] Furthermore, from another perspective, the step of generating the LLM answer may involve searching for documents that match the user's question through web browsing and then generating the LLM answer.

[0018] Furthermore, from another perspective, the interactive information delivery method may further include a step in which at least one processor displays the user's question and related additional content together with the answer to the user's question.

[0019] Furthermore, from another perspective, the additional content may consist of content mapped based on entities in response to the user's questions.

[0020] This invention provides a computer program that allows a computer device to execute an interactive information delivery method.

[0021] A computer device includes at least one processor that executes instructions readable by the computer device. The at least one processor searches for an answer to a user's question from a DB (database) storing questions and answers. If an answer to the user's question exists in the DB, the answer stored in the DB is displayed. If an answer to the user's question does not exist in the DB, an answer generated by an LLM (large language model) is displayed. A computer device is provided, which is characterized by the above.

[0022] A dialogue information providing system realized by a computer includes a DB (database) storing pre-generated questions and answers, and an LLM (large language model) learned by the questions and answers. As an answer to a user's question, either an answer retrieved from the DB or an answer generated by the LLM is displayed. A dialogue information providing system is provided, which is characterized by the above.

Advantages of the Invention

[0023] According to an embodiment of the present invention, when providing an answer to a user's question, by providing a hybrid service capable of using an answer using the DB and an answer using the LLM, infrastructure resources can be minimized.

Brief Description of the Drawings

[0024] [Figure 1] It is a diagram showing an example of a network environment in an embodiment of the present invention. [Figure 2] It is a block diagram showing an example of a computer device in an embodiment of the present invention. [Figure 3] It is a schematic diagram showing an example of a dialogue information providing system in an embodiment of the present invention. [Figure 4] It is a flowchart showing an example of a dialogue information providing method in an embodiment of the present invention. [Figure 5]This figure shows an example of an LLM-based question and answer generation flow in one embodiment of the present invention. [Figure 6] This figure shows an example of LLM-based prediction question generation in one embodiment of the present invention. [Figure 7] This figure shows an example of a question form for providing answers in one embodiment of the present invention. [Figure 8] This figure shows an example of determining whether or not a response is possible in one embodiment of the present invention. [Figure 9] This figure shows an example of a learning method for a response feasibility determination model in one embodiment of the present invention. [Figure 10] This figure shows an example of a response provision flow using hallucination control in one embodiment of the present invention. [Figure 11] This figure shows an example of additional content mapping in one embodiment of the present invention. [Figure 12] This figure shows an example of a screen for an interactive AI information provision service in one embodiment of the present invention. [Modes for carrying out the invention]

[0025] Embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0026] An interactive information provision system according to an embodiment of the present invention may be implemented by at least one computer device. In this case, a computer program according to one embodiment of the present invention may be installed and executed on the computer device implementing the interactive information provision system, and the computer device may execute an interactive information provision method according to an embodiment of the present invention in accordance with the control of the executed computer program. The above-described computer program may be recorded on a computer-readable recording medium in combination with the computer device to cause the computer to execute the interactive information provision method.

[0027] Figure 1 is a diagram showing an example of a network environment in one embodiment of the present invention. The network environment in Figure 1 shows an example that includes multiple electronic devices 110, 120, 130, 140, multiple servers 150, 160, and a network 170. Figure 1 is merely an example for the purpose of explaining the invention, and the number of electronic devices and servers is not limited to what is shown in Figure 1.

[0028] The multiple electronic devices 110, 120, 130, and 140 may be fixed terminals or mobile terminals implemented by a computer system. Examples of the multiple electronic devices 110, 120, 130, and 140 include smartphones, mobile phones, navigation systems, PCs (personal computers), notebook PCs, digital broadcasting terminals, PDAs (Personal Digital Assistants), PMPs (Portable Multimedia Players), tablets, game consoles, wearable devices, IoT (Internet of Things) devices, VR (virtual reality) devices, and AR (augmented reality) devices. As an example, Figure 1 shows a smartphone as an example of electronic device 110, but in embodiments of the present invention, electronic device 110 may mean one of a variety of physical computer systems that can communicate with other electronic devices 120, 130, 140 and / or servers 150, 160 via a network 170 using substantially wireless or wired communication methods.

[0029] The communication method is not limited, and it may include not only communication methods that utilize communication networks that can be included in network 170 (for example, mobile communication networks, wired internet, wireless internet, broadcasting networks, phase networks, etc.), but also short-range wireless communication between devices. For example, network 170 may include one or more arbitrary networks from among PAN (personal area network), LAN (local area network), CAN (campus area network), MAN (metropolitan area network), WAN (wide area network), BBN (broadband network), and the Internet. Furthermore, network 170 may include, but is not limited to, one or more arbitrary network topologies, including bus networks, star networks, ring networks, mesh networks, star-bus networks, tree or hierarchical networks.

[0030] Servers 150 and 160 may each be implemented by one or more computer devices that communicate with multiple electronic devices 110, 120, 130, and 140 via a network 170 to provide commands, code, files, content, services, etc. For example, server 150 may be a system that provides a first service to multiple electronic devices 110, 120, 130, and 140 connected via the network 170, and server 160 may also be a system that provides a second service to multiple electronic devices 110, 120, 130, and 140 connected via the network 170. As a more specific example, server 150 may provide the multiple electronic devices 110, 120, 130, and 140 as a first service through an application, which is a computer program installed and executed on the multiple electronic devices 110, 120, 130, and 140, with the service targeted by that application (for example, an interactive AI information provision service). As another example, server 160 may provide a second service that distributes files for installing and running the aforementioned application to multiple electronic devices 110, 120, 130, and 140.

[0031] Figure 2 is a block diagram showing an example of a computer device in one embodiment of the present invention. Each of the aforementioned electronic devices 110, 120, 130, and 140, as well as each of the servers 150 and 160, may be implemented by the computer device 200 shown in Figure 2.

[0032] Such a computer device 200 may include a memory 210, a processor 220, a communication interface 230, and an input / output interface 240, as shown in Figure 2. The memory 210 is a computer-readable recording medium and may include RAM (random access memory), ROM (read-only memory), and persistent mass storage devices such as disk drives. Here, persistent mass storage devices such as ROM and disk drives may be included in the computer device 200 as separate persistent storage devices distinct from the memory 210. The memory 210 may also store an operating system and at least one program code. Such software components may be loaded into the memory 210 from a computer-readable recording medium separate from the memory 210. Such a separate computer-readable recording medium may include computer-readable recording media such as floppy disks, disks, tapes, DVD / CD-ROM drives, and memory cards. In other embodiments, the software components may be loaded into the memory 210 through a communication interface 230 which is not a computer-readable recording medium. For example, software components may be loaded into the memory 210 of the computer device 200 based on a computer program installed by a file received via the network 170.

[0033] The processor 220 may be configured to process computer program instructions by performing basic arithmetic, logic, and input / output operations. Instructions may be provided to the processor 220 by memory 210 or a communication interface 230. For example, the processor 220 may be configured to execute instructions received according to program code stored in a recording device such as memory 210.

[0034] The communication interface 230 may provide a function for the computer device 200 to communicate with other devices (for example, the recording device described above) via the network 170. For example, requests, instructions, data, files, etc., generated by the processor 220 of the computer device 200 according to program code recorded in a recording device such as memory 210 may be transmitted to other devices via the network 170 under the control of the communication interface 230. Conversely, signals, instructions, data, files, etc., from other devices may be received by the computer device 200 via the network 170 through the communication interface 230 of the computer device 200. Signals, instructions, data, etc., received via the communication interface 230 may be transmitted to the processor 220 or memory 210, and files, etc., may be recorded on a recording medium (the persistent recording device described above) that the computer device 200 may further include.

[0035] The input / output interface 240 may be a means for interface with the input / output device 250. For example, the input device may include a microphone, keyboard, or mouse, and the output device may include a display or speaker. In another example, the input / output interface 240 may be a means for interface with a device that integrates input and output functions into one, such as a touchscreen. The input / output device 250 may consist of the computer device 200 and one other device.

[0036] In other embodiments, the computer device 200 may include fewer or more components than those shown in Figure 2. However, it is not necessary to explicitly show most of the conventional components in the figure. For example, the computer device 200 may be implemented to include at least some of the input / output devices 250 described above, and may further include other components such as transceivers and databases.

[0037] Figure 3 is a schematic diagram showing an example of an interactive information provision system according to one embodiment of the present invention. Figure 3 shows the interactive information provision system 310, a plurality of users 330, and a plurality of information providers 340.

[0038] The interactive information provision system 310 may correspond to a server (for example, server 150) that provides interactive AI information provision services to multiple users 330, and may be implemented by at least one computer device 200. Here, each of the multiple users 330 may be a physical device of a user that connects to the interactive information provision system 310 using the network 170 to receive the interactive AI information provision service, and such a physical device may be implemented by the computer device 200 described above.

[0039] The interactive AI information service provided by the interactive information provision system 310 to multiple users 330 may include search results corresponding to user input. The search results may be generated based on information that is searchable on the web. In particular, the interactive information provision system 310 may provide the interactive AI information service by including information data intended to be provided by at least one of the multiple information providers 340 in the search results. Here, the multiple information providers 340 may be advertisers providing brand information, and the information provided by the multiple information providers 340 may be advertising information, but is not limited to this.

[0040] On the other hand, the interactive information provision system 310 according to this embodiment may provide an interactive AI information provision service by including answers based on an artificial intelligence model such as LLM in the search results. For example, suppose the interactive information provision system 310 receives a natural language-based prompt from a specific user among a group of users 330. In this case, the interactive information provision system 310 may input the received prompt into LLM to generate an answer suitable for the prompt as an LLM result, and provide the user with search results including that answer. In this case, the search results may include at least a portion of existing diverse search results in addition to the LLM result. Furthermore, the interactive information provision system 310 may provide an interactive AI information provision service through a conversation between an LLM-based artificial intelligence and a user. When generating a dialogue-based answer between the artificial intelligence and the user, the interactive information provision system 310 may not simply provide the information provided by the information provider, but may generate an artificial intelligence-based answer using the user's prompt, the answer generated using LLM, the asset registered by the information provider, and / or the prompt registered by the information provider. Here, assets may include, for example, URLs (Uniform Resource Locators) associated with the content the information provider intends to provide, the title or identifier of the content, the category of the content, multimedia associated with the content, the content of the content, and the content of articles associated with the content. Here, multimedia associated with the content may include images and videos associated with the content. For example, if the information provider is an advertiser who wants to promote a specific product or service, the assets may include URLs associated with the product or service, product or service name, category of the product or service, product or service information, and article content associated with the product or service. In addition, prompts registered by the information provider may include information such as phrases or keywords that the information provider wants to emphasize in relation to the content they want to provide, and the tone and format of the information message to be provided as a response.

[0041] Figure 4 is a flowchart showing an example of an interactive information provision method in one embodiment of the present invention. The interactive information provision method according to this embodiment may be executed by a computer device 200 that implements the interactive information provision system 310 described above. In this case, the processor 220 of the computer device 200 may be implemented to execute control instructions from the operating system code contained in the memory 210 or the code of at least one computer program. Here, the processor 220 may control the computer device 200 so that the computer device 200 executes the steps included in the method of Figure 4 in accordance with the control instructions provided by the code recorded in the computer device 200.

[0042] The interactive information provision method according to this embodiment may include a preparation stage and a provision stage for an interactive AI information provision service.

[0043] During the service preparation phase, the computer device 200 may generate questions and answers for implementing a chatbot using LLM, and then store the generated questions and answers in the database (S411-S413). In this embodiment of the interactive AI information provision service, in order to reduce the cost of infrastructure resources and utilize database search, the questions and answers generated based on LLM are stored in the database during the service preparation phase.

[0044] Traditionally, a method was employed in which humans created predictive questions and corresponding answers, but this method consumed a lot of time and resources, and was prone to human errors such as typos. To improve this, this embodiment applies a method in which predictive questions are generated using LLM, and the corresponding answers are also generated using LLM.

[0045] During the service provision phase, the computer device 200 may first search the database for an answer corresponding to a question entered by the user and display the answer found in the database (S421-S427). If an answer to the user's question does not exist in the database, the computer device 200 may generate an answer using LLM (S441-S442).

[0046] If an LLM is called every time a question from an unspecified number of users is input into an interactive AI information provision service, it would require a significant amount of infrastructure resources, potentially leading to problems such as high operating costs. To improve this, this embodiment provides a hybrid service that allows for both database-based and LLM-based answers to minimize infrastructure requirements. In the service preparation phase, the system is designed to store answers generated by the LLM in the database beforehand. In the service delivery phase, answers to user questions are first processed using database searches, which have relatively low infrastructure resource costs. If an answer is not defined in the database, the LLM generates an answer in real time, thereby minimizing LLM calls during the service delivery phase and reducing LLM infrastructure costs in the long term.

[0047] Furthermore, in the case of questions entered in natural language, it may be difficult to properly grasp the user intent contained in the question, which can lead to a decrease in the quality of the answer. To improve this, in this embodiment, a function to revise the user's question may be supported (S422). In this case, a revised question to the user's question may be generated based on the information that the information provider intends to provide, or on a multi-turn including previous conversations with the user.

[0048] In the case of predefined answers, there is a risk of decreasing user interest due to the repeated display of the same content, and in the case of undefined content, there are limitations such as the inability to provide an answer. In this embodiment, the generative AI improves the hallucination problem, in which false information is generated and transmitted as if it were fact. The computer device 200, through the answerability determination process (S431), may display a message indicating that it cannot be answered if the user's question violates a defined theme or safety standard, rather than displaying an answer. In other words, it verifies whether the natural language-based question received from the user is a question in which it is acceptable to provide information from the information provider. If the information provider is an advertiser who wants to promote their own advertisement, the advertiser may not want their advertisement to be displayed in response to questions that request pre-set illegal or non-advertising information. Therefore, the first step is to verify whether the user's question is a question in which it is safe to provide information from the information provider. Subsequently, if the user's question does not violate any themes or safety standards, the computer device 200 searches for documents that match the question and then uses the searched documents to generate an answer based on LLM (S441-S442). At this time, the LLM answer provision method may be divided into two types. Specifically, one method is to search for documents that match the user's question from among the documents provided by the information provider and provide an answer, and the other method is to search for documents that match the user's question by web browsing if no documents are provided by the information provider and provide an answer. Furthermore, the computer device 200 may provide immersive information in a visualized form by mapping rich content such as images and videos as content related to the question through a matching process between entities derived from the user's question and content (S426).

[0049] Figure 5 shows an example of an LLM-based question and answer generation flow in one embodiment of the present invention. Figure 5 shows the question and answer generation process of an advertiser chatbot.

[0050] Referring to Figure 5, for example, if the information provider is an advertiser, the advertiser may use the registration function provided by server 150 to input various campaign information, such as the goals and plans of the advertising campaign, the message to be emphasized, the target audience, and the desired tone and manner when responding to the advertisement (S501).

[0051] Server 150 may generate predictive questions in relation to campaign information input from advertisers during the process of generating predictive questions on a specific theme using LLM (S502). Figure 6 shows an example of LLM-based predictive question generation.

[0052] Referring again to Figure 5, Server 150 may use LLM to generate an appropriate answer for each predicted question (S503). Server 150 may use documents provided by the advertiser, documents available on the web as documents related to the advertiser, documents available through databases or APIs related to the advertiser, etc., to generate a draft answer for each predicted question.

[0053] Server 150 may determine the appropriateness of the answers generated by the LLM (S504). Verification of the appropriateness of the answers to the predicted questions may be performed by the LLM, or, depending on the embodiment, verification may be performed by a human.

[0054] Server 150 may use the answers to each predicted question whose appropriateness has been verified to perform AI-based data learning (S505), and may generate an advertiser chatbot API for the conversational AI advertising service based on the AI ​​model learned from the answers to each predicted question (S506). At this time, Server 150 may store the answers to each predicted question whose appropriateness has been verified in the DB. In other words, the answers to each predicted question, after undergoing a verification process by LLM or humans, are used in DB answers or as LLM training data.

[0055] Therefore, this embodiment enables the automation of chatbot question and answer generation and verification using LLM.

[0056] Figure 7 shows an example of a question revision for providing answers in one embodiment of the present invention.

[0057] In this embodiment, a question reform function is supported to improve the quality of answers to user questions. For example, to provide appropriate answers to user questions on the "Nike" brand information page, reform questions may be used for purposes such as improving the intent of the question or providing multi-turn answers. As an example, server 150 may generate reform questions in response to user questions using LLM, utilizing the advertiser's campaign information. As another example, server 150 may generate reform questions in response to user questions using LLM, utilizing previous conversations. Using reform questions, either a DB answer or an LLM answer is provided.

[0058] Therefore, this embodiment makes it possible to accurately grasp the intent contained in a user's question by reforming it, provide a more appropriate answer, improve the quality of the answer, and provide an answer that is suitable for the user's intent.

[0059] As a solution for LLM-based hallucination control, this embodiment may include a DB-based response stage to provide the advertiser's intended answer, and an LLM-based response stage to provide answers to questions that the advertiser could not prepare in advance. Referring to Figure 4, the server 150 provides an answer to a user's input question after a DB search stage (S423), using a DB search which has relatively low infrastructure resource costs. However, if the user's question is not defined in the DB, a hybrid response method can be realized in which the server generates an answer using LLM (S442) and provides an LLM answer.

[0060] Figure 8 shows an example of determining whether or not a response is possible in one embodiment of the present invention.

[0061] In this embodiment, when providing LLM answers to questions that the advertiser could not prepare in advance, a function to determine whether an answer is acceptable may be added to preemptively eliminate questions unrelated to the advertiser or questions that violate safety standards, thereby performing harassment pre-control. For example, questions that cannot be answered may be identified by pre-setting specific themes or safety standards, or by classifying questions using a model that has learned data on questions that need to be avoided. In this case, the criteria for avoidance and the learning data may be input by the advertiser.

[0062] As a more concrete example, the answerability determination function may classify a user's question as either a "negative" question (meaning it cannot be answered) or an "affirmative" question (meaning it can be answered) using AI-based inference (for example, by using a trained answerability determination model). For user questions classified as "negative," the system may avoid answering or display a message indicating that the question cannot be answered (for example, a message directing the user to another question or a message stating that the question cannot be answered). For user questions classified as "affirmative," the system may display the LLM (Limited Liability Model) answer. Furthermore, it is possible to change the classification stage according to the level of answerability and provide appropriate processing or answers for each.

[0063] Figure 9 shows an example of a learning method for an answerability determination model in one embodiment of the present invention. As shown in Figure 9, the answerability determination model may be trained using data classified according to the degree to which an answer is possible, thereby classifying the level of answerability of a question in stages or determining whether an answer is possible or not.

[0064] Figure 10 shows an example of a response provision flow using hallucination control in one embodiment of the present invention.

[0065] Server 150 may use LLM to generate predicted questions and their answers based on information registered by advertisers, and store the question-and-answer pairs in the database.

[0066] Referring to Figure 10, the server 150 may first search the database for the answer to the question entered by the user (S901).

[0067] Server 150 may determine whether an answer corresponding to the user's question exists in the database by performing a database search (S902). Server 150 may search for a predicted question that matches the user's question from the predicted questions stored in the database and select the answer that is paired with this question as the database answer. For example, the embedding of the user's question may be compared with the embedding of the predicted question stored in the database, and a predicted question that matches the user's question may be found based on the similarity of the embeddings.

[0068] If a predicted question matching the user's question exists in the database, server 150 may display the database answer stored in the database as the answer corresponding to the user's question (S900).

[0069] If server 150 does not have a predicted question in the database that matches the user's question, it may determine that there is no database answer and first determine whether there is any document provided by the advertiser in order to provide an LLM answer (S903).

[0070] Server 150 may use LLM to generate answers to user questions using advertiser documents provided by the advertiser, such as during the campaign registration process (S904).

[0071] If no predicted question matching the user's question exists in the database, server 150 may generate an LLM answer based on the document provided by the advertiser and display it as the answer corresponding to the user's question (S900).

[0072] If no documents are available from the advertiser, Server 150 may search for documents matching the user's question in real time by browsing (web search), and then use LLM to generate an answer to the user's question using the web documents (S905).

[0073] If no predicted question matching the user's question exists in the database, server 150 may generate an LLM answer based on a web document that matches the user's question and display it as the answer corresponding to the user's question (S900).

[0074] New questions that do not exist in the database, and LLM answers generated based on advertiser documents or web documents as responses to them, may be added to the database and used for subsequent database answers.

[0075] As a result, this embodiment can gradually reduce the number of LLM calls by additionally storing LLM answers as DB answers.

[0076] Figure 11 shows an example of additional content mapping in one embodiment of the present invention.

[0077] When displaying an answer (DB answer or LLM answer) corresponding to a user's question, server 150 may map additional content to the user's question and display it.

[0078] Referring to Figure 11, the server 150 may use LLM to recommend entity 1002 in response to the user's question 1001 and map that entity 1002 to related content such as product information and rich media (images and videos). This allows for the real-time provision of visualized additional content along with text-based answers.

[0079] Figure 12 shows an example of a screen for an interactive AI information provision service in one embodiment of the present invention.

[0080] The interactive AI information service screen 1100 shows an example of a brand information page that includes a conversational interface with a brand chatbot that provides information data for a specific brand, such as "Nike."

[0081] Referring to Figure 12, when a user inputs a natural language-based question 1101 into the interactive AI information provision service screen 1100, the server 150 may display an answer 1110 corresponding to the user's question 1101. In this case, the answer 1110 may correspond to a DB answer retrieved from the DB, and in some cases, it may correspond to an LLM answer generated using an advertiser document or a web document.

[0082] Answer 1110 may consist of text-based content and may be displayed together with the user's question 1101 and related additional content 1120. The additional content 1120 may be content registered as an advertiser campaign for the relevant brand and may consist of, for example, content mapped based on entities for the user's question 1101.

[0083] When the server 150 provides an answer 1110 corresponding to the user's question 1101 through the interactive AI information provision service screen 1100, it may further provide the user with a suggested question (not shown) for the next conversation with the advertiser chatbot. In this case, the suggested question is a question generated by the LLM and may be generated based on at least one of the user's question 1101, the answer 1110, or the additional content 1120.

[0084] Thus, according to embodiments of the present invention, infrastructure resources can be minimized by providing a hybrid service that enables both DB-based and LLM-based answers when providing responses to user questions. Furthermore, according to embodiments of the present invention, hallucination problems can be solved by preemptively eliminating questions that violate themes or safety standards, and the quality of information can be improved by providing dynamic information by mapping not only text but also image and video-based rich content as answers to questions.

[0085] The above-described apparatus may be implemented by hardware components, software components, and / or combinations of hardware and software components. For example, the apparatus and components described in the embodiments may be implemented using one or more general-purpose or special-purpose computers, such as processors, controllers, ALUs (arithmetic logic units), digital signal processors, microcomputers, FPGAs (field programmable gate arrays), PLUs (programmable logic units), microprocessors, or various devices capable of executing and responding to instructions. The processing unit may execute an operating system (OS) and one or more software applications running on the OS. The processing unit may also respond to software execution, access data, record, manipulate, process, and generate data. For convenience of understanding, it may be described as if a single processing unit is used, but those skilled in the art will understand that the processing unit may include multiple processing elements and / or multiple types of processing elements. For example, the processing unit may include multiple processors or one processor and one controller. Other processing configurations, such as parallel processors, are also possible.

[0086] Software may include computer programs, code, instructions, or a combination of one or more of these, which may configure a processing unit to operate as desired, or which may instruct the processing unit independently or collectively. Software and / or data may be embodied in any kind of machine, component, physical device, computer recording medium, or device for interpretation based on the processing unit or for providing instructions or data to the processing unit. Software may be distributed across a networked computer system, and may be recorded or executed in a distributed manner. Software and data may be recorded on one or more computer-readable recording media.

[0087] The methods according to the embodiment may be implemented in the form of program instructions executable by various computer means and recorded on a computer-readable medium. In this case, the medium may continuously record computer-executable programs or may temporarily record them for execution or download. Furthermore, the medium may be various recording or storage means in the form of a combination of one or more hardware components, and may be a medium directly connected to a computer system or distributed on a network. Examples of mediums include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and devices configured to record program instructions such as ROM, RAM, and flash memory. Other examples of mediums include recording media and storage media managed by app stores that distribute applications, and sites and servers that supply and distribute various other software.

[0088] As described above, embodiments have been explained based on limited embodiments and drawings, but those skilled in the art will be able to make various modifications and variations from the above description. For example, the described technique may be performed in a different order than described, and / or the components of the described system, structure, apparatus, circuit, etc. may be combined or assembled in a different manner than described, or opposed or replaced by other components or equivalents, and still achieve suitable results.

[0089] Therefore, even if the embodiment is different, it falls within the scope of the attached claims if it is equivalent to the claims. [Explanation of symbols]

[0090] 310: Interactive Information Provision System 330: User 340: Provider

Claims

1. An interactive information provision method for a computer device including at least one processor, The process involves reforming the user's question using at least one of the aforementioned processors. The step of using at least one processor to search for the answer to the reformatted user's question from a DB in which questions and answers are stored. If the answer to the user's question exists in the DB, the at least one processor provides the answer stored in the DB, and An interactive information provision method comprising the step of displaying an answer generated by the LLM if the answer to the user's question is not found in the DB, using at least one processor.

2. The interactive information provision method according to claim 1, further comprising the step of using at least one processor to store the user's question and the answer generated by the LLM in the DB for use in subsequent answers.

3. The aforementioned renovation stage is, The interactive information provision method according to claim 1, further comprising the step of generating renovation questions in response to a user's question based on information registered by an information provider or a previous conversation with the user.

4. The interactive information provision method according to claim 1, further comprising the step of determining whether the user's question can be answered by the at least one processor.

5. The step of determining whether or not the user's question can be answered is: The interactive information provision method according to claim 4, further comprising the step of displaying a message indicating that the user's question cannot be answered if it is determined that the question cannot be answered.

6. An interactive information provision method for a computer device including at least one processor, The step of using at least one processor to search for the answer to the user's question from a DB in which questions and answers are stored. If the answer to the user's question exists in the DB, the at least one processor provides the answer stored in the DB, and The process includes, if the answer to the user's question is not found in the DB, displaying the answer generated by the LLM using at least one of the processors. The step of displaying the answers generated by the aforementioned LLM is: An interactive information delivery method, which includes the step of generating an LLM response using a document that matches the user's question.

7. The step of generating the LLM response is, The interactive information provision method according to claim 6, characterized in that it searches for a document that matches the user's question from documents provided by an information provider and generates the LLM answer.

8. The step of generating the LLM response is, The interactive information provision method according to claim 6, characterized in that it searches for documents that match the user's question by web browsing and generates the LLM answer.

9. The interactive information provision method according to claim 1, further comprising the step of displaying additional content related to the user's question, together with an answer to the user's question, using at least one processor.

10. The interactive information provision method according to claim 9, characterized in that the additional content consists of content mapped based on entities corresponding to the user's question.

11. A program for causing the computer device to execute the interactive information provision method described in any one of claims 1 to 10.

12. Includes at least one processor that executes instructions readable by a computer device, With the aforementioned at least one processor, Reform the user's question, From the database containing questions and answers, search for the answers to the questions of the user who has been remodeled. If an answer to the user's question exists in the DB, the answer stored in the DB is displayed. A computer device characterized by displaying an answer generated by LLM if the answer to the user's question does not exist in the DB.

13. With the aforementioned at least one processor, The computer device according to claim 12, characterized in that it stores the user's question and the answer generated by the LLM in the DB for use in subsequent answers.

14. With the aforementioned at least one processor, The computer device according to claim 12, characterized in that it rewrites the user's question based on information registered by the information provider or a previous conversation with the user.

15. With the aforementioned at least one processor, The computer device according to claim 12, characterized in that it determines whether or not the user's question can be answered.

16. Includes at least one processor that executes instructions readable by a computer device, With the aforementioned at least one processor, Search the database containing questions and answers for the user's question. If an answer to the user's question exists in the DB, the answer stored in the DB is displayed. If the answer to the user's question does not exist in the DB, the answer generated by LLM is displayed. A computer device characterized by generating an LLM response using a document that matches the user's question in order to display the response generated by the LLM.

17. With the aforementioned at least one processor, To display the answers generated by the aforementioned LLM, If a document provided by the information provider exists, the system searches for a document matching the user's question from the documents provided by the information provider and generates an LLM answer. The computer device according to claim 16, characterized in that, if no document provided by the information provider exists, it searches for a document matching the user's question by web browsing and generates the LLM answer.

18. With the aforementioned at least one processor, The computer device according to claim 12, characterized in that it displays additional content consisting of content mapped based on entities for the user's question, together with the answer to the user's question.

19. An interactive information provision system implemented by a computer device, A database containing pre-generated questions and answers, and The LLM learned from the above questions and answers, The computer device includes at least one processor, With the aforementioned at least one processor, Reform the user's question, From the aforementioned DB, search for answers to the user's questions regarding the renovations. If an answer to the user's question exists in the DB, the answer stored in the DB is displayed. An interactive information provision system characterized in that, if the answer to the user's question does not exist in the DB, it displays the answer generated by the LLM.

20. An interactive information provision system implemented by a computer device, A database containing pre-generated questions and answers, and The LLM learned from the above questions and answers, The computer device includes at least one processor, With the aforementioned at least one processor, From the aforementioned database, search for the answer to the user's question. If an answer to the user's question exists in the DB, the answer stored in the DB is displayed. If the answer to the user's question does not exist in the DB, the answer generated by the LLM is displayed. An interactive information provision system characterized by generating an LLM response using a document that matches the user's question in order to display the response generated by the LLM.

Images