Device and method for generative ai prompt generation in rag device and system

Abstract

To solve a problem with a conventional system in which the number of characters of irrelevant and harmful prompt information increases while appropriate answer contents and expressions cannot be narrowed down, causing an increase in a use charge and a deterioration in answer accuracy and response speed.SOLUTION: To increase the efficiency of managing knowledge and information, the knowledge and information are managed so as to be divided into a plurality of hierarchies and a plurality of data sets 22 in vector data storage means (2); and a similarity and relevant retrieval is performed between a question sentence and the data sets 22 and, in combining the knowledge and information, the combination is performed for only a data set prompt (5) associated therewith. In claim 2, the processing of determining how close mutually relevant information pieces are to be arranged is performed for optimization according to the capability of a generative AI of an output destination. A similarity calculation is preliminarily performed by "introducing a semantic label into a vector element" described in claim 3, which increases an answer accuracy even when a question sentence using a word that does not appear in the data sets 22 is received by input means (1).SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a prompt generation device and method for a generative AI based on RAG (Retrieval Augmented Generation), which stores knowledge and information as vector data to enable a generative AI such as ChatGPT, which has natural language dialogue capabilities, to respond based solely on specialized knowledge and local information, searches the vector data based on the similarity and relevance, and sends the found knowledge and information to the generative AI's API. [Background technology]

[0002] A method for generating prompts for a generative AI based on Retrieval Augmented Generation (RAG) is known, which stores knowledge and information as vector data to enable a generative AI such as ChatGPT, which has natural language dialogue capabilities, to respond based solely on specialized knowledge and local information, performs a search based on the similarity and relevance of the vector data, and sends the found knowledge and information to the API of the generative AI (Non-Patent Document 1).

[0003] Artificial intelligence (AI) technology has advanced rapidly in recent years, and generative AI, in particular, excels in its ability to automatically generate text and images. Generative AI is widely used in the field of natural language processing, where it is applied to tasks such as automatic text generation, translation, and summarization. However, general-purpose generative AI, which is pre-trained based on publicly available information, lacks the ability to respond to internal knowledge or local information found in questions such as, "What's the outside line number for our general affairs department?" This has limited its industrial applications. Prior to the advent of Retrieval Augmented Generation (RAG), response methods based on some mechanism for responding to inquiries in natural language text were publicly known, as seen in Patent Document 1, but they could only respond based on variations of predefined question and answer combinations, limiting their application.

[0004] To solve this problem, a method called RAG (Retrieval Augmented Generation) is described in Non-Patent Document 1. This method stores specialized knowledge and local information as vector data in advance, separate from the user's question, and then extracts only a portion of this data that is closely related to the question, sending several hundred to tens of thousands of characters to a generation AI. This has had some success in allowing the AI ​​to respond solely from the stored specialized knowledge and local information. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent Publication No. 2003-243086 [Title of the Invention] Reply sentence generation device, method, and program [Non-patent literature]

[0006] [Non-Patent Document 1] Patrick Lewis, Ethan Perez et al. ``Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks'' Summary of the Invention [Problem to be solved by the invention]

[0007] RAG devices constructed using known methods such as those described above have had issues with reduced accuracy, such as failing to extract data from vector data that is correctly related to the question and contains the correct answer, or extracting too much character data, which results in a large amount of irrelevant noise being sent to the generation AI, preventing the generation AI from providing the correct answer. Furthermore, when dealing with a wide variety of questions, such as those within a company where multiple prompts should be prepared to indicate different response styles for the general affairs, legal, accounting, and development departments, conventional RAG devices and methods are unable to appropriately narrow down the answer content and wording, resulting in an increase in the number of irrelevant and harmful prompt characters, which unnecessarily increases usage fees and reduces the generation AI's answer accuracy and response speed.

[0008] Therefore, the present invention solves the above problems by providing a prompt generation device and method that divides a vector database by genre of in-house knowledge and local information, divides prompts into their associated prompts (these divided prompts are called dataset prompts), and appropriately combines and constructs the data to be ultimately sent to the generation AI from the search results.The objectives of the device and method of the present invention are to quickly and accurately create data to be sent to the generation AI, reduce user costs, improve answer accuracy, and increase response speed. [Means for solving the problem]

[0009] In order to improve the efficiency of knowledge and information management, the present invention provides a mechanism for managing knowledge and information by dividing it into multiple hierarchies and multiple datasets 22 in a vector data storage means (2). At the top of the multiple hierarchies are global prompts 21, which are combined each time a question is combined with specialized knowledge and a portion of local information by the generation module (3) and sent to the output means (4), and provide instructions to the generation AI on how to respond. At the middle of the hierarchy are dataset prompts (5). At the bottom of the hierarchy are datasets 22, which contain actual knowledge and information (e.g., descriptions of specific parts of an internal company manual). Using cosine similarity between vectors, etc., a similarity / related search is performed between the question and the dataset. When the generation module (3) combines the top-ranked knowledge and information and sends it to the output means (4), only the dataset prompts (5) associated with the dataset 22 containing the relevant knowledge and information are subject to combination. [Effects of the Invention]

[0010] This eliminates dataset prompts (5) that are unrelated to the question, and the generation module (3) generates a generated result with a small amount of text. By increasing the knowledge and information that is highly relevant to the correct answer, the generation AI that receives the generated result from the output means (4) can generate a more appropriate answer.

[0011] Since the probability that correct knowledge and information and the contents of the dataset prompt 21 for correctly and appropriately interpreting it are included in a small amount of characters is increased, data to be sent to the generation AI can be created quickly and accurately, reducing user costs, improving answer accuracy, and increasing response speed. Furthermore, by having the generation module (3) perform the process described in claim 2, particularly the process of determining how closely related information should be placed, these effects can be optimized according to the capabilities of the output generation AI. Furthermore, by having the generation module (3) perform the process described in claim 3, in which the vector data storage means (2) "introduces semantic labels into vector elements" in advance and calculates similarity, even when the input means (1) receives a question sentence using words that do not appear in the dataset 22, the probability of assigning a high similarity to the correct answer increases, thereby improving answer accuracy. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is an explanatory diagram showing the system configuration and data processing flow of the present invention. DETAILED DESCRIPTION OF THE INVENTION [Example]

[0013] In Figure 1, which shows the system configuration and data processing flow of the present invention, in the learning phase, specialized knowledge and local information are received by the input means (1), and the vector data storage means (2) converts the specialized knowledge and local information into vector data with words as elements and stores it in the dataset 22 specified by the input means (1). In the operation phase, when the input means (1) receives a question, the question is converted into a vector with words as elements, and the vector data in the dataset 22 inside the vector data storage means (2) is appropriately combined with the top several vectors with high cosine similarity, and sent to the output means (4). Normally, instructions to the generation AI contained in the overall prompt 21 are also combined before being sent. This is the same operation as that of the known RAG device and method.

[0014] In claim 1, the contents of dataset prompts (5) associated with the top vector data with the highest cosine similarity to the question are also combined with labels such as "prerequisite knowledge" and sent to the generation AI via output means (4). In practice, datasets divided into fields such as accounting and legal affairs often contain domain-specific terminology definitions and local information supplements stored in dataset prompts (5). As a result, in known RAG devices and methods, terminology definitions and explanations that should only be sent to the generation AI for specific fields are stored in the overall prompt 21 and sent every time. This can lead to hallucination, which can lead to incorrect answers, or the sending of unnecessary text can slow the generation AI's response speed and increase fees. The processing of claim 1 eliminates this waste, improving the accuracy and speed of the generation AI and reducing fees. [Example]

[0015] In claim 2, in addition to claim 1, the generation module (3) analyzes input data, including specialized knowledge, local information, and questions, and performs a process to extract related information. Based on the extracted information, the generation module (3) places related information in close proximity when generating a string to be sent to the generation AI. Furthermore, the generation module (3) evaluates the appropriateness of the generated prompt, such as whether the prompt contains a context that could lead to misreading or misunderstanding, and performs corrections to eliminate misreadings as necessary. Finally, the generation module (3) determines how closely related information should be placed depending on the ability of the generation AI to associate information within a wide range of contexts (for example, the ability to associate information within a wide range of contexts increases as the number of parameters increases, such as when replacing GPT3.5 with GPT4). This allows a string optimized for the capabilities of the generation AI being used to be sent to the output means (4). [Example]

[0016] In claim 3, in addition to claim 1, the generation module (3) analyzes input data, including specialized knowledge, local information, and a question, and extracts not only words but also pre-trained semantic labels for each word, which are then added to the words and added to vector elements. When the generation module (3) determines the similarity and relevance of each knowledge record to the question using cosine values, etc., it determines the similarity and relevance of each knowledge record to the question using not only the cosine similarity of vectors consisting of word elements, but also the cosine similarity of vectors consisting of semantic label elements with a weight set by the user. As a result, even if a word in the question is not included in the dataset 22, if it has a synonym with the same semantic label, the generation module (3) appropriately extracts specialized knowledge and local information related to the question, combines them, and sends them to the generation AI via the output means (4), thereby increasing the probability of obtaining a highly accurate answer. [Industrial Applicability]

[0017] The prompt generation device and method for generative AI in the RAG device / method can be used in various industrial fields and business settings requiring so-called knowledge management. For example, it can be widely used for in-house knowledge support and customer knowledge support for products and services. Furthermore, it can be used to create services that respond to questions using the knowledge of store clerks, professionals, and experts in the metaverse, which has been developing in recent years, such as AI pharmacists. Expert knowledge provision services can also be used to train a large amount of knowledge over a certain period of time, and can also be used to evaluate and score the results. Therefore, it is expected to be used in a wide range of fields, including general education and self-study support. [Explanation of symbols]

[0018] 21 General prompt 22 datasets

Claims

1. A prompt generation device and method for a generative AI in a RAG device / system, comprising: an input means (1) for receiving input data; a vector data storage means (2) for storing specialized knowledge and local information, such as in-house manuals, in units of knowledge fragments as record units; a generation module (3) for creating prompts for the generative AI based on the input data; and an output means (4) for outputting prompts for the generative AI, wherein the vector data storage means (2) is divided into a plurality of datasets, and each dataset is accompanied by a dataset prompt (5) which is sent to the generative AI together with a label such as "prerequisite knowledge" when the knowledge record within the dataset shows a high similarity to the input data based on cosine values ​​or the like.

2. 2. A prompt generation device and method for a generation AI according to claim 1, characterized in that the generation module (3) performs the following operations: - Processing to analyze input data and extract related information - Process to automatically create prompts for the generation AI based on the extracted information - The process of evaluating the appropriateness of the prompts created and revising them as necessary - A process that determines how closely related information should be placed depending on the ability of the generating AI used to associate information within a broader context.

3. 2. A prompt generation device and method for a generative AI according to claim 1, characterized in that the generation module (3) creates prompts using the following techniques: When determining the similarity and relevance of each knowledge record to a question sentence using cosine values, etc., a process of introducing pre-trained semantic labels into vector elements, and determining the similarity and relevance of each knowledge record to a question sentence using not only the cosine similarity of vectors consisting of word elements but also the cosine similarity of vectors consisting of semantic label elements with a weight set by the user.

Images