Learning support system using retrieval-augmented generation

Abstract

[Problem] To increase the recording capacity of additional information by constructing a plurality of RAG databases for each related field according to the necessity of learning support, and to improve efficiency by narrowing down the objects of RAG retrieval. To eliminate omission of reference by referring to a text or an image of the whole original document page of a portion where relevant content is described, on the basis of a chunk group that is obtained by RAG retrieval, and at the same time, to reflect background information and relevant information that may be described in the periphery of a retrieved chunk in the context of a query. To provide a learning support system using retrieval-augmented generation for presenting image data that is difficult to convert into text to a user and enabling additional input to a query. [Solution] The invention comprises a test problem providing means, a page image acquiring means, a page text database recording means, a feature amount vector database recording means, a relevant feature amount vector extracting means, a duplicate reference address removing means, and a relevant text transcribing means.

Description

Learning support system using search extension generation

[0001] This invention relates to learning support using search extension generation, particularly to understanding test questions using multiple-choice options.

[0002] In education, it is common practice to administer exams after lectures to assess students' understanding of the material. Students with low scores are required to receive additional learning to address their weaknesses, while high-scoring students are expected to receive additional materials to deepen their understanding. This is where the application of machine learning is anticipated. In recent years, machine learning has advanced remarkably, with the widespread adoption of large language models (LLMs) in particular. Neural networks with tens of billions to trillions of parameters are trained on terabytes of data, enabling them to handle tasks such as translation, speech and image recognition, and text summarization. Furthermore, generative AI that generates images, music, and documents based on instructions (prompts) is also becoming increasingly practical. It has become known that scaling up models improves performance (scaling laws), and the scaling up of models is progressing rapidly.

[0003] As the applications of large-scale language models expand, knowledge data from various fields becomes necessary, and the incorporation of the latest knowledge data is required at all times, in line with social conditions and technological advancements in each field. However, training on large amounts of data requires massive computing resources, enormous electricity, and expense. For this reason, large-scale language models cannot be rebuilt frequently. A phenomenon called hallucination is known to occur when a large-scale language model is questioned and generates a response that is not based on facts, even if the knowledge necessary to answer the question is not stored internally. This is one of the reasons why the widespread use of large-scale language models is limited in fields such as medicine, where errors can directly lead to accidents.

[0004] In the practical application of large language models based on the latest information, additional acquisition of the latest information is required. Currently, two types of approaches are being taken. One is what is called additional learning (fine-tuning) or transfer learning, in which learning is performed on a part of the output layer (fine-tuning) or only the final layer (transfer learning) of an existing large language model using additional information to create a specialized large language model. Since it becomes a large language model specialized for a field, its usefulness is high. However, although it is not as much as training a large language model from scratch, costs and technical skills are still required for training.

[0005] The other approach is the Retrieval-Augmented Generation (RAG) approach used in the present invention. The model of the large language model itself is not changed. Additional information is separately stored in a database (RAG database), information necessary for solving a question sentence is retrieved from the RAG database, and the obtained information is added to the question sentence to obtain an answer sentence from the large language model. Since the additional information required for answering a certain question sentence is limited even if the amount of potentially additional data is large, only the limited additional information is retrieved from the RAG database, and the extracted additional information (context) is appended to the question sentence in the question input field of the large language model. In this approach, since no changes are made to the large language model itself through learning, only context information is added to the question input field, so costs and technical hurdles are low. Furthermore, it has excellent immediacy, such as being able to immediately reflect highly urgent information such as the emergency side effects of drugs.

[0006] Here, the RAG database is created by breaking down the additional document information into small fragments (chunks), and then vectorizing each chunk into feature vectors (chunk vectors) to create a database. When a question is asked, the question itself is also vectorized, and chunk vectors with a high similarity to the vector of the question are searched from the RAG database. The contents of the obtained chunk group are then added to the question. This makes it possible to handle specific fields and the latest information without requiring additional training of a costly large-scale language model, and by limiting the basis for the answer to the question to the RAG database, it is possible to prevent hallucinations. The following are prior art documents related to this application.

[0007] https: / / ja.wikipedia.org / wiki / %E5%A4%A7%E8%A6%8F%E6%A8%A1%E8%A8%80%E8%AA%9E%E3%83%A2%E3%83%87%E3%83%ABhttps: / / www.idnet.co.jp / column / page_308.html

[0008] Understanding learning items requires collecting information scattered across numerous books and documents. Traditionally, instructors manually collected information for each learning item and then edited it into a format easily understandable to students. However, the number of learning items is enormous, and it is difficult to say that sufficient information is provided for all of them. To solve this problem, the use of large-scale language models using search-enhanced generation is expected. However, when actually building a RAG database for learning support and attempting to search for information related to learning items, there are many unresolved problems, such as: (1) the RAG database itself becomes enormous, resulting in a relatively large amount of data unrelated to the question; (2) the content to be searched is not always appropriately contained within chunks, leading to missed searches of data related to the question; (3) background information and related information that may be present around the searched chunks are omitted; and (4) current large-scale language models are still insufficient in image recognition and searching. Furthermore, repeatedly performing RAG searches for similar questions wastes resources such as cost and time. Furthermore, many exams present multiple options and ask candidates to select the appropriate one. Explaining all the options together often involves mixing different pieces of information, which not only increases the length of the explanation but also makes focused analysis difficult.

[0009] The present invention was made to solve the aforementioned problems of the past, and its purpose is to provide a learning support system using search-enhanced generation, which involves constructing multiple RAG databases for each related field as needed for learning support to increase the capacity for recording additional information, improving efficiency by narrowing the target of RAG searches, eliminating missed references by referring to the text or image of the entire original document page containing related content based on the chunks obtained from RAG searches, and at the same time reflecting background information and related information that may be written around the searched chunks into the context of the question, presenting image data that is difficult to convert into text to the user and enabling additional information to be added to the question, enabling focused explanations by breaking down questions with multiple choices into individual choices and generating answers, and providing a learning support system using search-enhanced generation, which eliminates the need for costly and time-consuming re-response generation by creating a database of question and answer records.

[0010] As a means to achieve the above objective, the learning support system using search extension generation as described in claim 1 includes, in a large-scale language model, a test question provision means for recording test questions and providing them as question text, (1) a page image acquisition means for acquiring image images of individual pages of information sources to be referenced separately from the large-scale language model during inference, along with reference addresses to those image images, (2) a page text database recording means for extracting string text from the image images of the individual pages and recording it along with the reference addresses, (3) a feature vector database recording means for dividing the extracted text into small sections (chunks), calculating feature vectors, and recording them along with the reference addresses, (4) a related feature vector extraction means for extracting groups of feature vectors that are highly related to the test question from the feature vectors of the test question given to the learning support system using search extension generation, along with the reference addresses, and (5) a duplicate reference address removal means for removing duplicate reference address groups from the extracted related feature vectors. (6) The system is characterized by having related text transcription means that extracts the page text specified by the reference address after removing the duplicate reference address group from the page text database recording means and then transcribes it into the question input field together with the question text, thereby obtaining the answer text to the test question by performing the operations (1) to (6) above.

[0011] The learning support system using search extension generation according to claim 2 is characterized in that, in the learning support system using search extension generation according to claim 1, it is further characterized by comprising a page image database recording means that records each acquired page image together with a reference address to the page image.

[0012] The learning support system using the search extension generation described in claim 3 is characterized in that, in the learning support system using the search extension generation described in claim 1 or 2, it is provided with page image viewing means for displaying and making available for viewing the page image specified by the reference address.

[0013] The learning support system using search extension generation described in claim 4 is characterized in that, in the learning support system using search extension generation described in claim 2, after extracting a page image specified by a reference address from a page image database recording means, the user inputs comments for charts that have insufficient text conversion, and the comments are then transferred to the page text based on those inputs.

[0014] The learning support system using search extension generation according to claim 5 is characterized in that, in the learning support system using search extension generation according to claim 2, it comprises a plurality of page image acquisition means, a page text database recording means for the page image database recording means, and a plurality of feature vector database recording means for the page text database recording means, and a plurality of RAG database search means for searching and extracting related feature vectors for any of the feature vector database recording means.

[0015] The learning support system using search-extended generation according to claim 6 is characterized in that, in the learning support system using search-extended generation according to claim 1 or 2, it comprises a question-answer recording means for recording the question sentence and the obtained answer sentence, and for a new question sentence, the question-answer recording means first searches for a question sentence similar to the new question sentence, and if a similar question sentence is found, it comprises an F&Q database in which the answer record for the similar question sentence is used as the answer to the question sentence.

[0016] The learning support system using search extension generation according to claim 7 is characterized in that, in the learning support system using search extension generation according to claim 1 or 2, the test question providing means is equipped with a question-and-choice pair decomposition means that, in addition to a function of recording test questions and providing them as question texts, decomposes the questions into pairs of question texts and individual choices and provides them as question texts.

[0017] The learning support system using search-extended generation described in claim 1 includes a means for providing exam questions, so that exam questions are recorded and provided as question texts. It also includes a means for acquiring page image data, so that image data of individual pages of information sources to be referenced separately from the large-scale language model during inference is acquired along with the reference addresses to those image data. It also includes a page text database recording means, so that text strings are extracted from the image data of individual pages and recorded along with the reference addresses. It also includes a feature vector database recording means, so that the extracted text is divided into small segments (chunks), feature vectors are calculated, and recorded along with the reference addresses. It also includes a related feature vector extraction means, so that for the feature vectors of the exam questions given to the learning support system using search-extended generation, groups of feature vectors that are highly related to the exam questions are extracted along with their reference addresses. It also includes a duplicate reference address removal means, so that duplicate reference address groups are removed from the extracted related feature vectors. Finally, it includes a related text transcription means, so that the page text specified by the reference addresses after the duplicate reference address groups have been removed is extracted from the page text database recording means and then transcribed into the question input field along with the question text.

[0018] The learning support system using the search extension generation described in claim 2 includes a page image database recording means, so that each acquired page image is recorded along with a reference address to the page image.

[0019] The learning support system using the search extension generation described in claim 3 includes a means for viewing page images, so that the page image specified by the reference address can be displayed and viewed.

[0020] The learning support system using the search extension generation described in claim 4 includes a comment transcription means, which extracts a page image specified by a reference address from the image data recording means, inputs comments for figures and tables that have not been adequately converted to text, and transcribes them into the page text.

[0021] The learning support system using search extension generation described in claim 5 includes a plurality of page image acquisition means, a page text database recording means for the page image database recording means, and a plurality of feature vector database recording means for the page text database recording means. It also includes a plurality of RAG database search means for searching and extracting relevant feature vectors for any of the feature vector database recording means.

[0022] The learning support system using the search and extended generation described in claim 6 includes a question and answer recording means for recording a question and the obtained answer. Furthermore, for new question sentences, the question and answer recording means first searches for similar question sentences, and if similar question sentences are found, it includes an F&Q database that uses the answer records for those similar question sentences as answers to the question sentence.

[0023] In the learning support system using search extension generation described in claim 7, the test question providing means includes a function to record test questions and provide them as question texts, as well as a question text-option pair decomposition means that decomposes the questions into pairs of question texts and individual options and provides them as question texts.

[0024] This is an embodiment of the network configuration of the present invention. This is an example of a user interface in a Large-Scale Language Model (LLM). This shows the relationship between the Large-Scale Language Model and the RAG database. This shows the overall configuration of the extended RAG database of the present invention. This is a detailed diagram of the processes that constitute the extended RAG database. This shows the method for assigning reference addresses. This is the search procedure for the RAG database. This is an example of an exam question. This is an example of an exam question broken down into a combination of question text and answer choices. Multiple RAG databases are constructed for each domain. This is an explanatory diagram of the F&Q database.

[0025] The learning support system using search extension generation according to the present invention comprises a server device, a database, and a terminal. The server device is a known computer device and comprises an arithmetic unit, main memory, auxiliary storage device, input device, output device, and communication device. The arithmetic unit, main memory, auxiliary storage device, input device, output device, and communication device are connected to each other via a bus interface. The arithmetic unit comprises a known processor capable of executing an instruction set. The main memory comprises volatile memory such as RAM capable of temporarily storing the instruction set. The auxiliary storage device comprises non-volatile data storage capable of recording the OS and programs. The data storage may be an HDD or an SSD, for example. The input device may be a keyboard or mouse, for example. The output device may be a display such as an LCD, for example. The communication device comprises a network interface capable of connecting to a network. The server device includes means such as test question provision means, page image acquisition means, page text database recording means, feature vector database recording means, related feature vector extraction means, duplicate reference address removal means, related text transcription means, page image database recording means, page image viewing means, comment transcription means, multiple RAG database search means, question statement choice pair decomposition means, and an F&Q database. The processor of the server device exerts the effects of these means. The database according to the present invention may be configured in the auxiliary storage device of the server device, or in a separate auxiliary storage device independent of the server device. The database stores information handled by this system. The terminal according to the present invention has the same hardware configuration as a known computer as the server device. The server device, database, and terminal according to the present invention are communicable via a network.

[0026] Figure 1 shows a typical system configuration of the present invention. Large-scale language models consist of massive amounts of data, numerous CPUs (Central Processing Units), GPUs (Graphics Processing Units), and a high-speed network connecting them. Therefore, they are built on massive servers, such as cloud data centers, and provided via the Web. Educational institutions such as schools have numerous PC terminals connected via a Web-connected LAN (Local Area Network). Within educational institutions, there are also servers operating school administration databases that manage lecture and exam content, student grades, etc. In recent years, there has been an increase in instances of accessing cloud services such as large-scale language models and school administration database servers from both inside and outside educational institutions using mobile devices such as smartphones and tablets.

[0027] Servers, terminals, and mobile terminals all consist of memory for recording programs and data, recording media such as hard disks for persistently storing the programs and data as needed, a CPU for reading and processing the programs and data, a GPU for high-speed parallel processing as appropriate, and a communication module. As cloud services become more stable and inexpensive, there is an increasing trend to migrate some or all of the functions of school administration database servers to the cloud. Conversely, there is also a movement to move some or all of large-scale language models to terminals with increased processing power and memory capacity (edge ​​computing). Furthermore, the development of small language models (SLMs) with a reduced number of parameters is also progressing. It should be noted that even small-scale models are still sufficiently large compared to those before the emergence of large-scale language models, and all embodiments, including this form, are included in the present invention.

[0028] Figure 2 shows an example of a user interface for a Large-Scale Language Model (LLM). LLM is currently under rapid development, with numerous models being developed, including ChatGPT (a registered trademark of OpenAI), Bard, LaMDA (a registered trademark of Google), and LLaMA (a registered trademark of Meta). While the user interfaces naturally differ, the standard configuration, as shown in Figure 2, consists of a prompt input field for entering instructions and inquiries to the LLM, a field for displaying the response to that prompt (response display field), and a field for displaying the history of prompts and responses as a usage log (usage history field).

[0029] Recently, in addition to using LLM as a standalone application as described above, there has been an increase in cases where LLM itself is equipped with an API (Application Programming Interface), allowing external software to utilize LLM's functions. In this case, prompts, responses, and history are input and output between the external software and the LLM via the API, so the display format is controlled by the external software and is not limited to Figure 2.

[0030] In large-scale language models, a one-hot vector is used to represent a vocabulary word. This vector consists of zeros with the same number of dimensions as the number of vocabulary words used, and a single 1 is placed at the position corresponding to that vocabulary word. The vocabulary of a large amount of literature is replaced with vectors of this type, and the relationships (Attention) between each vocabulary vector are determined using deep learning. For a query (prompt), the model generates and adds vocabulary words one by one that are highly likely to follow the query and the already generated partial answer sentence to create the answer sentence. If the information used to generate the answer is contained in the large amount of literature, it is expected that a rational, useful, and correct sentence will be generated. However, if the information used to generate the answer sentence is not contained in the large amount of literature, the model mechanically selects the vocabulary words with high probability and proceeds with generating the answer sentence, which is known to result in the generation of false answer sentences that are not based on evidence (hallucination). When these hallucinations occur in settings such as healthcare and education, they can endanger patients' lives or cause students to believe in false information, which is one of the reasons why the application of large-scale language models to core business operations has not progressed.

[0031] Training large-scale language models requires massive servers, including a large number of parallel processing units (GPUs), as well as significant electricity resources and costs, to process vast amounts of documents and use deep learning to determine relationships between vocabulary words. While new documents are created daily, it is not realistic to reflect all of them in large-scale language models without any delay. Furthermore, the vast amount of documents processed mainly consists of publicly available documents on the web, but they do not include sensitive information such as internal company documents or electronic medical records from hospitals. Therefore, it is said that the documents that can be collected represent only a small fraction of all documents that exist on Earth.

[0032] To effectively utilize large-scale language models based on the latest information in the field, additional up-to-date information is required, and currently two approaches are being taken. One is called additional learning (fine-tuning) or transfer learning, where additional information is used to train only a part of the output layer (fine-tuning) or only the final layer (transfer learning) of an existing large-scale language model, creating a specialized large-scale language model. While this results in a domain-specific large-scale language model with high utility, it still requires considerable expense and technical skills, even if not as extensive as training a large-scale language model from scratch. Furthermore, sensitive information such as personal information and descriptions of medical conditions included in the additional information is used for training, posing a risk of it being accessed outside the organization. To prevent this, it is necessary to exclusively build and operate the additionally trained large-scale language model within the company or hospital.

[0033] Another approach is the Search-Augmented Generator (RAG) approach used in this invention. The model of the large-scale language model itself is not modified. Additional information is stored in a separate database (RAG database), and the information necessary to solve the question is retrieved from the RAG database. The text information of the obtained chunks is added to the question, and the large-scale language model is asked to produce an answer. Even if there is a large amount of data that could potentially be added, the additional information necessary to answer a given question is limited, so only that limited additional information is retrieved from the RAG database, and the extracted additional information is added to the question (context) in the question input field of the large-scale language model. In this approach, no changes are made to the model of the large-scale language model itself through learning, and only context information is added to the question input field, so the cost and technical hurdles are low. Furthermore, it is highly responsive, as it can immediately reflect urgent information such as urgent drug side effect information.

[0034] Figure 3 shows the flow of large-scale language model processing using a general search-enhanced generation (RAG) method. A broker program, such as a chat application, receives a question from the user (1) and searches the RAG database for information related to the question (2). The broker program then questions the LLM (Language Language Model) with the search results (3) along with the question (4) and receives an answer from the LLM (5). The received answer is then presented to the user (6). However, although the length of the chunk text string is variable in the settings, there are certain limitations, and it may not be possible to include all the necessary information. Conversely, if the chunk is too long, the focus of the feature vector may become blurred, and there is a risk that it will not be able to properly handle the search. Also, if a keyword is applied to the boundary between chunks, proper feature vectorization cannot be performed, making searching difficult. Furthermore, scanned images such as images and illustrations are not yet sufficiently recognized and converted into text, and there is a possibility that they will not be found in searches. Moreover, building a RAG database of a level that can withstand practical use requires a large-scale system based on a large number of document chunks. The required document set varies greatly depending on the field of interest. For example, the set of documents required in the medical field differs significantly from that required in history or literature. Building a RAG database from a set of documents covering all fields would result in a redundant configuration, potentially placing an unnecessary burden on database construction and retrieval.

[0035] Figure 4 shows the overall configuration of the extended RAG database of the present invention. Figure 5 is a detailed diagram of the processing flow in Figure 4. Scanned images of books, image images of PDF-converted web documents, etc., from additional potential information sources (information resources) such as books, ebooks, and PDF versions of web documents are recorded in the database, with each individual page image (page image) assigned a reference address such as "information resource name + page" (page image database). Subsequently, text is extracted from each page image (page text), and the page text is recorded along with the reference address (page text database). The text of each page is cut into small fragments (chunks), a feature vector is calculated for each chunk, and the reference address is assigned to record it in the feature vector database. Note that, if necessary, such as when there are strict constraints on recording capacity, the page text may be compressed and recorded using the summarization function of a large-scale language model.

[0036] Here, the reference address is formatted as a book title followed by page numbering, as is common in books, as shown in Figure 6. However, unlike books which have physical constraints, web documents do not require such strict display formatting. Therefore, depending on the granularity of the information, numbering can be done at the paragraph level, or conversely, at the section or chapter level. Page text can be unstructured plain text, but for complex content, tagged notation such as XML, JSON, or Markdown is preferable to clearly indicate the document structure. Different feature vector calculation software recommends different notations. This invention uses JSON notation, but any notation may be used.

[0037] Currently, it is not possible to perfectly represent images, videos, illustrations, etc., included in page images with text. Therefore, human judgment and understanding are ultimately required. It is also useful for users to judge the aforementioned images, videos, illustrations, etc., and add explanatory text to the corresponding page text as needed. Thus, while a page image database after extracting page text is not essential, it can be useful, and the relevant page images should be referenced as needed.

[0038] Figure 7 shows the search procedure for the RAG database. When a question is entered into the prompt input box, the broker program calculates the feature vector of the question and extracts a list of approximate feature vectors from the vector database (RAG database). The degree of similarity between feature vectors is often expressed by the magnitude of the dot product between the vectors (cosine similarity), but the Manhattan distance, which is the sum of the absolute values ​​of the differences between each element of both vectors, may also be used. The magnitude of the threshold for the degree of similarity used as the basis for extraction is variable depending on the situation. If the extracted list is too large, the threshold can be raised to narrow it down, and if the list is too small, the threshold can be lowered to increase the size of the list. In some cases, the threshold can be set to the top 10, for example, and the number of extracted items can be automatically adjusted to match the set value.

[0039] In conventional RAG systems, when a list of feature vectors corresponding to multiple chunks was extracted, the text of the chunk from which each feature vector originated was added to the question text. However, as mentioned above, there are limitations to the length of the chunk text strings, and sometimes the necessary information could not be included. Also, if a keyword was applied to the transition between chunks, proper feature vectorization was not possible. Furthermore, scanned images such as images and illustrations sometimes could not be adequately converted into text. To overcome these shortcomings, the present invention employs the following procedure.

[0040] The system organizes the reference addresses assigned to each feature vector, removes duplicate reference addresses, and transcribes the entire page text indicated by the remaining reference addresses into the prompt input box along with the question. The large-scale language model is then prompted to respond based on this. This procedure resolves the problem of gaps between chunks, where each chunk cannot contain all the necessary information, and also allows the text between chunks to be added to the question. This feature is useful in education for students and others, as it promotes deeper understanding by showing not only the solution to a specific problem but also the background information behind it.

[0041] If images, videos, illustrations, etc., are not adequately transcribed into text, the system searches for the image of the page in question in the page image database and displays it to the user. The user then adds a description of the image, video, or illustration to the question. If necessary, this description may also be added to the page text. This allows for the utilization of information that was previously underutilized, such as images, videos, and illustrations. As described above, by following the procedure shown in Figure 7, when a learning item is questioned, information is collected from a wide range of information resources, including numerous books and documents, provided to the LLM (Learning Language Master), and organized and summarized. This makes it easier to create explanatory texts, which previously relied on time-consuming manual work.

[0042] Figure 8 shows an example of a typical test question. While it would be possible to simply present the question in this form and request the creation of an answer and explanation, the presence of numerous independent items tends to result in redundant explanations. Furthermore, since there are countless combinations of answer choices, there would be a lot of duplication in the explanations for each question. To avoid this, as shown in Figure 9, by decomposing the question into pairs of the question and individual answer choices, and then querying each individual choice, a focused and concise explanation can be expected (question-choice pair decomposition means). With this configuration, it is easy to search for answer choices similar to the given choice, and it is easy to create review questions to check whether the understanding of an answer choice that was previously answered incorrectly has been sufficiently obtained.

[0043] When attempting to build a RAG database that can withstand practical use, it will become large-scale based on a large number of document chunks. For example, in the medical field, even in large fields such as internal medicine, surgery, and obstetrics and gynecology, there are easily over 50, and furthermore, there is a tendency towards further subdivision as medicine develops. If all of them are aggregated into one RAG database, it will cause a huge load on data storage and retrieval. Here, the necessary document groups vary greatly depending on the field of interest. For example, even within the medical field, the document groups of abdominal surgery and psychiatry, for example, have many non-overlapping parts. Furthermore, the document groups required for medicine are very different from those in the fields of history and literature. Therefore, as shown in FIG. 10, by constructing a plurality of RAG databases for each field of interest and having a broker program search for 1 to multiple RAG databases related to the question sentence as needed, it is possible to avoid the inefficiency of retrieval due to the enlargement of the RAG database. Of course, if the RAG database is overly subdivided, it will be necessary to execute a large number of searches and the retrieval efficiency will deteriorate. Needless to say, it is necessary to aggregate documents in highly related fields to build a RAG database.

[0044] In the educational field, when generating an explanatory text for the answer to a test question, for students who answered a certain question incorrectly and whose grades are not good, an explanatory text based on a general RAG database is appropriate. However, in this case, there is a risk that students with good grades will not have their intellectual curiosity satisfied. In this case, it is also useful to build a RAG database that contains more advanced and principle-based content and generate and provide an explanatory text with deep and advanced content from it.

[0045] While questions and their corresponding answers can be created each time, this incurs considerable computational resources and costs. As shown in Figure 11, pairs of questions and their corresponding answers are stored in a RAG database as an FAQ database. When a question is entered, the system first searches the FAQ database for answers closely related to that question. Only if the content of the found answers is unsatisfactory, a new answer is created using the procedure of the present invention, and the result is registered in the FAQ database. In this case, breaking down the problem into multiple-choice options as shown in Figure 9, rather than the format in Figure 8, is more likely to yield a more versatile and non-redundant explanation. When registering in the FAQ database, either registering only the feature vector of the question in the RAG database and linking it to the answer, or chunking the question and answer pairs and registering the feature vectors, is acceptable.

[0046] Furthermore, since there is a risk of information leakage if the content entered in the prompt is used to train a large-scale language model, it is useful to explicitly declare in the prompt that training is prohibited, or to use a paid version of a large-scale language model that guarantees that the content will not be used for training.

[0047] Although embodiments have been described above, the specific configuration of the present invention is not limited to the embodiments described above, and design changes and the like that do not depart from the gist of the invention are also included in the present invention. For example, although the question text is provided from the test question provision means, the test questions or learning items may also be directly entered into the prompt input field. In the present invention, the feature vector database of chunks has been mainly described, but if the information is useful for generation, it is also included in the present invention if information extracted from a normal relational database using SQL statements, reference information from the web, etc. is added to the prompt as appropriate, in addition to the feature vector database.

Claims

1. A large-scale language model is provided with a means for providing test questions, which records test questions and provides them as question texts, and includes: (1) a page image acquisition means for acquiring image images of individual pages of information sources that should be referenced separately from the large-scale language model during inference, along with reference addresses to those image images; (2) a page text database recording means for extracting string text from the image images of the individual pages and recording it along with the reference addresses; (3) a feature vector database recording means for dividing the extracted text into small segments (chunks), calculating feature vectors, and recording them along with the reference addresses; (4) a related feature vector extraction means for extracting groups of feature vectors that are highly related to the test question from the feature vectors of the test question given to a learning support system using search extension generation, along with the reference addresses; and (5) a duplicate reference address removal means for removing duplicate reference address groups from the extracted related feature vectors. (6) A learning support system using search extension generation, characterized in that it obtains an answer to the test question by performing the operations (1) to (6) above, by extracting the page text specified by the reference address after removing the group of duplicate reference addresses from the page text database recording means and then transferring it to the question input field together with the question text.

2. A learning support system using search extension generation according to claim 1, characterized by comprising a page image database recording means that records each acquired page image along with a reference address to the page image.

3. A learning support system using search extension generation according to claim 1 or 2, characterized by comprising a page image viewing means for displaying and providing access to the page image specified by the reference address.

4. The learning support system using the search extension generation described above, characterized in that, after extracting a page image specified by a reference address from a page image database recording means, the user inputs comments for figures and tables that have not been adequately converted to text, and the comment transcription means is used to transcribe the comments into the page text based on those inputs.

5. A learning support system using search extension generation according to claim 2, characterized in that it comprises a plurality of page image acquisition means, a page text database recording means for the page image database recording means, and a plurality of feature vector database recording means for the page text database recording means, and further comprises a plurality of RAG database search means for searching and extracting relevant feature vectors for any of the feature vector database recording means.

6. A learning support system using search-extended generation according to claim 1 or 2, comprising a question-answer recording means for recording the question and the obtained answer, wherein, with respect to a new question, the question-answer recording means first searches for a question similar to the new question, and if a similar question is found, the answer record for that similar question is used as the answer to the question, characterized in that it comprises an F&Q database.

7. The learning support system using search-extended generation according to claim 1 or 2, characterized in that the examination question providing means includes, in addition to a function of recording examination questions and providing them as question texts, a question text-choice pair decomposition means that decomposes the questions into pairs of question texts and individual choices and provides them as question texts.

Images