Computer system, and information processing method
The integrated management of document and vector stores in RAG systems through automated index data generation and unified directory structure addresses the separate management challenge, enhancing efficiency and reducing hallucination.
Patent Information
- Application Number
- JP2024068447
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2024-04-19
- Publication Date
- 2025-10-30
AI Technical Summary
In Retrieval Augmented Generation (RAG) systems, external information and its vectorized index data are managed separately, requiring separate operations for updates, leading to a significant workload and making it difficult to intuitively specify documents for search.
A computer system integrates document and vector store management, automatically generating and updating index data based on document operations, using meta-information to maintain a unified directory structure, thereby reducing the need for manual association and enhancing intuitive document specification.
This integration allows for centralized management of document databases and vector stores, reducing management costs and minimizing hallucination by enabling intuitive document retrieval.
Smart Images
Figure 2025164456000001_ABST
Abstract
Description
[Technical Field]
[0001] The present invention relates to a technique for reducing hallucination in generated AI. [Background technology]
[0002] In recent years, the use of generative AI has been progressing to improve and streamline business operations. However, hallucination, in which generative AI outputs incorrect answers, has become a problem.
[0003] Retrieval Augmented Generation (RAG) is a well-known technique for reducing hallucination. RAG is a technique that searches for information from external knowledge sources and adds the searched information to the prompts of a generation AI (see, for example, Non-Patent Document 1). [Prior art documents] [Non-patent literature]
[0004] [Non-Patent Document 1] Shohei Nagata and six others, "Introduction to Building a ChatGPT / LLM System with Azure OpenAI Service," Gijutsu Hyoronsha, January 24, 2024 Summary of the Invention [Problem to be solved by the invention]
[0005] In RAG, external information stored in a specified database is vectorized and stored in a vector store. Hereinafter, vectorized external information will be referred to as index data. Because external information and index data are managed separately, when external information is added, updated, or deleted, a separate operation is required to reflect the contents of the operation in the vector store, which is a significant workload. [Means for solving the problem]
[0006] A representative example of the invention disclosed in the present application is as follows: That is, a computer system includes a plurality of computers, each having a processor, a storage device connected to the processor, and a network interface connected to the processor, and manages a document database for storing documents and a vector store for storing index data obtained by vectorizing the documents, the index data being data referenced by a large-scale language model to generate a response to a requested task, the processor monitors operation events in the document database, and upon detecting the occurrence of an operation event for adding a first document to the document database, acquires the first document, acquires a path and identification information of the first document as first meta information, generates first index data by vectorizing the first document, generates first additional information for identifying a storage location of the first document in the document database based on the first meta information, and stores the first index data with the first additional information added in the vector store. [Effects of the Invention]
[0007] According to the present invention, the document database and the vector store can be managed in a unified manner. [Brief explanation of the drawings]
[0008] [Figure 1] FIG. 1 illustrates an example of a system configuration according to a first embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of a configuration of a computer for realizing the system of the first embodiment. [Figure 3] FIG. 2 is a diagram illustrating an example of meta information stored in a meta information DB according to the first embodiment. [Figure 4] FIG. 2 is a diagram illustrating an example of a user interface presented by the document management system of the first embodiment. [Figure 5]10 is a flowchart illustrating an example of processing executed by the document management system according to the first embodiment. [Figure 6] 10 is a flowchart illustrating an example of processing executed by the document management system according to the first embodiment. [Figure 7] 10 is a flowchart illustrating an example of processing executed by the index data management system according to the first embodiment. [Figure 8] 10 is a flowchart illustrating an example of processing executed by the RAG search system according to the first embodiment. [Figure 9] FIG. 2 is a diagram illustrating an example of a user interface presented by the RAG search system of the first embodiment. [Figure 10] FIG. 1 is a diagram illustrating an example of a configuration of a conventional system. DETAILED DESCRIPTION OF THE INVENTION
[0009] Hereinafter, examples of the present invention will be described with reference to the drawings. However, the present invention should not be construed as being limited to the description of the embodiments shown below. Those skilled in the art will readily understand that the specific configuration can be changed within the scope of the idea or purpose of the present invention.
[0010] In the configuration of the invention described below, the same or similar configurations or functions are denoted by the same reference numerals, and redundant explanations will be omitted.
[0011] In this specification, the terms "first," "second," "third," etc. are used to identify components and do not necessarily limit the number or order.
[0012] First, the details of the conventional problems will be described. Fig. 10 is a diagram showing an example of the configuration of a conventional system.
[0013] The system is composed of a document management system 1000, an index data management system 1001, a RAG search system 1002, an LLM system 1003, a document DB 1010, and a vector store 1011. The document management system 1000, the index data management system 1001, the RAG search system 1002, and the LLM system 1003 are connected to each other via a network (not shown).
[0014] Here, the processing involved in adding a document will be described. The document administrator operates the document management system 1000 to add a document to the document DB 1010 and requests the vector store administrator to update the vector store 1011. The vector store administrator obtains the added document from the document DB 1010, sets an ID for the index data, and then instructs the index data management system 1001 to vectorize the document and register the index. The index data management system 1001 vectorizes the document to generate index data and adds the index data to the vector store 1011.
[0015] In this way, in conventional systems, documents and index data are managed separately, so when a document is updated, the document administrator must request the vector store administrator to update the index data. In addition, the vector store administrator must set an ID for the index data and manage the correspondence between the index data and the document.
[0016] When entering a query, a user may need to specify the document to be searched. In this case, the directory structure (folder structure) of the vector store 1011 differs from the directory structure of the document DB 1010, making it difficult to intuitively specify the document to be searched. Here, intuitive specification means "specifying the document based on the directory structure of the document DB 1010." [Example]
[0017] Fig. 1 is a diagram illustrating an example of the configuration of a system according to the first embodiment. Fig. 2 is a diagram illustrating an example of the configuration of a computer for realizing the system according to the first embodiment.
[0018] The system is composed of a document management system 100, an index data management system 101, a RAG search system 102, an LLM system 103, a document DB 110, a meta information DB 111, and a vector store 112. The document management system 100, the index data management system 101, the RAG search system 102, and the LLM system 103 are connected to each other via a network (not shown).
[0019] The document DB 110 is a database that stores documents. Note that the present invention is not limited to the data format of the documents. The vector store 112 is a database that stores index data generated by vectorizing documents. The meta information DB 111 is a database that stores meta information of vectorized documents.
[0020] The document management system 100 is a system for managing documents, and registers documents in a document DB 110, and updates and deletes documents stored in the document DB 110.
[0021] The index data management system 101 is a system that manages index data. The index data management system 101 registers index data in a vector store 112, and updates and deletes the index data stored in the vector store 112.
[0022] The LLM system 103 is a system that provides an LLM (Large Language Model) 104. The LLM 104 is a natural language processing model constructed using large amounts of text data and can perform various language processing tasks. The LLM 104 accepts prompts containing task content, such as questions written in natural language, understands the meaning of the task content, and generates and outputs text that serves as a response.
[0023] The RAG search system 102 accepts a query from a user, retrieves related information from the vector store 112, and transmits a prompt (answer generation instruction) including the related information to the LLM system 103. The RAG search system 102 also transmits an answer generated by the LLM 104 of the LLM system 103 to the user. Note that the answer may also be transmitted from the LLM system 103 to the user.
[0024] Here, the related information may be the index data itself, or may be data extracted from the index data.
[0025] The document management system 100, the index data management system 101, the RAG search system 102, and the LLM system 103 are each configured, for example, by a computer 200 as shown in FIG.
[0026] The computer 200 includes a processor 201, a main memory device 202, a secondary memory device 203, and a network interface 204. The hardware elements are connected via a bus 205. The computer 200 may also include input devices such as a keyboard, a mouse, and a touch panel, as well as an output device such as a display.
[0027] FIG. 3 is a diagram illustrating an example of meta-information stored in the meta-information DB 111 according to the first embodiment.
[0028] The meta-information DB 111 stores meta-information 300 including, for example, a sequence number 301, a document ID 302, a document name 303, a path 304, a user 305, and a date and time 306.
[0029] The sequence number 301 is a field that stores a sequence number, which is identification information for the meta information 300 stored in the meta information DB 111. The document ID 302 is a field that stores the ID of the document. The document name 303 is a field that stores the name of the document. The path 304 is a field that stores the full path that indicates the storage location of the document.
[0030] User 305 is a field that stores information about the user who performed an operation such as adding, updating, or deleting a document. User 305 stores a user name, a user ID, etc. Date and Time 306 is a field that stores the date and time when the operation was performed.
[0031] FIG. 4 is a diagram showing an example of a user interface presented by the document management system 100 according to the first embodiment.
[0032] The document management system 100 presents a user interface 400 to the document manager. The document manager operates the user interface 400 to perform operations such as adding, updating, and deleting documents.
[0033] The user interface 400 includes buttons 401 and 402 , an input field 403 , and a table 404 .
[0034] An input field 403 is an input field for specifying a directory (folder). When a directory path (absolute path) is input in the input field 403, information about documents stored in the directory is displayed in a table 404.
[0035] The document manager adds or updates documents by dragging and dropping them into table 404. The document manager operates the check boxes in the table and operates button 401 to retrieve documents from document DB 110. The document manager also operates the check boxes in the table and operates button 402 to delete documents from document DB 110.
[0036] First, the link between document operations and index data operations will be described.
[0037] FIG. 5 is a flowchart illustrating an example of processing executed by the document management system 100 according to the first embodiment.
[0038] The document manager sets in advance the directories to be vectorized in the document management system 100. The designation and de-designation of the directories to be vectorized are set using, for example, text-format setting information. Here, the operations for the directories to be vectorized will be described.
[0039] The document management system 100 starts the process described below when it receives a document operation for a directory to be vectorized via the user interface 400. Note that document operations for directories not to be vectorized are similar to known processes, and therefore will not be described here.
[0040] The document management system 100 acquires meta information of the target document (step S501). Specifically, the document management system 100 acquires the document ID, document name, path, user, and date and time. Note that if the accepted operation is a document registration operation, the document ID is not acquired.
[0041] Next, the document management system 100 determines whether the received operation is a document registration operation (step S502).
[0042] If the accepted operation is a document registration operation, the document management system 100 generates a document ID and registers the document with the document ID assigned in the document DB 110 (step S503).
[0043] Next, the document management system 100 registers meta information including the generated document ID in the meta information DB 111 (step S504), and transmits a registration notification to the index data management system 101 (step S505). Thereafter, the document management system 100 ends the process. The registration notification includes the operation type, the document, and the meta information registered in the meta information DB 111.
[0044] If it is determined in step S502 that the received operation is not a document registration operation, the document management system 100 determines whether the received operation is a document update operation (step S506).
[0045] If the received operation is an operation to update a document, the document management system 100 updates the document in the document DB 110 (step S507). Specifically, the new document is overwritten on the document before the update.
[0046] Next, the document management system 100 registers the meta-information in the meta-information DB 111 (step S508) and transmits an update notification to the index data management system 101 (step S509). After that, the document management system 100 ends the process. The update notification includes the operation type, the document, and the meta-information registered in the meta-information DB 111.
[0047] If it is determined in step S506 that the received operation is a document deletion operation, the document management system 100 deletes the document from the document DB 110 (step S510).
[0048] Next, the document management system 100 acquires meta information corresponding to the document from the meta information DB 111, and then deletes the meta information stored in the meta information DB 111 (step S511). Specifically, the document management system 100 searches for meta information in which the document ID of the deleted document is stored in the document ID 302, and acquires the meta information with the latest date and time 306. The document management system 100 also deletes all of the searched meta information.
[0049] Next, the document management system 100 transmits a deletion notification to the index data management system 101 (step S512). After that, the document management system 100 ends the process. The deletion notification includes the operation type and meta information.
[0050] It should be noted that a method using a known technology such as Webhook may be used to send registration notifications, update notifications, and deletion notifications.
[0051] Fig. 6 is a flowchart illustrating an example of processing executed by the document management system 100 according to the embodiment 1. The processing shown in Fig. 6 is executed when the setting of the directory to be vectorized is changed.
[0052] The document management system 100 refers to the setting information and determines whether or not a directory that enables vectorization exists (step S601).
[0053] If a directory that enables vectorization does not exist, the document management system 100 proceeds to step S605.
[0054] If a directory that enables vectorization exists, the document management system 100 identifies the document stored in that directory and acquires the document from the document DB 110 (step S602).
[0055] Next, the document management system 100 acquires meta information of the identified document from the meta information DB 111 (step S603). Specifically, the document management system 100 searches for meta information in which the document ID of the identified document is stored in the document ID 302, and acquires meta information with the latest date and time 306.
[0056] Next, the document management system 100 transmits a registration notification to the index data management system 101 (step S604). After that, the document management system 100 proceeds to step S605. The registration notification includes the operation type, the document, and meta information.
[0057] In step S605, the document management system 100 refers to the setting information and determines whether or not a directory for which vectorization is to be disabled exists (step S605).
[0058] If no directory exists that disables vectorization, the document management system 100 ends the process.
[0059] If a directory for which vectorization is to be disabled exists, the document management system 100 identifies the documents stored in that directory (step S606).
[0060] Next, the document management system 100 acquires meta-information of the identified document from the meta-information DB 111 (step S607). The process of step S607 is the same as the process of step S603.
[0061] Next, the document management system 100 transmits a deletion notification to the index data management system 101 (step S608). After that, the document management system 100 ends the process. The deletion notification includes the operation type and meta information.
[0062] FIG. 7 is a flowchart illustrating an example of processing executed by the index data management system 101 according to the first embodiment.
[0063] When the index data management system 101 receives a notification from the document management system 100, it executes the process described below.
[0064] The index data management system 101 determines whether the received notification is a deletion notification (step S701).
[0065] If it is determined that the received notification is not a deletion notification, that is, if the received notification is a registration notification or an update notification, the index data management system 101 generates index data by vectorizing the document included in the notification (step S702). Note that since the index data can be generated using known technology, detailed description will be omitted.
[0066] The index data management system 101 stores the index data in the vector store 112 (step S703). After that, the index data management system 101 ends the process.
[0067] In this embodiment, the index data management system 101 generates additional information that allows the storage location of the document DB 110 to be identified based on the meta-information, and adds the additional information to the index data. Specifically, one of the following methods is adopted.
[0068] (First Method) The index data management system 101 generates an ID for index data from the path and document ID included in the meta information. Specifically, it generates an ID as a string by adding the document ID to the end of the path. The index data management system 101 stores the index data with the ID in the vector store 112. For example, if the path is " / Z / ZZ / ZZZ" and the document ID is "file_A", the index data ID will be " / Z / ZZ / ZZZ.file_A". In this method, the index data ID itself represents the storage location of the document in the document DB 110 and the document ID.
[0069] (Second Method) The index data management system 101 stores index data to which meta information has been added in the vector store 112. The ID of the index data is an ID that reflects the document ID.
[0070] When updating index data, new index data may be overwritten on the index data before the update, or the index data before the update may be left as is.
[0071] If it is determined in step S701 that the received notification is a deletion notification, the index data management system 101 identifies the index data to be deleted based on the meta information (step S704). In either the first method or the second method, the index data to be deleted can be identified based on the path and document included in the meta information.
[0072] The index data management system 101 deletes the index data to be deleted from the vector store 112 (step S705). After that, the index data management system 101 ends the process.
[0073] Next, the use of the LLM system 103 will be described.
[0074] Fig. 8 is a flowchart illustrating an example of processing executed by the RAG search system 102 according to the first embodiment. Fig. 9 is a diagram illustrating an example of a user interface presented by the RAG search system 102 according to the first embodiment.
[0075] When the RAG search system 102 receives access from a user, it executes the process described below.
[0076] The RAG search system 102 refers to the meta-information DB 111 and presents the user interface 900 (step S801).
[0077] Here, a description will be given of the user interface 900. The user interface 900 includes a search history display field 901, a chat field 902, and an index selection field 903.
[0078] The search history display field 901 is a field that displays the history of documents searched when generating an answer to a query. The chat field 902 is a field for entering questions and displaying answers generated by the LLM 104. The index selection field 903 is a field for specifying index data to reference. The index selection field 903 is displayed in the following procedure.
[0079] (Step 1) The RAG search system 102 refers to the meta-information DB 111 and acquires the latest meta-information 300 for each document.
[0080] (Step 2) The RAG search system 102 displays a list of documents corresponding to the index data in the directory structure of the document DB 110 in the index selection field 903 based on the path 304 of the meta information 300.
[0081] By specifying the document (index data) to be referenced in the index selection field 903, the occurrence of hallucination can be reduced.
[0082] The user operates the index selection field 903 to select a document to reference, and inputs a question in the chat field 902. This causes a query to be issued that includes information on the document to reference (reference information) and the question.
[0083] The RAG search system 102 receives a query via the user interface 900 (step S802).
[0084] The RAG search system 102 identifies the index data to be referenced based on the reference information included in the query (step S803).
[0085] The RAG search system 102 refers to the index data to be referenced based on the question included in the query, and extracts related information from any index data (step S804).
[0086] The RAG search system 102 generates a prompt including the question and related information, and transmits the prompt to the LLM system 103 as an instruction to generate an answer (step S805).
[0087] When the RAG search system 102 receives a response from the LLM system 103, it outputs the response to the user (step S806).
[0088] As described above, according to the present invention, index data is automatically manipulated when a document is manipulated, enabling centralized management of the document DB 110 and the vector store 112. Since there is no need to manually associate documents with index data, management costs can be reduced.
[0089] Furthermore, the index data is managed so as to be able to reproduce the directory structure of the document DB 110, allowing intuitive specification of the document to be referenced, thereby reducing the occurrence of hallucination.
[0090] The RAG search system 102 may monitor events in the document DB 110 and manage the meta-information DB 111 . [Example]
[0091] When updating a document, one method is to assign a version number to the document and manage it without overwriting it.
[0092] (Case 1) A case can be considered in which index data for both pre- and post-update documents is held. In this case, selection conditions for meta information are set in advance in the RAG search system 102. In case 1, the display processing of the index selection column 903 is partially different.
[0093] (Step 1) The RAG search system 102 references the meta information DB 111 and acquires the meta information 300 for each document. For each document, the RAG search system 102 selects the meta information 300 to use based on the document name 303 or the date and time 306. For example, the meta information 300 with the latest version number included in the document name or the latest date and time is selected.
[0094] (Step 2) The RAG search system 102 displays a list of documents corresponding to the index data in the directory structure of the document DB 110 in the index selection field 903 based on the path 304 of the meta information 300.
[0095] Note that filtering based on the document name 303 or the date and time 306 may be set as a selection condition for the meta information. For example, documents with older dates may be excluded from the selection targets, and documents with older version numbers may be excluded from the selection targets.
[0096] (Case 2) A case can be considered in which only the updated document is stored in the directory to be vectorized. In this case, the processing in step S507 is partially different. Specifically, the document management system 100 moves the pre-update document to another directory and stores the updated document in the target directory. Furthermore, in step S509, the document management system 100 sends a deletion notification of the pre-update document along with an update notification of the updated document.
[0097] The present invention is not limited to the above-described embodiments, but includes various modifications. The above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those including all of the described configurations. It is also possible to replace part of the configuration of one embodiment with the configuration of another embodiment, or to add the configuration of another embodiment to the configuration of one embodiment. It is also possible to add, delete, or replace part of the configuration of each embodiment with other configurations. [Explanation of symbols]
[0098] 100 Document Management Systems 101 Index Data Management System 102 RAG Search System 103 LLM System 104 LLM(Large Language Model) (1) 110 Document DB 111 Meta Information DB 112 Vector Store 200 calculator 201 processor 202 Main storage 203 Secondary storage device 204 Network Interface 205 Bus 300 Meta Information 400, 900 user interface
Claims
1. 1. A computer system comprising: a plurality of computers each having a processor, a storage device connected to the processor, and a network interface connected to the processor; Manage a document database that stores documents and a vector store that stores index data that vectorizes the documents; The index data is data that is referenced by a large-scale language model to generate an answer to a requested task, The processor: monitoring operational events in the document database; When detecting an occurrence of an operation event for adding a first document to the document database, acquiring the first document and acquiring a path and identification information of the first document as first meta information; generating first index data by vectorizing the first document; A computer system characterized by generating first additional information for identifying the storage location of the first document in the document database based on the first meta information, and storing the first index data with the first additional information attached in the vector store.
2. 2. The computer system of claim 1, The computer system is characterized in that the processor generates identification information for the first index data based on the path and identification information of the first document contained in the first meta information, and performs one of the following processes: a process of generating identification information for the first index data based on the path and identification information of the first document contained in the first meta information, and a process of adding the identification information of the first index data to the first index data as the first additional information; and a process of adding the first meta information to the first index data as the first additional information.
3. 3. The computer system according to claim 2, The processor: When detecting an occurrence of an operation event for updating the first document, acquiring the updated first document, and acquiring a path of the first document and identification information of the updated first document as second meta information; vectorizing the updated first document to generate second index data; A computer system characterized by generating second additional information for identifying the storage location of the updated first document in the document database based on the second meta information, and storing the second index data with the second additional information attached in the vector store.
4. 4. The computer system according to claim 3, The processor: When detecting the occurrence of an operation event for deleting the first document, the first meta information is acquired; A computer system characterized in that the first index data is identified based on the first meta-information, and the first index data is deleted.
5. 5. The computer system of claim 4, maintaining a meta-information database for storing meta-information including the path and identification information of the document; The processor: presenting an interface for instructing the large-scale language model to perform a task; A computer system characterized in that, based on the meta-information database, a list of the documents corresponding to the index data stored in the vector store in a directory structure of the document database is displayed on the interface.
6. 6. The computer system according to claim 5, A selection condition for the meta information is set; The processor: acquiring the meta information from the meta information database based on the selection conditions for the meta information; A computer system characterized in that, based on the acquired meta information, the user interface displays a list of the documents corresponding to the index data stored in the vector store in a directory structure of the document database.
7. 6. The computer system according to claim 5, The processor: receiving, via the interface, a task execution instruction including reference information specifying the index data to be referenced; referencing the vector store based on the reference information to obtain predetermined index data; A computer system comprising: inputting a prompt including the content of the task and the acquired index data into the large-scale language model.
8. 2. The computer system of claim 1, maintaining a meta-information database for storing meta-information including the path and identification information of the document; The processor: presenting a user interface for specifying a directory for which vectorization is enabled or a directory for which vectorization is disabled; When receiving information about the directory for which vectorization is enabled, refer to the document database to identify the document data stored in the directory, and acquire the identified document; obtaining the meta-information of the identified document from the meta-information database; vectorizing the identified documents to generate third index data; A computer system characterized by generating third additional information for identifying the storage location of the identified document in the document database based on the meta information of the identified document, and storing the third index data with the third additional information attached in the vector store.
9. 9. The computer system of claim 8, The processor: When receiving information about the directory for which vectorization is to be disabled, refer to the document database to identify the document data stored in the directory; obtaining the meta-information of the identified document from the meta-information database; Identifying the index data corresponding to the identified document based on the meta-information of the identified document; A computer system characterized in that the identified index data is deleted.
10. An information processing method executed by a computer system, comprising: The computer system a plurality of computers each having a processor, a storage device connected to the processor, and a network interface connected to the processor; Manage a document database that stores documents and a vector store that stores index data that vectorizes the documents; The index data is data that is referenced by a large-scale language model to generate an answer to a requested task, The information processing method includes: a first step in which the processor monitors manipulation events in the document database; a second step of acquiring the first document and acquiring a path and identification information of the first document as first meta information when the processor detects an occurrence of an operation event for adding a first document to the document database; a third step in which the processor generates first index data by vectorizing the first document; and a fourth step in which the processor generates first additional information for identifying the storage location of the first document in the document database based on the first meta information, and stores the first index data with the first additional information added in the vector store.
11. 11. The information processing method according to claim 10, The fourth step is an information processing method characterized in that the processor performs one of the following steps: generating identification information for the first index data based on the path and identification information of the first document contained in the first meta information, and assigning the identification information of the first index data to the first index data as the first additional information; and assigning the first meta information to the first index data as the first additional information.
12. 12. The information processing method according to claim 11, When the processor detects the occurrence of an operation event that updates the first document, the processor acquires the updated first document and acquires a path of the first document and identification information of the updated first document as second meta information; the processor vectorizing the updated first document to generate second index data; an information processing method characterized by including a step in which the processor generates second additional information for identifying the storage location of the updated first document in the document database based on the second meta information, and stores the second index data with the second additional information added in the vector store.
13. 13. The information processing method according to claim 12, acquiring the first meta information when the processor detects an occurrence of an operation event for deleting the first document; and a step of the processor identifying the first index data based on the first meta information and deleting the first index data.
14. 14. The information processing method according to claim 13, the computer system maintains a meta-information database for storing meta-information including paths and identification information of the documents; The information processing method includes a fifth step in which the processor presents an interface for instructing the large-scale language model to perform a task; The information processing method is characterized in that the fifth step includes a step in which the processor displays on the interface a list of the documents corresponding to the index data stored in the vector store in the directory structure of the document database based on the meta-information database.
15. 11. The information processing method according to claim 10, the computer system maintains a meta-information database for storing meta-information including paths and identification information of the documents; The information processing method includes: the processor presenting a user interface for specifying a directory for which vectorization is enabled or a directory for which vectorization is disabled; When the information of the directory that enables vectorization is received, the processor refers to the document database to identify the document data stored in the directory, acquires the identified document, acquires the meta information of the identified document from the meta information database, vectorizes the identified document to generate third index data, generates third additional information for identifying a storage location of the identified document in the document database based on the meta information of the identified document, and stores the third index data with the third additional information added in the vector store; an information processing method comprising the steps of: when receiving information of the directory that disables vectorization, the processor refers to the document database to identify the document data stored in the directory, obtains the meta information of the identified document from the meta information database, identifies the index data corresponding to the identified document based on the meta information of the identified document, and deletes the identified index data.