Information processing device, information processing method, and program

Abstract

This invention provides an information processing device, an information processing method, and a program that use a large-scale language model (LLM) to generate responses to inputs and improve the accuracy of responses in search extension generation. [Solution] In the information processing device, the database comprises multiple sub-databases. The model storage unit stores LLMs associated with each of the multiple sub-databases. The acquisition unit acquires a selected LLM, which is one LLM selected from among the multiple LLMs, and an input sentence that causes the selected LLM to generate an answer, from the user terminal used by the user. The LLM management unit causes the selected LLM to generate an answer to the input sentence by combining a search of the information stored in the sub-database associated with the selected LLM. Each of the multiple LLMs has a defined field of input sentences that it accepts in order to generate an answer, and each of the multiple sub-databases stores information about the field defined for the LLM it is associated with.

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and a program, and more particularly, to a technique for generating a response to an input using Large Language Models (LLMs).

Background Art

[0002] In recent years, generative AI (Artificial Intelligence), particularly Natural Language Processing (NLP) technology called LLM, has developed rapidly, and automatic generation of response sentences for input sentences from users has entered the stage of practical application. As one of such LLM technologies, Retrieval Augmented Generation (RAG) has also come to be utilized (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] RAG is a mechanism that allows an LLM to retrieve information such as regulations and business documents accumulated within a specific organization as external information. Since the LLM can extract external information and generate a response based on the extracted information, it has the advantage of being able to respond to the latest information and information that has not been used for learning.

[0005] On the other hand, in RAG, since the LLM generates answers based on external information, the accuracy and reliability of those answers can be affected by the quality of the external information. The inventors of this application have come to recognize the possibility of improving the accuracy of answers by appropriately managing the external information that the LLM refers to in order to generate answers to user input sentences in RAG.

[0006] This invention has been made in view of these points, and aims to provide a technology for improving the accuracy of responses in RAG. [Means for solving the problem]

[0007] A first aspect of the present invention is an information processing device. This information processing device comprises a database having a plurality of sub-databases, each storing different information; a model storage unit that stores a large-scale language model associated with each of the plurality of sub-databases; an acquisition unit that acquires a selected LLM, which is one LLM selected from the plurality of LLMs, and an input sentence that causes the selected LLM to generate an answer, from a user terminal used by the user; an LLM management unit that causes the selected LLM to generate an answer to the input sentence by combining a search of information stored in the sub-databases associated with the selected LLM; and an output unit that outputs the answer to the user terminal. Each LLM associated with the plurality of sub-databases has a defined field of input sentences that it accepts to generate an answer, and each of the plurality of sub-databases stores information about the field defined for the LLM it is associated with.

[0008] The acquisition unit may further acquire a search range that defines the range of information to be searched from the information stored in the sub-database associated with the selected LLM, and the information processing device may further include an extraction unit that refers to the sub-database associated with the selected LLM and extracts extracted information which is information related to the input sentence within the range specified by the search range, and the LLM management unit may input the input sentence and the extracted information into the selected LLM, and may acquire the answer generated by the selected LLM.

[0009] The information stored in the sub-database may be tagged, and the information processing device may further include a query generation unit that analyzes the input statement and generates a metadata identification query, and a similarity evaluation unit that evaluates the similarity between the metadata identification query and the tags, and the output unit may display the information tagged with a high similarity, as evaluated by the similarity evaluation unit, on the user terminal in a manner that the user can select.

[0010] The information stored in the sub-database may be tagged with tags indicating permissions that determine whether or not access is permitted, the information processing device may further include a user management unit that manages multiple users and attributes indicating the permissions of each user, and the LLM management unit may combine a search for information within the scope of access permitted by the user's permissions to cause the selected LLM to generate a response.

[0011] The acquisition unit may further receive information and a specification of a sub-database to store the information from the user terminal; the LLM management unit may have the LLM associated with the sub-database analyze the received information to identify a tag to be attached to the information; and the information processing device may further include a database management unit that attaches the identified tag to the received information and stores it in the specified sub-database.

[0012] The information processing device may further include a learning data storage unit that stores combined data of input sentences and search ranges acquired by the acquisition unit from the user terminal, and a learning unit that generates a learning model based on the combined data that is trained to output a corresponding search range when an input sentence is input.

[0013] The acquisition unit may also acquire the output of the learning model as the search range when it inputs the input sentence acquired from the user terminal into the learning model.

[0014] A second aspect of the present invention is an information processing method. This information processing method involves a processor performing the following steps: obtaining a selected LLM, which is one LLM selected from among a plurality of LLMs associated with each of a plurality of sub-databases, each storing different information, from a user terminal used by a user; an input statement that causes the selected LLM to generate a response; a step of causing the selected LLM to generate a response to the input statement by combining a search of information stored in the sub-databases associated with the selected LLM; and a step of outputting the response to the user terminal. Each of the plurality of sub-databases associated with an LLM has a defined field of input statements that it accepts to generate a response, and each of the plurality of sub-databases stores information about the field defined for the LLM it is associated with.

[0015] A third aspect of the present invention is a program. This program provides a computer with the following functions: a function to obtain from a user terminal used by a user a selected LLM, which is one LLM selected from among a plurality of LLMs associated with each of a plurality of sub-databases, each storing different information; an input sentence that causes the selected LLM to generate an answer; a function to cause the selected LLM to generate an answer to the input sentence by combining a search of information stored in the sub-databases associated with the selected LLM; and a function to output the answer to the user terminal. Each of the plurality of sub-databases associated with an LLM has a defined field of input sentences that it accepts to generate an answer, and each of the plurality of sub-databases stores information about the field defined for the LLM it is associated with.

[0016] To provide this program, or to update a part of it, a computer-readable recording medium on which this program is recorded may be provided, and this program may also be transmitted over a communication line.

[0017] Furthermore, any combination of the above components, as well as conversions of the expression of the present invention between methods, apparatus, systems, computer programs, data structures, recording media, etc., are also valid embodiments of the present invention. [Effects of the Invention]

[0018] According to the present invention, the accuracy of responses in RAG can be improved. [Brief explanation of the drawing]

[0019] [Figure 1] This is a diagram illustrating the outline of the processing performed by an information processing system according to an embodiment of the present invention. [Figure 2] This diagram schematically shows the functional configuration of the information processing device according to the embodiment. [Figure 3] Figures 3(a)-(b) illustrate the acquisition of the selection assistant and input text by the acquisition unit. [Figure 4] This is a diagram for explaining acquisition of a search range by an acquisition unit. [Figure 5] This is a diagram schematically illustrating an example of a user interface for tagging information. [Figure 6] This is a diagram for explaining an example of query generation by a query generation unit and similarity evaluation by a similarity evaluation unit. [Figure 7] This is a diagram for explaining acquisition of a search range based on the similarity evaluated by the similarity evaluation unit. [Figure 8] This is a diagram schematically illustrating the data structure of an authority database referred to by a user management unit. [Figure 9] This is a diagram schematically illustrating an example of a user interface for setting a validity period of information. [Figure 10] This is a flowchart for explaining the flow of information processing executed by the information processing apparatus according to the embodiment. [Mode for Carrying Out the Invention]

[0020] <Overview of Embodiment> Figure 1 is a diagram for explaining the outline of processing executed by the information processing system S according to the embodiment of the present invention. Hereinafter, the outline of the embodiment will be described with reference to Figure 1.

[0021] The information processing system S according to the embodiment includes an information processing apparatus 1 and user terminals T, which are connected to each other in a mutually communicable manner via a known communication network N such as the Internet or a LAN (Local Area Network). Although Figure 1 shows an example in which the information processing system S includes two user terminals T, namely a first user terminal T1 and a second user terminal T2, the number of user terminals T is not limited to two.

[0022] The information processing device 1 is a device that uses RAG technology to have the LLM respond to input sentences obtained from the user via the user terminal T. Therefore, the information processing device 1 can access the database 20 that stores external information. Figure 1 shows an example where the information processing device 1 itself has the database 20, but the database 20 may be stored in an external storage device separate from the information processing device 1.

[0023] Information processing device 1 changes the LLM (Language-Likely Modeling) that generates the answer according to the field to which the content of the input sentence obtained from the user belongs. In other words, each LLM used by information processing device 1 has a predetermined specialized field to which it will provide an answer. Here, the field to which the content of the input sentence belongs varies depending on the usage scenario of information processing device 1. For example, when information processing device 1 is used for business advice in a company, examples of fields include legal affairs, human resources, sales, development, general affairs, etc. Also, when information processing device 1 is used for searching for documents in an administrative organization such as a city hall, examples of fields include family registration, disaster prevention, insurance, education, finance, sports promotion, etc.

[0024] Each LLM used by the information processing device 1 according to this embodiment has a predetermined area of ​​expertise for which it will answer questions. Therefore, users of the information processing device 1 can use the LLMs as if they were designating experts in each field and asking them questions. In this specification, LLMs may also be referred to as "Assistant A" in the sense that each LLM can assist in the work of its respective area of ​​expertise. Figure 1 shows an example in which the information processing device 1 uses at least two LLMs (Assistant A), namely the first Assistant A1 and the second Assistant A2. Note that the number of Assistant A's used by the information processing device 1 is not limited to two and may be more than two.

[0025] As described above, the information processing device 1 uses RAG technology to have Assistant A generate answers. Each Assistant A used by the information processing device 1 in this embodiment has a predetermined area of ​​expertise for which it will provide answers, and each Assistant A is associated with a dedicated database that it refers to for searching external information. In Figure 1, the database 20 accessible by the information processing device 1 comprises a first sub-database 200a and a second sub-database 200b, each associated with the first Assistant A1 and the second Assistant A2, respectively. This allows Assistant A to exclude information outside of its area of ​​expertise from the search, thereby suppressing the inclusion of noise in the search results.

[0026] Thus, the information processing device 1 according to this embodiment generates an answer by combining the retrieval of information stored in a dedicated sub-database 200 associated with Assistant A. As a result, Assistant A can improve the accuracy of its external information retrieval, and consequently, improve the accuracy of its answers in RAG.

[0027] <Functional configuration of the information processing device 1 according to the embodiment> Figure 2 is a schematic diagram showing the functional configuration of an information processing device 1 according to an embodiment. The information processing device 1 comprises a storage unit 2, a communication unit 3, and a control unit 4. In Figure 2, the arrows indicate the main data flow, and there may be data flows not shown in Figure 2. In Figure 2, each functional block shows a functional unit configuration, not a hardware (device) unit configuration. Therefore, the functional blocks shown in Figure 2 may be implemented in a single device, or they may be implemented separately in multiple devices. Data exchange between functional blocks may be performed via any means, such as a data bus, network, or portable storage medium.

[0028] The memory unit 2 is a large-capacity storage device such as an HDD (Hard Disk Drive) or SSD (Solid State Drive) that stores various information, including a ROM (Read Only Memory) that stores the BIOS (Basic Input Output System) of the computer that implements the information processing device 1, a RAM (Random Access Memory) that serves as the working area of ​​the information processing device 1, an OS (Operating System), application programs, and a database 20 that is referenced when the application programs are executed.

[0029] Communication unit 3 is a communication interface for the information processing device 1 to communicate with external devices, and is implemented using known communication modules such as LAN modules and Wi-Fi® modules. Hereafter in this specification, it is assumed that when the information processing device 1 communicates with external devices, it does so via communication unit 3, and the description of communication unit 3 may be omitted.

[0030] The control unit 4 is a processor such as the CPU (Central Processing Unit), GPU (Graphics Processing Unit), or NPU (Neural Network Processing Unit) of the information processing device 1, and functions as an acquisition unit 40, an LLM management unit 41, an output unit 42, an extraction unit 43, a query generation unit 44, a similarity evaluation unit 45, a database management unit 46, a user management unit 47, and a learning unit 48 by executing programs stored in the memory unit 2.

[0031] Figure 2 shows an example where the information processing device 1 is composed of a single device. However, the information processing device 1 may be implemented using multiple computing resources such as processors and memory, for example, in a cloud computing system. In this case, each part constituting the control unit 4 is implemented by at least one of the multiple different processors executing a program.

[0032] The memory unit 2 comprises a database 20, a model memory unit 21, an authorization database 22, and a training data memory unit 23. The database 20 also includes multiple sub-databases 200, each storing different information. The model memory unit 21 stores multiple LLMs, or Assistant A, associated with each of the multiple sub-databases 200. Details of the authorization database 22 and the training data memory unit 23 will be described later.

[0033] The acquisition unit 40 acquires from the user terminal T used by the user a selection assistant (selection LLM), which is one assistant A selected from among multiple assistants A, and an input sentence that causes the selection assistant to generate an answer.

[0034] Figures 3(a) and 3(b) illustrate the acquisition of the selection assistant and input text by the acquisition unit 40. Specifically, Figure 3(a) schematically shows an example of the selection screen of Assistant A displayed on the display unit of the user terminal T. Figure 3(b) schematically shows an example of the input screen of the input text displayed on the display unit of the user terminal T. As an example, although not limited to this, the information processing device 1 according to the embodiment functions as a web server and provides the Assistant A service to the user terminal T in the form of a web application. When a user accesses the information processing device 1 via the user terminal T, the screens shown in Figures 3(a) and 3(b) are displayed on the web browser running on the user terminal T.

[0035] In the example selection screen shown in Figure 3(a), Assistant A, who specializes in sales, and Assistant A, who specializes in legal affairs and intellectual property, are displayed. The user operates the user terminal T to select Assistant A. In Figure 3(a), the user has selected Assistant A, who specializes in sales. The acquisition unit 40 acquires Assistant A, who specializes in sales, selected by the user, from the user terminal T via the communication network N as the selected assistant.

[0036] When the user selects one of the assistants A, the screen displayed on the user terminal T transitions to the input screen shown in Figure 3(b). The input screen shown in Figure 3(b) shows an example where the user inputs "Please tell me the summary of the consumer survey results regarding product A from last year" to Assistant A, who specializes in sales and is the selected assistant. When the user operates the user terminal T and sends the input, the acquisition unit 40 receives and acquires the input from the user terminal T via the communication network N. Although not shown in the figures, the acquisition unit 40 may also acquire the input by having the user select from inputs prepared in advance in the form of templates.

[0037] The LLM management unit 41 combines searching the information stored in the sub-database 200 associated with the selection assistant to cause selection assistant A to generate a response to the input statement. In the example shown in Figures 3(a)-(b), the user has selected assistant A, whose specialty is sales, as the selection assistant. In this case, the sub-database 200 associated with the selection assistant stores only sales-related information, such as product information and sales materials of the company using the information processing device 1. In other words, the sub-database 200 associated with the selection assistant does not store information other than sales-related information, such as legal or financial information. Therefore, assistant A, whose specialty is sales, will only search for information related to its specialty as external information, and will not extract any other information. For this reason, the response generated by the selection assistant can be expected to be a highly accurate response that combines information from the selection assistant's specialty.

[0038] The output unit 42 outputs the response generated by the selection assistant in the LLM management unit 41 to the user terminal T. In this way, each of the multiple assistants A associated with each of the multiple sub-databases 200 provided in the information processing device 1 according to this embodiment has a defined field of input sentences that it accepts in order to generate a response, and each of the multiple sub-databases 200 stores information about the field defined in the assistant A associated with it. As a result, the information processing device 1 according to this embodiment can improve the accuracy of the response in RAG.

[0039] As described above, in the information processing device 1 according to this embodiment, the user has an assistant A specializing in a particular field generate an answer according to the field of the input sentence entered by the user. Furthermore, as shown in Figure 3(a), when the user selects a selection assistant from among multiple assistants A to provide an answer, it can be expected that the user themselves has some kind of foresight regarding the field of the input sentence. Therefore, if there is a mechanism to feed back the foresight information that the user possesses to the selection assistant, it can be expected to contribute to improving the accuracy of the answer. Accordingly, the acquisition unit 40 may further acquire a search range that defines the range of the search target from the information stored in the sub-database 200 associated with the selection assistant.

[0040] Figure 4 is a diagram illustrating the acquisition of the search range by the acquisition unit 40. The sub-database 200 associated with the selection assistant has a hierarchical structure, which is a structure commonly referred to as a folder or directory (hereinafter referred to as "folder" in this specification). The information stored in the sub-database 200 is pre-classified and recorded in one or more folders, and Figure 4 shows a list of folder names.

[0041] Figure 4 shows that the sub-database 200 contains 11 types of folders, from folder A to folder K. In Figure 4, folders A, D, F, G, and H are folders selected by the user. The acquisition unit 40 acquires the folders selected by the user as the search range from the information stored in the sub-database 200, which is used by the selection assistant to define the search target range.

[0042] The extraction unit 43 refers to the sub-database 200 associated with the selection assistant and extracts extraction information, which is information related to the input sentence within the range specified by the search range obtained by the acquisition unit. The LLM management unit 41 inputs the input sentence and the extraction information into the selection assistant and obtains the answer generated by the selection assistant. In this way, the extraction unit 43 can suppress the extraction of information unintended by the user by limiting the source of external information to the search range specified by the user. As a result, the information processing device 1 can improve the accuracy of the answers in RAG.

[0043] Here, the information stored in the sub-database 200 according to the embodiment is tagged. For example, the fact that the information stored in the sub-database 200 is pre-classified and recorded in one or more folders is equivalent to each piece of information being tagged to indicate that it should be stored in a specific folder. In addition, the information stored in the sub-database 200 is tagged with one or more tags indicating its respective attributes.

[0044] For example, information stored in sub-database 200 is tagged with information such as date and time information indicating when the information was stored, data type information indicating whether the information is a document or an image, and information indicating the size of the information. As another example, information stored in sub-database 200 that stores sales-related information is tagged with information indicating the content of the information, such as whether it is information about the company's products, information about sales materials, or information about the results of consumer surveys.

[0045] Furthermore, since folders within the sub-database 200 are also a type of information, tags can be applied not only to individual files stored in each folder, but also to folders themselves. Figure 5 schematically shows an example of a user interface for tagging information, and specifically, it shows a user interface for tagging folders.

[0046] In the example shown in Figure 5, "Name," "Region," and "Genre" are examples of tags that the user has assigned to each folder. For example, a folder with the name tag "Folder A" has the region tag "X City" and the genre tag "Sports." The user can also add new tags by selecting "Add." These tags can be set by the user by selecting from several pre-defined tags, or by the user directly entering them. Furthermore, the user can set two or more tags for a single piece of information (a folder in the example in Figure 5). The user interface shown in Figure 5 is displayed on the user terminal T by the output unit 42 under the control of the database management unit 46. The database management unit 46 acquires the information entered into the user interface displayed on the user terminal T as tags.

[0047] Let's return to the explanation of Figure 4. Figure 4 shows an example where the user manually specifies the search range, but it would be convenient if the folders to be specified could be prioritized using some indicator based on the tags attached to the information stored in the sub-database 200. For this reason, the information processing device 1 is equipped with a query generation unit 44 and a similarity evaluation unit 45. Specifically, first, the query generation unit 44 analyzes the input sentence acquired by the acquisition unit 40 and generates a metadata identification query. The similarity evaluation unit 45 evaluates the similarity between the metadata identification query generated by the query generation unit 44 and the tags attached to the information.

[0048] Figure 6 illustrates an example of query generation by the query generation unit 44 and similarity evaluation by the similarity evaluation unit 45. Figure 6 assumes that the information processing device 1 is used by an administrative agency such as a city hall, and shows an example where the acquisition unit 40 acquires the sentence "Please tell me some materials regarding the autumn citizens' sports competition" as input sentence I from the user. In this example, the query generation unit 44 analyzes the input sentence I and generates strings such as "autumn," "fall," "sports," "exercise," "materials," and "poster" as metadata specific query Q. The query generation unit 44 may be implemented using known technologies such as morphological analysis or thesaurus, or it may generate metadata specific query Q using LLM. In the latter case, the query generation unit 44 may instruct the LLM management unit 41 to have assistant A generate metadata specific query Q.

[0049] The similarity evaluation unit 45 vectorizes the metadata identification query Q generated by the query generation unit 44 using known embedding techniques in NLP. Similarly, the similarity evaluation unit 45 vectorizes the tag L attached to the information. Figure 6 shows an example where the similarity evaluation unit 45 converts the metadata identification query Q and the tag L into high-dimensional (e.g., 1024-dimensional) vectors V.

[0050] The similarity evaluation unit 45 evaluates the similarity between the vectorized metadata identification query Q and the vectorized tag L using a known vector similarity measurement method, such as cosine similarity. In the example shown in Figure 6, the similarity evaluation unit 45 determines that the tag L "marathon" is similar to the metadata identification query Q "sports". Similarly, the similarity evaluation unit 45 also shows that it determined that the tag L "Tanabata festival" is not similar to any of the multiple metadata identification queries Q. As a result, the query generation unit 44 and the similarity evaluation unit 45 can identify the tag L that is similar to the metadata identification query Q extracted from the input sentence I among the tag L attached to the information stored in the sub-database 200.

[0051] The output unit 42 displays information tagged with high similarity, as evaluated by the similarity evaluation unit 45, on the user terminal T in a manner that allows the user to select it. Figure 7 is a diagram illustrating the acquisition of a search range based on the similarity evaluated by the similarity evaluation unit 45. Specifically, Figure 7 shows an example where multiple folders contained in the sub-database 200 are sorted in order of the amount of information with high similarity, as evaluated by the similarity evaluation unit 45, and displayed in a manner that allows the user to select it. This makes it easier for the user to select a folder from among multiple folders that contains a large amount of information with a high probability of being related to the input sentence I entered by the user.

[0052] Here, the information stored in the sub-database 200 is not necessarily information that anyone can freely access. For example, documents containing trade secrets are highly confidential, and it may be preferable to limit the users who are permitted to view them. Therefore, the information stored in the sub-database 200 is tagged with permissions that determine whether or not access is permitted. The user management unit 47 manages multiple users by linking them with attributes that indicate the permissions of each user.

[0053] Figure 8 schematically shows the data structure of the authorization database 22 referenced by the user management unit 47. The authorization database 22 is stored in the storage unit 2 and managed by the user management unit 47. In the example authorization database 22 shown in Figure 8, access rights for each sub-database 200 are stored in association with a user identifier that uniquely identifies the user. For example, a user with user identifier UID0001 has access rights of level 1 for the first sub-database 200a, the second sub-database 200b, and the X sub-database 200x. The same applies to other user identifiers. In the example shown in Figure 8, a higher number indicating the level of access rights indicates that access to more information is permitted.

[0054] The LLM management unit 41 combines searching for information within the scope of the user's access privileges to cause the selected LLM to generate an answer. As a result, the information processing device 1 can refer to external information within the scope of the access privileges granted to the user and present the answer to the user.

[0055] The above explains how the information processing device 1 references external information stored in the sub-database 200 to have Assistant A generate an answer. Next, we will explain the process that the information processing device 1 executes when a user stores external information in the sub-database 200.

[0056] When a user attempts to store new information in the sub-database 200, there is a high probability that the user understands the content of that information. If the user understands the content of the information, it also means that the user has knowledge of the field in which that information is located. In such cases, it is preferable to utilize the user's knowledge when selecting the sub-database 200 in which the information should be stored. Therefore, the acquisition unit 40 receives the information that the user will newly store and the specification of the sub-database 200 in which that information should be stored from the user terminal T.

[0057] The LLM management unit 41 instructs Assistant A, which is associated with the sub-database 200, to analyze the received information and identify the tags to be attached to that information. The database management unit 46 attaches the tags identified by Assistant A to the information received by the acquisition unit 40 and stores it in the sub-database 200 specified by the user. As a result, the information processing device 1 can use the user's knowledge of the information content to automatically attach tags to information to be newly stored in the sub-database 200 without requiring any action from the user.

[0058] In this case, depending on the information that the user newly stores, it is preferable to specify the period during which that information is valid. For example, if information about the articles of a law that is scheduled to be amended is stored in the sub-database 200 which stores information about laws and regulations, the effective date will be the start date of the validity period, and if a repeal date has been set, that date will be the end date of the validity period. Therefore, the database management unit 46 displays a user interface on the user terminal T in the output unit 42 for setting the validity period of the information.

[0059] Figure 9 schematically illustrates an example of a user interface for setting the validity period of information. Specifically, Figure 9 shows an example of a user interface for setting the start and end dates of the validity period for document information related to a business outsourcing contract with XX Company. The user can set the start and end dates by checking the corresponding checkboxes in the user interface.

[0060] If the information has an expiration date, the extraction unit 43 extracts the information based on the condition that the date of extraction falls within the information's expiration period, in addition to the search range specified by the acquisition unit 40. This allows the information processing device 1 to further improve the accuracy of the answers in the RAG.

[0061] Furthermore, the database management unit 46 may delete information whose validity period has expired from the sub-database 200. This removes unnecessary information from the sub-database 200, allowing for more efficient use of the sub-database's capacity.

[0062] As explained with reference to Figures 4 and 7, the user who entered the input sentence I also manually specified the search range. In other words, the information processing device 1 acquires combined data, which is data that combines the user's input sentence I and the search range at that time. By accumulating this combined data, the information processing device 1 can use it to analyze statistical information regarding the likelihood of selecting the input sentence I and the corresponding search range.

[0063] Therefore, the learning data storage unit 23 stores and accumulates combination data of input sentences and search ranges acquired by the acquisition unit 40 from the user terminal T. Based on the combination data stored in the learning data storage unit 23, the learning unit 48 generates a learning model that has been trained using known machine learning techniques so that when an input sentence is entered, it outputs the corresponding search range. The learning unit 48 stores the generated learning model in the model storage unit 21.

[0064] When a learning model has been generated by the learning unit 48, the acquisition unit 40 acquires the output of the learning model as the search range when it inputs the input sentence obtained from the user terminal T into the learning model. This allows the information processing device 1 to automatically specify the search range or its candidates simply by the user inputting the input sentence I.

[0065] <Processing flow of the information processing method executed by the information processing device 1> Figure 10 is a flowchart illustrating the flow of information processing performed by the information processing device 1 according to this embodiment. The processing in this flowchart starts, for example, when the information processing device 1 is started up.

[0066] The acquisition unit 40 acquires a selection assistant, which is one LLM selected from among multiple LLMs, from the user terminal T used by the user (S2). The acquisition unit 40 also acquires an input sentence I from the user terminal T that causes the selection assistant to generate a response (S4).

[0067] The LLM management unit 41 instructs the selection assistant to generate a response to input statement I by combining a search of the information stored in the sub-database 200 associated with the selection assistant (S6). The output unit 42 outputs the response generated by the selection assistant to the user terminal T (S8). Once the output unit 42 outputs the response to the user terminal T, the process in this flowchart is completed.

[0068] <Effects of the information processing device 1 according to the embodiment> As described above, the information processing device 1 according to the embodiment can improve the accuracy of responses in RAG.

[0069] Although the present invention has been described above using embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes are possible within the scope of its gist. For example, all or part of the apparatus can be configured by functionally or physically distributing and integrating in any unit. Furthermore, new embodiments resulting from any combination of multiple embodiments are also included in the embodiments of the present invention. The effects of the new embodiments resulting from the combinations are combined with the effects of the original embodiments. Such modifications will be described below.

[0070] <First variation> The above describes the cases in which the user manually specifies the search range, the case in which the search range is determined based on the similarity evaluated by the similarity evaluation unit 45, and the case in which the learning model outputs the search range. The method of obtaining the search range is not limited to these, and other methods may be used. For example, the LLM associated with each sub-database 200 may determine the search range by engaging in dialogue with the user to narrow down the search range.

[0071] <Second variation> The above primarily described the case where one LLM is associated with one sub-database 200. However, two or more LLMs may be associated with a single sub-database 200. For example, an LLM specializing in labor law may need access to legal information in order to generate answers. To address this, an LLM specializing in labor law may be associated with the sub-database 200 that stores legal information, in addition to an LLM specializing in legal law. This allows the LLM specializing in labor law to generate more accurate answers.

[0072] <Third variation> The above mainly describes the case in which the similarity evaluation unit 45 evaluates the similarity between the vectorized metadata specific query Q and the vectorized tag L using a known vector similarity measurement method such as cosine similarity. Alternatively, the similarity evaluation unit 45 may determine the similarity between the metadata specific query Q and the tag L by evaluating whether they are a perfect match or a partial match. In this case, the similarity evaluation unit 45 should evaluate that the metadata specific query Q and the tag L are similar if they are a perfect match or a partial match, and dissimilar otherwise. [Explanation of Symbols]

[0073] 1. Information Processing Device 2...Storage section 20...Database 200 Sub-databases 21...Model Memory Unit 22. Authority Database 23. Learning Data Storage Unit 3. Communications Department 4. Control Unit 40...Acquisition part 41...Management Department 42. Output section 43...Extraction part 44. Query generation section 45. Similarity Evaluation Department 46. ​​Database Management Department 47. User Management Department 48. Learning Department N... Communications Network S... Information Processing System T...User terminal

Claims

1. A database having multiple sub-databases, each storing different information, A model storage unit that stores large language models (LLMs) associated with each of the aforementioned sub-databases, An acquisition unit that acquires, from the user terminal used by the user, a selected LLM which is one LLM selected from the plurality of LLMs, and an input sentence which causes the selected LLM to generate a response, An LLM management unit for causing the selected LLM to generate a response to the input sentence by combining a search of information stored in a sub-database associated with the selected LLM, The system includes an output unit that outputs the aforementioned answer to the user terminal, Each of the aforementioned sub-databases has a defined domain of input sentences it accepts to generate responses. Each of the aforementioned sub-databases stores information about the field defined in the LLM to which it is associated. Information processing device.

2. The acquisition unit further acquires a search range that defines the scope of the search target from the information stored in the sub-database associated with the selected LLM, The aforementioned information processing device is The system further includes an extraction unit that references a sub-database associated with the selected LLM and extracts extracted information, which is information related to the input statement within the range specified by the search range. The LLM management unit inputs the input sentence and the extracted information into the selection LLM and obtains the response generated by the selection LLM. The information processing apparatus according to claim 1.

3. The information stored in the aforementioned sub-database is tagged, The aforementioned information processing device is A query generation unit that analyzes the aforementioned input sentence and generates a metadata identification query, The system further comprises a similarity evaluation unit that evaluates the similarity between the metadata specific query and the tag, The output unit displays the information tagged with high similarity, as evaluated by the similarity evaluation unit, on the user terminal in a manner selectable by the user. The information processing apparatus according to claim 1 or 2.

4. The information stored in the aforementioned sub-database is tagged with a permission tag that determines whether or not it can be accessed. The aforementioned information processing device further comprises a user management unit that manages multiple users and attributes indicating the permissions of each user, The aforementioned LLM management unit combines a search for information within the scope of access permitted by the user's authority to cause the selected LLM to generate an answer. The information processing apparatus according to claim 3.

5. The acquisition unit further receives information and the specification of a sub-database to store the information from the user terminal. The LLM management unit analyzes the information received by the LLM associated with the sub-database to identify the tag to be attached to the information, The information processing device further comprises a database management unit that attaches the identified tag to the information received and stores it in the designated sub-database. The information processing apparatus according to claim 3.

6. The aforementioned information processing device is The acquisition unit includes a learning data storage unit that stores combined data, which is a combination of the input sentence and the search range acquired from the user terminal. The system further includes a learning unit that generates a learning model trained to output a corresponding search range when an input sentence is given, based on the aforementioned combination data. The information processing apparatus according to claim 2.

7. When the acquisition unit inputs the input sentence acquired from the user terminal into the learning model, it acquires the output of the learning model as the search range. The information processing apparatus according to claim 6.

8. The processor, The steps include obtaining a selected LLM (Limited Literacy Machine) from a user terminal used by the user, which is one LLM selected from among multiple LLMs associated with each of several sub-databases that store different information, and an input statement that causes the selected LLM to generate a response. The steps include: causing the Select LLM to generate a response to the input sentence by combining a search of information stored in a sub-database associated with the Select LLM; The steps of outputting the above answer to the user terminal are performed, Each of the aforementioned sub-databases has a defined domain of input sentences it accepts to generate responses. Each of the aforementioned sub-databases stores information about the field defined in the LLM to which it is associated. Information processing methods.

9. On the computer, A function to obtain, from the user terminal used by the user, a selected LLM (Limited Literacy Machine), which is one LLM selected from among multiple LLMs associated with each of several sub-databases that store different information, and an input sentence that causes the selected LLM to generate a response. A function to cause the selection LLM to generate a response to the input sentence by combining a search of information stored in a sub-database associated with the selection LLM, The function of outputting the above answer to the user terminal is implemented, Each of the aforementioned sub-databases has a defined domain of input sentences it accepts to generate responses. Each of the aforementioned sub-databases stores information about the field defined in the LLM to which it is associated. program.

Images