Information processing device, information processing system, and information processing method

Abstract

To obtain accurate answers while streamlining the search for external information to include in query information for language models. [Solution] A first similarity is calculated by referring to a database containing multiple databases relating to external information of different information categories, each storing multiple first embedding vectors representing the external information, and calculating the similarity between a second embedding vector representing a question and the first embedding vector. A second similarity is calculated by correcting the first similarity based on the attributes of the databases storing the first embedding vectors. First embedding vectors with higher second similarities are given priority and selected. Query information including the second embedding vector and the selected first embedding vector is output to a language model, and an answer to the query information is obtained from the language model.

Description

Technical Field

[0005] , ,

[0001] This application relates to an information processing apparatus, an information processing system, and an information processing method, for example, an information processing system for obtaining an answer to a question.

Background Art

[0002] Retrieval-Augmented Generation (RAG) is a technique for text generation using a Large Language Model (LLM) based on query information constructed by adding external information obtained by referring to a database for a question. RAG is spreading as a method for improving the output accuracy from an LLM trained using limited information.

[0003] For example, RAG is applied to the information processing system described in Patent Document 1. The information processing system described in Patent Document 1 obtains information regarding output candidates and a plurality of target information, calculates first intermediate data by inputting the information regarding the output candidates into a machine learning model, and uses at least a part of the intermediate data to execute an inference process using the machine learning model for each of the plurality of target information, thereby generating output information for each of the plurality of target information.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

[0006] Furthermore, when searching for external information related to a question from a database that stores a large amount of external information, a diverse range of external information is collected. Querying the LLM using query information containing this diverse range of external information can also contribute to a decrease in output accuracy. Therefore, it is advisable to perform the following two-stage processing prior to querying the LLM. The first stage involves identifying the category of the question information from the retrieved external information by referring to a database that stores external information belonging to all categories. The second stage involves searching for external information related to the question information by referring to category-specific external information data pertaining to the identified category within a category-specific external information database that stores external information belonging to each individual category.

[0007] In searching for external information, a method that compares the query information with all external information stored in the database can achieve high output accuracy, but it requires a lot of computation, thus reducing processing speed. Another approach is to pre-classify the external information stored in the database into multiple clusters. For each cluster, the representative value of the external information is compared with the query information, a cluster is selected based on the comparison result, and each piece of external information belonging to the selected cluster is compared with the query information. In this case, the computational processing involved in the search is reduced, improving processing speed, but output accuracy decreases. [Means for solving the problem]

[0008] This invention was made to solve the above problems, and an information processing device according to one embodiment refers to a database which is a plurality of databases relating to external information of different information categories, each storing a plurality of first embedding vectors that represent the external information, calculates a first similarity which is the similarity between a second embedding vector that represents a question and the first embedding vector, calculates a second similarity by correcting the first similarity based on the attributes of the databases that store the first embedding vectors, selects the first embedding vector with a higher second similarity, outputs query information including the second embedding vector and the selected first embedding vector to a language model, and obtains an answer to the query information from the language model.

[0009] In an information processing device according to one embodiment, the attribute of the database is the size of the database, and the first similarity may be adjusted to be higher for first embedded vectors stored in smaller databases.

[0010] In one embodiment of the information processing device, the attributes of the database may be corrected to be higher in proportion to the first similarity of the first embedded vector stored in the database with a high utilization rate.

[0011] In an information processing device according to one embodiment, the attribute of the database is the similarity to past questions, and the attribute may be corrected to be higher for the first similarity of the first embedded vector stored in the database with a high similarity.

[0012] In an information processing device according to one embodiment, the first embedded vector is classified into one of a plurality of clusters in the database, a third similarity is calculated which is the similarity between the representative value of the first embedded vector for each cluster and the second embedded vector, clusters with higher third similarities are selected with priority, and a first similarity is calculated between the first embedded vector and the second embedded vector for each of the first embedded vectors classified into the selected cluster.

[0013] An information processing system according to one embodiment may include the above-described information processing device and the above-described plurality of databases.

[0014] An information processing method according to one embodiment involves referring to a database which contains multiple databases relating to external information of different information categories, each storing multiple first embedding vectors that represent the external information, to calculate a first similarity which is the similarity between a second embedding vector representing a question and the first embedding vector, correcting the first similarity based on the attributes of the databases storing the first embedding vectors to calculate a second similarity, prioritizing the selection of first embedding vectors with higher second similarities, outputting query information including the second embedding vector and the selected first embedding vector to a language model, and obtaining an answer to the query information from the language model. [Effects of the Invention]

[0015] According to the embodiment of the present invention, accurate answers can be obtained while streamlining the search for external information to be included in query information to the language model. [Brief explanation of the drawing]

[0016] [Figure 1] This is a schematic block diagram showing an example configuration of the information processing system according to this embodiment. [Figure 2] This is a schematic block diagram showing an example of the functional configuration of the terminal device according to this embodiment. [Figure 3] This schematic block diagram shows an example of the functional configuration of the application server and external database server according to this embodiment. [Figure 4] This is a list illustrating the information categories for each database according to this embodiment. [Figure 5] This is a sequence diagram illustrating the response processing according to this embodiment. [Figure 6] This is a flowchart showing a first example of the first similarity correction method according to this embodiment. [Figure 7] This is an explanatory diagram showing a first embodiment of the first similarity correction method according to this embodiment. [Figure 8] It is a flowchart showing a second example of the method for correcting the first similarity according to this embodiment. [Figure 9] It is an explanatory diagram showing a second embodiment of the method for correcting the first similarity according to this embodiment. [Figure 10] It is a flowchart showing a third example of the method for correcting the first similarity according to this embodiment. [Figure 11] It is an explanatory diagram showing a third embodiment of the method for correcting the first similarity according to this embodiment. [Figure 12] It is a schematic block diagram showing a configuration example of the computer system according to this embodiment.

Mode for Carrying Out the Invention

[0017] Hereinafter, embodiments of the present application will be described with reference to the drawings. A configuration example of the information processing system S1 according to this embodiment will be described. FIG. 1 is a schematic block diagram showing a configuration example of the information processing system S1 according to this embodiment. The information processing system S1 includes a terminal device 10, an application server 20, an external database server 30, and a language model server 40. The terminal device 10, the application server 20, the external database server 30, and the language model server 40 are interconnected so as to be communicable using a network. The information processing system S1 uses RAG to obtain an answer from a question.

[0018] The terminal device 10 acquires question information indicating a question from a user. The terminal device 10 outputs the acquired question information to the application server 20. The terminal device 10 receives answer information for the question information from the application server 20. The terminal device 10 presents the received answer information. That is, the terminal device 10 executes steps of a user inputting a question and presenting an answer to the user. The terminal device 10 may be realized in any form such as, for example, a personal computer (PC: Personal Computer), a mobile phone, a tablet terminal device, or the like.

[0019] The application server 20 selects one or more pieces of external information from a large amount of external information stored in the external database server 30 based on a question received from the terminal device 10. In other words, the application server 20 performs the step of searching for external information related to the question. The application server 20 is configured, for example, as a web server.

[0020] The external database server 30 has multiple databases. Each database stores external information and a vector (sometimes referred to as the "first embedded vector" in this application) that represents its contents, in association with each other. Different categories of external information are stored in each database.

[0021] The application server 20 calculates the similarity (sometimes referred to as the "first similarity") between a vector representing the content of the received query (sometimes referred to as the "second embedded vector" in this application) and the first embedded vector. The application server 20 then corrects the first similarity based on the attributes of the database where the first embedded vector is stored to calculate the second similarity. The application server 20 then selects the first embedded vector, prioritizing those with a higher second similarity. The application server 20 generates query information (prompt) including the second embedded vector and the selected first embedded vector.

[0022] The application server 20 sends the generated query information to the language model server 40. The application server 20 receives a response from the language model server 40 as a reply to the query information. The application server 20 sends the received response to the terminal device 10.

[0023] The language model server 40 generates answers to query information using a predetermined large language model (LLM). That is, the language model server 40 performs the process of generating answers based on a question and external information. The language model server 40 uses the LLM to generate text representing the answer from query information including a question indicated by a second embedding vector and external information indicated by a first embedding vector. The LLM may be, for example, GPT (Generative Pre-trained Transformer) (registered trademark), Chat-GPT (registered trademark), etc. The language model server 40 sends the generated answer to the application server 20. The language model server may be connected to the internet so that it can be accessed from other devices, for example. The language model server may also be configured as a cloud server.

[0024] Next, an example of the functional configuration of the terminal device 10 according to this embodiment will be described. Figure 2 is a schematic block diagram showing an example of the functional configuration of the terminal device 10 according to this embodiment. The terminal device 10 comprises a user interface unit 102, an input device 112, and a display 114.

[0025] The user interface unit 102 displays a display screen on the display 114. The display screen consists of display fields that represent questions and answers. The display screen is configured, for example, as a chat screen. The user interface unit 102 receives an input of an operation signal indicating text (text input) from the input device 112 and acquires a question expressed in the received text. The user interface unit 102 displays the acquired question in the display field of the display screen and also outputs it to the application server 20.

[0026] The user interface unit 102 receives an answer from the application server 20 as a response to a question. The user interface unit 102 displays the received answer in the display area of ​​the display screen. Each time new information of either the question or the answer is acquired, the user interface unit 102 sequentially places the newly acquired information in the display area, following the existing information.

[0027] The input device 112 detects user operations, generates operation signals according to the detected operations, and outputs them to the user interface unit 102. The input device 112 may be, for example, a keyboard, a touchpad, or a combination of either. The display 114 displays a screen based on various display data input from the user interface unit 102. The display 114 may be, for example, a liquid crystal display (LCD) or an organic light-emitting diode (OLED) display.

[0028] Next, an example of the functional configuration of the application server 20 and external database server 30 according to this embodiment will be described. Figure 3 is a schematic block diagram showing an example of the functional configuration of the application server 20 and external database server 30 according to this embodiment. The application server 20 comprises an application processing unit 202 and a storage unit 204. The functions of the application processing unit 202 are realized by the computer system of the application server 20 executing a predetermined application. The external database server 30 has N databases (where N is a predetermined integer of 2 or more). In this application, each database is distinguished by being called the first DB (Database) 30-1, the second DB 30-2, and so on.

[0029] The application processing unit 202 receives a question from the terminal device 10 and derives a second embedding vector corresponding to the received question. The application processing unit 202 decomposes the text representing the question into a sequence of tokens consisting of one or more tokens (tokenization). In tokenization, the application processing unit 202 performs morphological analysis, for example, to decompose the token sequence into a sequence of words. The application processing unit 202 converts the decomposed token sequence into a second embedding vector using a predetermined embedding model. The second embedding vectors corresponding to each token sequence are derived such that the more similar the content (e.g., the meaning of words or sentences) is between multiple token sequences, the closer the values ​​in the vector space will approximate them. The embedding model may be, for example, BERT (Bidirectional Encoder Representations from Transformers), Sentence-BERT, or any other model.

[0030] The application processing unit 202 refers to the N databases (i.e., the first DB30-1 to the Nth DB30-N) located on the external database server 30 and searches for the first embedding vector based on the derived second embedding vector. Each database 30-i (where i is an integer between 1 and N) has a set of external information consisting of multiple pieces of external information. Each piece of external information is divided into units called documents. Each document is assigned a first embedding vector that indicates its content. The unit used to divide each document may be, for example, a sentence, a paragraph, or a section. Each database has a set of external information belonging to a different information category. Each database may be formed as a corpus. Each database may be generated based on existing documents.

[0031] In the example in Figure 4, each database consists of text from different documents. The first database 30-1 represents the basic operation manual for the business application. The second database 30-2 represents the instruction manual for the camera application. The third database 30-3 represents manufacturer support information. Each database has a large number of documents, and one first embedding vector is associated with each document. The first embedding vector is pre-calculated for the text contained in the document using the same embedding model used to generate the first embedding vector. The Li (typically Li is an integer of several tens or more) first embedding vectors contained in each database 30-i may be pre-classified (clustered) into Mi (typically Mi is an integer of about √Li) clusters. Each cluster contains multiple first embedding vectors whose values ​​are similar to each other. A database that has not been clustered can be considered a database with one cluster. In addition, a representative value (e.g., mean) of the first embedding vector may be pre-calculated for each cluster. Furthermore, a correction factor for the first similarity may be pre-set depending on the database attributes (described later).

[0032] Returning to Figure 3, the application processing unit 202 calculates a first similarity score, which is the similarity between the first embedding vector and the second embedding vector, when searching for the first embedding vector. Any of the following can be used as the similarity metric: for example, the absolute value of the difference, the dot product, etc. The application processing unit 202 calculates the first similarity score between each first embedding vector and the second embedding vector. The application processing unit 202 calculates the second similarity by correcting the first similarity based on the attributes of the database where the first embedding vector is stored. For example, the application processing unit 202 calculates the second similarity by multiplying the first similarity by a correction coefficient related to the database. The application processing unit 202 selects a predetermined number of first embedding vectors in descending order of the calculated second similarity, or first embedding vectors whose second similarity is higher than a predetermined standard value for the second similarity.

[0033] When the first embedded vector search range is set to the entirety of N databases, the amount of processing involved in the search tends to be large. Therefore, the application processing unit 202 may reduce the processing related to searching for the first embedding vector by performing the following two-step process. The application processing unit 202 calculates the similarity between the representative value of the first embedding vector for each cluster and the second embedding vector as the third similarity. The application processing unit 202 selects a predetermined number of clusters or clusters where the third similarity is higher than a predetermined threshold value for the third similarity, in descending order of the calculated third similarity. The application processing unit 202 calculates the first similarity between each first embedding vector belonging to the selected cluster and the second embedding vector. The application processing unit 202 calculates the second similarity by correcting the first similarity based on the attributes of the database where the first embedding vector is stored. The application processing unit 202 selects a predetermined number of second embedding vectors or clusters where the similarity is higher than a predetermined threshold value for the second similarity, in descending order of the calculated second similarity.

[0034] The application processing unit 202 generates a prompt including the second embedding vector and the second embedding vector obtained through the search. The prompt represents the question and query information indicating external information related to that question. The application processing unit 202 sends the generated prompt to the language model server 40. The application processing unit 202 receives an answer from the language model server 40 as a response to a prompt. The application processing unit 202 sends the received answer to the terminal device 10 as a response to the question.

[0035] The application processing unit 202 may configure question processing information for each received question and sequentially store the configured question processing information in the storage unit 204. The question processing information includes the question, the time the question was received, a first embedded vector retrieved using the question, and information from the database where the first embedded vector is stored, and these are associated with each other. The storage unit 204 accumulates the individual question processing information and stores it as a question history. The application processing unit 202 may receive questions from multiple users or multiple terminal devices 10. In this case, the application processing unit 202 may store question processing information for each user in the storage unit 204 and save a question history for each user. Each user can be identified using identification information such as a user ID, the device ID of the terminal device 10, or an email address.

[0036] Next, an example of the response processing according to this embodiment will be described. Figure 5 is a sequence diagram illustrating the response processing according to this embodiment. (Step S102) The user interface unit 102 of the terminal device 10 acquires a question based on the operation signal input from the input device 112. The user interface unit 102 transmits the acquired question to the application server 20. (Step S104) The application processing unit 202 of the application server 20 calculates a first embedding vector using a predetermined embedding model based on the question received from the terminal device 10.

[0037] (Step S106) The application processing unit 202 selects a cluster from each of the first DB30-1 to the NDB30-N of the external database server 30 based on the representative value of the first embedding vector for each cluster and the third similarity between the second embedding vector. (Step S108) The application processing unit 202 calculates a first similarity between each first embedding vector belonging to the selected cluster and a second embedding vector. The application processing unit 202 then corrects the first similarity based on the attributes of the database where the first embedding vector is stored and calculates a second similarity. (Step S110) The application processing unit 202 selects the second embedding vectors in descending order of the second similarity, which is the corrected similarity.

[0038] (Step S112) The application processing unit 202 generates a prompt that includes the calculated second embedding vector and the selected first embedding vector. (Step S114) The application processing unit 202 sends the generated prompt to the language model server 40. (Step S116) The language model server 40 generates text indicating the answer based on the prompt using a predetermined LLM. (Step S118) The language model server 40 sends the generated response to the application server 20. The application processing unit 202 of the application server 20 sends the response received from the language model server 40 to the terminal device 10. (Step S120) The user interface unit 102 of the terminal device 10 displays the response received from the application server 20 on the display 114. After that, the process shown in Figure 5 is terminated.

[0039] Next, we will explain an example of a method for correcting the first similarity between the first and second embedding vectors. The first example of a method for correcting the first similarity is a technique that corrects the first similarity as an attribute for each database, such that the second similarity becomes larger for first embedding vectors stored in smaller databases.

[0040] Figure 6 is a flowchart showing a first example of the first similarity correction method according to this embodiment. The process in Figure 6 is performed for each first embedding vector to be processed. In the example in Figure 6, the number of clusters is used as the size of the database.

[0041] (Step S122) The application processing unit 202 of the application server 20 identifies a database for storing the first embedding vector used in the calculation of the first similarity. In step S108 of Figure 5, the application processing unit 202 can identify a database having the cluster identified in the processing of step S106 as the database for storing the first embedding vector used in the calculation of the first similarity.

[0042] (Step S124) The application processing unit 202 determines a correction factor based on the number of clusters in the identified database. The application processing unit 202 may pre-set a correction factor that decreases as the number of clusters in each database increases. The application processing unit 202 can determine a correction factor corresponding to the identified database.

[0043] (Step S126) The application processing unit 202 calculates the second similarity by correcting it using a correction coefficient related to the database from which the first similarity was identified. The application processing unit 202 calculates the second similarity by multiplying the first similarity by the correction coefficient. After that, the process shown in Figure 6 is terminated.

[0044] According to the processing shown in Figure 6, the first similarity based on the first embedding vector stored in databases with a large number of clusters is corrected to be relatively lower. In the example in Figure 7, the first DB30-1, second DB30-2, and third DB30-3 have 3, 2, and 1 clusters, respectively, and their correction coefficients are set to 0.8, 0.9, and 1.0. From the first DB30-1, two clusters are selected out of three, and four first embedding vectors relating to external information are selected from the two selected clusters based on the first similarity. From the second DB30-2, one cluster is selected out of two, and one first embedding vector relating to external information is selected from the selected cluster based on the first similarity. From the third DB30-3, one first embedding vector relating to external information is selected from the entire database based on the first similarity.

[0045] Here, the first similarity between each of the four first embedding vectors related to external information stored in the first DB30-1 and the second embedding vector related to the question is calculated to be 0.9, 0.7, 0.84, and 0.82. The first similarity between the first embedding vector related to the one external information stored in the second DB30-2 and the second embedding vector related to the question is calculated to be 0.85. The first similarity between the first embedding vector related to the one external information stored in the third DB30-3 and the third embedding vector related to the question is calculated to be 0.8. Multiplying the first similarity for the first DB30-1 by a correction factor of 0.8 yields the second similarity scores of 0.72, 0.56, 0.672, and 0.656, respectively. Multiplying the first similarity for the second DB30-2 by a correction factor of 0.9 yields the second similarity score of 0.765. When the first similarity score for DB30-3 is multiplied by a correction factor of 1, the second similarity score is calculated to be 0.8. As a result, the first embedding vector extracted from DB30-3, which has the highest second similarity score, is selected with the highest priority.

[0046] In the first example of the first similarity correction method according to this embodiment, a case was used in which clusters were used as an indicator of the size of the database, but this is not limited to this. For example, the number of external information items (corresponding to the number of first embedding vectors) may be used as an indicator of the size of the database.

[0047] Furthermore, while Figures 6 and 7 show a case where the application processing unit 202 calculates the second similarity by multiplying the first similarity by a correction coefficient corresponding to an indicator of database size, the application processing unit 202 is not limited to this case. Instead of a correction coefficient corresponding to an indicator of database size, the application processing unit 202 may be given a pre-set adjustment amount for the first similarity and calculate the second similarity by adding the adjustment amount corresponding to the indicator of database size to the first similarity.

[0048] Next, we will explain a second example of a method for correcting the first similarity. This second example of a method for correcting the first similarity is a technique that corrects the first similarity so that the second similarity becomes larger for first embedding vectors stored in databases with high user usage rates, as an attribute for each database.

[0049] Figure 8 is a flowchart showing a second example of the similarity correction method according to this embodiment. The process in Figure 8 is performed for each first embedding vector to be processed. However, the application processing unit 202 of the application server 20 stores the question history for each user in the storage unit 204 and pre-sets the adjustment amount for each database based on the question history.

[0050] For example, the application processing unit 202 refers to the question history of each user, identifies a database where the first embedded vector selected for each question is stored, and counts the frequency identified for each database. Then, the application processing unit 202 normalizes the frequency counted for each database by dividing it by the number of questions up to that point (the current point) to calculate the user usage rate of that database for that user. The application processing unit 202 is configured to set an adjustment amount for the first similarity such that it increases as the calculated user usage rate increases. Since the process in step S132 is the same as the process in step S122 (Figure 6), we will use the explanation from that step.

[0051] (Step S134) The application processing unit 202 identifies the user of the terminal device 10 that sent the question, and determines an adjustment amount based on the user usage rate for the database that stores the first embedded vector for the identified user from among the pre-set adjustment amounts. (Step S136) The application processing unit 202 corrects the first similarity using the adjustment amount related to the identified database and calculates the second similarity. The application processing unit 202 adds the adjustment amount to the first similarity and calculates the second similarity. After that, the process shown in Figure 8 is terminated.

[0052] According to the processing shown in Figure 8, the first similarity is adjusted so that it increases relatively for first embedding vectors stored in databases with high user utilization. In the example in Figure 9, it is assumed that six first embedding vectors are selected from three databases based on the first similarity, similar to the example in Figure 7. However, the user utilization increases in the order of second DB30-2, first DB30-1, and third DB30-3. The adjustment amounts for first DB30-1, second DB30-2, and third DB30-3 are set to 0, -0.1, and 0.2, respectively.

[0053] When an adjustment amount of 0 is added to the first similarity for DB30-1, the second similarity is calculated to be 0.9, 0.7, 0.84, and 0.82, respectively. When an adjustment amount of -0.1 is added to the first similarity for DB30-2, the second similarity is calculated to be 0.75. When an adjustment amount of 0.1 is added to the first similarity for DB30-3, the second similarity is calculated to be 0.9. As a result, the first embedding vectors extracted from DB30-1 and DB30-3, each having the highest second similarity (a total of two), are selected with the highest priority.

[0054] The second example of the first similarity correction method uses the frequency with which the first embedding vector is selected for each question as an indicator of database user usage, but is not limited to this. For example, the frequency with which a cluster is selected for each question may be used as an indicator of database user usage.

[0055] Furthermore, while Figures 8 and 9 show the case where the application processing unit 202 calculates the second similarity by adding an adjustment amount corresponding to the database user utilization index to the first similarity, it is not limited to this case. The application processing unit 202 may be instructed to pre-set a correction coefficient for the first similarity instead of an adjustment amount corresponding to the database user utilization index, and the second similarity may be calculated by multiplying the first similarity by the correction coefficient corresponding to the database user utilization index.

[0056] Next, we will explain a third example of the first similarity correction method. The third example of the first similarity correction method is a technique that corrects the first similarity so that the second similarity increases for first embedding vectors stored in databases with high user usage rates and high similarity to question trends composed of past questions, as attributes for each database.

[0057] Figure 10 is a flowchart showing a third example of the similarity correction method according to this embodiment. The process in Figure 10 is performed for each first embedding vector to be processed. However, the application processing unit 202 of the application server 20 stores the question history for each user in the storage unit 204 and pre-sets the adjustment amount for each database based on the question history.

[0058] For example, the application processing unit 202 refers to the question history of each user, calculates the similarity between the second embedding vector and the first embedding vector corresponding to each question, and selects the first embedding vector whose calculated similarity is equal to or greater than a predetermined similarity threshold. The application processing unit 202 counts the number of first embedding vectors selected for each database as an example of an indicator of similarity between each question and the database.

[0059] The application processing unit 202 then calculates the sum of the number of first embedding vectors selected for each database across questions as an example of database similarity. The application processing unit 202 sets an adjustment amount for the first similarity so that it increases as the database similarity calculated for each database increases. Since the process in step S142 is the same as the process in step S122 (Figure 6), we will use the explanation from that step.

[0060] (Step S144) The application processing unit 202 identifies the user of the terminal device 10 that sent the question, and determines an adjustment amount based on the question history for the database that stores the first embedded vector for the identified user from among the pre-set adjustment amounts. (Step S146) The application processing unit 202 calculates the second similarity by correcting the first similarity using the adjustment amount related to the identified database. The application processing unit 202 calculates the second similarity by adding the adjustment amount to the first similarity. After that, the process shown in Figure 10 is terminated.

[0061] According to the processing shown in Figure 10, the first similarity is adjusted so that it increases relatively as the first embedding vector stored in the database with a high relevance to previously asked questions is increased. In the example in Figure 11, it is assumed that six first embedding vectors are selected from three databases based on the first similarity, similar to the examples in Figures 7 and 9. However, the similarity to past questions increases in the order of second DB30-2, first DB30-1, and third DB30-3, and the adjustment amounts for first DB30-1, second DB30-2, and third DB30-3 are set to 0, -0.1, and 0.2, respectively.

[0062] Therefore, when an adjustment amount of 0 is added to the first similarity for DB30-1, the second similarity is calculated to be 0.9, 0.7, 0.84, and 0.82, respectively. When an adjustment amount of -0.1 is added to the first similarity for DB30-2, the second similarity is calculated to be 0.75. When an adjustment amount of 0.1 is added to the first similarity for DB30-3, the second similarity is calculated to be 0.9. As a result, the first embedding vectors extracted from DB30-1 and DB30-3, each having the highest second similarity (a total of two), are selected with the highest priority.

[0063] In the third example of the second similarity correction method, we used a case where the total number of first embedding vectors whose similarity to the second embedding vector representing past questions is above a predetermined threshold value as an indicator of similarity to the database's question trends, but this is not the only example. The total number of clusters that give a representative value of the first embedding vectors whose similarity to the second embedding vector representing past questions is above a predetermined threshold value as an indicator of similarity to the database's question trends may also be used.

[0064] Furthermore, Figures 10 and 11 show a case where the application processing unit 202 calculates the second similarity by adding an adjustment amount corresponding to the similarity index with the database question trends to the first similarity, but it is not limited to this. The application processing unit 202 may be given a correction coefficient for the first similarity in advance instead of an adjustment amount corresponding to the similarity index with the database question trends, and the second similarity may be calculated by multiplying the first similarity by the correction coefficient corresponding to the similarity index with the database question trends.

[0065] The terminal device 10, application server 20, external database server 30, and language model server 40 described above can each be configured as an information processing device equipped with a computer system. Figure 12 is a schematic block diagram showing an example configuration of the computer system 50 according to this embodiment.

[0066] The computer system 50 is comprised of, for example, a processor 52, storage 54, input / output interface 56, communication interface 58, ROM 62, and RAM 64. The processor 52, storage 54, input / output interface 56, communication interface 58, ROM 62, and RAM 64 are connected to each other using a bus BS, allowing various types of data to be entered and exited.

[0067] The processor 52 reads programs and various data stored in the ROM 62, executes the programs, and controls the operation of the computer system 50 and the devices equipped with the computer system 50. For example, the processors of the terminal device 10 and the application server 20 execute predetermined application programs to realize the functions of the user interface unit 102 and the application processing unit 202, respectively. In this application, "executing a program" or "executing a program" includes the meaning of executing the processes instructed by the commands written in the program. The processor 52 is, for example, a CPU (Central Processing Unit). In addition to a CPU, the processor 52 may also include a GPU (Graphics Processing Unit).

[0068] Storage 54 is an auxiliary storage device that can continuously store various types of data in a read-write manner. Storage 54 may be, for example, an HDD (Hard-disk Drive) or an SSD (Solid State Drive). The I / F56 input / output interface connects to other devices via wired or wireless connections, enabling the input and output of various types of data in accordance with the specified input / output standards. The communication interface 58 connects to other devices via wired or wireless connections, enabling input and output of various types of data, in accordance with a predetermined communication standard.

[0069] ROM62 stores programs for execution by, for example, the processor 52 or various devices. RAM64 is used as a main memory medium that functions as a working area for temporarily storing various data and programs used by the processor 52, for example.

[0070] In the above explanation, the terminal device 10, application server 20, external database server 30, and language model server 40 are assumed to be independent devices, but this is not limited to this configuration. Any combination of the components of these devices may be implemented as a single device. Furthermore, some components or functions may be omitted, or other components or functions may be added to any of the devices. For example, the application server 20 may be configured as a single device comprising the configuration of an external database server 30. The terminal device 10 may be configured as a single device comprising the configuration of the application server 20 or both the application server 20 and the external database server 30. The application server 20 may be configured as a single device comprising the configuration of a language model server 40 or both the application server 20 and the external database server 30.

[0071] As described above, the information processing device (e.g., application server 20) according to this embodiment refers to a plurality of databases relating to external information of different information categories, each storing a plurality of first embedding vectors representing the external information (e.g., DB30-1 to NDB30-N), and calculates a first similarity, which is the similarity between a second embedding vector representing a question and the first embedding vector. The information processing device corrects the first similarity based on the attributes of the databases storing the first embedding vectors to calculate a second similarity. The information processing device prioritizes selecting first embedding vectors with higher calculated second similarities. The information processing device outputs query information (e.g., a prompt) including the second embedding vector and the selected first embedding vector to a language model (e.g., language model server 40), and obtains a response to the query information from the language model. Furthermore, this embodiment may be implemented as an information processing system comprising the information processing device and the multiple databases (for example, an external database server 30). In this configuration, the first embedding vector related to external information with a high relevance to the question is given priority, and the query information, which is constructed by adding the selected external information to the question, is queried to the language model. The first similarity score, which indicates the similarity between the question and each piece of external information, is corrected based on the attributes of the database that stores the first embedding vector. By correcting the relevance to the question according to the attributes of the database that stores the referenced external information, the likelihood of obtaining a more accurate answer from the language model can be improved.

[0072] The attribute of the database is the size of the database, and the information processing device may adjust the first similarity score for the first embedding vector stored in a smaller database to be higher. With this configuration, the similarity is evaluated such that the first embedding vector stored in a smaller database receives a higher score, thus more accurately selecting external information that is highly relevant to the question.

[0073] The attributes of a database represent the degree of user utilization of that database, and the information processing device may adjust the first similarity to the first embedding vector stored in a database with a high degree of utilization. With this configuration, the similarity is evaluated such that the first embedded vector stored in a database with high usage rates receives a higher score, thus more accurately selecting external information that is highly relevant to the question.

[0074] The database attribute is the similarity to past questions, and the information processing device may adjust the first similarity to the first embedding vector stored in the database with a high similarity, so that the first similarity is higher. With this configuration, the similarity is evaluated such that the first embedded vector stored in a database with high usage rates receives a higher score, thus more accurately selecting external information that is highly relevant to the question.

[0075] In some or all of the above database, the first embedding vector is classified into one of several clusters, and the information processing device may calculate a third similarity, which is the similarity between the representative value of the first embedding vector for each cluster and the second embedding vector, and select clusters with higher third similarities as preferred, and calculate a first similarity with the second embedding vector for each first embedding vector classified into the selected cluster. In this configuration, clusters are prioritized for selection based on their similarity in the representative value of the first embedding vector to the second embedding vector corresponding to the query. Then, from the selected clusters, the first embedding vector similar to the second embedding vector is selected. Therefore, the selection of a first embedding vector similar to the second embedding vector requires less computation than a brute-force search of the entire database. As a result, the process from query acquisition to query information generation can be made more efficient.

[0076] Although embodiments of the present invention have been described in detail above with reference to the drawings, the specific configurations are not limited to the embodiments described above, and include designs and the like that do not depart from the spirit of this invention. The configurations described in the embodiments described above can be combined in any way. [Explanation of Symbols]

[0077] S1…Information processing system, 10…Terminal device, 20…Application server, 30…External database server, 30-1…First DB, 30-2…Second DB, 30-3…Third DB, 30-N…Nth DB, 40…Language model server, 50…Computer system, 52…Processor, 54…Storage, 56…Input / Output I / F, 58…Communication I / F, 62…ROM, 64…RAM, 102…User interface unit, 112…Input device, 114…Display, 202…Application processing unit, 204…Storage unit

Claims

1. Multiple databases relating to external information of different information categories, Referencing a database that stores multiple first embedded vectors, each representing external information. A first similarity score is calculated, which is the similarity between the second embedding vector representing the question and the first embedding vector. The first similarity is corrected based on the attributes of the database where the first embedded vector is stored, and the second similarity is calculated. The first embedding vector with a higher second similarity is given priority for selection. The query information, including the second embedding vector and the selected first embedding vector, is output to the language model. Obtain the response to the inquiry information from the language model. Information processing device.

2. The attribute of the aforementioned database is the size of the database, The first similarity is corrected to be higher for the first embedding vector stored in the small-scale database. The information processing apparatus according to claim 1.

3. The attribute of the aforementioned database is the degree of user utilization of that database. The first similarity is adjusted to be higher for the first embedding vector stored in the database with high utilization. The information processing apparatus according to claim 1.

4. The attribute of the aforementioned database is the similarity to past questions. The first embedding vector stored in the database with high similarity is corrected to have a higher first similarity. The information processing apparatus according to claim 1.

5. In the aforementioned database, the first embedded vector is classified into one of several clusters. A third similarity is calculated, which is the similarity between the representative value of the first embedding vector for each cluster and the second embedding vector. The clusters with higher third similarity are selected with priority. For each of the first embedding vectors classified into the selected cluster, calculate the first similarity with the second embedding vector. The information processing apparatus according to claim 1.

6. The information processing apparatus according to claim 1, The plurality of databases, Information processing system.

7. An information processing method in an information processing device, Multiple databases relating to external information of different information categories, Referencing a database that stores multiple first embedded vectors, each representing external information. A first similarity score is calculated, which is the similarity between the second embedding vector representing the question and the first embedding vector. The first similarity is corrected based on the attributes of the database where the first embedded vector is stored, and the second similarity is calculated. The first embedding vector with a higher second similarity is given priority for selection. The query information, including the second embedding vector and the selected first embedding vector, is output to the language model. Obtain the response to the inquiry information from the language model. Information processing methods.

Images