Information processing system, information processing method, and program

Abstract

To provide an information processing system, an information processing method, and a program.SOLUTION: An information processing method includes: acquiring user classification item about a question sentence 26; converting the question sentence 26 into a question vector; retrieving a prescribed number of document vectors having a high degree of association with a question vector from among a plurality of document numbers, a document vector attached with a content classification item corresponding to a user classification item related to the question vector being regarded as a retrieval object in the retrieval; generating a prompt to be inputted to a large-scale language model 14, the prompt including the question sentence 26 and a content of a conversion source of the prescribed number of document vectors to be context; inputting the prompt into the large-scale language model 14; and outputting an answer sentence 27 generated on the basis of a document outputted by the large-scale language model 14.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an information processing system, an information processing method, and a program. [Background technology]

[0002] In recent years, question-and-answer systems using large-scale language models (LLMs) have been provided. LLMs are language models specialized for natural language processing (NLP) and are a type of generative AI (artificial intelligence). Generally, when a command sentence called a prompt is input into a large-scale language model, the model outputs a sentence called a completion, which is generated based on the command.

[0003] Generative AI using large-scale language models often outputs answers that lack accuracy to questions. For example, Patent Document 1 discloses a technology that generates better prompts to input into large-scale language models by adding reference information to the question sentence. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 7313757 Summary of the Invention [Problem to be solved by the invention]

[0005] To optimize the prompt, it is necessary to add more appropriate information to the question. Therefore, in order to output more appropriate answers using a general-purpose large-scale language model, it is necessary to establish a method for selecting the information to be included in the prompt along with the question.

[0006] An object of the present disclosure is to provide an information processing system, an information processing method, and a program that can output more appropriate answer sentences using a general-purpose large-scale language model. [Means for solving the problem]

[0007] An information processing system according to one aspect of the present disclosure is an information processing system configured to output an answer sentence generated using a large-scale language model in response to a question sentence input from each of a plurality of users, the information processing system comprising at least one memory configured to store a program and at least one processor configured to execute processing based on the program, the at least one processor acquiring content included in a plurality of electronic documents, the content including type content assigned any of a plurality of content classification items based on content classification, converting the content into a plurality of document vectors, the plurality of document vectors including assigning the same content classification item as the content from which the content was converted, and acquiring user classification items related to the question sentence, the the user classification item is one of a plurality of user classification items based on a user classification, and the user classification corresponds to the content classification; converting the question sentence into a question vector; searching for a predetermined number of document vectors from the plurality of document vectors that have a high degree of relevance to the question vector, wherein the search includes searching for document vectors that are assigned the content classification item that corresponds to the user classification item related to the question vector; generating a prompt to be input into the large-scale language model, wherein the prompt includes the question sentence and content from which the predetermined number of document vectors are converted, which serves as context; inputting the prompt into the large-scale language model; and outputting the answer sentence generated based on a sentence output by the large-scale language model.

[0008] An information processing method according to one aspect of the present disclosure is an information processing method for outputting an answer sentence generated using a large-scale language model in response to a question sentence input from each of a plurality of users, the information processing method comprising: acquiring content contained in a plurality of electronic documents into one or more computers, the content including type content assigned with any of a plurality of content classification items based on content classification; converting the content into a plurality of document vectors, the method including assigning to each of the plurality of document vectors the same content classification item as the content from which the content was converted; and acquiring a user classification item related to the question sentence, the user classification item being any of a plurality of user classification items based on user classification, the user classification being a content classification item of the content from the plurality of document vectors. the question sentence corresponds to a content classification; converting the question sentence into a question vector; searching for a predetermined number of document vectors from among the plurality of document vectors that are highly related to the question vector, wherein the search targets document vectors that are assigned the content classification item that corresponds to the user classification item related to the question vector; generating a prompt to be input to the large-scale language model, wherein the prompt includes the question sentence and the content from which the predetermined number of document vectors are converted, which serves as a context; inputting the prompt to the large-scale language model; and outputting the answer sentence generated based on a sentence output by the large-scale language model.

[0009] An information processing system according to one aspect of the present disclosure is an information processing system configured to output an answer sentence generated using a vector database and a large-scale language model in response to a question sentence input from each of a plurality of users, wherein the vector database is configured to store a plurality of document vectors, each of which is a vector conversion of content to which one of a plurality of content classification items based on content classification is assigned, and the information processing system includes at least one memory configured to store a program and at least one processor configured to execute processing based on the program, and the at least one processor acquires a user classification item related to the question sentence, and the user classification item is one of a plurality of user classification items based on the user classification. the user classification corresponds to the content classification; converting the question sentence into a question vector; searching for a predetermined number of document vectors from among the plurality of document vectors that are highly related to the question vector, wherein the search includes searching for document vectors to which the content classification item corresponding to the user classification item related to the question vector is assigned; generating a prompt to be input into the large-scale language model, wherein the prompt includes the question sentence and content from which the predetermined number of document vectors are converted, which serves as a context; inputting the prompt into the large-scale language model; and outputting the answer sentence generated based on a sentence output by the large-scale language model.

[0010] A program according to one aspect of the present disclosure is a computer program configured to output an answer sentence generated using a vector database and a large-scale language model in response to a question sentence input from each of a plurality of users, wherein the vector database is configured to store a plurality of document vectors, each of the plurality of document vectors being vector-converted content to which one of a plurality of content classification items based on content classification is assigned, and the computer program causes a computer to acquire a user classification item related to the question sentence, the user classification item being one of a plurality of user classification items based on user classification, and the user classification corresponding to the content classification. converting the question sentence into a question vector; searching for a predetermined number of document vectors from the plurality of document vectors that have a high degree of relevance to the question vector, wherein the search targets document vectors that are assigned the content classification item that corresponds to the user classification item related to the question vector; generating a prompt to be input into the large-scale language model, wherein the prompt includes the question sentence and content from which the predetermined number of document vectors are converted, which serves as context; inputting the prompt into the large-scale language model; and outputting the answer sentence generated based on a sentence output by the large-scale language model. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a schematic diagram of an information processing system according to an embodiment. [Figure 2] FIG. 2 is a diagram showing an example of correspondence between user classification and content classification. [Figure 3] FIG. 3 is a diagram illustrating the configuration of the first and second information processing devices included in the information processing system of FIG. [Figure 4] FIG. 4 is a diagram showing a method for constructing a vector database by the information processing system of FIG. [Figure 5]FIG. 5 is a diagram showing a method for outputting a response by the information processing system of FIG. DETAILED DESCRIPTION OF THE INVENTION

[0012] Examples of the information processing system 11, information processing method, and program of the present disclosure will be described with reference to Figures 1 to 5. The present invention is not limited to these examples, but is defined by the claims, and is intended to include all modifications within the meaning and scope equivalent to the claims.

[0013] [Overall configuration of information processing system] 1, an information processing system 11 includes one or more information processing devices 20. In the following, a case will be described in which the information processing system 11 includes two information processing devices 20 (a first information processing device 20A and a second information processing device 20B).

[0014] Each of the information processing devices 20A and 20B may be realized as a computer including, for example, one or more processors 21, one or more memories 22, and a communication interface (IF) 23. The configurations of the multiple information processing devices 20 may be partially or entirely the same, or may be different from each other.

[0015] The one or more memories 22 store programs and data for realizing various functions. The programs include applications and operating systems. The one or more processors 21 execute processes based on the programs to realize various functions. The communication interface 23 enables communication with other devices via the network 12. The network 12 includes, for example, the Internet, a wide area network (WAN), a local area network (LAN), a provider terminal, a wireless communication network, a wireless base station, a dedicated line, etc.

[0016] One or more information processing devices 20 may be able to communicate with one or more web servers 30 via the network 12. Each web server 30 may be realized as a computer including one or more processors 31, one or more memories 32, and a communication interface (IF) 33. The communication interface 33 enables communication with other devices via the network 12.

[0017] The processors 21 and 31 are, for example, a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor unit (MPU), a field-programmable gate array (FPGA), or other arithmetic devices. The processors 21 and 31 are, for example, processing circuits configured to execute various software processes. The processing circuits may include dedicated hardware circuits (e.g., ASICs) that process at least part of the software processes. In other words, the software processes may be executed by processing circuitry that includes at least one or more software processing circuits and one or more dedicated hardware circuits.

[0018] The memories 22 and 32 may include, for example, a random access memory (RAM) or other volatile memory. The memories 22 and 32 may be configured to temporarily store programs and data. The memories 22 and 32 may also include storage for permanently storing data including programs. The storage may be, for example, a read-only memory (ROM), a hard disk drive, a flash memory, or other non-volatile storage device. The storage may also be a removable storage device such as a memory card. The communication interfaces 23 and 33 may be realized, for example, as a LAN or other wired communication IF.

[0019] The web server 30 is configured to provide, for example, an in-house homepage for a company that has branches or sales offices in multiple countries and regions. A memory 32 of the web server 30 stores multiple electronic documents .

[0020] The information processing system 11 may include a vector database 13. The vector database 13 may be stored in a server device (not shown). Alternatively, the information processing devices 20A and 20B may use data in the vector database 13, which is not included in the information processing system 11, via the network 12. Alternatively, the information processing devices 20A and 20B may include the vector database 13.

[0021] The information processing system 11 may include a large-scale language model 14. Alternatively, the second information processing device 20B may use a large-scale language model 14 that is not included in the information processing system 11 via the network 12. The large-scale language model 14 is a natural language processing model trained using a large amount of text data. The large-scale language model 14 is a general-purpose language model that can be adapted to various natural language processing tasks, such as information extraction, text summarization, text generation, or question and answering.

[0022] The large-scale language model 14 is configured to, when a prompt 45 (see FIG. 3) containing some instruction is input, create a sentence according to the instruction and output it as a completion 46 (see FIG. 3). In this example, the second information processing device 20B generates the prompt 45 and inputs it to the large-scale language model 14, and then the second information processing device 20B edits the completion 46 of the large-scale language model 14 as necessary and outputs it.

[0023] [Electronic Documents and Content Classification] The plurality of electronic documents 34 are, for example, HTML, PDF, or plain text files containing content created for internal viewing, but the file format of the electronic documents 34 is not limited to these. Each of the plurality of electronic documents 34 is provided with metadata 35 about the corresponding electronic document 34. The metadata 35 may include, for example, information such as the title or URL (Uniform Resource Locator) of the content, and may further include information such as a document number (source) for internal management (see FIG. 4).

[0024] Each of the multiple electronic documents 34 includes content that is primarily composed of character strings. This content is prepared to provide information to multiple users. Examples of content include, but are not limited to, various company rules, instructions for company equipment, or Q&A (Questions and Answers) related to these, all of which are intended for viewing by multiple users who are multiple employees.

[0025] The content may include type content to which one of multiple content classification items based on the content classification is assigned. Here, "content classification" refers to dividing the content into multiple content groups according to the type, and the name of each content group is indicated as a "content classification item." The content classification items may be written as titles within the content of the electronic document 34, or may be written as comments in source code such as HTML.

[0026] As shown in Figure 2, the multiple content classification items may include multiple content type items corresponding to multiple types of content, respectively, and a common content item corresponding to content regardless of the type of content. If one of the multiple content type items is assigned to a portion of content, content that does not have a content type item assigned may be classified as a common content item. Content assigned a content type item is content for which a target user is set, and content assigned a common content item is content for all users.

[0027] [User Classification] Some or all of the multiple employees (users) are classified based on user classifications corresponding to the content classifications. Here, "user classification" refers to dividing multiple users into multiple user groups according to their type, and the names of each user group are indicated as "user classification items." User classifications may be defined based on the type of employee. The type of employee may be defined based on at least one of, for example, department, job type, assigned region, or job title.

[0028] 1, a user database 24 including employee information (e.g., name or employee ID) of each of a plurality of employees and employee type information may be stored in the memory 22 of the information processing device 20. The type information may include information that serves as a user classification item, such as, but is not limited to, department, job type, assigned region, or job title.

[0029] As shown in FIG. 2, the multiple user classification items may include multiple user type items corresponding to user types, and an all user item. The all user item is the name of a user group corresponding to all users, and can overlap with multiple user type items. A user whose type is undetermined or unknown may be classified as an all user item. Classification of users into multiple user type items may be overlap-enabled classifications that allow overlap, or may be unique classifications that do not allow overlap (alternative). The multiple user type items may match or may be different from the multiple content type items. For example, the number of user type items may be greater or less than the number of content type items.

[0030] For example, the user classification may be defined based on the country or region to which each of the multiple employees (users) belongs. For example, the multiple user classification items may include multiple user type items corresponding to each of two or more countries or regions, and an all-user item that includes all of the two or more countries or regions.

[0031] The "region" in this disclosure may be a region such as "Europe" that includes multiple countries, or may be multiple regions within a single country. This "region" is not limited to regions separated by official boundaries such as countries, prefectures, or cities, and may be set arbitrarily depending on the content or the user.

[0032] In this case, the plurality of content classification items may include a plurality of content type items corresponding to two or more countries or regions, and a common content item that is not dependent on the country or region. Contents assigned with a content type item are specific to the corresponding country or region, and contents assigned with a common content item are common to all countries and regions.

[0033] For example, if content classification is defined based on "region," content type items may include "Japan," "Asia," "Americas (North, South, and Central)," and "EMEA (Europe, Middle East, and Africa)." In this case, the common content item may be "All regions." Corresponding to this content classification, user classification may also be defined based on "region," and user type items may include "Japan," "Asia," "Americas," and "EMEA," just like the content type items.

[0034] In this case, users classified as "Japan" may also be classified as "Asia," or users from Asia other than "Japan" may be classified as "Asia" to create a unique alternative classification. The "All Users" item may also be "Global." The user classification and content classification may have different item names but essentially refer to the same subject, such as "Global" and "All Areas."

[0035] In addition to classification based on "region," users and content may be further classified based on at least one of the employee's department, job type, assigned region, or job title. Alternatively, or in addition, "EMEA" for either the user or content may be further classified as "Europe," "Middle East," and "Africa." In this way, content classification items and user classification items may correspond "one to two or more" or "two or more to one."

[0036] [First information processing device] 3, the first information processing device 20A operates as a device having functional units 51 to 55 by the processor 21 executing processing based on the program 25. The functional units 51 to 55 may be program code. The first information processing device 20A operates as a device that constructs the vector database 13 by having the functional units 51 to 55.

[0037] The functional unit 51 is a content acquisition unit 51. The content acquisition unit 51 is configured to acquire a plurality of electronic documents 34 from the memory 32 of the web server 30. Each of the plurality of electronic documents 34 includes content. In addition, each of the plurality of electronic documents 34 has metadata 35 attached thereto.

[0038] The functional unit 52 is a metadata management unit 52. The metadata management unit 52 is configured to edit the metadata 35 attached to a plurality of electronic documents 34. For example, the metadata management unit 52 can add new data to the metadata 35, or attach the metadata 35 attached to the original electronic document 34 to data generated based on the electronic document 34.

[0039] The functional unit 53 is a content splitter 53. The content splitter 53 is configured to split the content acquired by the content acquisition unit 51 into appropriate sections or into pieces of an appropriate length. In this case, the content splitter 53 may generate multiple digital documents in a unified format, each of which includes the split content.

[0040] The functional unit 54 is a vector conversion unit 54. The vector conversion unit 54 is configured to convert the content divided into appropriate lengths into vectors. This conversion is also called embedding. A vector is a numerical representation of a concept that allows a computer to understand the relationship between the concepts. In this disclosure, a vector that embeds content is called a "document vector."

[0041] The functional unit 55 is a data management unit 55. The data management unit 55 is configured to store document vectors in the vector database 13 and update the document vectors stored in the vector database 13.

[0042] [Second information processing device] The second information processing device 20B operates as a device having functional units 61 to 68 by the processor 21 executing processing based on the program 25. The second information processing device 20B provides a chatbot for responding to inquiries by having the functional units 61 to 68. The functional units 61 to 68 may be program code for the chatbot.

[0043] As shown in Figure 1, the chatbot is configured to output answer sentences 27 generated using a large-scale language model 14 in response to questions 26 input by each of multiple users. The chatbot is configured to display a chat window 40 on the display 16 of the terminal 15.

[0044] Each of the multiple users who use the chatbot communicates with the second information processing device 20B through the terminal 15. The terminal 15 is, for example, but not limited to, a personal computer, a tablet, or a smartphone. The terminal 15 may be a user terminal used or owned by an individual, or may be a shared terminal used by multiple users.

[0045] 3 , the functional unit 61 is a user category acquisition unit 61. The user category acquisition unit 61 is configured to acquire the question 26 or a user category item related to the user who sent the question 26. The functional unit 62 is a question acquisition unit 62. The question acquisition unit 62 is configured to acquire the question 26 sent through the chat window 40.

[0046] The functional unit 63 is a vector conversion unit 63. The vector conversion unit 63 is configured to perform vector conversion (embedding) on the question sentence 26. In the present disclosure, the vector in which the question sentence 26 is embedded is referred to as a "question vector."

[0047] The functional unit 64 is a vector search unit 64. The vector search unit 64 is configured to search the vector database 13 for document vectors that are highly relevant to the question vector. This search is a semantic search. The vector search unit 64 is configured to filter document vectors to be searched using user classification items and content classification items, and to extract document vectors that are highly relevant to the question vector from among the document vectors that are assigned the target content classification items.

[0048] For example, if the query vector is marked with the all-users item ("Global"), all document vectors marked with the content type items ("Japan," "Asia," "America," and "EMEA") and document vectors marked with the common content item ("Global") will be searched. Then, the vector search unit 64 extracts a specified number of document vectors that are highly relevant to the query vector from among the document vectors marked with "Japan," "Asia," "America," "EMEA," or "Global."

[0049] As another example, if a question vector is assigned one user type item (e.g., "Japan") among multiple user type items, document vectors assigned a common content item ("All Regions") and document vectors assigned a content type item ("Japan") corresponding to the one user type item ("Japan") will be searched. In this case, document vectors assigned content type items ("Asia," "America," "EMEA") other than the content type item ("Japan") corresponding to the one user type item ("Japan") will not be searched. Then, the vector search unit 64 extracts a specified number of document vectors that are highly relevant to the question vector from the document vectors assigned "Japan" or "All Regions."

[0050] The number of document vectors to be acquired (prescribed number) can be set according to the token length of the chunk 36 and the length limit of the prompt 45 to be input to the large-scale language model 14. The prescribed number may be, for example, multiple (e.g., 2, 3, 5, or 10), or may be changed according to the user type item attached to the question vector.

[0051] Functional unit 65 is a prompt generator 65. Prompt generator 65 is configured to generate a prompt 45 for input to large scale language model 14. Functional unit 66 is a prompt input unit 66. Prompt input unit 66 is configured to input a prompt 45 to large scale language model 14.

[0052] The functional unit 67 is an answer acquisition unit 67. The answer acquisition unit 67 is configured to acquire the completion 46 output by the large-scale language model 14 as an answer. The functional unit 68 is an answer output unit 68. The answer output unit 68 is configured to add to or edit the completion 46 as necessary to generate an answer sentence 27, and then output the generated answer sentence 27 to the chat window 40.

[0053] [Chatbot for inquiries] 1, the chat window 40 has an input field 41 for inputting the question 26. The chat window 40 may include a type button 42, a send button 43, and a clear button 44. The type button 42 is operated to input the type of the question 26. The send button 43 is operated when transmitting the question 26 input in the input field 41 to the information processing device 20. The clear button 44 is operated to clear the input content or the displayed content.

[0054] The type button 42 may be multiple-choice. The type of question 26 may match the user's classification items. For example, if the user's classification items include "Japan," "Asia," "America," "EMEA," and "Global," the same options may be selectable using the type button 42. The user may be able to select a type of question 26 that is different from their own type. For example, if a user currently in Japan is transferring to the United States, they may need information about the country they will be transferring to before the transfer.

[0055] The type button 42 may be used to select the user's department, job type, position, or name, or an input field may be added to the chat window 40 for freely inputting the type of user or question 26. In this case, the information processing device 20 may obtain corresponding type information as a user classification item from the user database 24 based on the selected or input content.

[0056] Alternatively, the user category acquisition unit 61 may automatically acquire a user category item corresponding to the type information stored in the user database 24 based on employee information (user information) such as the ID of the terminal 15 or the login ID to the terminal 15. In this case, instead of or in addition to acquiring type information for each question, user type information may be acquired as the type of question 26 for each login or each activation of the chatbot. For example, type information may be acquired from the user database 24 at the time of login, and if the type button 42 is not operated, the user category item corresponding to the type information may be assigned to the question 26, whereas if the type button 42 is operated, the selected user category item may be assigned to the question 26.

[0057] The chatbot may respond to inquiries from users (e.g., employees) regarding the contents of multiple electronic documents 34 provided by the web server 30. That is, the chatbot may be configured to generate an answer 27 to a user's question 26 by using a specific electronic document 34 as the basis for the answer and utilizing the sentence decoding and sentence creation functions of the large-scale language model 14.

[0058] For example, confidential documents that cannot be made public outside the company are not preferably used as training data for the large-scale language model 14 provided by an external organization. Therefore, the information processing system 11 constructs a vector database 13 dedicated to internal company documents and generates answer sentences 27 based on the contents of electronic documents 34 without having the large-scale language model 14 learn the internal company documents.

[0059] [How to generate a vector database] With reference to FIG. 4, a method for the first information processing device 20A to construct the vector database 13 will be described using an example in which the content classification and user classification are "area."

[0060] First, in step S11, the content acquisition unit 51 acquires a plurality of electronic documents 34 from the memory 32 of the web server 30. Each electronic document 34 is attached with metadata 35. Since each of the plurality of electronic documents 34 includes content, the content is acquired in step S11.

[0061] In the next step S12, the content acquisition unit 51 acquires a content classification item ("Japan," "Asia," "America," "EMEA," and "All Regions") for each of the acquired electronic documents 34 or content. A content classification item may be assigned to each electronic document 34, or one electronic document 34 may include multiple pieces of content to which different content classification items are assigned.

[0062] In step S13, if one electronic document 34 includes multiple pieces of content with different content classification items, the content division unit 53 divides the content into each content classification item. In step S14, the metadata management unit 52 adds the content classification items to the metadata 35 of the electronic document 34 as region information.

[0063] In step S15, the content division unit 53 divides each piece of content to which a content classification item has been assigned so that the amount of data (token length) falls within a specified range. At this time, each of the divided pieces of content is called a chunk 36. At this time, the content may be divided so that each chunk 36 has an appropriate token length (for example, approximately 2000 characters). The metadata management unit 52 assigns metadata 35 of the content before division to each chunk 36. In the following step S16, the metadata management unit 52 adds corresponding chunk information to the metadata 35 of each of the multiple chunks 36.

[0064] In step S17, the vector conversion unit 54 converts the content contained in the multiple chunks 36 into multiple document vectors. The metadata management unit 52 assigns the same metadata 35 as the source content (chunks 36) to each of the multiple document vectors. As a result, the corresponding content classification item is assigned to each document vector.

[0065] In step S18, the data management unit 55 stores the chunk 36 (content before conversion) and the corresponding document vector as a set in the vector database 13, and then ends the process. This constructs the vector database 13 that stores document vectors to which content classification items are assigned.

[0066] The document vectors stored in the vector database 13 may be managed in a separate history database (not shown) as a set with data related to vectorization, such as the metadata 35 and the acquisition date of the electronic document 34. When the electronic document 34, which is the original data of the document vector, or the content contained therein is updated on the web server 30, the data management unit 55 may update the document vector in the vector database 13 only for the updated portion, based on the history database.

[0067] [How to output the answer] A method for the second information processing device 20B to output the answer sentence 27 will be described with reference to FIG. When the send button 43 is operated with the question 26 entered in the input field 41, in step S21 the user classification acquisition unit 61 acquires user classification items related to the question 26. The user classification acquisition unit 61 may acquire information selected by the type button 42, or may acquire the user classification items by any of the methods described above. For example, after acquiring employee information related to the user (employee), the user classification acquisition unit 61 may refer to the user database 24 and acquire user classification items corresponding to the employee type information corresponding to the acquired employee information.

[0068] In step S22, the question sentence acquisition unit 62 acquires the question sentence 26 entered in the input field 41. The question sentence acquisition unit 62 may attach the user classification item acquired in step S21 to the question sentence 26 acquired in step S22. Step S22 may be executed before step S21 or may be executed simultaneously with step S21.

[0069] In step S23, the question acquisition unit 62 or another functional unit (translation unit) not shown may translate the question 26 input in the first language into the second language. For example, if the language used in the main training data of the large-scale language model 14 is the second language (e.g., English), the completion 46 is generated with higher accuracy if the prompt 45 is written in the second language. Therefore, if the question 26 is written in a language other than the second language, which is the training language of the large-scale language model 14, it is recommended to machine-translate the question 26 into the second language.

[0070] In step S24, the vector conversion unit 63 converts the question sentence 26 into a question vector. Here, if the main content (electronic document 34) before vectorization is written in a first language, the question sentence 26 written in the first language may be vectorized in step S24. Alternatively, if the main content before vectorization is written in a second language, the question sentence 26 translated into the second language may be vectorized in step S24. In other words, the question vector and document vector used for search may be vector-converted from character strings in the same language.

[0071] In step S25, the vector search unit 64 searches the vector database 13 based on the question vector generated in step S24 and the user classification items acquired in step S21, with conditions set so that document vectors with content classification items corresponding to the user classification items are included in the search targets.

[0072] Then, in step S26, the vector conversion unit 63 acquires, as a search result, a specified number of document vectors (for example, chunks 36 of the top 10 search results) that are highly related to the question vector from among the multiple document vectors. The contents of the acquired specified number of document vectors before vector conversion become information sources (also called contexts) for generating the answer sentence 27.

[0073] In step S27, the prompt generation unit 65 uses the question sentence 26 and the source content (the information source) of the document vectors acquired in step S26 to generate a prompt 45 to be input to the large-scale language model 14. At this time, a template for the prompt 45 may be prepared in advance. Regardless of whether a template is used, the prompt 45 includes the question sentence 26 and the source content of a predetermined number of document vectors that serve as context.

[0074] Prompt 45 may be written in a second language to improve the accuracy of generating completion 46. In this case, question sentence 26 translated into the second language in step S23 may be included in prompt 45. Furthermore, if context content is written in the first language, the content may be machine-translated into the second language and then included in prompt 45.

[0075] The template may be, for example, an instruction such as "Generate an answer to the question using the following context," where the "question" part is replaced with a question sentence 26 and the "context" part is replaced with the content that serves as the source of the information to complete the prompt 45. This prompt 45 (template) may include, for example, an instruction to output, in the answer sentence 27, a link (URL) to the electronic document 34 that is the source of the content used, as well as instructions on the formal output method, such as the number of characters in the output sentence or the written language.

[0076] In step S28, the prompt input unit 66 inputs the generated prompt 45 to the large-scale language model 14. The large-scale language model 14 then outputs a completion 46 using the question sentence 26 and the content serving as the information source as a context.

[0077] In step S29, the answer acquisition unit 67 acquires the completion 46 (answer) output by the large-scale language model 14. If the question sentence 26 is translated in step S23, in step S30, the answer output unit 68 or another functional unit (translation unit) not shown may translate the completion 46 written in the second language into the first language to generate the answer sentence 27.

[0078] The translation into the first language may be performed by the large-scale language model 14 by including an instruction to output an answer in the first language in the prompt 45. In this case, since the completion 46 written in the first language is acquired in step S29, the translation into the first language in step S30 can be omitted. The answer output unit 68 may output the completion 46 as the answer sentence 27 as is, or may generate the answer sentence 27 by editing the completion 46 or adding some information (for example, the date and time of the answer). In step S31, the answer sentence 27 generated by the answer output unit 68 is output to the chat window 40, and the process ends.

[0079] After viewing the answer sentence 27 output in the chat window 40, the user can continue by sending the next question sentence 26. In this case, the second information processing device 20B repeats steps S21 to S31. When questions and answers are repeatedly sent in this manner, the second information processing device 20B may take into consideration the previous question sentence 26 and the user classification items attached thereto, as well as the content of the answer sentence 27, when creating the second or subsequent answer sentence 27. For example, in step S27, the prompt generation unit 65 may add past chat history to the prompt 45 in addition to the question sentence 26 and the content serving as the information source.

[0080] [Effects of the present disclosure] Building a high-performance large-scale language model requires a huge amount of training data, advanced computers for training the data, long processing times, and significant costs. For this reason, it is not realistic for ordinary individuals or companies to develop large-scale language models, and so large-scale language models provided as open source are often used.

[0081] Training a large-scale language model with training data for a specific field can improve the accuracy of the output for that field. However, open-source large-scale language models do not allow general users to select the training data. In addition, since internal company documents cannot be made public, they are not suitable as training data for open-source large-scale language models used by an unspecified number of users.

[0082] It is also possible to input all relevant content from internal documents into a large-scale language model as a prompt along with an instruction sentence, and have the model output an answer based on the instruction. However, since there is a limit to the token length that can be input as a prompt, it is not possible to include all content in the prompt.

[0083] Therefore, the information processing system 11 of the present disclosure includes some content obtained from the electronic document 34 as context in the prompt 45, and then causes the large-scale language model 14 to output a completion 46 for the question sentence 26. In this way, by compensating for the knowledge deficiency of the large-scale language model 14 with the content necessary for the answer, it becomes possible to generate a more accurate answer sentence 27 while utilizing the general-purpose sentence creation function of the large-scale language model 14.

[0084] For example, suppose a company with employees in multiple countries publishes manuals for in-house equipment (such as electronic devices like printers) or administrative procedures like attendance reports on its internal website as electronic documents 34. Depending on the content, these manuals may differ from country to country, may be rarely needed, and may be updated irregularly. Therefore, reading through all of the electronic documents 34 can be a burden for the average user.

[0085] Therefore, the information processing system 11 provides a chatbot as an artificial intelligence that is familiar with the contents of the electronic document 34 and answers the user's questions, thereby reducing the burden on the user (employee). However, the specifications of devices such as printers may differ slightly from country to country, and it is difficult to train a machine learning model to learn such subtle differences.

[0086] For this reason, the information processing system 11 classifies the content of electronic documents 34 by type in advance, and attaches the classification information (content classification items) to the content as metadata 35. By dividing such content with classification information into chunks 36 and then vectorizing it into document vectors, it can be used as a search target for a semantic search that matches the meaning of the search statement (question statement 26).

[0087] Furthermore, by adding classification information (user classification items) corresponding to the content classification items to the question sentence 26, it becomes possible to search for related content more accurately. In other words, the document vectors to be searched are filtered by the user classification items and content classification items. This makes it possible to distinguish subtle differences in content that are difficult to read in the large-scale language model 14 based on the metadata 35.

[0088] For example, if the content is a manual on how to set up a printer, whether the manual is for "the United States" or "Japan" may be indicated in the file name, page name, or page title, but is often not indicated within the content. Furthermore, when the content is broken down into chunks 36, most chunks 36 do not include region information such as "the United States" or "Japan." Therefore, it is very difficult for the large-scale language model 14 to determine the target region based on the content contained in the chunks 36, resulting in inaccurate answers.

[0089] In this regard, filtering can be performed efficiently by attaching the content type to the document vector as metadata 35. That is, by selecting content of the content classification item corresponding to the user classification item of the question sentence 26 and obtaining a document vector that is highly related to the question vector from among the content, it is possible to prevent content of different types from being included in the answer sentence 27.

[0090] Here, the classification information for the question 26 is a user classification item based on the user classification. By adopting a classification based on the type of user who inputs the question 26 rather than the type of the question 26 itself, it becomes easier to narrow down the content to be searched. In particular, if the electronic document 34 is an internal company document, employee information on employees is easier to obtain than personal information on an unspecified number of users. Furthermore, since it is easy to predict the employees who will be viewing the content type, it is easy to match the content classification with the user classification.

[0091] In this way, by linking content classification information to the content that is the source of the answer and by linking user classification information to the question sentence 26, it becomes possible to compensate for inaccuracies in the completion 46.

[0092] [Effects of this disclosure] According to the present disclosure, the following effects can be achieved. (1) By adding a user classification item to the question sentence 26, it is possible to search for content (chunks 36) that have content classification items corresponding to the user classification item. By including content that is highly relevant to the question sentence 26 in the prompt 45, it is possible to output a more appropriate answer sentence 27 using the general-purpose large-scale language model 14.

[0093] (2) Content includes not only content of a type that is expected to be targeted to a target user, but also content that is common to all users. By assigning common content items to such content that is common to all users, appropriate searches can be performed.

[0094] (3) If the question vector has an all-users item, the question 26 relates to content common to all users. In this case, by searching all document vectors with content type items and document vectors with common content items, it is possible to search for all content highly related to the question.

[0095] (4) When a question vector is assigned a user type item, by searching document vectors assigned a content type item corresponding to the user type item and document vectors assigned a common content item, it is possible to search for all content highly related to the question. In addition, by excluding document vectors assigned content type items other than the content type item from the search, it is possible to prevent the contents of different types of content from being included in the answer sentence 27.

[0096] (5) When multiple users are multiple employees, the information processing device 20 can obtain the user classification items to be assigned to the question 26 by referencing the user database 24. This saves the employee the trouble of performing an operation to input a user classification item (e.g., selecting the type button 42) each time they input the question 26. In addition, output of an incorrect answer due to an operation error (e.g., selecting the wrong type button 42) can be avoided.

[0097] (6) In internal documents (electronic documents 34), it is often not stated in the content itself which content is intended for which employee, and it is also difficult for employees themselves to determine. In this regard, if content classification items according to the department, job type, assigned area, or position are attached to the content, it is possible to output an appropriate answer 27 according to the department, job type, assigned area, or position of the user (employee) who sent the question 26.

[0098] (7) By including translation processing of the question sentence 26 and the answer sentence 27 in the series of processes performed by the chatbot, the large-scale language model 14 and the information processing device 20 can be processed in a second language suitable for them, and at the same time, the user can ask questions and receive answers in the first language that they are familiar with.

[0099] (8) Products or services offered in multiple countries or regions may have the same basic structure or content, but may differ in detail depending on the country or region. Therefore, categorizing content and users based on region is useful when providing information about products or services offered in a wider region.

[0100] (9) Once the information processing device 20 acquires the user information of the user operating the chatbot, it can acquire the user classification items to be assigned to the question 26 by referring to the user database 24. This saves the user the trouble of having to input the user classification items each time they input the question 26.

[0101] [Example of change] This embodiment can be modified as follows: This embodiment and the following modifications can be combined and implemented within the scope of technical compatibility.

[0102] [Change Example 1] The functional units 51 to 55 and 61 to 68 may be realized by one information processing device 20, or may be realized by three or more information processing devices 20. Furthermore, steps S11 to S18 and S21 to 31 may be realized by different functional units (program codes).

[0103] [Change Example 2] The information processing system 11 may not include the first information processing device 20A. That is, the information processing system 11 may not execute the construction of the vector database 13 shown in steps S11 to S17. In this case, the information processing device 20 may be configured to generate the answer sentence 27 by using a vector database 13 that has been constructed in advance. The vector database 13 used at this time may or may not be included in the information processing system 11.

[0104] [Change Example 3] In the present disclosure, content type information is included in the metadata 35, but the data format and method of adding the type information can be changed as desired as long as the type information is associated with the chunk and document vector. For example, the type information may be added as an index, or the document vector and type information may be numbered and linked to each other using a separate database or the like.

[0105] [Change Example 4] If the electronic document 34 is a confidential document, the content classification may be determined based on a confidentiality level such as "confidential," "secret," or "top secret." In this case, it is preferable to determine a user classification based on the target users (employees) who are permitted to view each confidential document. In addition, in step S30, the answer acquisition unit 67 may generate the answer statement 27 by adding the confidentiality level of the output information or precautions for handling the information to the completion 46.

[0106] [Change Example 5] The electronic document 34 is not limited to confidential documents such as internal company documents, but may also include content to be provided to registered users of various services, such as e-commerce sites, electronic payments, or point services. For example, if the content is a manual for each service, the service provider can obtain user information (e.g., user ID) of the registered user from login information for the service site or the application for providing the service. In this case, the information processing device 20 can refer to the user database 24 to obtain user classification items corresponding to the obtained user information. The user classification items may be, for example, the user's membership rank or membership type.

[0107] [Change Example 6] The electronic document 34 may include content related to services provided by a store with multiple branches. If the business hours, products, or services offered vary from store to store, the names of multiple branches may be used as classification items for user classification and content classification. In this case, the terminal 15 is not limited to a device owned by a user, but may also be a store terminal located in a store and made available for use by users.

[0108] [Change Example 7] If the electronic document 34 contains content related to a product or service that has multiple versions, the content classification may be defined according to the version. In this case, the user classification may also be defined with the user of each version in mind. For example, the type button 42 may be used to select the version of the product or service related to the question. In this way, the user classification may match the content classification, and multiple user classification items may completely match multiple content classification items. In this case, "user classification" and "user classification item" can be rephrased as "content classification" and "content classification item," respectively.

[0109] [Change Example 8] As in Modification Example 7, the classification of the question sentence 26 may be based on the classification of the question itself or the content classification, rather than on the classification by user type. In this case, the "user classification" and "user classification item" in the present disclosure can be rephrased as "query classification" and "query classification item," respectively.

[0110] [Change Example 9] The user category item related to the question 26 may be acquired by a chat function in the chat window 40. For example, when the chat window 40 is launched, a message inquiring about the user's type (user category item) may be sent as a chat, or a message inquiring about the user's type (user category item) may be sent in response to the question 26. If multiple user categories are set (for example, for "area" and "occupation"), multiple messages inquiring about the user's type related to each user category may be sent.

[0111] [Change Example 10] The plurality of electronic documents 34 may be stored in any device, such as a data server, memory, or storage, not limited to the web server 30. Furthermore, the information processing device 20 may acquire the electronic documents 34 by downloading an HTML file published as a website, or may acquire the electronic documents 34 via a removable storage medium (e.g., a memory card or storage) without going through the network 12.

[0112] [Change Example 11] The information processing system 11 may be realized by one or more information processing devices 20 that utilize an external vector database 13 and a large-scale language model 14 to realize various functions of the present disclosure.

[0113] The following are some aspects that can be understood based on the above-described embodiment and modifications. [1] An information processing system configured to output an answer sentence generated using a large-scale language model in response to a question sentence input from each of a plurality of users, the information processing system comprising: at least one memory configured to store a program; at least one processor configured to execute processing based on the program; the at least one processor: Acquiring content included in a plurality of electronic documents, the content including type content assigned with any of a plurality of content classification items based on content classification; converting the content into a plurality of document vectors, which includes assigning the same content classification item as the original content to each of the plurality of document vectors; acquiring a user classification item related to the question sentence, the user classification item being one of a plurality of user classification items based on a user classification, the user classification corresponding to the content classification; converting the query sentence into a query vector; searching for a predetermined number of document vectors that are highly related to the question vector from among the plurality of document vectors, the search including targeting document vectors to which the content classification item corresponding to the user classification item related to the question vector is assigned; generating a prompt for input to the large-scale language model, the prompt including the question sentence and content from which the predetermined number of document vectors are converted as context; inputting the prompt into the large-scale language model; outputting the answer sentence generated based on the sentence output by the large-scale language model; An information processing system configured to execute the

[0114] [2] The user classification is a classification based on the type of the user; the plurality of user classification items include a plurality of user type items corresponding to the plurality of user types, respectively, and an all user item corresponding to all users; the content classification is a classification according to the type of the content, and the type of the content is set based on the type of the user; the plurality of content classification items include a plurality of content type items corresponding to the plurality of types of content, respectively, and a common content item corresponding to content regardless of the type of the content, The content to which the content type item is assigned is content for a user classified into the user type item corresponding to the content type item, and the content to which the common content item is assigned is content that does not depend on the type of user. The information processing system according to [1] above.

[0115] [3] The searching includes searching all the document vectors to which the content type item is assigned and the document vectors to which the common content item is assigned when the all-user item is assigned to the question vector. The information processing system according to [2] above.

[0116] [4] The searching includes, when one user type item among the plurality of user type items is assigned to the question vector, searching for the document vector to which the common content item is assigned and the document vector to which one content type item corresponding to the one user type item is assigned, and excluding the document vector to which a content type item other than the one content type item is assigned from the search targets. The information processing system according to [2] or [3] above.

[0117] [5] the plurality of users are a plurality of employees, The user classification is defined based on the employee type; The at least one memory stores a user database including employee information of each of the plurality of employees and the user classification items; The acquiring of the user classification items includes: acquiring the employee information; referencing the user database to acquire the user classification items corresponding to the acquired employee information; Including, The information processing system according to any one of [1] to [4] above.

[0118] [6] The type of employee is defined based on at least one of the following: department, job type, assigned region, or job title; The information processing system according to [5] above.

[0119] [7] translating the query entered in the first language into a second language; translating the sentences output by the large-scale language model in the second language into the first language; Including, The information processing system according to any one of [1] to [6] above.

[0120] [8] The user classification is defined based on a country or region to which each of the plurality of users belongs; the plurality of user classification items include a plurality of user type items corresponding to each of two or more countries or regions, and an all-user item; the plurality of content classification items include a plurality of content type items corresponding to each of the two or more countries or regions and a common content item; The content to which the content type item is assigned is content specific to the corresponding country or region, and the content to which the common content item is assigned is content that is not country or region specific. The information processing system according to any one of [1] to [7] above.

[0121] [9] The at least one memory stores a user database including user information of each of the plurality of users and the user classification items; The acquiring of the user classification items includes: obtaining the user information; referencing the user database to acquire the user classification items corresponding to the acquired user information; Including, The information processing system according to any one of [1] to [8] above.

[0122]

[10] An information processing method for outputting answers generated using a large-scale language model in response to questions input from each of a plurality of users, comprising: On one or more computers, Acquiring content included in a plurality of electronic documents, the content including type content assigned with any of a plurality of content classification items based on content classification; converting the content into a plurality of document vectors, which includes assigning the same content classification item as the original content to each of the plurality of document vectors; acquiring a user classification item related to the question sentence, the user classification item being one of a plurality of user classification items based on a user classification, the user classification corresponding to the content classification; converting the query sentence into a query vector; searching for a predetermined number of document vectors that are highly related to the question vector from among the plurality of document vectors, the search including targeting document vectors to which the content classification item corresponding to the user classification item related to the question vector is assigned; generating a prompt for input to the large-scale language model, the prompt including the question sentence and content from which the predetermined number of document vectors are converted as context; inputting the prompt into the large-scale language model; outputting the answer sentence generated based on the sentence output by the large-scale language model; An information processing method, including causing a computer to execute the above.

[0123]

[11] An information processing system configured to output answers generated using a vector database and a large-scale language model in response to questions input by each of a plurality of users, the vector database is configured to store a plurality of document vectors, each of which is obtained by vector-converting content to which one of a plurality of content classification items based on content classification is assigned; The information processing system includes: at least one memory configured to store a program; at least one processor configured to execute processing based on the program; the at least one processor: acquiring a user classification item related to the question sentence, the user classification item being one of a plurality of user classification items based on a user classification, the user classification corresponding to the content classification; converting the query sentence into a query vector; searching for a predetermined number of document vectors that are highly related to the question vector from among the plurality of document vectors, the search including targeting document vectors to which the content classification item corresponding to the user classification item related to the question vector is assigned; generating a prompt for input to the large-scale language model, the prompt including the question sentence and content from which the predetermined number of document vectors are converted as context; inputting the prompt into the large-scale language model; outputting the answer sentence generated based on the sentence output by the large-scale language model; An information processing system configured to execute the

[0124]

[12] A computer program configured to output answers generated using a vector database and a large-scale language model in response to questions input by each of a plurality of users, the vector database is configured to store a plurality of document vectors, each of which is obtained by vector-converting content to which one of a plurality of content classification items based on content classification is assigned; The computer program includes: acquiring a user classification item related to the question sentence, the user classification item being one of a plurality of user classification items based on a user classification, the user classification corresponding to the content classification; converting the query sentence into a query vector; searching for a predetermined number of document vectors that are highly related to the question vector from among the plurality of document vectors, the search including targeting document vectors to which the content classification item corresponding to the user classification item related to the question vector is assigned; generating a prompt for input to the large-scale language model, the prompt including the question sentence and content from which the predetermined number of document vectors are converted as context; inputting the prompt into the large-scale language model; outputting the answer sentence generated based on the sentence output by the large-scale language model; A program that executes. [Explanation of symbols]

[0125] 11...information processing system, 12...network, 13...vector database, 14...large-scale language model, 15...terminal, 16...display, 20, 20A, 20B...information processing device, 21...processor, 22...memory, 23...communication interface, 24...user database, 25...program, 26...question statement, 27...answer statement, 30...web server, 31...processor, 32...memory, 33...communication interface, 34...electronic document, 35...metadata, 36...chunk, 40...Chat window, 41...Input field, 42...Type button, 43...Send button, 44...Clear button, 45...Prompt, 46...Completion, 51...Content acquisition unit, 52...Metadata management unit, 53...Content division unit, 54...Vector conversion unit, 55...Data management unit, 61...User classification acquisition unit, 62...Question acquisition unit, 63...Vector conversion unit, 64...Vector search unit, 65...Prompt generation unit, 66...Prompt input unit, 67...Answer acquisition unit, 68...Answer output unit.

Claims

1. An information processing system configured to output an answer sentence generated using a large-scale language model in response to a question sentence input from each of a plurality of users, the information processing system comprising: at least one memory configured to store a program; at least one processor configured to execute processing based on the program; the at least one processor: Acquiring content included in a plurality of electronic documents, the content including type content assigned with any of a plurality of content classification items based on content classification; converting the content into a plurality of document vectors, which includes assigning the same content classification item as the original content to each of the plurality of document vectors; acquiring a user classification item related to the question sentence, the user classification item being one of a plurality of user classification items based on a user classification, the user classification corresponding to the content classification; converting the query sentence into a query vector; searching for a predetermined number of document vectors that are highly related to the question vector from among the plurality of document vectors, the search including targeting document vectors to which the content classification item corresponding to the user classification item related to the question vector is assigned; generating a prompt for input to the large-scale language model, the prompt including the question sentence and content from which the predetermined number of document vectors are converted as context; inputting the prompt into the large-scale language model; outputting the answer sentence generated based on the sentence output by the large-scale language model; An information processing system configured to execute the

2. the user classification is a classification based on the type of the user, the plurality of user classification items include a plurality of user type items corresponding to the plurality of user types, respectively, and an all user item corresponding to all users; the content classification is a classification according to the type of the content, and the type of the content is set based on the type of the user; the plurality of content classification items include a plurality of content type items corresponding to the plurality of types of content, respectively, and a common content item corresponding to content regardless of the type of the content, The content to which the content type item is assigned is content for a user classified into the user type item corresponding to the content type item, and the content to which the common content item is assigned is content that does not depend on the type of user. The information processing system according to claim 1 .

3. The searching includes, when the all-user item is assigned to the question vector, searching all the document vectors assigned with the content type item and the document vectors assigned with the common content item. The information processing system according to claim 2 .

4. The searching includes, when one user type item of the plurality of user type items is assigned to the question vector, searching for the document vector assigned with the common content item and the document vector assigned with one content type item corresponding to the one user type item, and excluding the document vector assigned with a content type item other than the one content type item from the search targets. The information processing system according to claim 2 .

5. the plurality of users are a plurality of employees, The user classification is defined based on the employee type; The at least one memory stores a user database including employee information of each of the plurality of employees and the user classification items; The acquiring of the user classification items includes: acquiring the employee information; referencing the user database to acquire the user classification items corresponding to the acquired employee information; Including, 5. The information processing system according to claim 1.

6. The type of employee is defined based on at least one of department, job type, assigned region, or job title; The information processing system according to claim 5 .

7. translating the query entered in a first language into a second language; translating the sentences output by the large-scale language model in the second language into the first language; Including, 5. The information processing system according to claim 1.

8. the user classification is defined based on a country or region to which each of the plurality of users belongs; the plurality of user classification items include a plurality of user type items corresponding to each of two or more countries or regions, and an all-user item; the plurality of content classification items include a plurality of content type items corresponding to each of the two or more countries or regions and a common content item; The content to which the content type item is assigned is content specific to the corresponding country or region, and the content to which the common content item is assigned is content that is not country or region specific.

5. The information processing system according to claim 1.

9. a user database including user information of each of the plurality of users and the user classification items is stored in the at least one memory; The acquiring of the user classification items includes: obtaining the user information; referencing the user database to acquire the user classification items corresponding to the acquired user information; Including, 5. The information processing system according to claim 1.

10. An information processing method for outputting an answer sentence generated using a large-scale language model in response to a question sentence input from each of a plurality of users, the method comprising: On one or more computers, Acquiring content included in a plurality of electronic documents, the content including type content assigned with any of a plurality of content classification items based on content classification; converting the content into a plurality of document vectors, which includes assigning the same content classification item as the original content to each of the plurality of document vectors; acquiring a user classification item related to the question sentence, the user classification item being one of a plurality of user classification items based on a user classification, the user classification corresponding to the content classification; converting the query sentence into a query vector; searching for a predetermined number of document vectors that are highly related to the question vector from among the plurality of document vectors, the search including targeting document vectors to which the content classification item corresponding to the user classification item related to the question vector is assigned; generating a prompt for input to the large-scale language model, the prompt including the question sentence and content from which the predetermined number of document vectors are converted as context; inputting the prompt into the large-scale language model; outputting the answer sentence generated based on the sentence output by the large-scale language model; An information processing method, including causing a computer to execute the above.

11. An information processing system configured to output answer sentences generated using a vector database and a large-scale language model in response to question sentences input by each of a plurality of users, the vector database is configured to store a plurality of document vectors, each of which is obtained by vector-converting content to which one of a plurality of content classification items based on content classification is assigned; The information processing system includes: at least one memory configured to store a program; at least one processor configured to execute processing based on the program; the at least one processor: acquiring a user classification item related to the question sentence, the user classification item being one of a plurality of user classification items based on a user classification, the user classification corresponding to the content classification; converting the query sentence into a query vector; searching for a predetermined number of document vectors that are highly related to the question vector from among the plurality of document vectors, the search including targeting document vectors to which the content classification item corresponding to the user classification item related to the question vector is assigned; generating a prompt for input to the large-scale language model, the prompt including the question sentence and content from which the predetermined number of document vectors are converted as context; inputting the prompt into the large-scale language model; outputting the answer sentence generated based on the sentence output by the large-scale language model; An information processing system configured to execute the

12. A computer program configured to output an answer sentence generated using a vector database and a large-scale language model in response to a question sentence input from each of a plurality of users, the vector database is configured to store a plurality of document vectors, each of which is obtained by vector-converting content to which one of a plurality of content classification items based on content classification is assigned; The computer program causes a computer to: acquiring a user classification item related to the question sentence, the user classification item being one of a plurality of user classification items based on a user classification, the user classification corresponding to the content classification; converting the query sentence into a query vector; searching for a predetermined number of document vectors that are highly related to the question vector from among the plurality of document vectors, the search including targeting document vectors to which the content classification item corresponding to the user classification item related to the question vector is assigned; generating a prompt for input to the large-scale language model, the prompt including the question sentence and content from which the predetermined number of document vectors are converted as context; inputting the prompt into the large-scale language model; outputting the answer sentence generated based on the sentence output by the large-scale language model; A program that executes.

Images