Information processing device, information processing method, and program

Abstract

The present invention provides an information processing device, an information processing method, and a program that can suppress a decrease in response accuracy even when a generative model is used to perform tasks related to a specific field. [Solution] One aspect of the information processing device of the present disclosure includes an acquisition unit that acquires the execution result of a task performed by a generation model based on expert knowledge data which is data of knowledge used professionally in a particular field, and a prompt which includes at least a first prompt having an instruction to perform a task relating to the particular field, and an output unit that outputs the execution result of the task.

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, an information processing method, and a program.

Background Art

[0002] Conventionally, a technique for improving the response accuracy of chat systems such as chatbots using generative artificial intelligence (AI) has been known. For example, Patent Document 1 discloses a technique for obtaining an answer by inputting a question sentence into a generative AI when the degree of relevance between the question sentence input into a question-answering system and the answer sentence output from the question-answering system is low in a frequently asked questions (FAQ) service using a chatbot.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Generally, generative AI learns a vast amount of data such as information and knowledge publicly available on the web. However, depending on the field, as the specialization increases, the learning data becomes insufficient and the amount of learning tends to be insufficient. Therefore, when generative AI answers an instruction regarding such a field, it may answer content that does not actually exist, resulting in a decrease in response accuracy.

[0005] In view of the above circumstances, the present disclosure aims to provide an information processing apparatus, an information processing method, and a program capable of suppressing a decrease in response accuracy even when a task related to a specific field is executed by a generative model.

Means for Solving the Problems

[0006] One aspect of the information processing device of the present disclosure includes an acquisition unit that acquires the execution result of a task performed by a generation model based on expert knowledge data which is data of knowledge used professionally in a particular field, and a prompt which includes at least a first prompt having an instruction to perform a task relating to the particular field, and an output unit that outputs the execution result of the task.

[0007] One aspect of the information processing method of the present disclosure includes an acquisition step in which an acquisition unit acquires the result of an execution of a task performed by a generation model based on expert knowledge data which is data of knowledge used professionally in a particular field and a prompt which includes at least a first prompt having an instruction to execute a task relating to the particular field, and an output step in which an output unit outputs the result of the execution of the task.

[0008] One aspect of the program of this disclosure causes a computer to perform an acquisition step of acquiring the result of an execution of a task performed by a generative model based on expert knowledge data, which is data of knowledge used professionally in a particular field, and a prompt which includes at least a first prompt having instructions to perform a task relating to the particular field, and an output step of outputting the result of the execution of the task. [Effects of the Invention]

[0009] According to this disclosure, it is possible to provide an information processing device, an information processing method, and a program that can suppress a decrease in response accuracy even when a generative model is made to perform a task related to a specific field. [Brief explanation of the drawing]

[0010] [Figure 1] Figure 1 is a block diagram showing an example of the system configuration of the book selection service system of this embodiment. [Figure 2] Figure 2 is a block diagram showing an example of the hardware configuration of the information processing device of this embodiment. [Figure 3]Figure 3 is a block diagram showing an example of the functional configuration of the book selection service system of this embodiment. [Figure 4] Figure 4 shows an example of the first data included in the expert knowledge data of this embodiment. [Figure 5] Figure 5 shows an example of the second data included in the expert knowledge data of this embodiment. [Figure 6] Figure 6 shows an example of a system prompt in this embodiment. [Figure 7] Figure 7 is a flowchart showing an example of preprocessing performed on the administrator terminal in this embodiment. [Figure 8] Figure 8 is a flowchart showing an example of reference processing performed in the information processing device of this embodiment. [Figure 9] Figure 9 shows an example of the reference results for the comparative example that was the correct answer. [Figure 10] Figure 10 shows an example of a reference result for this embodiment that was found to be correct. [Figure 11] Figure 11 shows an example of the reference results for a comparative example in which hallucination occurred. [Figure 12] Figure 12 shows an example of a reference result for this embodiment that was deemed correct. [Figure 13] Figure 13 shows an example of the reference results for both this embodiment and comparative examples from 20 cases that were verified. [Figure 14] Figure 14 shows an example of a system prompt in Modification Example 1. [Figure 15] Figure 15 shows an example of the results of vibiotherapy in a comparative example. [Figure 16] Figure 16 shows an example of the results of vibiotherapy in Modification Example 1. [Modes for carrying out the invention]

[0011] Hereinafter, embodiments of the present disclosure (hereinafter simply referred to as "the present embodiment") will be described in detail with reference to the drawings. Note that the present disclosure is not limited to the following embodiments. Also, the following embodiments and each modification can be combined as appropriate.

[0012] Hereinafter, the present disclosure will be described by taking as an example the book selection service using a generative AI. However, the present disclosure is not limited thereto.

[0013] Generally, bibliographic information such as the title, author, and publisher of a book is general information publicly available and republished on the web by publishers and multiple sales sites, etc., and it is considered that the generative AI has also sufficiently learned it. Therefore, for questions that can be answered only from bibliographic information, such as explicitly stating the title of a book and asking it to answer the author, it is considered that there is a low possibility that the generative AI will answer information that is not a fact or content that does not actually exist.

[0014] By the way, even if a user is looking for a book, the user may not necessarily be aware of the bibliographic information of the book, and may only have a fragmentary or vague memory of the title or the like. Also, there are cases where the user only has a determined purpose and wants to know whether there is a book that meets the purpose.

[0015] For this reason, in a library or the like, as a service by a librarian, a reference is provided to assist in searching for a book that is expected to meet the conditions from fragmentary or vague information of the user, or conditions such as the purpose. Note that a librarian is an expert in books and, in addition to the bibliographic information of a huge number of books, also has a corresponding understanding of the content thereof, so a reference can be provided. Thus, the reference is a service that depends on the specialized knowledge of the librarian.

[0016] However, even if we try to get a generative AI to perform the kind of references described above, the generative AI is prone to hallucination, where it provides answers that do not actually exist, because it lacks sufficient data to represent the specialized knowledge of librarians.

[0017] Therefore, in this embodiment, we will describe an example of providing a book reference service that utilizes not only a generating AI but also specialized knowledge data, which is data of knowledge specifically used in book selection services.

[0018] First, the configuration of the book selection service system in this embodiment will be described.

[0019] Figure 1 is a block diagram showing an example of the system configuration of the book selection service system 1 of this embodiment. As shown in Figure 1, the book selection service system 1 comprises an administrator terminal 10, a generation model 20, a database 30, and an information processing device 40.

[0020] The administrator terminal 10, the generation model 20, the database 30, and the information processing device 40 are connected via network 2. Network 2 can be implemented by at least one of the following: the Internet and / or a LAN (Local Area Network). Network 2 may be a wired network, a wireless network, or a combination of wired and wireless networks.

[0021] The administrator terminal 10 is a terminal device used by administrators such as operators of the book selection service system 1, and examples include PCs (personal computers), tablet devices, or smartphones. The administrator of the book selection service system 1 uses the administrator terminal 10 to manage and configure the generation model 20 and the database 30 to work together to provide book reference services.

[0022] The generative model 20 is a natural language processing model, also known as generative AI (Generative Artificial Intelligence), and provides services using natural language processing. In this embodiment, the generative model 20 is described as an AI chat service that has been fine-tuned from a large language model (LLM), but it is not limited to this.

[0023] Database 30 is a database that stores specialized knowledge data (an example of specialized knowledge data, which is data of knowledge used professionally in a specific field) that is data of knowledge professionally used in the book selection service. Examples of database 30 include, but are not limited to, one or more server devices for databases. In this embodiment, it is assumed that the specialized knowledge data is uploaded to database 30 from the administrator terminal 10 by the administrator of the book selection service system 1, but is not limited to this.

[0024] In this embodiment, the expert knowledge data is data on books introduced by book experts such as librarians, booksellers, and book curators, and includes, but is not limited to, first data which lists one or more recommended books and their recommendations for each of several arbitrary themes.

[0025] The theme can be a predetermined theme or a theme chosen by the book expert introducing the book. Examples of themes chosen by the book expert include, but are not limited to, themes that users may be interested in. The recommendation content may include, but is not limited to, at least one of the following: the reason for recommending the book or a description of it. The first data may omit the recommendation content. The first data may be publicly available data published on the web, limited access data made available only to members, or private data. Furthermore, the expert knowledge data may also include second data, which associates the book's bibliographic information with vaguely remembered information about the book. Details of the first and second data will be described later.

[0026] Furthermore, in this embodiment, the expert knowledge data is expected to be used in RAG (Retrieval Augmented Generation), and is structured in formats such as JSON (Javascript Object Notation) or XML (Extensible Markup Language), and further chunked into certain units, but is not limited to these. RAG is a method that generates prompts based on the contents of pre-specified documents or databases and has the generating AI respond.

[0027] The information processing device 40 is a terminal device used by users of the book reference service, and may include, but is not limited to, a smartphone, tablet device, or PC.

[0028] Figure 2 is a block diagram showing an example of the hardware configuration of the information processing device 40 of this embodiment. As shown in Figure 2, the information processing device 40 comprises a control device 41, a main memory 42, an auxiliary storage device 43, a communication device 44, an input device 45, a display device 46, and various buses 49. The control device 41, main memory 42, auxiliary storage device 43, communication device 44, input device 45, and display device 46 are connected via various buses 49. Thus, the information processing device 40 of this embodiment has a general hardware configuration using a normal computer.

[0029] The control device 41 controls the overall operation of the information processing device 40. The control device 41 may be, for example, at least one of a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit), but is not limited to these. There may be one or more CPUs or GPUs, and they may be single-core or multi-core.

[0030] Examples of main memory 42 include, but are not limited to, ROM (Read Only Memory) and RAM (Random Access Memory). ROM stores various programs, such as programs for controlling the information processing device 40 and programs for realizing book reference services. RAM is used as a work area when the control device 41 performs various controls based on the programs stored in ROM.

[0031] The auxiliary storage device 43 stores the various programs described above, as well as various data used to realize the book reference service. The various programs described above only need to be stored in at least one of the main storage device 42 and the auxiliary storage device 43. Examples of the auxiliary storage device 43 include, but are not limited to, at least one of existing storage devices capable of magnetic, electrical, or optical storage, such as an HDD (Hard Disk Drive), SSD (Solid State Drive), and DVD (Digital Versatile Disc). The auxiliary storage device 43 may be built into the information processing device 40, or it may be externally connected to the information processing device 40 via an interface such as USB (Universal Serial Bus). Furthermore, the auxiliary storage device 43 may be a NAS (Network Attached Storage) connected via a network such as a LAN or WAN (Wide Area Network).

[0032] The communication device 44 is used to communicate with the generative model 20 and the like via the network 2. Examples of the communication device 44 include, but are not limited to, a communication device for a wired LAN or a wireless communication device for a wireless LAN.

[0033] The input device 45 is used by the user for various inputs, selections, and specifications, and serves as a user interface between the user and the system. Examples of input devices 45 include, but are not limited to, keyboards, mice, and touch panels. The input device 45 may be built into the information processing device 40 or connected externally to the information processing device 40 via an interface such as USB.

[0034] The display device 46 displays various screens and serves as a user interface between the user and the device. Examples of the display device 46 include, but are not limited to, liquid crystal displays, organic electro-luminescence (OLED) displays, and touch panel displays. The display device 46 may be an internal display built into the information processing device 40, or an external display connected to the information processing device 40 via a display interface such as HDMI®.

[0035] In addition to the above configuration, the information processing device 40 may further include hardwired circuits such as ICs (Integrated Circuits), ASICs (Application Specific Integrated Circuits), and FPGAs (Field-Programmable Gate Arrays) that are specific to the information processing device 40.

[0036] Furthermore, although a detailed explanation will be omitted in this embodiment, the administrator terminal 10 is assumed to have the same hardware configuration as the information processing device 40.

[0037] Figure 3 is a block diagram showing an example of the functional configuration of the book selection service system 1 of this embodiment, and mainly shows an example of the functional configuration of the information processing device 40. As shown in Figure 3, the information processing device 40 includes a reception unit 401, an input unit 403, an acquisition unit 405, a verification unit 407, and an output unit 409. The reception unit 401, input unit 403, acquisition unit 405, verification unit 407, and output unit 409 can be realized by, for example, the control device 41, main memory 42, and communication device 44 described in Figure 2. For example, the control device 41 reads a program for realizing a book reference service stored in the main memory 42 (ROM) or auxiliary memory 43 and expands it into the main memory 42 (RAM). The control device 41 realizes each of the above-mentioned functional units by executing various processes according to the expanded program. Here, the case in which each of the above-mentioned functional units is realized as software has been explained as an example, but at least a part of each of the above-mentioned functional units may be realized as hardware. In this case, the functional components to be implemented as hardware can be implemented, for example, by the hardwired circuits described above. Alternatively, any of the above-mentioned functional components may be implemented through the cooperation of software and hardware.

[0038] It should be noted that the information processing device 40 of this embodiment does not necessarily require all of the above-described functional units to be part of the configuration; at least some of the functional units can be omitted. For example, the verification unit 407 can be omitted.

[0039] The administrator terminal 10 uploads the expert knowledge data to the database 30 to build the RAG. The administrator terminal 10 may upload the expert knowledge data directly to the database 30 or via the generative model 20. Figure 4 shows an example of the first data included in the expert knowledge data of this embodiment, and Figure 5 shows an example of the second data included in the expert knowledge data of this embodiment.

[0040] In the example shown in Figure 4, the first data set is a file that associates bibliographic information with a recommendation list. The bibliographic information includes ID (IDentification), author, title, publisher, and publication year. The recommendation list includes theme, reason for recommendation, and introduction. The theme is defined by the book expert mentioned above. The books indicated by the bibliographic information are books selected (recommended books) by the book expert mentioned above based on this theme. The reason for recommendation is the reason why the book expert mentioned above recommends the books indicated by the bibliographic information. The introduction is an introduction to the books indicated by the book expert mentioned above.

[0041] In the example shown in Figure 4, the first data is presented in a tabular format with bibliographic information as the key, but the data format is not limited to this. For example, the first data could be expressed in a tabular format with the theme as the key. If a book expert introduces multiple books on a theme they have defined, the first data would be in a format where the bibliographic information and recommendation content (reasons for recommendation and introduction) of each of the introduced books are associated with each theme. Furthermore, the first data could be expressed structurally, for example, in JSON format, rather than in a tabular format.

[0042] In the example shown in Figure 5, the second data set consists of data (files) that associate queries with correct answers. The queries are vaguely remembered information about books, such as titles that visitors might ask about when a librarian is performing reference work. The correct answers are the information about the book that the librarian correctly identified from the associated query (the book the visitor was looking for), specifically bibliographic information such as author, title, publisher, and publication year. For example, for the vaguely remembered information, "And Noda has no beard," the correct answer is the title "Rough and wild, but not vulgar" by Saburo Shiroyama. The meanings of the two titles are completely different, but because they sound similar, it is thought that the visitor vaguely remembered "Rough and wild, but not vulgar" by its sound and understood it to be a book with a title like "And Noda has no beard."

[0043] Regarding the second set of data, the operator of the book selection service system 1 may create it independently, or they may use publicly available data from external sources. Examples of publicly available data from external sources include the collection of misremembered titles published by the Fukui Prefectural Library (http: / / www.library-archives.pref.fukui.lg.jp / tosyo / category / shiraberu / 368.html).

[0044] Furthermore, the administrator terminal 10 inputs (sets) a system prompt (an example of a second prompt) to the generation model 20 that contains instructions for the generation model 20 to perform librarian functions. In this embodiment, in order to have the generation model 20 perform reference services, the administrator terminal 10 sets the role, the use of RAG, and the output format, and also sets a system prompt to the generation model 20 that applies Few-shot Learning, in order for the generation model 20 to perform librarian functions.

[0045] Figure 6 shows an example of a system prompt in this embodiment. In the prompt shown in Figure 6, code 51 indicates a prompt to set a role, code 53 indicates a prompt to apply fusion learning, code 55 indicates a prompt to set the use of RAG, and code 57 indicates a prompt to set the output format. The prompt indicated by code 51 is a prompt to set the role of performing reference as a librarian. The prompt indicated by code 53 is a prompt to set a specific example of reference as fusion learning. The prompt indicated by code 55 is a prompt to use (reference) the files "ReferenceBooks" and "MistakenBooks" as RAG when performing reference. Note that the file "ReferenceBooks" corresponds to the first data, and the file "MistakenBooks" corresponds to the second data. In addition to setting the output format, the prompt indicated by code 57 includes instructions to prohibit hallucination and instructions to verify the reliability of the reference execution results in the second and third paragraphs. Note that you may include either one of these instructions in the prompt, or you may omit both.

[0046] The reception unit 401 receives prompts, which are instruction statements for the generation model 20, entered by the user using an input device 45 or the like. Specifically, the reception unit 401 receives user prompts (an example of a first prompt) that have instructions for executing a task related to a book selection service (an example of a specific field). The task may include suggesting books that are expected to satisfy the user's instructions based on the user's instructions. For example, a reference may suggest books that are expected to satisfy at least one of the following conditions: fragmented information from the user, vaguely remembered information from the user, and the user's purpose.

[0047] Suggesting books that are expected to satisfy a user based on fragmented or vaguely remembered information could involve, for example, finding the book the user intended based on vaguely remembered or fragmented titles, as mentioned above. Suggesting books that are expected to satisfy a user based on their objectives could involve, for example, recommending books that would be helpful in achieving a user's objective, such as researching the origins of Japanese era names.

[0048] The input unit 403 inputs the prompt received by the reception unit 401 into the generation model 20. Specifically, the input unit 403 inputs the user prompt received by the reception unit 401 into the generation model 20.

[0049] When the generative model 20 receives a prompt from the information processing device 40 that has an instruction to execute a task, it executes the task based on the prompt and the expert knowledge data stored in the database 30. For example, the generative model 20 executes the task by referring to the expert knowledge data stored in the database 30.

[0050] More specifically, the generative model 20 performs a reference based on expert knowledge data stored in the database 30 and prompts including user prompts input from the information processing device 40 and system prompts set by the administrator terminal 10. Specifically, when a user prompt is input from the information processing device 40, the generative model 20 refers to the expert knowledge data, extracts the portion related to the content of the reference included in the user prompt as context, includes it in the user prompt, and performs a reference according to the user prompt, system prompts, and learned knowledge.

[0051] The acquisition unit 405 acquires the execution result of a task performed by the generation model 20 based on the expert knowledge data stored in the database 30 and prompts that have at least an instruction to execute a task related to a specific field, which are input by the input unit 403.

[0052] For example, when the generative model 20 receives a user prompt with the instruction to perform a reference, such as "Tell me the fairy tale about kissing a frog and turning him into a prince," it refers to the expert knowledge data, extracts the part related to the requested reference content as context, and includes it in the user prompt. The generative model 20 then performs the reference using the user prompt, system prompt, and learned knowledge, including the context. For example, as a result of performing the reference, the generative model 20 generates the answer, "Isn't the fairy tale you're looking for 'The Frog Prince' by the Brothers Grimm?", and the retrieval unit 405 retrieves this answer.

[0053] The verification unit 407 verifies the reliability of the task execution results obtained by the acquisition unit 405. The verification unit 407 verifies, for example, whether the generative model 20 executed the task without hallucination by performing a web search. For example, in the case of the above-mentioned answer, "Isn't the fairy tale you're looking for 'The Frog Prince' by the Brothers Grimm?", the verification unit 407 performs a web search to verify whether a book called "The Frog Prince" by the Brothers Grimm actually exists. As mentioned above, the verification unit 407 may be omitted.

[0054] The output unit 409 outputs the task execution results acquired by the acquisition unit 405. The output unit 409 outputs the task execution results acquired by the acquisition unit 405 to a display device 46, for example. The output unit 409 may output the task execution results if the reliability of the task execution results is confirmed by the verification unit 407, and may not output the task execution results if reliability is not confirmed. If reliability is not confirmed, the output unit 409 may output a message such as "The book you are looking for was not found" or "We will search again, please provide more detailed information."

[0055] Next, the operation of the book selection service system of this embodiment will be described.

[0056] Figure 7 is a flowchart showing an example of preprocessing performed at the administrator terminal 10 in this embodiment.

[0057] First, the administrator terminal 10 uploads expert knowledge data to the database 30 (step S101) and builds the RAG.

[0058] Next, the administrator terminal 10 sets a system prompt on the generation model 20 that contains instructions for the generation model 20 to perform its librarian function (step S103).

[0059] Note that the processing order of each step in the flowchart shown in Figure 7 is not limited to the example described above. For example, the processing order of step S101 and step S103 may be swapped.

[0060] Figure 8 is a flowchart showing an example of reference processing performed by the information processing device 40 in this embodiment.

[0061] First, the reception unit 401 receives a user prompt that has an instruction to perform a reference regarding the book selection service (step S201).

[0062] Next, the input unit 403 inputs the user prompt received by the reception unit 401 to the generation model 20 (step S203).

[0063] Next, when a user prompt containing an instruction to execute a reference is input, the generative model 20 refers to the expert knowledge data, extracts the portion of the reference content contained in the user prompt as context, includes it in the user prompt, and executes the reference according to the user prompt, system prompts, and learned knowledge (step S205).

[0064] Next, the acquisition unit 405 acquires the result of the reference execution performed by the generation model 20 (step S207).

[0065] Next, the output unit 409 outputs the execution result of the reference acquired by the acquisition unit 405 (step S209). Note that if the verification unit 407 verifies the reliability of the execution result of the reference, the output unit 409 may output the execution result of the reference if reliability is confirmed, and may not output the execution result of the reference if reliability is not confirmed.

[0066] Next, the effects of the book selection service system of this embodiment will be described.

[0067] In the following, we will explain the effectiveness (superiority) of the book selection service system of this embodiment by comparing the reference results of this embodiment, which uses specialized knowledge data, with the reference results of a comparative example that does not use specialized knowledge data.

[0068] The reference for this embodiment, which uses specialized knowledge data, shall be performed as described in this embodiment. Specifically, the first data shown in Figure 4 and the second data shown in Figure 5 shall be used as the specialized knowledge data, and the prompt shown in Figure 6 shall be used as the system prompt. On the other hand, the reference for the comparative example, which does not use specialized knowledge data, shall not use either the specialized knowledge data or the system prompt. Instead, the prompt shown in Figure 6, excluding the prompt for setting the use of RAG shown by reference numeral 55, shall be used as the user prompt, which shall be entered in advance. The remaining conditions, such as the generative model used, shall be the same. Below, the effects (advantages) of the book selection service system of this embodiment will be explained by describing one example from 20 cases verified under these conditions.

[0069] First, we will explain cases where both the reference for this embodiment and the reference for the comparative example were correct. Figure 9 shows an example of the reference result for the comparative example that was correct, and Figure 10 shows an example of the reference result for this embodiment that was correct.

[0070] In the examples shown in Figures 9 and 10, a user prompt (indicated by code 61 in Figure 9 and code 71 in Figure 10) with the instruction to execute the reference "Tell me about a story in Showa-era pure literature about setting fire to a temple in Nara" is input to the generative model 20. In response, in both examples shown in Figures 9 and 10, the generative model 20 answers with Yukio Mishima's "The Temple of the Golden Pavilion" (indicated by code 63 in Figure 9 and code 73 in Figure 10), and both references are correct answers.

[0071] Next, we will describe a case in which the reference for this embodiment was correct, but hallucination occurred in the reference for the comparative example. Figure 11 shows an example of the reference result for the comparative example in which hallucination occurred, and Figure 12 shows an example of the reference result for this embodiment, which was correct.

[0072] In the examples shown in Figures 11 and 12, the generative model 20 is input a user prompt (indicated by code 65 in Figure 11 and code 75 in Figure 12) that has the instruction to execute the reference "Tell me the fairy tale about kissing a frog and turning him into a prince." In response, in the example shown in Figure 11, the generative model 20 answers with Kamogawa's "The Frog King," as indicated by code 67, but such a book does not exist, resulting in hallucination. On the other hand, in the example shown in Figure 12, the generative model 20 answers with the Brothers Grimm's "The Frog King," as indicated by code 77, which is the correct answer.

[0073] Furthermore, there were no instances in the reference of this embodiment where hallucination occurred, while in the comparative example reference where it was the correct answer.

[0074] Figure 13 shows an example of the reference results for both this embodiment and the comparative example for 20 cases that were verified. As shown in Figure 13, in the reference results for the comparative example, which does not use specialized knowledge data, out of 20 cases, 10 cases were answered correctly on the first attempt, 6 cases were answered correctly on subsequent attempts, and 4 cases resulted in hallucination. On the other hand, in the reference results for this embodiment, which uses specialized knowledge data, out of 20 cases, 12 cases were answered correctly on the first attempt, 6 cases were answered correctly on subsequent attempts, and 2 cases resulted in hallucination. Cases where the answer was correct on subsequent attempts refer to cases where the first answer was incorrect, but the answer was for an existing book and did not cause hallucination, so the questioning continued and the answer was correct on the second or subsequent attempts.

[0075] As described above, compared to the reference results of the comparative example that did not use specialized knowledge data, the reference results of this embodiment that used specialized knowledge data showed a reduction of approximately 10% in hallucination, indicating that the decrease in response accuracy was suppressed.

[0076] Therefore, according to this embodiment, even when a generative model is made to perform a task related to a specific field, it is possible to suppress a decrease in the accuracy of the response.

[0077] (Variation 1) In the above embodiment, a reference service was used as an example of a book selection service. However, in Modification 1, bibliotherapy (reading therapy) is used as another example of a book selection service.

[0078] Bibliotherapy is a type of psychotherapy that involves suggesting (prescribing) books that are expected to have a psychologically supportive effect on the user. Bibliotherapists, like librarians, are experts in books and possess a considerable understanding of not only the bibliographic information of a vast number of books but also their content, enabling them to provide bibliotherapy. Thus, bibliotherapy is a service that relies on the specialized knowledge of the bibliotherapist.

[0079] However, even if we were to have a generative AI perform the bibliophile therapy described above, the generative AI would likely suffer from hallucination, producing answers that do not actually exist, because it lacks sufficient data to represent the specialized knowledge of a bibliophile.

[0080] Therefore, Modification 1 describes an example of providing a book bibliophile therapy service using specialized knowledge data, which is data of knowledge professionally used in book selection services, in addition to the generating AI. In Modification 1, the parts that differ from the above embodiment will be mainly described, and the parts that are the same as the above embodiment will not be explained.

[0081] In variation 1, the expert knowledge data only needs to include the first data, but does not need to include the second data.

[0082] In variation 1, the administrator terminal 10 inputs (sets) a system prompt (an example of a third prompt) into the generation model 20 that contains instructions for the generation model 20 to perform its function as a bibliotherapist.

[0083] Figure 14 shows an example of a system prompt in Modification 1. In the prompts shown in Figure 14, reference numeral 81 and 85 indicate prompts to set the use of RAG, reference numeral 83 indicates a prompt to set a role, and reference numeral 87 indicates a prompt to set the output format. The prompts indicated by reference numeral 81 and 85 are prompts to set the use (reference) of the file "ReferenceBooks" as RAG when performing reference. The file "ReferenceBooks" corresponds to the first data. The prompt indicated by reference numeral 83 is a prompt to set the role of performing bibliotherapy as a bibliotherapist. In addition, the prompt indicated by reference numeral 57 may include, as in the above embodiment, at least one of an instruction to prohibit hallucination and an instruction to verify the reliability of the reference execution result, in addition to setting the output format.

[0084] In variation 1, the task included in the user prompt is bibliotherapy, which involves suggesting books that are expected to have a psychologically supportive effect on the user based on their psychological state. Suggesting books that are expected to have a psychologically supportive effect on the user based on their psychological state could, for example, involve finding a book that will motivate a user who is feeling down.

[0085] The generation model 20 of Modified Example 1 performs bibliotherapy based on expert knowledge data stored in the database 30 and prompts including user prompts input from the information processing device 40 and system prompts set by the administrator terminal 10. Specifically, when a user prompt is input from the information processing device 40, the generation model 20 of Modified Example 1 refers to the expert knowledge data, extracts the portion related to the content of bibliotherapy included in the user prompt as context, includes it in the user prompt, and performs bibliotherapy according to the user prompt, system prompts, and learned knowledge.

[0086] For example, when generative model 20 receives a user prompt input containing a bibliotherapy instruction such as "I'm feeling down, so please recommend a book that will help me feel more positive," it refers to expert knowledge data, extracts the relevant portion of the requested bibliotherapy content as context, and includes it in the user prompt. Generative model 20 then performs the bibliotherapy using the user prompt, system prompt, and learned knowledge, including the context.

[0087] Below, we will explain the effectiveness (superiority) of the book selection service system of Modification 1 by comparing the bibliotherapy results of this embodiment, which uses specialized knowledge data, with the bibliotherapy results of a comparative example that does not use specialized knowledge data.

[0088] Furthermore, the bibliotherapy in Modification 1, which uses specialized knowledge data, will be performed as described in Modification 1. Specifically, the first data shown in Figure 4 will be used as the specialized knowledge data, and the prompts shown in Figure 14 will be used as the system prompts. On the other hand, the bibliotherapy in the Comparative Example, which does not use specialized knowledge data, will not use either the specialized knowledge data or the system prompts. Instead, the user prompts will be pre-entered as the prompts shown in Figure 14, excluding the prompts for setting the use of RAG shown by reference numerals 81 and 85. The remaining conditions, such as the generative model used, will be the same. Below, the effects (advantages) of the book selection service system of this embodiment will be explained by describing an example of a case in which verification was performed under these conditions.

[0089] Figure 15 shows an example of the bibliotherapy results for the comparative example, and Figure 16 shows an example of the bibliotherapy results for the modified example 1. In both the examples shown in Figure 15 and Figure 16, a user prompt (indicated by code 91 in Figure 15 and code 95 in Figure 16) containing the instruction to perform bibliotherapy, "I am a salaried worker in my 50s. I do office work at a large company. When I have many long working days in a row, I tend to feel depressed even on my days off. Please list three books that will brighten my mood and help me maintain a positive attitude when I read them," is input to the generative model 20. In response to this, in the example shown in Figure 15, the generative model 20 answers "The Four Promises" by Don Miguel Ruiz, as indicated by code 93. On the other hand, in the example shown in Figure 16, the generative model 20 answers "The Hospitality Division of the Prefectural Government" by Hiro Arikawa, as indicated by code 97.

[0090] While a detailed explanation is omitted, in Modification 1, compared to the Bibliotherapy results of the Comparative Example without using specialized knowledge data, the occurrence of hallucination was suppressed and the decrease in response accuracy was mitigated in the Bibliotherapy results of Modification 1 using specialized knowledge data. Furthermore, compared to the Bibliotherapy results of the Comparative Example without specialized knowledge data, it appears that the Bibliotherapy results of Modification 1 using specialized knowledge data were able to suggest books that were expected to be effective in improving the user's psychological state.

[0091] Therefore, even in Modification 1, it is possible to suppress a decrease in response accuracy even when the generative model is made to perform tasks related to a specific field. In particular, in both the above embodiment and Modification 1, it is possible to suppress a decrease in response accuracy, and it has been confirmed that the first data included in the expert knowledge data, that is, data in which one or more recommended books and their recommendations are listed for each of several arbitrary themes, is useful data for selecting books.

[0092] (Modification 2) In the above embodiment and modification 1, an example of using expert knowledge data as RAG was described, but the invention is not limited to this. For example, the expert knowledge data may be used to fine-tune the generative model 20, or the expert knowledge data may be included in the prompts. In this way, it is expected that the same effects as in the above embodiment and modification 1 will be achieved.

[0093] (Variation 3) In the above embodiments and their variations, hallucination was described assuming that it refers to providing information that does not actually exist. However, it is not limited to this, and hallucination may also include providing information that is not factual.

[0094] (Modification 4) In the above embodiments and their variations, examples were described in which the administrator terminal 10 is used to build a RAG using expert knowledge data and to set system prompts. However, the invention is not limited to this, and at least one of these may be performed by the information processing device 40.

[0095] (Variation 5) In the above embodiments and their variations, the example of implementing the generation model 20 as a cloud service was used for explanation, but the generation model 20 may also be included in the information processing device 40. In this case, the generation model 20 can be implemented, for example, as an AI chat service that fine-tunes a small language model (SLM).

[0096] (program) The programs executed by the information processing device and administrator terminal of the above embodiments and each of the above modifications are provided as installable or executable files stored on a computer-readable storage medium such as a CD-ROM, CD-R, memory card, DVD, or flexible disk (FD).

[0097] Furthermore, the programs executed by the information processing device and administrator terminal of the above embodiments and each of the modified versions may be stored on a computer connected to a network such as the Internet and provided by downloading them via the network. Alternatively, the programs executed by the information processing device and administrator terminal of the above embodiments and each of the modified versions may be provided or distributed via a network such as the Internet. Furthermore, the programs executed by the information processing device and administrator terminal of the above embodiments and each of the modified versions may be pre-installed in ROM or the like and provided.

[0098] The information processing device and administrator terminal executed in the above embodiments and their respective modifications have a modular configuration for realizing the above-described parts on a computer. In actual hardware, for example, the CPU reads the program from the HDD into RAM and executes it, thereby realizing the above-described parts on the computer.

[0099] The above embodiments and their respective modifications are merely examples of how this disclosure may be implemented, and they do not restrict the technical scope of this disclosure. Therefore, this disclosure can be implemented in various ways without departing from its essence or its main features. For example, the above embodiments and their respective modifications may be combined as appropriate on a component basis. Also, for example, some components may be removed from the total components in the above embodiments and their respective modifications.

[0100] This disclosure also includes the following aspects:

[0101] (1) An acquisition unit that acquires the execution result of a task performed by a generative model based on expert knowledge data, which is data of knowledge used professionally in a specific field, and a prompt which includes at least a first prompt having an instruction to perform a task related to the specific field; An output unit that outputs the execution result of the aforementioned task, An information processing device equipped with the following features.

[0102] (2) The aforementioned expert knowledge data includes, for each of several arbitrary themes, a first data set in which one or more recommended books are listed. The information processing device described in (1) above.

[0103] (3) The first data further includes the recommendation content for each of the one or more recommended books. The information processing device described in (2) above.

[0104] (4) The task is to suggest a book that is expected to satisfy the user's instructions, based on the user's instructions. An information processing device as described in any one of the above items (1) to (3).

[0105] (5) The task is a reference that proposes books that are expected to satisfy at least one of the following conditions: fragmentary information of the user, vaguely remembered information of the user, and the user's purpose. The information processing device described in (4) above.

[0106] (6) The prompt further includes a second prompt which has instructions for causing the generative model to perform librarian functions, The information processing device described in (5) above.

[0107] (7) The second prompt further includes at least one of the following: an instruction to prohibit hallucination and an instruction to verify the reliability of the result of performing the reference. The information processing device described in (6) above.

[0108] (8) The aforementioned expert knowledge data further includes second data which associates the bibliographic information of the book with vaguely remembered information about the book, The information processing device described in (5) above.

[0109] (9) The task is a bibliotherapy in which the user is asked to propose a book that is expected to have a psychologically supportive effect on the user based on the user's psychological state. The information processing device described in (4) above.

[0110] (10) The prompt further includes a third prompt for causing the generative model to function as a bibliotherapist, The information processing device described in (9) above.

[0111] (11) The third prompt further includes at least one of the following: an instruction to prohibit hallucination and an instruction to verify the reliability of the results of the vibriotherapy. The information processing device described in (10) above.

[0112] (12) The system further comprises a verification unit for verifying the reliability of the results of the execution of the task, The output unit outputs the execution result of the task if the reliability is confirmed. The information processing device described in (1) above.

[0113] (13) The aforementioned expert knowledge data is stored in a database, The system further includes an input unit that inputs the first prompt to the generation model, The generative model performs the task by referring to the expert knowledge data. The information processing device described in (1) above.

[0114] (14) An acquisition step in which the acquisition unit acquires the result of an execution of a task performed by a generation model based on expert knowledge data which is data of knowledge used professionally in a specific field and a prompt which includes at least a first prompt having an instruction to execute a task relating to the specific field, The output unit performs an output step that outputs the execution result of the task, Information processing methods including

[0115] (15) An acquisition step to acquire the result of a task performed by a generative model based on expert knowledge data, which is data of knowledge used professionally in a specific field, and a prompt which includes at least a first prompt having instructions to perform a task relating to the specific field; An output step that outputs the execution result of the aforementioned task, A program that causes a computer to execute something. [Explanation of Symbols]

[0116] 1. Book Selection Service System 10 Administrator terminal 20 Generative Models 30 databases 40 Information Processing Devices 41 Control device 42 Main storage 43 Auxiliary storage device 44 Communication equipment 45 Input device 46 Display device 49 Various types of buses 401 Reception Desk 403 Input section 405 Acquisition Department 407 Verification Department 409 Output section

Claims

1. An acquisition unit that acquires the execution result of a task performed by a generative model based on expert knowledge data, which is data of knowledge used professionally in a specific field, and a prompt that includes at least a first prompt having an instruction to execute a task related to the specific field, An output unit that outputs the execution result of the aforementioned task, An information processing device equipped with the following features.

2. The aforementioned expert knowledge data includes, for each of several arbitrary themes, a first data set in which one or more recommended books are listed. The information processing apparatus according to claim 1.

3. The first data further includes the recommendation content for each of the one or more recommended books. The information processing apparatus according to claim 2.

4. The task is to suggest books that are expected to satisfy the user's instructions, based on the user's instructions. The information processing apparatus according to any one of claims 1 to 3.

5. The task is to provide a reference that suggests books expected to satisfy at least one of the following conditions: fragmentary information of the user, vaguely remembered information of the user, and the user's purpose. The information processing apparatus according to claim 4.

6. The prompt further includes a second prompt which has instructions to cause the generative model to perform librarian functions. The information processing apparatus according to claim 5.

7. The second prompt further includes at least one of the following: an instruction to prohibit hallucination, and an instruction to verify the reliability of the result of performing the reference. The information processing apparatus according to claim 6.

8. The aforementioned expert knowledge data further includes a second set of data that associates the bibliographic information of a book with vaguely remembered information about that book. The information processing apparatus according to claim 5.

9. The aforementioned task is a bibliotherapy method in which a book is proposed that is expected to have a psychologically supportive effect on the user based on the user's psychological state. The information processing apparatus according to claim 4.

10. The prompt further includes a third prompt to cause the generative model to function as a bibliotherapist. The information processing apparatus according to claim 9.

11. The third prompt further includes at least one of the following: an instruction to prohibit hallucination, and an instruction to verify the reliability of the results of the vibriotherapy. The information processing apparatus according to claim 10.

12. The system further includes a verification unit for verifying the reliability of the execution results of the aforementioned task, The output unit outputs the execution result of the task if the reliability is confirmed. The information processing apparatus according to claim 1.

13. The aforementioned expert knowledge data is stored in a database. The system further includes an input unit that inputs the first prompt to the generation model, The generative model performs the task by referring to the expert knowledge data. The information processing apparatus according to claim 1.

14. The acquisition step involves an acquisition unit acquiring the result of a task performed by a generative model based on expert knowledge data, which is data of knowledge used professionally in a specific field, and a prompt that includes at least a first prompt having an instruction to perform a task related to the specific field. The output unit performs an output step that outputs the execution result of the task, Information processing methods including

15. An acquisition step to acquire the result of a task performed by a generative model based on expert knowledge data, which is data of knowledge used professionally in a specific field, and a prompt which includes at least a first prompt having instructions to perform a task related to the specific field; An output step that outputs the execution result of the aforementioned task, A program that causes a computer to execute something.

Images