Information process system, information processing method, and program

Abstract

To improve the quality of responses based on the output of a language model while taking privacy protection into consideration.SOLUTION: An information processing system is configured to: acquire text including one or a plurality of pieces of privacy information; determine query information including at least a part of the one or the plurality of pieces of privacy information contained in the text on the basis of the text; search a knowledge database to acquire knowledge information on the basis of the query information; replace each of the one or the plurality of pieces of privacy information with concealment information different from the one or the plurality of pieces of privacy information; acquire a response from a language model by inputting the text where the privacy information has been replaced and the knowledge information into the language model; and output information based on the acquired response.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to an information processing system, an information processing method, and a program.

Background Art

[0002] The performance of language models such as large language models (LLMs) has been significantly improved. Large language models can answer questions in natural language. Large language models are used, for example, in chatbots such as ChatGPT (registered trademark).

[0003] Patent Document 1 discloses a technique for generating a summary text desired by a user using a large language model obtained by machine learning a vast amount of unlabeled text.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] The inventor is considering a system that answers questions using a language model provided as a service. Depending on the physical arrangement of such a language model and the policies of the service provider, it may be difficult to input information related to personal privacy protection. On the other hand, if such information is not input into the language model, the quality of the answers output by the language model may decrease.

[0006] The present disclosure provides a technique for improving the quality of answers based on the output of a language model while taking privacy protection into consideration.

Means for Solving the Problems

[0007] (1) An information processing system including: a query acquisition means for acquiring a query including one or more pieces of privacy information based on an input from a user; a concealment means for replacing each of the one or more pieces of privacy information included in the query with concealment information different from the one or more pieces of privacy information; an answer acquisition means for requesting a language model to create an answer to the query with the privacy information replaced, and acquiring an answer from the language model; a restoration means for replacing the concealment information included in the acquired answer with the privacy information replaced by the concealment information in the query; and an answer means for sending information based on the answer with the concealment information replaced to the user.

[0008] (2) The information processing system according to (1), wherein the concealment means replaces each of the one or more pieces of privacy information included in the query with concealment information corresponding to the type of the privacy information.

[0009] (3) The information processing system according to (2), wherein the types of the one or more pieces of privacy information include at least a part of name, age, gender, and medical history.

[0010] (4) The information processing system according to any one of (1) to (3) further includes an association management means for storing each of the one or more pieces of privacy information in association with the concealment information replaced with the privacy information in a database, and the restoration means replaces the concealment information with the privacy information associated with the concealment information when the answer includes the concealment information associated with any of the one or more pieces of privacy information.

[0011] (5) In any one of (1) to (4), when the inquiry includes the age of the user, range selection means for selecting an age range to which the age of the user belongs from a plurality of predetermined age ranges, and age selection means for selecting any one of a plurality of ages within the selected age range as a masked age for the age of the user, and the masking means replaces the age of the user included in the inquiry with the masked age for the age of the user, an information processing system.

[0012] (6) In (5), the age selection means randomly selects any one of a plurality of ages within the selected age range as a masked age for the age of the user, an information processing system.

[0013] (7) In (5) or (6), further including basis acquisition means for acquiring, based on at least a part of the one or more pieces of privacy information, any one of a plurality of answer basis information that targets the age range to which the age of the user belongs, and the answer acquisition means requests a language model to create an answer to the question in which the privacy information is replaced, and acquires an answer from the language model, an information processing system.

[0014] (8) A step of obtaining an inquiry including one or more pieces of privacy information based on an input from a user, a step of replacing each of the one or more pieces of privacy information included in the inquiry with masked information different from the one or more pieces of privacy information, a step of requesting a language model to create an answer to the inquiry in which the privacy information is replaced, and acquiring an answer from the language model, a step of replacing the masked information included in the acquired answer with the privacy information replaced with the masked information in the inquiry, and a step of sending information based on the answer in which the masked information is replaced to the user, an information processing method.

[0015] A program for causing a computer to function as follows: a query acquisition means for acquiring a query including one or more pieces of privacy information based on an input from a user; a masking means for replacing each of the one or more pieces of privacy information included in the query with masking information different from the one or more pieces of privacy information; a response acquisition means for requesting a language model to create a response to the query with the privacy information replaced and acquiring the response from the language model; a restoration means for replacing the masking information included in the acquired response with the privacy information replaced with the masking information in the query; and a response means for sending information based on the response with the masking information replaced to the user.

Advantages of the Invention

[0016] According to the present invention, it is possible to improve the quality of responses based on the output of a language model while considering privacy protection.

Brief Description of the Drawings

[0017]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Best Mode for Carrying Out the Invention

[0018] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Duplicate descriptions for components with the same reference numerals will be omitted.

[0019] FIG. 1 is a diagram showing an example of elements related to an inquiry management system 2 according to an embodiment of the present invention. The inquiry management system 2 receives an inquiry including information related to user privacy protection (privacy information) from a user terminal 1 operated by a user, inputs an instruction based on the inquiry to a large language model service 3, and outputs an answer according to the text output by the large language model service 3 to the user terminal 1. Hereinafter, as an example of the inquiry management system 2, the inquiry management system 2 mainly for recommending insurance products to users will be described.

[0020] The user terminal 1 is a computer having a user interface, such as a personal computer or a smartphone.

[0021] The large-scale language model service 3 includes a general-purpose large-scale language model implemented by a computer. The large-scale language model service 3 receives an instruction from the inquiry management system 2 and delivers the output obtained by inputting the instruction into the large-scale language model to the inquiry management system 2. This instruction is in text format and is also called a prompt. Hereinafter, the text-format instruction among these instructions is also described as an instruction text. This general-purpose large-scale language model is learned from data in a wide range of fields. The large-scale language model service 3 may be a service such as ChatGPT (registered trademark).

[0022] Hereinafter, when simply described as the "large-scale language model", it refers to the large-scale language model included in the large-scale language model service 3. The inquiry management system 2 executes the process of inputting information into the large-scale language model and obtaining the output from the large-scale language model by using the API provided by the large-scale language model service 3. The inquiry management system 2 does not necessarily input all the information in a single API call, and may input the information in parts by a plurality of API calls. Note that the large-scale language model service 3 may be provided inside the inquiry management system 2. In the present embodiment, the inquiry management system 2 inputs information for requesting the creation of some kind of answer to the large-scale language model and obtains the output of the large-scale language model as the answer. Hereinafter, inputting information for requesting the creation of some kind of answer to the large-scale language model is also described as requesting the large-scale language model to create an answer.

[0023] The inquiry management system 2 includes one or more computers (such as server computers). The inquiry management system 2 includes one or more processors 21, one or more storages 22, and one or more communication units 23. The inquiry management system 2 may include a plurality of computers each including one or more processors 21, storages 22, and communication units 23, or may include one computer having one or more processors 21 and storages 22. Note that the inquiry management system 2 may be implemented on one or more virtual servers or container platforms.

[0024] The processor 21 operates according to a program (also referred to as instruction code) stored in the storage 22. The processor 21 also controls the communication unit 23. The processor 21 includes, for example, a CPU (Central Processing Unit), and may further include a GPU (Graphic Processing Unit) and an NPU (Neural Processing Unit). Note that the above program may be provided via the Internet or the like, or may be provided by being stored in a computer-readable storage medium such as a flash memory or a DVD-ROM.

[0025] The storage 22 is composed of memory elements such as RAM and flash memory, and external storage devices such as a hard disk drive (HDD) and a solid state drive (SSD). The storage 22 stores the above program. The storage 22 also stores information and calculation results input from the processor 21 and the communication unit 23.

[0026] The communication unit 23 is a communication interface for communicating with other devices, such as a network interface card. The communication unit 23 is composed of, for example, an integrated circuit, an antenna, a communication terminal, etc. that implement a wireless LAN or a wired LAN. Based on the control of the processor 21, the communication unit 23 inputs information received from other devices via a network to the processor 21 and the storage 22, and transmits information to other devices.

[0027] Note that the hardware configuration of the inquiry management system 2 is not limited to the above example. For example, the inquiry management system 2 may include a device that reads a computer-readable information storage medium (e.g., an optical disk drive or a memory card slot), or a device that inputs and outputs data with an external device (e.g., a USB port). The external device may be an input device or an output device.

[0028] Next, the functions provided by the inquiry management system 2 will be described. FIG. 2 is a block diagram showing the functions realized by the inquiry management system 2. Functionally, the inquiry management system 2 includes a management unit 50 and a knowledge database 60. Further, the management unit 50 functionally includes an input acquisition unit 51, a privacy management unit 52, a concealment unit 53, a knowledge acquisition unit 54, an answer acquisition unit 55, a restoration unit 56, and an answer output unit 57. The privacy management unit 52 functionally includes a privacy extraction unit 58 and an association management unit 59. The management unit 50 and the knowledge database 60 are realized by the processor 21 executing a program corresponding to each function stored in the storage 22 and controlling the communication unit 23 and the like.

[0029] The knowledge database 60 is a database in which knowledge information in a certain knowledge field is stored. The knowledge database 60 acquires information indicating the characteristics of the input from the user from the management unit 50, searches for knowledge information corresponding to the information, and delivers the knowledge information found by the search to the management unit 50. In the knowledge database 60, the knowledge information and the feature vector serving as the index of the knowledge information are stored in the storage 22 in association with each other. The knowledge database 60 acquires a feature vector serving as a query based on the input from the user and acquires the knowledge information corresponding to the query.

[0030] Here, the knowledge information is a text-formatted document, a link string to a website on the Internet (e.g., a URL). In the present embodiment, the knowledge domain is an insurance product sold by a certain insurance company, and the feature vector serving as the query is generated from the age group, gender, and medical history of the user and the user's family members. In this case, in the knowledge database 60, not only the information of the insurance product as knowledge information and the feature vector of the knowledge information, but also the information indicating the age group, gender, and medical history that are the application conditions (usage conditions of the insurance product) of the knowledge information may be associated and stored in the storage 22. The knowledge database 60 may handle information in other knowledge domains as long as the knowledge can be classified by information related to privacy. The knowledge information is used as information (answer basis information) that serves as the basis when the large language model creates an answer to a question.

[0031] In the search for knowledge information, the knowledge database 60 may search for a feature vector similar to the query from a plurality of feature vectors stored in the storage 22, and output the knowledge information stored in association with the similar feature vector. Also, the query may include at least a part of the privacy information. In this case, the knowledge database 60 may search for knowledge information that satisfies the usage conditions of the query. Further, the knowledge database 60 may search for a feature vector similar to the query among one or a plurality of feature vectors associated with the knowledge information that satisfies the usage conditions of the query. The knowledge database 60 may use, for example, the cosine similarity between the feature vector of the query and the feature vectors in the storage 22 as the similarity. The knowledge database 60 may select the feature vector with the largest similarity as the similar feature vector, and acquire the knowledge information associated with the selected feature vector.

[0032] Here, the feature vector may be generated by a feature extraction model which is a machine learning model. The feature extraction model is trained using document data for training. During training, the feature extraction model extracts the feature vector of the query and the feature vector of the document data corresponding to the query. Then, the feature extraction model is trained so that the similarity between the feature vector of the query and the feature vector of the corresponding document data increases, and the similarity for non-corresponding document data decreases. The document data in training may be only the part corresponding to the topic or question among the knowledge documents. This training may be performed based on the methods shown in the following two papers.

[0033] Paper 1: Wataru Sakata, Tomohide Shibata, Ribeka Tanaka, and Sadao Kurohashi. 2019. FAQ Retrieval using Query-Question Similarity and BERT-Based Query-Answer Relevance. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'19). Association for Computing Machinery, New York, NY, USA, 1113-1116.

[0034] Paper 2: Seo, J.; Lee, T.; Moon, H.; Park, C.; Eo, S.; Aiyanyo, I.D.; Park, K.; So, A.; Ahn, S.; Park, J. Dense-to-Question and Sparse-to-Answer: Hybrid Retriever System for Industrial Frequently Asked Questions. Mathematics 2022, 10, 1335.

[0035] The management unit 50 obtains an inquiry including one or more pieces of privacy information from the user terminal 1, and obtains knowledge information corresponding to the inquiry from the knowledge database 60, while concealing the privacy information included in the inquiry. Further, the management unit 50 requests the large language model service 3 to generate an answer to the inquiry with the privacy information concealed, and outputs an output sentence based on the answer to the user terminal 1 via the network.

[0036] The input acquisition unit 51 obtains an inquiry including one or more pieces of privacy information based on the input from the user. The inquiry may be a sentence or may include information other than a sentence. The input from the user is information that the user operates the user terminal 1 to input and receives from the user terminal 1. The input acquisition unit 51 may obtain the sentence as the input from the user as the inquiry as it is, or may generate a sentence as the inquiry by processing the privacy information and free text obtained interactively as the input from the user.

[0037] The privacy management unit 52 extracts one or more pieces of privacy information included in the inquiry and manages the extracted privacy information. The privacy extraction unit 58 included in the privacy management unit 52 extracts the privacy information included in the inquiry. When the inquiry includes the user's age, the privacy extraction unit 58 selects the age range to which the user's age belongs from a plurality of predetermined age ranges. When the inquiry includes the age of another person, the privacy extraction unit 58 selects the age range to which the single person's age belongs from a plurality of predetermined age ranges.

[0038] The related management unit 59 included in the privacy management unit 52 determines concealment information corresponding to each of the one or more pieces of privacy information included in the inquiry. Further, the related management unit 59 associates each of the one or more pieces of privacy information included in the inquiry with the concealment information corresponding to that privacy information and stores it in the storage 22. The concealment information is information that replaces the privacy information included in the inquiry. The concealment information may be information (tags or dummy information) corresponding to the type of privacy information (e.g., name, age, gender, medical history).

[0039] In determining the concealment information, when the inquiry includes the user's age, the related management unit 59 may select any one (which may be random) of a plurality of ages within the age range selected by the privacy extraction unit 58 as the concealed age for the user's age. When the inquiry includes the ages of the user and other persons and they belong to the same age range, the related management unit 59 may select the concealed age so that the concealed age for the user does not overlap with the concealed age for the other persons.

[0040] The concealment unit 53 replaces each of the one or more pieces of privacy information included in the inquiry with concealment information different from that privacy information. When the inquiry includes the user's age, the concealment unit 53 may replace the user's age included in the inquiry with the concealed age for that user. When the inquiry includes the age of another person, the concealment unit 53 may replace the age of the other person included in the inquiry with the concealed age for that other person.

[0041] The knowledge acquisition unit 54 sends a query based on an inquiry to the knowledge database 60, and acquires knowledge information retrieved from the knowledge database 60 based on the query. When the inquiry includes the user's age, the knowledge acquisition unit 54 acquires, from the knowledge database 60, knowledge information targeted at the age range to which the user's age belongs, from among a plurality of pieces of knowledge information stored in the storage 22. Targeting an age range may mean that the age range is included within the application conditions of the knowledge information, or that a character string indicating that the age range is targeted is included within the knowledge information.

[0042] The answer acquisition unit 55 requests a large language model to create an answer to the inquiry with the privacy information replaced, and acquires the answer from the large language model. The answer may be in the form of text. Here, the answer acquisition unit 55 may request a large language model to create an answer to the inquiry with the privacy information replaced based on the knowledge information. This large language model may be connected to the inquiry management system 2 via the Internet. Note that when the answer acquired from the large language model includes an inappropriate character string (e.g., a URL), the answer acquisition unit 55 may filter out that character string from the answer and use the filtered answer for subsequent processing.

[0043] The restoration unit 56 replaces the concealed information included in the answer acquired from the large language model with the privacy information replaced by the concealed information in the inquiry. More specifically, when the answer includes concealed information associated with any one of the one or more pieces of privacy information included in the inquiry, the restoration unit 56 replaces the concealed information with the privacy information stored in the storage 22 associated with the concealed information.

[0044] The answer output unit 57 sends information based on the answer in which the concealed information is replaced to the user. More specifically, the answer output unit 57 may process the answer in which the concealed information is replaced and send the processed answer to the user. The processing may be formatting the format of the text included in the answer, or adding a hyperlink to the knowledge information included in the answer or information related thereto. Further, the answer output unit 57 may determine whether the replaced or processed answer is consistent with the acquired knowledge information. When it is determined that they are consistent, the answer output unit 57 may add a hyperlink to the knowledge information or information related thereto to the answer, and when it is determined that they are not consistent, the answer output unit 57 may add a warning message to the answer. Here, the answer output unit 57 may determine the consistency based on whether each of a plurality of words included in the answer is included in the words included in the knowledge information.

[0045] FIG. 3 is a flowchart showing an example of the processing of the inquiry management system 2. The processing of the management unit 50 is mainly described in FIG. 3.

[0046] First, the input acquisition unit 51 acquires an inquiry sentence including one or more pieces of privacy information based on the information input from the user terminal 1 based on the user's operation (S101). The inquiry sentence is mainly an inquiry including text.

[0047] FIG. 4 is a diagram showing an example of an input screen to the inquiry management system 2. The screen shown in FIG. 4 is output to the user terminal 1. On the screen of FIG. 4, an inquiry sentence input by the user is displayed. In the example of FIG. 4, the inquiry sentence is a description of the user's self and family members input by the user to select an appropriate insurance. In the example of FIG. 4, the input acquisition unit 51 acquires the sentence input in one input field as the inquiry sentence. The input acquisition unit 51 may further detect whether there are any missing items in the sentences input so far, and if an item is detected, output information prompting the input of the detected item to the user terminal 1. In this case, the input acquisition unit 51 may acquire the sentence input later and generate an inquiry sentence from the sentences input so far. Note that the inquiry information may include other privacy information such as an image of the user's face.

[0048] Next, the privacy extraction unit 58 identifies the privacy information included in the inquiry sentence (S102). The privacy extraction unit 58 also identifies the types of privacy information included in the inquiry sentence. In the example of this embodiment, the types of privacy information include the user's name, age, gender, medical history, married / unmarried, and phone number, and the family members' names, ages, genders, medical histories, married / unmarried, and phone numbers. The types of privacy information actually handled may be a part of this, or different classifications may be made. The privacy extraction unit 58 may extract proper nouns and nouns from the inquiry sentence by morphological analysis, and identify the privacy information and types by comparing the extracted proper nouns and nouns with a dictionary created in advance. Also, the privacy information may be identified based on the output when the inquiry sentence is input to a machine learning model learned by learning data including the sentence, the position of the privacy information included in the sentence, and correct data including the type of the privacy information.

[0049] FIG. 5 is a diagram showing an example of privacy information extracted from an inquiry sentence. In the example of FIG. 5, the "Field" column indicates the type of privacy information, and the "value" column indicates the privacy information itself. FIG. 5 may further include information indicating the position of the privacy information in the inquiry sentence, or the types of privacy information may be subdivided according to the classification of users and family relationships.

[0050] When the privacy information is specified, the privacy extraction unit 58 converts the privacy information according to its type and determines query information (S103). When the privacy information specified from the inquiry sentence includes the user's age (including privacy information of which the type is the user's age), the privacy extraction unit 58 selects the age range to which the user's age belongs from a plurality of predetermined age ranges. The plurality of age ranges are set, for example, as 0 - 5 years old, 6 - 17 years old, 18 - 19 years old, 20 - 29 years old, 30 - 39 years old, 40 - 49 years old, 50 - 59 years old, 60 - 69 years old so as to have no overlap or gap. The privacy extraction unit 58 may convert gender or disease into codes.

[0051] FIG. 6 is a diagram showing an example of a query based on privacy information. In the example of FIG. 6, the query includes items of an age range (age_range), a code indicating gender (gender_code), and a code indicating disease (disease_code).

[0052] The association management unit 59 determines the masking information corresponding to each of the one or more privacy information included in the inquiry sentence, associates them, and stores them in the storage 22 (S104). The association management unit 59 determines the masking information corresponding to the privacy information according to the type of the privacy information. Also, each of the privacy information is stored in the storage 22 in association with the corresponding masking information. The association management unit 59 may determine masking information different from the privacy information for some of the types among the one or more privacy information included in the inquiry sentence.

[0053] When the inquiry sentence includes privacy information about the age of the user or another person (including privacy information whose type is the age of the user or another person), the privacy extraction unit 58 selects any one of a plurality of ages within the age range selected for that age as the concealed age for that age. This concealed age is specifically related to age among the concealed information. When the specified privacy information is the name, address, or phone number of the user or another person (the type is the name, address, or phone number of the user or another person), the related management unit 59 may determine a dummy string such as a name associated with that type in advance as the concealed information.

[0054] Figure 7 is a diagram showing an example of the relationship between privacy information and concealed information. Figure 7 is an example of privacy information and concealed information stored in the storage 22. The relation column indicates the type of person having the privacy information, and together with the field column, indicates the type of privacy information. Self indicates the user himself / herself, and husband indicates the male spouse. The original_value column is the column of privacy information included in the inquiry sentence, and the encoded_value column is the column of concealed information. Figure 7 also shows privacy information that cannot be substantially replaced by concealed information.

[0055] When the concealed information is determined, the concealing unit 53 replaces each of the one or more pieces of privacy information included in the inquiry with the concealed information (S105). As a result, at least a part of the privacy information is replaced with concealed information different from the privacy information. The concealing unit 53 may search for the character string of the privacy information extracted from the inquiry sentence and replace the found character string with the corresponding character string of the concealed information, or the concealing unit 53 may replace the character string at that position with the concealed information based on the information indicating the position of the privacy information.

[0056] FIG. 8 is a diagram showing an example of an inquiry in which privacy information is replaced. FIG. 8 is an example of the case where the processes of S102 to S105 are performed on the inquiry sentence shown in FIG. 4. Comparing FIG. 8 with FIG. 4, my name and the ages of my husband and I are replaced with masked information.

[0057] Also, the knowledge acquisition unit 54 acquires knowledge information from the knowledge database 60 based on the query information determined in S103 (S106). The knowledge acquisition unit 54 may send a query based on the query information determined in S103 to the knowledge database 60 via the API, and acquire the knowledge information found by the knowledge database 60 through the search of the query via the API. This knowledge information is input into the large language model and used as information for the basis of the answer. The process of S106 may be executed in parallel with the processes of S104 and S105 or may be executed in a different order.

[0058] Then, the answer acquisition unit 55 inputs an instruction text including the inquiry sentence with the privacy information replaced and the knowledge information into the large language model, and acquires the output (answer) of the large language model (S107). The answer acquisition unit 55 requests the large language model to create an answer to the inquiry sentence by inputting the instruction text.

[0059] FIG. 9 is a diagram showing an example of an instruction text input into the large language model. In FIG. 9, there are character strings such as {article} and {user_introduction}, but in reality, knowledge information (for example, insurance product information) and the inquiry sentence are set at the positions of those character strings respectively. The instruction text shown in FIG. 9 requests the large language model to generate an answer recommending an appropriate insurance product searched based on the privacy information based on the inquiry information. Note that a hyperlink (URL) to the information may be input into the large language model as knowledge information, or in the case of an application where an answer can be created even without specific knowledge information, the acquisition of knowledge information and the input into the large language model may not be performed.

[0060] The inquiry sentences included in the instruction text input to the large language model do not contain information related to personal identification, especially among privacy information. Therefore, it is possible to prevent such information from being passed to the large language model service 3 including the large language model. As a result, the risk of leakage of privacy information by the large language model service 3 can be significantly reduced, and it can be easily compliant with privacy-related rules. In addition, by including a dummy of privacy information as concealed information in the inquiry sentence, the influence on the generation of the answer can be suppressed.

[0061] FIG. 10 is a diagram showing an example of an answer output from the large language model. FIG. 10 is an example of an answer generated when instruction text including the information shown in FIGS. 8 and 9 and the knowledge information of "Cancer Insurance A" is input. The large language model may generate an answer including concealed information such as an item of name as shown in FIG. 10 in some cases. Also, in some cases, an answer including concealed information such as age and medical history may be generated.

[0062] When the answer is acquired, the restoration unit 56 replaces each of the one or more pieces of concealed information included in the acquired answer with the privacy information corresponding to the concealed information (S108). For example, the restoration unit 56 searches for whether each of the one or more pieces of concealed information stored in the storage 22 and associated with the privacy information is included in the answer. Then, when the string of the concealed information is included in the answer, the restoration unit 56 replaces the string with the privacy information associated with the concealed information.

[0063] When the concealed information is replaced, the answer output unit 57 outputs information based on the answer with the concealed information replaced to the user terminal 1 operated by the user (S109). The answer output unit 57 outputs the information obtained by processing the answer as the information based on the answer.

[0064] FIG. 11 is a diagram showing an example of a response in which the concealed information has been replaced. FIG. 11 is an example of the case where the process by the restoration unit 56 is executed on the response shown in FIG. 10, and a hyperlink to information regarding the knowledge information is further added. In the example of FIG. 11, the name described in the first line of the response has reverted to the user's name.

[0065] As described above, it is possible to prevent the leakage of privacy information by concealing the information related to privacy from the inquiry sentence input to the large language model. On the other hand, there is a risk that the response output from the large language model may include concealed information and become an unnatural response. In the present embodiment, by restoring the concealed information included in the response to the original privacy information, the inquiry management system 2 can output a natural response. Further, by making the concealed information dummy information closer to the actual situation, the response itself generated by the large language model can be made more natural. Further, when replacing the age with a dummy concealed age, the concealed age is set to be within the same age range, and further, the setting of the age range is adjusted according to the setting of the age range in the knowledge information. Thereby, the influence on the response due to the change in age can be minimized.

[0066] Note that the concealment method in the present embodiment is not limited to those described above. For example, as the concealed information, instead of the dummy information according to the type of privacy information, a fixed character string such as a tag indicating the type of privacy information may be used.

[0067] FIG. 12 is a diagram showing another example of an inquiry sentence with private information replaced. By performing the processes of S102 to S105 on the inquiry sentence shown in FIG. 4, the inquiry sentence shown in the example of FIG. 12 is obtained. However, the concealed information for which private information is replaced is not dummy information imitating actual private information, but tag information indicating the type of private information itself. In this case, in S104, the process of selecting an age based on the age range is not performed, and the age of the private information may simply be associated with the tag of the concealed information indicating the age. Also, in this example, the types of private information may be simply types such as name, age, gender, and medical history, without including the classification of users and family members.

[0068] FIG. 13 is a diagram showing another example of an instruction text input to a large language model. Actually, knowledge information (for example, insurance product information) and an inquiry sentence as shown in FIG. 12 are set at the positions of the character strings {article} and {user_introduction} in FIG. 13, respectively. In the example of FIG. 13, different from the example of FIG. 9, information explaining the meaning of the tags in the inquiry sentence is described in the instruction text input to the large language model. Even in this way, it is possible to generate an answer without inputting sensitive private information to the large language model, and the quality of the answer can be ensured to a certain extent.

[0069] In this embodiment, a large language model is used, but its implementation and the scale of the number of parameters are not particularly limited. The present invention is applicable to a machine learning model (language model) that handles natural language.

Explanation of Signs

[0070] 1 User terminal, 2 Inquiry management system, 3 Large language model service, 21 Processor, 22 Storage, 23 Communication unit, 50 Management unit, 51 Input acquisition unit, 52 Privacy management unit, 53 Concealment unit, 54 Knowledge acquisition unit, 55 Answer acquisition unit, 56 Restoration unit, 57 Answer output unit, 58 Privacy extraction unit, 59 Relevance management unit, 60 Knowledge database.

Claims

1. An acquisition means for acquiring text including one or more pieces of privacy information; a query determining means for determining, based on the text, query information including at least a portion of the one or more pieces of privacy information contained in the text; a knowledge acquisition means for searching a knowledge database based on the query information and acquiring knowledge information; a concealment means for replacing each of the one or more pieces of privacy information with concealment information different from the one or more pieces of privacy information; an answer acquisition means for acquiring an answer from a language model by inputting the text in which the privacy information has been replaced and the knowledge information into the language model; an output means for outputting information based on the acquired answer; An information processing system comprising:

2. 2. The information processing system according to claim 1, The concealment means replaces each of the one or more pieces of privacy information included in the text with concealment information corresponding to a type of the privacy information. Information processing system.

3. 3. The information processing system according to claim 2, The one or more types of privacy information include at least some of name, age, gender, and medical history; Information processing system.

4. 2. The information processing system according to claim 1, The method further includes a restoration means for replacing the concealed information included in the obtained answer with privacy information that has been replaced with the concealed information in the text. Information processing system.

5. 5. The information processing system according to claim 4, The method further includes an association management means for storing each of the one or more pieces of privacy information in a database in association with concealment information replaced with the privacy information, When the answer includes concealment information associated with any of the one or more pieces of privacy information, the restoration means replaces the concealment information with the privacy information associated with the concealment information. Information processing system.

6. 2. The information processing system according to claim 1, a range selection means for selecting, when the text includes an age, an age range to which the age belongs from a plurality of predetermined age ranges; an age selection means for selecting one of a plurality of ages within the selected age range as a concealed age for the age included in the text; The concealment means replaces the age included in the text with a concealed age for that age. Information processing system.

7. 7. The information processing system according to claim 6, the age selection means randomly selects one of a plurality of ages within the selected age range as a concealed age for the age included in the text; Information processing system.

8. 8. The information processing system according to claim 6, The query determination means determines query information including at least a part of the one or more pieces of privacy information included in the text and an age range to which the age included in the text belongs. Information processing system.

9. An information processing system including one or more processors, obtaining text including one or more pieces of privacy information; determining query information based on the text, the query information including at least a portion of the one or more pieces of privacy information contained in the text; searching a knowledge database based on the query information to obtain knowledge information; replacing each of the one or more pieces of privacy information with concealment information different from the one or more pieces of privacy information; obtaining an answer from a language model by inputting the text in which the privacy information has been replaced and the knowledge information into the language model; outputting information based on the obtained answers; An information processing method comprising:

10. An acquisition means for acquiring text including one or more pieces of privacy information; a query determining means for determining, based on the text, query information comprising at least a portion of the one or more pieces of privacy information contained in the text; a knowledge acquisition means for searching a knowledge database based on the query information and acquiring knowledge information; a concealment means for replacing each of the one or more pieces of privacy information with concealment information different from the one or more pieces of privacy information; an answer acquisition means for acquiring an answer from a language model by inputting the text in which the privacy information has been replaced and the knowledge information into the language model; and an output means for outputting information based on the acquired answer; A program that makes a computer function as a

Images