Information processing system, information processing method, and program

Abstract

To improve the quality of a response based on the output of a language model while considering privacy protection.SOLUTION: An information processing system is configured to: acquire a query including one or a plurality of privacy information on the basis of input from a user; replace each of the one or plurality of privacy information included in the query with concealment information different from the one or plurality of privacy information; request a language model to generate a response to the query in which the privacy information has been replaced and acquire the response from the language model; replace the concealment information included in the acquired response with the privacy information which has been replaced in the query; and send information based on the response in which the concealment information has been replaced to the user.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to an information processing system, an information processing method, and a program. [Background technology]

[0002] The performance of language models such as large language models (LLMs) has improved significantly. Large language models can answer questions in natural sentences. Large language models are used in chatbots such as ChatGPT (registered trademark).

[0003] Patent Document 1 discloses a technology for generating a summary sentence desired by a user using a large-scale language model obtained by machine learning of a huge amount of unlabeled text. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2023-73095 Summary of the Invention [Problem to be solved by the invention]

[0005] The inventor is considering a system that answers questions using a language model provided as a service. Depending on the physical location of such a language model and the policy of the service provider, it may be difficult to input information related to protecting personal privacy. On the other hand, if such information is not input to the language model, the quality of the answers output by the language model may be reduced.

[0006] The present disclosure provides a technique for improving the quality of answers based on the output of a language model while respecting privacy protection. [Means for solving the problem]

[0007] (1) An information processing system including: a question acquisition means for acquiring an inquiry including one or more pieces of privacy information based on an input from a user; a concealment means for replacing each of the one or more pieces of privacy information included in the inquiry with concealment information different from the one or more pieces of privacy information; an answer acquisition means for requesting a language model to create an answer to the inquiry in which the privacy information has been replaced, and acquiring the answer from the language model; a restoration means for replacing the concealment information included in the acquired answer with the privacy information that has been replaced with the concealment information in the inquiry; and an answering means for sending information based on the answer in which the concealment information has been replaced to the user.

[0008] (2) In the information processing system according to (1), the concealment means replaces each of the one or more pieces of privacy information included in the inquiry with concealment information corresponding to a type of the privacy information.

[0009] (3) In (2), the one or more types of privacy information include at least some of name, age, gender, and medical history.

[0010] (4) In any one of (1) to (3), the information processing system further includes an association management means for associating each of the one or more pieces of privacy information with concealed information replaced with the one or more pieces of privacy information and storing the one or more pieces of privacy information in a database, and the restoration means replaces the one or more pieces of concealed information with the privacy information associated with the one or more pieces of concealed information when the answer includes the one or more pieces of concealed information associated with the one or more pieces of privacy information.

[0011] (5) In any of (1) to (4), an information processing system further includes a range selection means for selecting an age range to which the user's age belongs from a plurality of predetermined age ranges when the inquiry includes the user's age, and an age selection means for selecting one of a plurality of ages within the selected age range as a concealed age for the user's age, wherein the concealment means replaces the user's age included in the inquiry with the concealed age for the user's age.

[0012] (6) In the information processing system according to (5), the age selection means randomly selects one of a plurality of ages within the selected age range as a concealed age for the user's age.

[0013] (7) In the information processing system of (5) or (6), the system further includes a basic acquisition means for acquiring, based on at least a portion of the one or more pieces of privacy information, one of a plurality of pieces of answer basic information, the answer basic information being targeted at an age range to which the user belongs, and the answer acquisition means requests a language model to create an answer to the question in which the privacy information has been replaced based on the acquired answer basic information, and acquires the answer from the language model.

[0014] (8) An information processing method including: acquiring an inquiry including one or more pieces of privacy information based on an input from a user; replacing each of the one or more pieces of privacy information included in the inquiry with concealment information different from the one or more pieces of privacy information; requesting a language model to create an answer to the inquiry in which the privacy information has been replaced, and acquiring the answer from the language model; replacing the concealment information included in the acquired answer with the privacy information that has been replaced with the concealment information in the inquiry; and sending information based on the answer in which the concealment information has been replaced to the user.

[0015] (9) A program for causing a computer to function as a question acquisition means for acquiring an inquiry including one or more pieces of privacy information based on an input from a user, a concealment means for replacing each of the one or more pieces of privacy information included in the inquiry with concealment information different from the one or more pieces of privacy information, an answer acquisition means for requesting a language model to create an answer to the inquiry in which the privacy information has been replaced and acquiring the answer from the language model, a restoration means for replacing the concealment information included in the acquired answer with the privacy information that was replaced with the concealment information in the inquiry, and a response means for sending information based on the answer in which the concealment information has been replaced to the user. Effect of the Invention

[0016] According to the present invention, it is possible to improve the quality of answers based on the output of a language model while taking into consideration privacy protection. [Brief description of the drawings]

[0017] [Figure 1] FIG. 2 is a diagram illustrating an example of elements related to an inquiry management system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a block diagram showing functions realized by the inquiry management system. [Diagram 3] 13 is a flowchart illustrating an example of a process of an inquiry management system. [Figure 4] FIG. 13 is a diagram showing an example of an input screen for an inquiry management system. [Diagram 5] FIG. 11 is a diagram illustrating an example of privacy information extracted from a query text. [Figure 6] FIG. 13 is a diagram illustrating an example of a query based on privacy information. [Figure 7] FIG. 13 is a diagram illustrating an example of a relationship between concealment information and privacy information. [Figure 8] FIG. 13 is a diagram illustrating an example of a query sentence in which privacy information has been replaced. [Figure 9]FIG. 2 is a diagram showing an example of instruction text input to a large-scale language model. [Figure 10] FIG. 13 is a diagram illustrating an example of an answer output from a large-scale language model. [Figure 11] FIG. 13 is a diagram showing an example of a reply in which hidden information has been replaced. [Figure 12] FIG. 13 is a diagram showing another example of a query text in which privacy information has been replaced. [Figure 13] FIG. 13 is a diagram showing another example of instruction text input to the large-scale language model. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. Configurations with the same reference numerals will not be described repeatedly.

[0019] 1 is a diagram showing an example of elements related to an inquiry management system 2 according to an embodiment of the present invention. The inquiry management system 2 receives an inquiry including information related to the protection of the user's privacy (privacy information) from a user terminal 1 operated by a user, inputs a command based on the inquiry to a large-scale language model service 3, and outputs a response corresponding to a sentence output by the large-scale language model service 3 to the user terminal 1. In the following, as an example of the inquiry management system 2, an inquiry management system 2 for mainly recommending insurance products to a user will be described.

[0020] The user terminal 1 is a computer having a user interface, such as a personal computer or a smartphone.

[0021] The large-scale language model service 3 includes a general-purpose large-scale language model realized by a computer. The large-scale language model service 3 receives an instruction from the query management system 2, and passes the output obtained by inputting the instruction into the large-scale language model to the query management system 2. This instruction is in text format and is also called a prompt. Hereinafter, among these instructions, an instruction in text format is also referred to as an instruction text. This general-purpose large-scale language model is trained with data from a wide range of fields. The large-scale language model service 3 may be, for example, a service such as ChatGPT (registered trademark).

[0022] In the following, when simply described as "large-scale language model", it refers to the large-scale language model included in the large-scale language model service 3, and the query management system 2 executes the process of inputting information to the large-scale language model and acquiring the output from the large-scale language model by using the API provided by the large-scale language model service 3. The query management system 2 does not necessarily input all information with a single API call, and may input information part by part through multiple API calls. The large-scale language model service 3 may be provided in the query management system 2. In this embodiment, the query management system 2 inputs information requesting the large-scale language model to create a response, and acquires the output of the large-scale language model as the response. In the following, inputting information requesting the creation of a response into the large-scale language model is also described as requesting the large-scale language model to create a response.

[0023] The inquiry management system 2 includes one or more computers (e.g., server computers). The inquiry management system 2 includes one or more processors 21, one or more storages 22, and one or more communication units 23. The inquiry management system 2 may include multiple computers each including one or more processors 21, storages 22, and communication units 23, or may include a single computer having one or more processors 21 and storages 22. The inquiry management system 2 may be implemented on one or more virtual server or container platforms.

[0024] The processor 21 operates according to a program (also called an instruction code) stored in the storage 22. The processor 21 also controls the communication unit 23. The processor 21 includes, for example, a CPU (Central Processing Unit), and may further include a GPU (Graphic Processing Unit) and an NPU (Neural Processing Unit). The program may be provided via the Internet or the like, or may be provided by being stored in a computer-readable storage medium such as a flash memory or a DVD-ROM.

[0025] The storage 22 is composed of memory elements such as RAM and flash memory, and external storage devices such as a hard disk drive (HDD) and a solid state drive (SSD). The storage 22 stores the above programs. The storage 22 also stores information input from the processor 21 and the communication unit 23 and calculation results.

[0026] The communication unit 23 is a communication interface, such as a network interface card, for communicating with other devices. The communication unit 23 is composed of an integrated circuit, an antenna, a communication terminal, etc., which realize a wireless LAN or a wired LAN. The communication unit 23 inputs information received from other devices to the processor 21 or the storage 22 via a network under the control of the processor 21, and transmits information to other devices.

[0027] The hardware configuration of the inquiry management system 2 is not limited to the above example. For example, the inquiry management system 2 may include a device for reading a computer-readable information storage medium (e.g., an optical disk drive or a memory card slot) and a device for inputting and outputting data to and from an external device (e.g., a USB port). The external device may be an input device or an output device.

[0028] Next, the functions provided by the inquiry management system 2 will be described. Fig. 2 is a block diagram showing the functions realized by the inquiry management system 2. The inquiry management system 2 functionally includes a management unit 50 and a knowledge database 60. Furthermore, the management unit 50 functionally includes an input acquisition unit 51, a privacy management unit 52, a concealment unit 53, a knowledge acquisition unit 54, an answer acquisition unit 55, a restoration unit 56, and an answer output unit 57. The privacy management unit 52 functionally includes a privacy extraction unit 58 and an association management unit 59. The management unit 50 and the knowledge database 60 are realized by the processor 21 executing programs corresponding to each function stored in the storage 22 and controlling the communication unit 23 and the like.

[0029] The knowledge database 60 is a database in which knowledge information in a certain knowledge field is stored. The knowledge database 60 acquires information indicating the characteristics of an input from a user from the management unit 50, searches for knowledge information corresponding to the information, and passes the knowledge information found by the search to the management unit 50. In the knowledge database 60, the knowledge information and a feature vector serving as an index of the knowledge information are stored in the storage 22 in association with each other. The knowledge database 60 acquires a feature vector serving as a query based on an input from a user, and acquires knowledge information corresponding to the query.

[0030] Here, the knowledge information may include at least a part of a text document or a link character string (e.g., URL) to a site on the Internet. In this embodiment, the knowledge field is insurance products sold by an insurance company, and the feature vector serving as the query is generated from the age group, gender, and medical history of the user and the user's family. In this case, in the knowledge database 60, not only the information on the insurance product as knowledge information and the feature vector of the knowledge information, but also information indicating the age group, gender, and medical history that are application conditions of the knowledge information (use conditions of the insurance product) may be associated and stored in the storage 22. The knowledge database 60 may handle information in other knowledge fields as long as the knowledge can be classified by information related to privacy. The knowledge information is used as basic information (basic answer information) when the large-scale language model creates an answer to a question.

[0031] In searching for knowledge information, the knowledge database 60 may search for a feature vector similar to the query from among a plurality of feature vectors stored in the storage 22, and output knowledge information stored in association with the similar feature vector. The query may also include at least a portion of privacy information. In this case, the knowledge database 60 may search for knowledge information for which the query satisfies the use conditions. Furthermore, the knowledge database 60 may search for a feature vector similar to the query among one or more feature vectors associated with the knowledge information for which the query satisfies the use conditions. The knowledge database 60 may use, for example, a cosine similarity between the feature vector of the query and the feature vector in the storage 22 as the similarity. The knowledge database 60 may select the feature vector with the largest similarity as the similar feature vector, and obtain knowledge information associated with the selected feature vector.

[0032] Here, the feature vector may be generated by a feature extraction model, which is a machine learning model. The feature extraction model is trained using document data for training. During training, the feature extraction model extracts a feature vector of a query and a feature vector of document data corresponding to the query. Then, the feature extraction model is trained so that the similarity between the feature vector of the query and the feature vector of the corresponding document data increases and the similarity between the non-corresponding document data decreases. The document data in training may be only a part of the knowledge document that corresponds to a topic or a question. This training may be performed based on the methods shown in the following two papers.

[0033] Paper 1: Wataru Sakata, Tomohide Shibata, Ribeka Tanaka, and Sadao Kurohashi. 2019. FAQ Retrieval using Query-Question Similarity and BERT-Based Query-Answer Relevance. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'19). Association for Computing Machinery, New York, NY, USA, 1113-1116.

[0034] Paper 2: Seo, J.; Lee, T.; Moon, H.; Park, C.; Eo, S.; Aiyanyo, ID; Park, K.; So, A.; Ahn, S.; Park, J. Dense-to-Question and Sparse-to-Answer: Hybrid Retriever System for Industrial Frequently Asked Questions. Mathematics 2022, 10, 1335.

[0035] The management unit 50 acquires an inquiry including one or more pieces of privacy information from the user terminal 1, and acquires knowledge information corresponding to the inquiry from the knowledge database 60, while concealing the privacy information included in the inquiry. The management unit 50 also requests the large-scale language model service 3 to generate an answer to the inquiry in which the privacy information is concealed, and outputs an output sentence based on the answer to the user terminal 1 via the network.

[0036] The input acquisition unit 51 acquires an inquiry including one or more pieces of privacy information based on an input from a user. The inquiry may be a sentence, or may include information other than the sentence. The input from the user is information that the user inputs by operating the user terminal 1 and receives from the user terminal 1. The input acquisition unit 51 may acquire the sentence as an input from the user as it is as an inquiry, or may generate the sentence as an inquiry by processing the privacy information and free text acquired interactively as an input from the user.

[0037] The privacy management unit 52 extracts one or more pieces of privacy information included in the inquiry and manages the extracted privacy information. The privacy extraction unit 58 included in the privacy management unit 52 extracts privacy information included in the inquiry. When the inquiry includes the user's age, the privacy extraction unit 58 selects an age range to which the user's age belongs from a plurality of predetermined age ranges. When the inquiry includes the age of another person, the privacy extraction unit 58 selects an age range to which the age of the single person belongs from a plurality of predetermined age ranges.

[0038] The association management unit 59 included in the privacy management unit 52 determines concealment information corresponding to each of one or more pieces of privacy information included in the query. Furthermore, the association management unit 59 associates each of one or more pieces of privacy information included in the query with concealment information corresponding to the privacy information, and stores the information in the storage 22. The concealment information is information that is replaced with the privacy information included in the query. The concealment information may be information (tag or dummy information) according to the type of privacy information (e.g., name, age, sex, medical history).

[0039] In determining the concealment information, when the inquiry includes the user's age, the association management unit 59 may select one of a plurality of ages (which may be random) within the age range selected by the privacy extraction unit 58 as a concealment age for the user's age. When the inquiry includes the ages of the user and another person and they belong to the same age range, the association management unit 59 may select a concealment age such that the concealment age for the user does not overlap with the concealment age for the other person.

[0040] The concealment unit 53 replaces one or more pieces of privacy information included in the inquiry with concealment information different from the privacy information. If the inquiry includes the user's age, the concealment unit 53 may replace the user's age included in the inquiry with a concealment age for the user's age. If the inquiry includes the age of another person, the concealment unit 53 may replace the age of the other person included in the inquiry with a concealment age for the age of the other person.

[0041] The knowledge acquisition unit 54 transmits a query based on the inquiry to the knowledge database 60, and acquires knowledge information searched based on the query from the knowledge database 60. When the inquiry includes the user's age, the knowledge acquisition unit 54 acquires, from the knowledge database 60, knowledge information targeting the age range to which the user's age belongs, from a plurality of pieces of knowledge information stored in the storage 22. Targeting an age range may mean that the age range is included in the application conditions of the knowledge information, or may mean that the knowledge information includes a character string indicating that the age range is targeted.

[0042] The answer acquisition unit 55 requests the large-scale language model to create an answer to the inquiry in which the privacy information has been replaced, and acquires the answer from the large-scale language model. The answer may be a sentence. Here, the answer acquisition unit 55 may request the large-scale language model to create an answer to the inquiry in which the privacy information has been replaced based on the knowledge information. This large-scale language model may be connected to the inquiry management system 2 via the Internet. Note that, if the answer acquired from the large-scale language model includes an inappropriate character string (e.g., a URL), the answer acquisition unit 55 may filter the character string from the answer, and use the filtered answer for subsequent processing.

[0043] The restoration unit 56 replaces the hidden information included in the answer acquired from the large-scale language model with the privacy information replaced with the hidden information in the query. More specifically, when the answer includes hidden information associated with one or more pieces of privacy information included in the query, the restoration unit 56 replaces the hidden information with the privacy information stored in the storage 22 in association with the hidden information.

[0044] The answer output unit 57 sends information based on the answer in which the hidden information has been replaced to the user. More specifically, the answer output unit 57 may process the answer in which the hidden information has been replaced and send the processed answer to the user. The processing may be shaping the format of the sentence included in the answer, or may be adding a hyperlink to the knowledge information or information related thereto to the answer. The answer output unit 57 may also determine whether the replaced or processed answer is consistent with the acquired knowledge information. If it is determined that the answer is consistent, the answer output unit 57 may add a hyperlink to the knowledge information or information related thereto to the answer, and if it is determined that the answer is not consistent, may add a warning message to the answer. Here, the answer output unit 57 may determine the consistency based on whether each of the multiple words included in the answer is included in the words included in the knowledge information.

[0045] 3 is a flowchart showing an example of the processing of the inquiry management system 2. FIG. 3 mainly shows the processing of the management unit 50.

[0046] First, the input acquisition unit 51 acquires a query sentence including one or more pieces of privacy information based on information input from the user terminal 1 based on a user's operation (S101). The query sentence is a query that mainly includes text.

[0047] FIG. 4 is a diagram showing an example of an input screen to the inquiry management system 2. The screen shown in FIG. 4 is output to the user terminal 1. The screen in FIG. 4 displays an inquiry sentence input by the user. In the example of FIG. 4, the inquiry sentence is an introduction of the user and his / her family input by the user to select an appropriate insurance. In the example of FIG. 4, the input acquisition unit 51 acquires a sentence input in one input field as an inquiry sentence. The input acquisition unit 51 may further detect whether there is a missing item in the sentence input so far, and when an item is detected, output information prompting the input of the detected item to the user terminal 1. In this case, the input acquisition unit 51 may acquire a sentence input thereafter and generate an inquiry sentence from the sentence input so far. Note that the inquiry information may include other privacy information such as an image of the user's face.

[0048] Next, the privacy extraction unit 58 identifies the privacy information included in the inquiry text (S102). The privacy extraction unit 58 also identifies the type of privacy information included in the inquiry text. In the example of this embodiment, the type of privacy information includes the user's name, age, sex, medical history, marital status / unmarried status, and phone number, and the names, ages, sex, medical history, marital status / unmarried status, and phone numbers of family members. The type of privacy information actually handled may be a part of this, or may be classified differently. The privacy extraction unit 58 may extract proper nouns and nouns from the inquiry text by morphological analysis, and identify the privacy information and the type by matching the extracted proper nouns and nouns with a dictionary created in advance. In addition, the privacy information may be identified based on the output when the inquiry text is input to a machine learning model trained with learning data including a sentence and correct answer data including the position of the privacy information and the type of the privacy information included in the sentence.

[0049] Fig. 5 is a diagram showing an example of privacy information extracted from a query. In the example of Fig. 5, the field column indicates the type of privacy information, and the value column indicates the privacy information itself. Fig. 5 may further include information indicating the position of the privacy information in the query, and the type of privacy information may be subdivided according to the classification of the user and family relationship.

[0050] When the privacy information is specified, the privacy extraction unit 58 converts the privacy information according to its type and determines the query information (S103). When the privacy information specified from the query includes the user's age (including the privacy information whose type is the user's age), the privacy extraction unit 58 selects an age range to which the user's age belongs from a plurality of predetermined age ranges. The plurality of age ranges are set so as not to overlap or separate, for example, 0-5 years old, 6-17 years old, 18-19 years old, 20-29 years old, 30-39 years old, 40-49 years old, 50-59 years old, and 60-69 years old. The privacy extraction unit 58 may convert gender and illness into codes.

[0051] Fig. 6 is a diagram showing an example of a query based on privacy information. In the example of Fig. 6, the query includes items of an age range (age_range), a code indicating a gender (gender_code), and a code indicating a disease (disease_code).

[0052] The association management unit 59 determines concealment information corresponding to each of one or more pieces of privacy information included in the query, and associates them with each other and stores them in the storage 22 (S104). The association management unit 59 determines concealment information corresponding to the privacy information according to the type of the privacy information. Also, the association management unit 59 associates each piece of privacy information with the corresponding concealment information and stores it in the storage 22. The association management unit 59 may determine concealment information different from the privacy information for some types of one or more pieces of privacy information included in the query.

[0053] When the query includes privacy information of the age of the user or other person (including privacy information whose type is the age of the user or other person), the association management unit 59 selects one of a plurality of ages within the age range selected by the privacy extraction unit 58 for that age as a concealed age for that age. This concealed age is particularly related to age among the concealed information. When the identified privacy information is the name, address, or phone number of the user or other person (the type is the name, address, or phone number of the user or other person), the association management unit 59 may determine a dummy character string such as a name previously associated with that type as the concealed information.

[0054] FIG. 7 is a diagram showing an example of the relationship between the privacy information and the hidden information. FIG. 7 is an example of the privacy information and the hidden information stored in the storage 22. The relation is a column showing the type of the person having the privacy information, and indicates the type of the privacy information together with the field column. The self column shows the user himself, and the husband column shows the male spouse. The original_value column shows the privacy information included in the query, and the encoded_value column shows the hidden information. FIG. 7 also shows the privacy information that is not substantially replaced by the hidden information.

[0055] When the concealment information is determined, the concealment unit 53 replaces each of one or more pieces of privacy information included in the query with the concealment information (S105). As a result, at least a part of the privacy information is replaced with concealment information different from the privacy information. The concealment unit 53 may search for a character string of the privacy information extracted from the query text and replace the found character string with a character string of the corresponding concealment information, or the concealment unit 53 may replace a character string at a position of the privacy information with the concealment information based on information indicating the position of the privacy information.

[0056] Fig. 8 is a diagram showing an example of an inquiry in which privacy information has been replaced. Fig. 8 shows an example in which the processes of S102 to S105 have been performed on the inquiry text shown in Fig. 4. Comparing Fig. 8 with Fig. 4, my name and my and my husband's ages have been replaced with concealed information.

[0057] Further, the knowledge acquisition unit 54 acquires knowledge information from the knowledge database 60 based on the query information determined in S103 (S106). The knowledge acquisition unit 54 may send a query based on the query information determined in S103 to the knowledge database 60 via the API, and acquire knowledge information found by the knowledge database 60 searching for the query via the API. This knowledge information is input to the large-scale language model and used as information that forms the basis of an answer. The process of S106 may be executed in parallel with the processes of S104 and S105, or may be executed in a different order.

[0058] Then, the answer acquisition unit 55 inputs the query sentence in which the privacy information has been replaced and the command text including the knowledge information to the large-scale language model, and acquires the output (answer) of the large-scale language model (S107). By inputting the command text, the answer acquisition unit 55 requests the large-scale language model to create an answer to the query sentence.

[0059] FIG. 9 is a diagram showing an example of an instruction text input to a large-scale language model. In FIG. 9, there are character strings {article} and {user_introduction}, but in reality, knowledge information (e.g., information on an insurance product) and a query are set in the places of these character strings, respectively. The instruction text shown in FIG. 9 requests the large-scale language model to generate an answer that recommends an appropriate insurance product searched for based on privacy information, based on the query information. Note that a hyperlink (URL) to the information may be input to the large-scale language model as knowledge information, and in the case of an application in which an answer can be created without specific knowledge information, it is not necessary to acquire knowledge information and input it to the large-scale language model.

[0060] The query sentence included in the command text input to the large-scale language model does not include any privacy information, particularly information related to identifying an individual. Therefore, it is possible to prevent such information from being passed to the large-scale language model service 3 that includes the large-scale language model. This makes it possible to significantly reduce the risk of privacy information leaking from the large-scale language model service 3, and also to easily comply with privacy rules. In addition, since the query sentence includes dummy privacy information as concealed information, the impact on the creation of an answer can be reduced.

[0061] Fig. 10 is a diagram showing an example of an answer output from a large-scale language model. Fig. 10 is an example of an answer generated when an instruction text including the information shown in Figs. 8 and 9 and the knowledge information of "cancer insurance A" is input. In some cases, the large-scale language model generates an answer including hidden information such as a name item, as shown in Fig. 10. In other cases, an answer including hidden information such as age and medical history may be generated.

[0062] When the answer is acquired, the restoration unit 56 replaces each of one or more pieces of hidden information included in the acquired answer with the privacy information corresponding to the hidden information (S108). For example, the restoration unit 56 searches for each of one or more pieces of hidden information stored in the storage 22 and associated with the privacy information in the answer to see if the answer contains a character string of the hidden information. If the answer contains a character string of the hidden information, the restoration unit 56 replaces the character string with the privacy information associated with the hidden information.

[0063] When the hidden information is replaced, the answer output unit 57 outputs information based on the answer for which the hidden information has been replaced to the user terminal 1 operated by the user (S109). The answer output unit 57 outputs information in which the answer has been processed as the information based on the answer.

[0064] Fig. 11 is a diagram showing an example of an answer in which hidden information has been replaced. Fig. 11 shows an example in which the restoration unit 56 has executed processing on the answer shown in Fig. 10, and further a hyperlink to information related to knowledge information has been added. In the example of Fig. 11, the name written in the first line of the answer has been restored to the user's name.

[0065] As described above, the leakage of privacy information can be prevented by concealing privacy-related information from the query sentence input to the large-scale language model, but on the other hand, there is a risk that the answer output from the large-scale language model will contain the concealed information, resulting in an unnatural answer. In this embodiment, the query management system 2 can output a natural answer by returning the concealed information contained in the answer to the original privacy information. In addition, by making the concealed information dummy information closer to the actual situation, the answer itself generated by the large-scale language model can be made more natural. In addition, when replacing the age with a dummy concealed age, the concealed age is set to be in the same age range, and further, the age range setting is matched to the age range setting in the knowledge information. This makes it possible to minimize the impact on the answer due to the change in age.

[0066] The concealment method in this embodiment is not limited to the above-described method. For example, a fixed character string such as a tag indicating the type of privacy information may be used as the concealment information instead of dummy information corresponding to the type of privacy information.

[0067] FIG. 12 is a diagram showing another example of an inquiry text in which privacy information has been replaced. The inquiry text shown in FIG. 4 is subjected to the processes of S102 to S105, resulting in the inquiry text shown in the example of FIG. 12. However, the concealment information in which the privacy information is replaced is not dummy information resembling actual privacy information, but tag information indicating the type of privacy information itself. In this case, the process of selecting an age based on an age range is not performed in S104, and the age of the privacy information may simply be associated with the tag of the concealment information indicating the age. Also, in this example, the type of privacy information may not include a classification of user or family, but may simply be a type such as name, age, sex, and medical history.

[0068] Fig. 13 is a diagram showing another example of an instruction text input to a large-scale language model. In reality, knowledge information (e.g., information on an insurance product) and a query sentence as shown in Fig. 12 are set in the character strings {article} and {user_introduction} in Fig. 13, respectively. In the example of Fig. 13, unlike the example of Fig. 9, information explaining the meaning of the tag in the query sentence is described in the instruction text input to the large-scale language model. Even in this way, it is possible to generate an answer without inputting sensitive privacy information to the large-scale language model, and the quality of the answer can be ensured to a certain degree.

[0069] In this embodiment, a large-scale language model is used, but there is no particular limitation on the scale of its implementation and the number of parameters. The present invention can be applied to a machine learning model (language model) that handles natural language. [Explanation of symbols]

[0070] 1 User terminal, 2 Query management system, 3 Large-scale language model service, 21 Processor, 22 Storage, 23 Communication unit, 50 Management unit, 51 Input acquisition unit, 52 Privacy management unit, 53 Concealment unit, 54 Knowledge acquisition unit, 55 Answer acquisition unit, 56 Restoration unit, 57 Answer output unit, 58 Privacy extraction unit, 59 Association management unit, 60 Knowledge database.

Claims

1. A obtaining means for obtaining a query including one or more pieces of privacy information based on an input from a user; a concealment means for replacing each of the one or more pieces of privacy information included in the inquiry with concealment information different from the one or more pieces of privacy information; an answer acquisition means for requesting a language model to create an answer to the inquiry in which the privacy information has been replaced, and acquiring the answer from the language model; a restoration means for replacing the concealed information included in the obtained response with the privacy information that was replaced with the concealed information in the inquiry; a reply means for sending information based on the reply in which the hidden information has been replaced to the user; An information processing system comprising:

2. 2. The information processing system according to claim 1, the concealment means replaces each of the one or more pieces of privacy information included in the inquiry with concealment information corresponding to a type of the privacy information; Information processing system.

3. 3. The information processing system according to claim 2, The one or more types of privacy information include at least a portion of name, age, gender, and medical history; Information processing system.

4. 2. The information processing system according to claim 1, The method further includes an association management means for storing each of the one or more pieces of privacy information in a database in association with concealment information replaced with the privacy information, When the answer includes concealment information associated with any of the one or more pieces of privacy information, the restoration means replaces the concealment information with the privacy information associated with the concealment information. Information processing system.

5. 2. The information processing system according to claim 1, a range selection means for selecting an age range to which the user's age belongs from a plurality of predetermined age ranges when the inquiry includes the user's age; and an age selection means for selecting one of a plurality of ages within the selected age range as a concealed age for the user's age, The concealment means replaces the user's age included in the inquiry with a concealed age for the user's age. Information processing system.

6. 6. The information processing system according to claim 5, The age selection means randomly selects one of a plurality of ages within the selected age range as a concealed age for the user's age. Information processing system.

7. 7. The information processing system according to claim 5, The method further includes a basic acquisition means for acquiring answer basic information, the answer basic information being targeted at an age range to which the user belongs, from among a plurality of pieces of answer basic information based on at least a part of the one or more pieces of privacy information; the answer acquisition means requests a language model to create an answer to the question in which the privacy information has been replaced based on the acquired answer basic information, and acquires the answer from the language model. Information processing system.

8. obtaining a query including one or more pieces of privacy information based on input from a user; replacing each of the one or more pieces of privacy information included in the query with concealment information different from the one or more pieces of privacy information; requesting a language model to generate an answer to the query with the replaced privacy information, and obtaining the answer from the language model; replacing the concealment information included in the obtained response with the privacy information that was replaced with the concealment information in the inquiry; sending information based on the answer with the hidden information replaced to the user; An information processing method comprising:

9. A obtaining means for obtaining a query including one or more pieces of privacy information based on an input from a user; a concealment means for replacing each of the one or more pieces of privacy information included in the inquiry with concealment information different from the one or more pieces of privacy information; an answer acquisition means for requesting a language model to create an answer to the inquiry in which the privacy information has been replaced, and acquiring the answer from the language model; a restoration means for replacing the concealed information included in the obtained response with the privacy information that was replaced with the concealed information in the inquiry; and a reply means for sending information based on the reply in which the hidden information has been replaced to the user; A program that makes a computer function as a

Images