Information processing systems, information processing methods, and programs

Abstract

To improve the quality of responses based on the output of a language model while taking privacy protection into consideration.SOLUTION: An information processing system is configured to: acquire text including one or a plurality of pieces of privacy information; determine query information including at least a part of the one or the plurality of pieces of privacy information contained in the text on the basis of the text; search a knowledge database to acquire knowledge information on the basis of the query information; replace each of the one or the plurality of pieces of privacy information with concealment information different from the one or the plurality of pieces of privacy information; acquire a response from a language model by inputting the text where the privacy information has been replaced and the knowledge information into the language model; and output information based on the acquired response.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to an information processing system, an information processing method, and a program. [Background Art]

[0002] The performance of language models such as large language models (LLMs) has been significantly improved. Large language models can answer questions in natural sentences. Large language models are used in chatbots such as ChatGPT (registered trademark), for example.

[0003] Patent Document 1 discloses a technique for generating a summary sentence requested by a user using a large language model obtained by machine learning on a huge amount of unlabeled text. [Prior Art Documents] [Patent Documents]

[0004] [Patent Document 1] Japanese Unexamined Patent Application Publication No. 2023-73095 [Summary of the Invention] [Problem to be Solved by the Invention]

[0005] The inventor has studied a system that answers questions using a language model provided as a service. Depending on the physical deployment of such a language model and the policies of the service provider, it may be difficult to input information related to personal privacy protection. On the other hand, if such information is not input to the language model, there is a risk that the quality of answers output by the language model may decrease.

[0006] The present disclosure provides a technique for improving the quality of answers based on the output of a language model while giving consideration to privacy protection. [Means for Solving the Problem]

[0007] (1) An information processing system comprising: a question acquisition means for acquiring a question containing one or more pieces of privacy information based on input from a user; a concealment means for replacing each of the one or more pieces of privacy information contained in the question with concealment information different from the one or more pieces of privacy information; a response acquisition means for requesting a language model to create a response to the question in which the privacy information has been replaced and acquiring a response from the language model; a restoration means for replacing the concealment information contained in the acquired response with the privacy information that was replaced with the concealment information in the question; and a response means for sending information based on the response in which the concealment information has been replaced to the user.

[0008] (2) In (1), the concealment means is an information processing system that replaces each of the one or more pieces of privacy information included in the query with concealment information corresponding to the type of privacy information.

[0009] (3)(2) The information processing system in which the one or more types of privacy information include at least some of name, age, gender, and medical history.

[0010] (4) An information processing system in any of (1) to (3) that further includes association management means for storing each of the one or more pieces of privacy information in a database in association with the concealed information that replaces the privacy information, wherein the restoration means replaces the concealed information with the privacy information associated with the concealed information if the response includes concealed information to which any of the one or more pieces of privacy information is associated.

[0011] (5) An information processing system in which, in any of (1) to (4), if the query includes the user's age, the system further includes a range selection means for selecting an age range to which the user's age belongs from a predetermined range of age ranges, and an age selection means for selecting one of the selected age ranges as a concealed age for the user's age, wherein the concealment means replaces the user's age included in the query with the concealed age for the user's age.

[0012] (6)(5) The age selection means is an information processing system that randomly selects one of a plurality of ages within the selected age range as a concealed age for the user's age.

[0013] (7)(5) or (6), the information processing system further includes a basic acquisition means for acquiring, based on at least a portion of the one or more pieces of privacy information, one of a plurality of answer basic pieces of information, which covers an age range to which the user's age belongs, wherein the answer acquisition means requests a language model to create an answer to a question in which the privacy information has been replaced, based on the acquired answer basic piece of information, and acquires an answer from the language model.

[0014] (8) An information processing method comprising the steps of: obtaining a query containing one or more pieces of privacy information based on input from a user; replacing each of the one or more pieces of privacy information contained in the query with concealed information that is different from the one or more pieces of privacy information; requesting a language model to create a response to the query in which the privacy information has been replaced, and obtaining a response from the language model; replacing the concealed information contained in the obtained response with the privacy information that was replaced in the query; and sending information based on the response in which the concealed information has been replaced to the user.

[0015] (9) A program for causing a computer to function as: a question acquisition means for acquiring a query containing one or more pieces of privacy information based on input from a user; a concealment means for replacing each of the one or more pieces of privacy information contained in the query with concealment information different from the one or more pieces of privacy information; a response acquisition means for requesting a language model to create a response to the query in which the privacy information has been replaced and acquiring a response from the language model; a restoration means for replacing the concealment information contained in the acquired response with the privacy information that was replaced with the concealment information in the query; and a response means for sending information based on the response in which the concealment information has been replaced to the user. [Effects of the Invention]

[0016] This invention makes it possible to improve the quality of responses based on the output of a language model while taking privacy protection into consideration. [Brief explanation of the drawing]

[0017] [Figure 1] This figure shows an example of elements related to an inquiry management system according to an embodiment of the present invention. [Figure 2] This block diagram shows the functions that the inquiry management system provides. [Figure 3] A flowchart illustrating an example of the processing in an inquiry management system. [Figure 4] This figure shows an example of an input screen for the inquiry management system. [Figure 5] This figure shows an example of privacy information extracted from an inquiry. [Figure 6] This figure shows an example of a query based on privacy information. [Figure 7] This diagram illustrates an example of the relationship between concealed information and private information. [Figure 8] This figure shows an example of a query with privacy information replaced. [Figure 9]It is a diagram illustrating an example of instruction text input to a large language model. [Figure 10] It is a diagram illustrating an example of an answer output from a large language model. [Figure 11] It is a diagram illustrating an example of an answer in which concealed information has been replaced. [Figure 12] It is a diagram illustrating another example of an inquiry sentence in which privacy information has been replaced. [Figure 13] It is a diagram illustrating another example of instruction text input to a large language model.

Mode for Carrying Out the Invention

[0018] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Duplicate descriptions of components denoted by the same reference numerals will be omitted.

[0019] FIG. 1 is a diagram illustrating an example of elements related to an inquiry management system 2 according to an embodiment of the present invention. The inquiry management system 2 receives an inquiry including information related to user privacy protection (privacy information) from a user terminal 1 operated by a user, inputs an instruction based on the inquiry to a large language model service 3, and outputs an answer corresponding to the text output by the large language model service 3 to the user terminal 1. Hereinafter, as an example of the inquiry management system 2, the inquiry management system 2 mainly for recommending insurance products to users will be described.

[0020] The user terminal 1 is a computer having a user interface, such as a personal computer or a smartphone, for example.

[0021] Large-scale language model service 3 includes a general-purpose large-scale language model implemented by a computer. Large-scale language model service 3 receives commands from query management system 2 and passes the output obtained by inputting those commands into the large-scale language model back to query management system 2. These commands are in text format and are also called prompts. Hereafter, these text-format commands will also be referred to as command text. This general-purpose large-scale language model is trained using data from a wide range of fields. Large-scale language model service 3 may be a service such as ChatGPT®.

[0022] In the following, when simply referred to as "large-scale language model," it refers to the large-scale language model included in the large-scale language model service 3. The inquiry management system 2 performs the process of inputting information into the large-scale language model and obtaining output from the large-scale language model by using the API provided by the large-scale language model service 3. The inquiry management system 2 is not required to input all information in a single API call; it may input information in parts through multiple API calls. The large-scale language model service 3 may also be provided within the inquiry management system 2. In this embodiment, the inquiry management system 2 inputs information requesting the creation of some kind of answer from the large-scale language model and obtains the output of the large-scale language model as that answer. In the following, inputting information requesting the creation of some kind of answer from the large-scale language model will also be referred to as requesting the large-scale language model to create an answer.

[0023] The inquiry management system 2 includes one or more computers (e.g., server computers). The inquiry management system 2 includes one or more processors 21, one or more storage 22, and one or more communication units 23. The inquiry management system 2 may include multiple computers, each containing one or more processors 21, storage 22, and communication units 23, or it may include one computer having one or more processors 21 and storage 22. The inquiry management system 2 may be implemented on one or more virtual servers or container infrastructure.

[0024] The processor 21 operates according to the program (also called instruction code) stored in the storage 22. The processor 21 also controls the communication unit 23. The processor 21 may include, for example, a CPU (Central Processing Unit), and may also include a GPU (Graphic Processing Unit) or an NPU (Neural Processing Unit). The program may be provided via the internet or the like, or it may be provided stored on a computer-readable storage medium such as flash memory or DVD-ROM.

[0025] The storage 22 consists of memory elements such as RAM and flash memory, and external storage devices such as hard disk drives (HDDs) and solid-state drives (SSDs). The storage 22 stores the above-mentioned program. The storage 22 also stores information and calculation results input from the processor 21 and the communication unit 23.

[0026] The communication unit 23 is a communication interface that communicates with other devices, such as a network interface card. The communication unit 23 is composed of integrated circuits, antennas, and communication terminals that implement wireless LAN or wired LAN, for example. Based on the control of the processor 21, the communication unit 23 inputs information received from other devices via the network to the processor 21 and storage 22, and transmits the information to the other devices.

[0027] Note that the hardware configuration of the inquiry management system 2 is not limited to the example above. For example, the inquiry management system 2 may include devices for reading computer-readable information storage media (e.g., optical disc drives or memory card slots) and devices for inputting and outputting data with external devices (e.g., USB ports). External devices may be input devices or output devices.

[0028] Next, the functions provided by the inquiry management system 2 will be described. Figure 2 is a block diagram showing the functions realized by the inquiry management system 2. Functionally, the inquiry management system 2 includes a management unit 50 and a knowledge database 60. Functionally, the management unit 50 includes an input acquisition unit 51, a privacy management unit 52, a concealment unit 53, a knowledge acquisition unit 54, a response acquisition unit 55, a restoration unit 56, and a response output unit 57. Functionally, the privacy management unit 52 includes a privacy extraction unit 58 and a related management unit 59. The management unit 50 and the knowledge database 60 are realized by the processor 21 executing programs corresponding to each function stored in the storage 22 and controlling the communication unit 23, etc.

[0029] The knowledge database 60 is a database that stores knowledge information for a particular knowledge field. The knowledge database 60 obtains information from the management unit 50 that indicates the characteristics of user input, searches for knowledge information corresponding to that information, and passes the knowledge information found through the search back to the management unit 50. In the knowledge database 60, knowledge information and the feature vectors that serve as indexes for that knowledge information are associated with each other and stored in the storage 22. The knowledge database 60 obtains a feature vector that serves as a query based on user input and obtains knowledge information corresponding to that query.

[0030] Here, knowledge information is in text format, and includes links to websites on the internet. It may include at least a portion of (for example, a URL). In this embodiment, the knowledge field is an insurance product sold by an insurance company, and the feature vector used for the query is generated from the user's and the user's family's age group, gender, and medical history. In this case, the knowledge database 60 may store not only the information about the insurance product as knowledge information and its feature vector, but also information indicating the age group, gender, and medical history that constitute the application conditions (conditions for using the insurance product) of that knowledge information in the storage 22. The knowledge database 60 may also handle information from other knowledge fields if it is possible to classify the knowledge by privacy-related information. The knowledge information is used as the basis for the large-scale language model to create answers to questions (answer base information).

[0031] In retrieving knowledge information, the knowledge database 60 may search for a feature vector similar to the query from a plurality of feature vectors stored in the storage 22 and output the stored knowledge information associated with that similar feature vector. The query may also include at least a portion of the privacy information. In this case, the knowledge database 60 may search for knowledge information that satisfies the usage conditions of the query. Furthermore, the knowledge database 60 may search for a feature vector similar to the query among one or more feature vectors associated with the knowledge information that satisfies the usage conditions of the query. The knowledge database 60 may use, for example, the cosine similarity between the feature vector of the query and the feature vectors in the storage 22 as the similarity metric. The knowledge database 60 may select the feature vector with the highest similarity as the similar feature vector and obtain the knowledge information associated with the selected feature vector.

[0032] Here, the feature vectors may be generated by a feature extraction model, which is a machine learning model. The feature extraction model is trained using training document data. During training, the feature extraction model extracts the feature vector of the query and the feature vector of the document data corresponding to that query. The feature extraction model is then trained so that the similarity between the feature vector of the query and the feature vector of the corresponding document data increases, and the similarity for uncorresponding document data decreases. The document data used for training may consist only of the parts of the knowledge document that correspond to the topic or question. This training may be carried out based on the methods shown in the following two papers.

[0033] Paper 1: Wataru Sakata, Tomohide Shibata, Ribeka Tanaka, and Sadao Kurohashi. 2019. FAQ Retrieval using Query-Question Similarity and BERT-Based Query-Answer Relevance. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'19). Association for Computing Machinery, New York, NY, USA, 1113-1116.

[0034] Paper 2: Seo, J.; Lee, T.; Moon, H.; Park, C.; Eo, S.; Aiyanyo, ID; Park, K.; So, A.; Ahn, S.; Park, J. Dense-to-Question and Sparse-to-Answer: Hybrid Retriever System for Industrial Frequently Asked Questions. Mathematics 2022, 10, 1335.

[0035] The management unit 50 receives a query containing one or more pieces of private information from the user terminal 1, retrieves knowledge information corresponding to the query from the knowledge database 60, while concealing the private information contained in the query. The management unit 50 also requests the large-scale language model service 3 to generate a response to the query with the private information concealed, and outputs an output sentence based on the response to the user terminal 1 via the network.

[0036] The input acquisition unit 51 acquires a query containing one or more pieces of privacy information based on user input. The query may be in text form or may contain information other than text. User input is information that the user inputs by operating the user terminal 1 and is received from the user terminal 1. The input acquisition unit 51 may acquire the text input from the user as is as a query, or it may generate a query text by processing the privacy information and free text acquired interactively as user input.

[0037] The Privacy Management Unit 52 extracts one or more pieces of privacy information included in the inquiry and manages the extracted privacy information. The Privacy Extraction Unit 58, included in the Privacy Management Unit 52, extracts the privacy information included in the inquiry. If the inquiry includes the user's age, the Privacy Extraction Unit 58 selects the age range to which the user's age belongs from among several predetermined age ranges. If the inquiry includes the age of another person, the Privacy Extraction Unit 58 selects the age range to which the age of that person belongs from among several predetermined age ranges.

[0038] The related management unit 59, included in the privacy management unit 52, determines the concealed information corresponding to each of the one or more pieces of privacy information included in the query. The related management unit 59 also stores each of the one or more pieces of privacy information included in the query in storage 22, associating it with the concealed information corresponding to that privacy information. The concealed information is information that replaces the privacy information included in the query. The concealed information may be information (tags or dummy information) depending on the type of privacy information (e.g., name, age, gender, medical history).

[0039] In determining the information to be concealed, if the inquiry includes the user's age, the relevant management unit 59 may select one of several ages within the age range selected by the privacy extraction unit 58 (which may be random) as the concealed age for the user's age. If the inquiry includes the ages of the user and other persons, and they belong to the same age range, the relevant management unit 59 may select the concealed ages so that the concealed age for the user and the concealed age for the other persons do not overlap.

[0040] The concealment unit 53 replaces each of the one or more pieces of privacy information included in the inquiry with a different piece of concealment information. If the inquiry includes the user's age, the concealment unit 53 may replace the user's age included in the inquiry with a concealed age for that user. If the inquiry includes the age of another person, the concealment unit 53 may replace the age of the other person included in the inquiry with a concealed age for that other person.

[0041] The knowledge acquisition unit 54 sends a query based on the inquiry to the knowledge database 60 and retrieves knowledge information retrieved from the knowledge database 60 based on that query. If the inquiry includes the user's age, the knowledge acquisition unit 54 retrieves knowledge information from the knowledge database 60, from among multiple pieces of knowledge information stored in the storage 22, that covers the age range to which the user's age belongs. Covering an age range means that the age range is included in the application conditions of the knowledge information, or that the knowledge information contains a string indicating that it covers that age range.

[0042] The response acquisition unit 55 requests the large-scale language model to create a response to the inquiry in which privacy information has been replaced, and obtains the response from the large-scale language model. The response may be in the form of text. Here, the response acquisition unit 55 may request the large-scale language model to create a response to the inquiry in which privacy information has been replaced based on knowledge information. This large-scale language model may be connected to the inquiry management system 2 via the Internet. If the response obtained from the large-scale language model contains an inappropriate string (e.g., a URL), the response acquisition unit 55 may filter that string from the response and use the filtered response for subsequent processing.

[0043] The restoration unit 56 replaces the concealed information contained in the response obtained from the large-scale language model with the privacy information that replaced the concealed information in the query. More specifically, if the response contains concealed information associated with one or more pieces of privacy information contained in the query, the restoration unit 56 replaces that concealed information with the privacy information stored in the storage 22 that is associated with that concealed information.

[0044] The response output unit 57 sends information to the user based on the response in which the hidden information has been replaced. More specifically, the response output unit 57 may process the response in which the hidden information has been replaced and send the processed response to the user. Processing may include formatting the text included in the response, or adding hyperlinks to knowledge information or related information for the response. The response output unit 57 may also determine whether the replaced or processed response is consistent with the acquired knowledge information. If it is determined that the response is consistent, the response output unit 57 may add hyperlinks to the knowledge information or related information to the response, and if it is determined that the response is not consistent, it may add a warning message to the response. Here, the response output unit 57 may determine consistency based on whether each of the multiple words included in the response is included in the words included in the knowledge information.

[0045] Figure 3 is a flowchart illustrating an example of the processing performed by the inquiry management system 2. Figure 3 primarily shows the processing performed by the management unit 50.

[0046] First, the input acquisition unit 51 acquires one or more inquiry statements containing privacy information based on the information entered from the user terminal 1 based on the user's operation (S101). The inquiry statements are mainly inquiries containing text.

[0047] Figure 4 shows an example of an input screen for the inquiry management system 2. The screen shown in Figure 4 is output to the user terminal 1. The screen in Figure 4 displays the inquiry text entered by the user. In the example in Figure 4, the inquiry text is an introduction of the user and their family entered by the user to help them select appropriate insurance. In the example in Figure 4, the input acquisition unit 51 acquires the text entered in one input field as the inquiry text. The input acquisition unit 51 may further detect whether there are any missing items in the text entered so far, and if items are detected, it may output information to the user terminal 1 prompting the user to enter the detected items. In this case, the input acquisition unit 51 may acquire the text entered thereafter and generate an inquiry text from the text entered so far. Note that the inquiry information may include other private information such as an image of the user's face.

[0048] Next, the privacy extraction unit 58 identifies the privacy information contained in the inquiry statement. (S102). The privacy extraction unit 58 also identifies the types of privacy information contained in the query. In this embodiment, the types of privacy information include the user's name, age, gender, medical history, marital status, and telephone number, as well as the names, ages, genders, medical history, marital status, and telephone numbers of family members. The types of privacy information actually handled may be only a part of these, or different classifications may be made. The privacy extraction unit 58 may also extract proper nouns and nouns from the query by morphological analysis and identify the privacy information and its types by comparing the extracted proper nouns and nouns with a pre-created dictionary. Alternatively, the privacy information may be identified based on the output when the query is input to a machine learning model trained on training data including the text and ground truth data including the location and type of privacy information contained in the text.

[0049] Figure 5 shows an example of privacy information extracted from a query. In the example in Figure 5, the "Field" column indicates the type of privacy information, and the "value" column shows the privacy information itself. Figure 5 may further include information indicating the location of the privacy information in the query, and the type of privacy information may be subdivided by user and family relationship categories.

[0050] Once privacy information is identified, the privacy extraction unit 58 converts the privacy information according to its type and determines the query information (S103). If the privacy information identified from the query statement includes the user's age (i.e., includes privacy information of type user's age), the privacy extraction unit 58 selects an age range to which the user's age belongs from a predetermined set of age ranges. The age ranges are set so as not to overlap or be separated, for example, 0-5 years old, 6-17 years old, 18-19 years old, 20-29 years old, 30-39 years old, 40-49 years old, 50-59 years old, and 60-69 years old. The privacy extraction unit 58 may convert gender and illness into codes.

[0051] Figure 6 shows an example of a query based on privacy information. In the example in Figure 6, the query includes fields for age range, gender code, and disease code.

[0052] The related management unit 59 determines the concealment information corresponding to each of the one or more pieces of privacy information included in the query statement and stores them in the storage 22 in association (S104). The related management unit 59 determines the concealment information corresponding to the privacy information according to the type of privacy information. It also stores each piece of privacy information in the storage 22 in association with the corresponding concealment information. The related management unit 59 may determine concealment information that is different from the privacy information for some of the types of the one or more pieces of privacy information included in the query statement.

[0053] If the inquiry contains private information such as the age of a user or another person (privacy information of type being the age of a user or another person), the related management unit 59 selects one of several ages within the age range selected by the privacy extraction unit 58 for that age as the concealed age for that age. This concealed age is specifically related to age among the concealed information. If the identified private information is the name, address, or telephone number of a user or another person (type being the name, address, or telephone number of a user or another person), the related management unit 59 may predetermine a dummy string such as a name associated with that type as the concealed information.

[0054] Figure 7 shows an example of the relationship between privacy information and concealed information. Figure 7 shows an example of privacy information and concealed information stored in storage 22. The relation column indicates the type of person who possesses the privacy information, and together with the field column, it indicates the type of privacy information. self indicates the user themselves, and husband indicates the male spouse. original_value is the column for privacy information included in the query statement, and encoded_value is the column for concealed information. Figure 7 also shows privacy information that is not effectively replaced by concealed information.

[0055] Once the concealed information is determined, the concealment unit 53 replaces each of the one or more pieces of privacy information included in the query with the concealed information (S105). As a result, at least some of the privacy information is replaced with concealed information that is different from that privacy information. The concealment unit 53 may search for the string of privacy information extracted from the query statement and replace the found string with the string of the corresponding concealed information, or the concealment unit 53 may replace the string at the location of the privacy information with the concealed information based on information indicating the location of the privacy information.

[0056] Figure 8 shows an example of a query in which privacy information has been replaced. Figure 8 shows an example of what happens when the query shown in Figure 4 is processed in steps S102 to S105. Comparing Figure 8 with Figure 4, you can see that my name and my and my husband's ages have been replaced with concealed information.

[0057] Furthermore, the knowledge acquisition unit 54 acquires knowledge information from the knowledge database 60 based on the query information determined in S103 (S106). The knowledge acquisition unit 54 may send a query based on the query information determined in S103 to the knowledge database 60 via API, and the knowledge database 60 may acquire the knowledge information found by searching that query via API. This knowledge information is input into the large-scale language model and used as the basis for the answer. The process in S106 may be executed in parallel with the processes in S104 and S105, or in a different order.

[0058] The response acquisition unit 55 then inputs the inquiry statement with the privacy information replaced and the command text containing the knowledge information into the large-scale language model and obtains the output (response) of the large-scale language model (S107). By inputting the command text, the response acquisition unit 55 requests the large-scale language model to create a response to the inquiry statement.

[0059] Figure 9 shows an example of instruction text input to a large-scale language model. In Figure 9, there are strings {article} and {user_introduction}, but in reality, knowledge information (e.g., information about insurance products) and a query statement are set in place of these strings, respectively. The instruction text shown in Figure 9 requests the large-scale language model to generate a response that recommends appropriate insurance products searched based on the query information and privacy information. Note that the large-scale language model may also be input with hyperlinks (URLs) to that information as knowledge information, and if the application is such that a response can be generated without specific knowledge information, the acquisition of knowledge information and input into the large-scale language model may not be performed.

[0060] The query statements included in the command text input to the large-scale language model do not contain any personally identifiable information, especially private information. Therefore, it is possible to prevent such information from being passed to the large-scale language model service 3, which includes the large-scale language model itself. This significantly reduces the risk of private information being leaked by the large-scale language model service 3 and makes it easier to comply with privacy-related rules. In addition, by including dummy private information as concealed information in the query statements, the impact on the creation of responses can be minimized.

[0061] Figure 10 shows an example of a response output from a large-scale language model. Figure 10 is an example of a response generated when an instruction text containing the information shown in Figures 8 and 9, as well as the knowledge information for "cancer insurance A," is input. In some cases, the large-scale language model may generate responses that include hidden information, such as the name field, as shown in Figure 10. In some cases, responses may also be generated that include hidden information such as age and medical history.

[0062] Once a response is obtained, the restoration unit 56 replaces each of the one or more pieces of concealed information contained in the obtained response with the corresponding privacy information (S108). For example, the restoration unit 56 searches the response for each of the one or more pieces of concealed information stored in the storage 22 and associated with the privacy information, to see if the string of that concealed information is included. If the string of that concealed information is included in the response, the restoration unit 56 replaces that string with the privacy information associated with that concealed information.

[0063] When the concealed information is replaced, the response output unit 57 outputs information based on the response in which the concealed information has been replaced to the user terminal 1 operated by the user (S109). The response output unit 57 outputs information that has been processed from that response as information based on the response.

[0064] Figure 11 shows an example of a response in which the hidden information has been replaced. Figure 11 shows an example in which the response shown in Figure 10 is processed by the restoration unit 56 and a hyperlink to information related to the knowledge information is added. In the example in Figure 11, the name written in the first line of the response has been reverted to the user's name.

[0065] As explained above, while it is possible to prevent the leakage of privacy information by concealing privacy-related information from the query text input to the large-scale language model, there is a risk that the concealed information may be included in the response output by the large-scale language model, resulting in an unnatural response. In this embodiment, by converting the concealed information included in the response back to the original privacy information, the query management system 2 can output a natural response. Furthermore, by using dummy information that is closer to reality as the concealed information, the response generated by the large-scale language model itself can be made more natural. In addition, when replacing age with a dummy concealed age, the concealed age is set to be within the same age range, and the setting of that age range is matched to the setting of the age range in the knowledge information. This minimizes the impact on the response when age is changed.

[0066] It should be noted that the concealment methods in this embodiment are not limited to those described above. For example, instead of dummy information corresponding to the type of privacy information, a fixed string of characters, such as a tag indicating the type of privacy information, may be used as concealment information.

[0067] Figure 12 shows another example of a query statement with replaced privacy information. Processing S102-S105 on the query statement shown in Figure 4 results in the query statement shown in the example in Figure 12. However, the concealed information on which the privacy information is replaced is not dummy information that resembles the actual privacy information, but tag information that indicates the type of privacy information itself. In this case, in S104, the process of selecting an age based on an age range is not performed, and the age of the privacy information may simply be associated with a tag of concealed information that indicates age. Also, in this example, the type of privacy information may not include distinctions between user and family, but may simply be a type such as name, age, gender, or medical history.

[0068] Figure 13 shows another example of the command text input to a large-scale language model. In reality, the strings {article} and {user_introduction} in Figure 13 are replaced with knowledge information (e.g., information about insurance products) and a query statement like the one shown in Figure 12, respectively. Unlike the example in Figure 9, the command text input to the large-scale language model in Figure 13 includes information explaining the meaning of the tags in the query statement. Even in this way, it is possible to generate answers without inputting sensitive privacy information into the large-scale language model, and to ensure a certain level of quality in those answers.

[0069] Although a large-scale language model is used in this embodiment, there are no particular limitations on its implementation or the number of parameters. The present invention can be applied to machine learning models (language models) that handle natural language. [Explanation of Symbols]

[0070] 1. User terminal, 2. Inquiry management system, 3. Large-scale language model service, 21 Processor, 22 Storage, 23 Communication Unit, 50 Management Unit, 51 Input Acquisition Unit, 52 Privacy Management Unit, 53 Concealment Unit, 54 Knowledge Acquisition Unit, 55 Response Acquisition Unit, 56 Restoration Unit, 57 Response Output Unit, 58 Privacy Extraction Unit, 59 Related Management Unit, 60 Knowledge Database.

Claims

1. A means for obtaining text containing one or more pieces of private information, A query determination means that determines query information including at least a portion of the one or more pieces of privacy information contained in the text, based on the text, A knowledge acquisition means that searches a knowledge database based on the aforementioned query information and obtains knowledge information, Concealment means for replacing each of the one or more pieces of privacy information with concealment information different from the one or more pieces of privacy information, A response acquisition means that obtains a response from a language model by inputting the text in which the privacy information has been replaced and the knowledge information into the language model, An output means for outputting information based on the acquired response, An information processing system that includes this.

2. In the information processing system described in Claim 1, The query determination means determines query information which includes information in which at least a portion of the one or more pieces of privacy information has been transformed according to the type of privacy information, The concealment means replaces each of the one or more pieces of privacy information with concealment information that is different from the information transformed by the query determination means. Information processing system.

3. In the information processing system described in Claim 1, The privacy information included in the aforementioned query information includes at least some of the following: age, gender, and medical history. Information processing system.

4. In the information processing system described in Claim 1, The above text is a query based on user input. Information processing system.

5. In the information processing system described in claim 1, The concealment means replaces each of the one or more pieces of privacy information contained in the text with concealment information corresponding to the type of privacy information. Information processing system.

6. In the information processing system described in claim 5, The aforementioned one or more types of privacy information include at least some of name, age, gender, and medical history. Information processing system.

7. In the information processing system described in claim 1, The restoration means further includes replacing the concealed information contained in the acquired response with the privacy information that replaced the concealed information in the text, Information processing system.

8. In the information processing system described in claim 7, The system further includes related management means for storing each of the one or more pieces of privacy information in a database in association with the concealed information that replaces the privacy information, The restoration means, if the answer includes concealed information to which one or more of the aforementioned privacy information is associated, replaces the concealed information with the privacy information associated with the concealed information. Information processing system.

9. In the information processing system described in claim 1, If the text includes an age, a range selection means is provided to select the age range to which the age belongs from a predetermined set of age ranges. The text includes an age selection means for selecting one of a plurality of ages within the selected age range as a concealed age for the age contained in the text, The concealment means replaces the age contained in the text with a concealed age for that age. Information processing system.

10. In the information processing system described in claim 9, The age selection means randomly selects one of several ages within the selected age range as a concealed age for the age contained in the text. Information processing system.

11. In the information processing system according to claim 9 or 10, The query determination means determines query information that includes at least a portion of the one or more pieces of privacy information contained in the text and the age range to which the age contained in the text belongs. Information processing system.

12. An information processing system including one or more processors, The steps include obtaining text containing one or more pieces of private information, A step of determining query information that includes at least a portion of the one or more pieces of privacy information contained in the text, based on the text; The steps include searching the knowledge database based on the aforementioned query information and obtaining knowledge information, A step of replacing each of the one or more pieces of privacy information with concealed information that is different from the one or more pieces of privacy information, The steps include: inputting the text with the privacy information replaced and the knowledge information into a language model to obtain a response from the language model; The steps include outputting information based on the acquired responses, Information processing methods including

13. means for obtaining text containing one or more pieces of private information, A query determination means that determines query information including at least a portion of the one or more pieces of privacy information contained in the text, based on the text. A knowledge acquisition means that searches a knowledge database based on the aforementioned query information and obtains knowledge information. Concealment means for replacing each of the one or more pieces of privacy information with concealment information different from the one or more pieces of privacy information, A response acquisition means that obtains a response from a language model by inputting the text with the privacy information replaced and the knowledge information into the language model, and Output means for outputting information based on the acquired response, A program that makes a computer function.

Images