Information processing device, information processing method, and program
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Filing Date
- 2025-12-24
- Publication Date
- 2026-03-26
Abstract
Description
Information processing device, information processing method, and recording medium
[0001] The present disclosure relates to an information processing device, an information processing method, and a storage medium.
[0002] In recent years, AI (Artificial Intelligence) services, including large-scale language models, have become widespread. Non-Patent Document 1 describes a technology that prevents users from sending confidential or sensitive information to AI services. Specifically, the system described in Non-Patent Document 1 reduces the risk of information leakage by users by detecting and identifying "information that should not be released to the outside" from information input by users using a dictionary and machine learning model prepared in advance.
[0003] Conventional information detection methods are mainly divided into "dictionary-based" and "learning-based" methods. Dictionary-based information detection searches for prohibited words in the information entered by the user based on a list of prohibited words prepared in advance, and warns the user, thereby reducing the risk of information leakage. On the other hand, learning-based information detection creates a classification model that "determines whether or not the information can be released to the outside" based on a list of documents that should not be released to the outside and documents that can be released to the outside, and uses the classification model to warn users about how to handle the information.
[0004] "exaBase Generative AI built with GPT-4", [online], April 26, 2023, [Retrieved June 23, 2023], Internet <URL: https: / / exawizards.com / archives / 23982>
[0005] However, in the method using a dictionary or a machine learning model as in Non-Patent Document 1, it is necessary to prepare data that takes into account all possible patterns in advance. Moreover, over time, the information in the dictionary and the information used for learning may become outdated, which may result in missed detections or overdetection.
[0006] One object of the present disclosure is to provide an information processing device that can prevent information leakage when using AI services.
[0007] In one aspect of the present disclosure, an information processing device includes: a search means for searching for documents related to a prompt, which is an input sentence of a machine learning model; an estimation means for estimating, for each of the related documents, the disclosure range of the document; and a disclosure determination means for determining whether the prompt can be made public based on the estimated disclosure range.
[0008] In another aspect of the present disclosure, an information processing method includes searching for documents related to a prompt, which is an input sentence of a machine learning model; estimating a disclosure range for each of the related documents; and determining whether to disclose the prompt based on the estimated disclosure range.
[0009] In yet another aspect of the present disclosure, a recording medium records a program that causes a computer to execute a process of searching for documents related to a prompt, which is an input sentence of a machine learning model, estimating the disclosure range of each related document, and determining whether to disclose the prompt based on the estimated disclosure range.
[0010] According to the present disclosure, it is possible to prevent information leakage when using AI services.
[0011] 1 is a block diagram showing a schematic configuration of an information leakage prevention system. 2 is a block diagram showing a hardware configuration of an information processing device. 3 is a block diagram showing a functional configuration of an information processing device. 4 is an example of information relating to an internal company document. 5 is a flowchart of a disclosure possibility determination process. 6 is a block diagram showing a schematic configuration of an information leakage prevention system according to a second embodiment. 7 is a block diagram showing a functional configuration of an information processing device according to the second embodiment. 8 is a flowchart of a disclosure possibility determination process according to the second embodiment. 9 is an example of a display by a terminal device. 10 is a block diagram showing a functional configuration of an information processing device according to a third embodiment. 11 is a flowchart of a disclosure possibility determination process according to the third embodiment. 12 is another example of a display by a terminal device. 13 is a block diagram showing a functional configuration of an information processing device according to a fourth embodiment. 14 is a flowchart of a process by an information processing device according to the fourth embodiment.
[0012] Preferred embodiments of the present disclosure will now be described with reference to the drawings. <First Embodiment> [Background: Service Forms and Issues of Large-Scale Language Models] To facilitate understanding of the invention, the premise of this embodiment will be described. Large-scale, high-precision LLMs (Large Language Models), such as GPTs (Generative Pre-trained Transformers), are sometimes provided as Software as a Service (SaaS). This is because creating and operating LLMs is expensive, and economies of scale are realized when a few companies exclusively develop and operate them. As such, it has become commonplace for LLMs to be provided as services on a large scale and at a global level. However, in such services, users send information entered into the LLM to a third party via a network. This creates a risk, for example, that users, such as employees, may unintentionally send personal information, insider information, or confidential information outside the company.
[0013] To solve this problem, it is necessary to clarify the rules and policies for managing input information and sending destinations, and then implement appropriate training and monitoring. It is also important for users to take appropriate security measures regarding LLM servers. In conclusion, when utilizing large-scale, high-precision LLMs, it is essential that both users and providers take appropriate security measures to reduce the risk of information leaks. This will enable the vast potential of LLMs to be safely utilized.
[0014] 1 shows the overall configuration of an information leakage prevention system according to the present disclosure. The information leakage prevention system 1 includes a terminal device 5, an information processing device 10, and an external server 20.
[0015] The terminal device 5 is configured, for example, by a personal computer or a tablet terminal, and communicates with the information processing device 10 via wired or wireless communication. The terminal device 5 is operated by an LLM user, such as an employee, and transmits input information (hereinafter also referred to as a "prompt") to the LLM to the information processing device 10.
[0016] The information processing device 10 controls the transmission of a prompt to the external server 20 between the terminal device 5 and the external server 20. Specifically, if the prompt received from the terminal device 5 contains confidential information or sensitive information, the information processing device 10 does not transmit the prompt to the external server 20 but transmits an alert to the terminal device 5. On the other hand, if the prompt received from the terminal device 5 does not contain confidential information or sensitive information, the information processing device 10 transmits the prompt to the external server 20.
[0017] The external server 20 is realized by a server device of a company that provides the LLM, etc. When the external server 20 receives a prompt from the information processing device 10, it generates an answer to the prompt and transmits the answer to the information processing device 10. The information processing device 10 then transmits the answer received from the external server 20 to the terminal device 5.
[0018] The information processing device 10 of this embodiment comprehensively searches all documents within a company to determine whether a prompt contains confidential or sensitive information, thereby enabling it to detect confidential or sensitive information that may be missed by dictionary-based or learning-based information detection.
[0019] [Regarding the Language Model] The language model assumed in this embodiment will be described. The language model is a machine learning model that has learned the relationships between words in a sentence and generates related strings related to a target string from the target string. By using a language model that has learned sentences and phrases from various contexts, it is possible to generate related strings with appropriate content related to the target string. For example, a case will be described in which the language model is used in question answering. The language model receives an input of the question, "What kind of country is Japan?" as the target string. The language model generates a string such as "Japan is an island country in the Northern Hemisphere..." as an answer to the question.
[0020] The learning method of the language model is not particularly limited, but as an example, the language model may be trained to output at least one sentence including an input character string. As a specific example, the language model may be GPT-2 (Generative Pre-Training-2) or GPT-3, which outputs a sentence including an input character string by predicting a character string that is likely to follow the input character string. Other examples of language models include T5 (Text-to-Text Transfer Transformer), BERT (Bidirectional Encoder Representations from Transformers), RoBERTa (Robustly optimized BERT approach), and ELECTRA (Efficiently Learning an Encoder that Classifies Token Replacements Accurately).
[0021] Furthermore, the character strings generated by the language model are not limited to natural languages. For example, the language model may output an artificial language (such as program source code) for a character string input in a natural language. For example, the language model may accept an input question such as "How do I retrieve data containing a specific character string from a database?" as a target character string. The language model may output program source code for performing database processing. Alternatively, the language model may output a natural language corresponding to a character string input in an artificial language. Furthermore, the content generated by the language model is not limited to character strings. For example, the language model may generate image data, video data, audio data, or other data formats corresponding to the input character string.
[0022] 2 is a block diagram showing the hardware configuration of the information processing device 10. As shown in the figure, the information processing device 10 includes a processor 11, an interface (IF) 12, a read-only memory (ROM) 13, a random access memory (RAM) 14, and a storage device 15. Each component is connected to the other via a bus 16, for example.
[0023] The processor 11 is a computer such as a CPU (Central Processing Unit), and executes a program prepared in advance to control the entire information processing device 10. Specifically, the processor 11 may be a CPU, a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), an MPU (Micro Processing Unit), an FPU (Floating Point number Processing Unit), a PPU (Physics Processing Unit), a TPU (Tensor Processing Unit), a quantum processor, a microcontroller, or a combination thereof.
[0024] The processor 11 also loads programs stored in the ROM 13, the storage device 15, etc., and executes each process coded in the program. The processor 11 also functions as a part or all of the information processing device 10. The processor 11 then executes a disclosure permission determination process, which will be described later.
[0025] The IF 12 transmits and receives information between the terminal device 5 and the external server 20. Specifically, the IF 12 receives a prompt input by a user from the terminal device 5 and transmits the received prompt to the external server 20. The IF 12 also receives a response to the prompt from the external server 20 and transmits the received response to the terminal device 5.
[0026] The ROM 13 stores various programs executed by the processor 11. The RAM 14 is used as a working memory while the processor 11 is executing various processes.
[0027] The storage device 15 is a non-volatile, non-transitory storage device such as a disk-shaped recording medium or a semiconductor memory. The storage device 15 may be configured to be detachable from the information processing device 10. The storage device 15 stores various programs executed by the processor 11. The storage device 15 may store machine learning models, learning data, and the like. The storage device 15 may also include a database in which information related to in-house documents is recorded.
[0028] In addition to the above, the information processing device 10 may also include a display device such as a liquid crystal display or a projector, and an input device such as a keyboard or a mouse. These display devices and input devices are used by, for example, an administrator of the information processing device 10 to perform necessary management.
[0029] [Functional Configuration] Fig. 3 is a block diagram showing the functional configuration of the information processing device 10 according to the first embodiment. Functionally, the information processing device 10 includes a related document search unit 111, a storage unit 112, a document disclosure range estimation unit 113, a disclosure permission determination unit 114, and an output unit 115. The storage unit 112 is realized by the storage device 15 shown in Fig. 2. The related document search unit 111, the document disclosure range estimation unit 113, the disclosure permission determination unit 114, and the output unit 115 are configured by the processor 11 shown in Fig. 2.
[0030] The terminal device 5 transmits the prompt input by the user to the information processing device 10. The information processing device 10 receives the prompt from the terminal device 5. The prompt is input to the related document search unit 111 and the output unit 115.
[0031] The related document search unit 111 searches for and acquires documents related to the prompt (hereinafter also referred to as "related documents") from the storage unit 112. Note that the storage unit 112 stores information about internal company documents. The related document search unit 111 outputs the related documents to the document disclosure range estimation unit 113.
[0032] Specifically, the related document searching unit 111 generates a search query based on a prompt input by the user. For example, the related document searching unit 111 may use the prompt as the search query as is, or may use a characteristic word in the prompt as the search query. The related document searching unit 111 may also generate the search query using a language model that generates a search query from the prompt. Then, the related document searching unit 111 searches the storage unit 112 for documents related to the search query.
[0033] FIG. 4 shows an example of information about internal documents stored in the storage unit 112. In the table data in FIG. 4, each row represents one document, and each column indicates information about that document. The table data in FIG. 4 includes an ID, document content, update date and time, creator, disclosure level, and file storage location. "ID" is an identifier for uniquely identifying a document. "Document content" indicates the document title or subject of the document. Note that "Document content" may also include information about the entire document. "Update date and time" indicates the date and time the document was last updated. "Creator" indicates the creator of the document or the person who last updated the document. "Disclosure level" indicates the scope of disclosure of the document. For example, "Confidential" indicates information that is only known to individuals within a specific department of the company and is not disclosed outside the company. "Confidential" indicates information that is only known to company employees and is not disclosed outside the company. "Public" indicates information that is disclosed outside the company. "File storage location" indicates the location where each document is stored. For example, the "file storage location" stores the URL of a cloud storage, an in-house server, etc. Note that the information about the document in Fig. 4 is an example, and may include other information that can be managed by the document management system, such as the document version, the project or department related to the document, tags or categories, and approval status.
[0034] The related document search unit 111 outputs documents related to the search query as related documents to the document disclosure range estimation unit 113. Note that the related document search unit 111 may also output documents related to the search query and information about those documents as related documents to the document disclosure range estimation unit 113.
[0035] The related document search unit 111 may acquire a relevance score in addition to acquiring documents related to the search query. The relevance score indicates the degree of relevance between the search query and the documents in the search results. A higher relevance score indicates a stronger relevance between the search query and the documents in the search results. For example, if a search is performed for "how to raise cats," documents that specifically explain how to raise cats will have a high relevance score. On the other hand, documents that are not related to cats will have a low relevance score. The related document search unit 111 may output documents having a relevance score equal to or greater than a predetermined threshold as related documents to the document disclosure range estimation unit 113. Alternatively, the related document search unit 111 may acquire a predetermined number of related documents in descending order of relevance score and output them to the document disclosure range estimation unit 113.
[0036] The document disclosure range estimation unit 113 receives the related documents as input from the related document search unit 111. The document disclosure range estimation unit 113 estimates the disclosure range of the related documents and outputs the estimation result to the disclosure permission determination unit 114. Specifically, the document disclosure range estimation unit 113 can estimate the disclosure range of the related documents using the following method.
[0037] (Estimation Method 1) The document disclosure range estimation unit 113 may estimate the disclosure range using the permission values of related documents. Permissions refer to access rights set for files and folders, and operations that can be performed on files and folders (e.g., read, write, delete) are preset for specific users or groups. For example, if a related document has access rights set to "confidential" or if only specific departments or project members are permitted to access it, the document disclosure range estimation unit 113 estimates that the disclosure range of the related document is "private." On the other hand, if a related document is permitted to be accessed by all employees, the document disclosure range estimation unit 113 estimates that the disclosure range of the related document is "internal disclosure."
[0038] (Estimation Method 2) The document disclosure scope estimation unit 113 may estimate the disclosure scope using a folder structure. Specifically, the document disclosure scope estimation unit 113 estimates the disclosure scope of a related document based on the file name of the related document and the folder name of the folder in which the related document is stored. For example, if the file name of the related document or the folder name in which the related document is stored contains words such as "confidential" or "internal transaction information," or if the file name of the related document or the folder name in which the related document is stored are named in a specific format (e.g., "XXX_confidential_YYY"), the document disclosure scope estimation unit 113 estimates that the related document or the related documents stored in that folder are likely to be confidential documents and estimates that the disclosure scope of the related document is "private."
[0039] Furthermore, if the related documents are stored in a subfolder, the document disclosure range estimation unit 113 performs the same process as described above for the subfolder to estimate the disclosure range of the related documents stored in the subfolder. Note that if the folder name of the parent folder includes a phrase such as "confidential," the document disclosure range estimation unit 113 estimates the disclosure range of the related documents stored in the subfolder to be "private," regardless of the folder name of the subfolder or the file name of the related documents stored in the subfolder. For example, if a subfolder called "Project A" exists in the "Confidential" folder, the document disclosure range estimation unit 113 estimates the disclosure range of the related documents stored in "Project A" to be "private."
[0040] (Estimation Method 3) The document disclosure range estimation unit 113 may estimate the disclosure range based on information contained in the related document. For example, the document disclosure range estimation unit 113 may estimate the disclosure range of the related document based on phrases such as "confidential" or "internal use only" contained in the related document, or on "department name," "person's name," or "job title" contained in the related document.
[0041] The above-described methods for estimating the disclosure range are merely examples, and the present invention is not limited to these.
[0042] The disclosure determination unit 114 receives an estimation result from the document disclosure range estimation unit 113. The disclosure determination unit 114 determines whether to disclose the prompt based on the estimation result. Specifically, the disclosure determination unit 114 determines whether to disclose the prompt based on predetermined rules. Examples of predetermined rules are shown below. Example 1: If at least one of the multiple related documents has a disclosure range limited to within the company, the disclosure is determined to be "not possible." Example 2: If the creation date or update date of all related documents is a certain period ago, the disclosure is determined to be "possible." Example 3: If more than a certain percentage of the multiple related documents have access rights set only to specific departments or project teams, the disclosure is determined to be "not possible." Example 4: If more than a certain percentage of the multiple related documents have access rights limited to specific departments or project teams, the disclosure is determined to be "not possible." Example 5: If more than a certain percentage of the multiple related documents have already been disclosed, the disclosure is determined to be "possible."
[0043] In addition, when there is a document set of related documents that mention the same matter, the disclosure advisability determination unit 114 may determine the disclosure range by selecting the document with the most recent update date and time among the document set. This makes it possible to deal with situations in which the disclosure range changes over time, such as when an important fact that constitutes insider information is disclosed and then no longer constitutes insider information.
[0044] The disclosure determination unit 114 outputs the determination result as to whether the prompt can be disclosed to the output unit 115 .
[0045] The method for determining whether a prompt should be made public is not limited to the above method. The public / non-public determination unit 114 can determine whether a prompt should be made public using various methods that use the prompt and related documents. For example, the public / non-public determination unit 114 may determine whether a prompt should be made public using an online optimization algorithm such as Bayesian optimization. The information that a user wants to keep secret may depend on the user's preferences. By using online optimization, the public / non-public determination unit 114 can determine whether a prompt should be made public in a manner that reflects the user's preferences.
[0046] Here, we will explain Bayesian optimization as an example of an online optimization algorithm. Bayesian optimization is an optimization method that uses Bayesian statistics, and it makes it possible to use user feedback to determine whether or not to publish information while reflecting the user's preferences.
[0047] The publication advisability determination unit 114 determines whether to publish the input prompt using Bayesian optimization, taking uncertainty into consideration. The user inputs feedback regarding the determination result output by the publication advisability determination unit 114, such as whether the determination result is appropriate. The publication advisability determination unit 114 then processes the feedback input by the user based on a Bayesian optimization algorithm. The publication advisability determination unit 114 may also accumulate user feedback data and build a machine learning model. The publication advisability determination unit 114 may also predict user responses using a trained machine learning model. This enables the publication advisability determination unit 114 to determine whether to publish a prompt while taking into consideration the preferences of each individual user, thereby increasing user satisfaction.
[0048] The output unit 115 receives an input of the determination result of whether the prompt can be made public from the publication possibility determination unit 114. If the determination result is "public," the output unit 115 transmits the prompt to the external server 20. On the other hand, if the determination result is "public," the output unit 115 does not transmit the prompt to the external server 20, but transmits an alert to the terminal device 5.
[0049] In the above configuration, the related document search unit 111 and the memory unit 112 are examples of a search means, the document disclosure range estimation unit 113 is an example of an estimation means, the disclosure eligibility determination unit 114 is an example of a disclosure eligibility determination means, and the output unit 115 is an example of an output means.
[0050] [Disclosure Determination Process] Next, the disclosure determination process will be described. Fig. 5 is a flowchart of the disclosure determination process performed by the information processing device 10. This process is realized by the processor 11 shown in Fig. 2 executing a program prepared in advance and operating as each element shown in Fig. 3.
[0051] First, the terminal device 5 transmits a prompt input by the user to the information processing device 10. The information processing device 10 receives the prompt from the terminal device 5. The related document searching unit 111 acquires the prompt (step S11). Next, the related document searching unit 111 searches for and acquires related documents from the storage unit 112 (step S12). Then, the related document searching unit 111 outputs the related documents to the document disclosure range estimation unit 113.
[0052] Next, the document disclosure range estimation unit 113 estimates the disclosure range of the related document (step S13). For example, the document disclosure range estimation unit 113 may estimate the disclosure range using the permission value of the document, may estimate the disclosure range using the folder structure, or may estimate the disclosure range based on information included in the related document. The document disclosure range estimation unit 113 outputs the estimation result to the disclosure permission determination unit 114.
[0053] Next, the disclosure determination unit 114 determines whether the prompt should be made public based on the estimation result (step S14). The disclosure determination unit 114 may determine whether the prompt should be made public based on, for example, predetermined rules, or may determine whether the prompt should be made public using an online optimization algorithm such as Bayesian optimization. The disclosure determination unit 114 then outputs the determination result to the output unit 115.
[0054] If the determination result is "not open to the public" (step S15: No), the output unit 115 transmits an alert to the terminal device 5 (step S16). On the other hand, if the determination result is "open to the public" (step S15: Yes), the output unit 115 transmits a prompt to the external server 20 (step S17). Then, the process ends.
[0055] [Modification] Next, a modification of the first embodiment will be described. The information processing device of the first embodiment may be applied to the medical or healthcare field.
[0056] In the fields of medicine and healthcare, a lot of confidential information, such as personal information of patients, is handled. Therefore, when medical professionals at hospitals and medical institutions use external LLM services, there is a risk of information leakage, as confidential information may be transmitted to external parties. In response to this, by providing a monitoring service that implements the information processing device of this embodiment, it is possible to reduce the risk of information leakage.
[0057] For example, a medical professional sends a prompt to an information processing device via a terminal device. The prompt includes, for example, questions about a patient's condition or treatment method. The information processing device determines whether the prompt corresponds to confidential information or sensitive information. For example, if the prompt includes the patient's personal information (such as name, address, or telephone number), the information processing device determines that the prompt corresponds to confidential information or sensitive information.
[0058] If the prompt does not correspond to confidential or sensitive information, the information processing device transmits the prompt to an external LLM service. On the other hand, if the prompt corresponds to confidential or sensitive information, the information processing device transmits an alert to the medical professional's terminal device. The alert may include, for example, information indicating that the prompt contains confidential or sensitive information, or information instructing the medical professional to delete or obscure the confidential or sensitive information. The alert may also include information instructing the medical professional to use another LLM, such as an LLM available on a server within the hospital. This allows the medical professional to understand that the prompt corresponds to confidential or sensitive information. The medical professional can also modify the prompt according to the content of the alert.
[0059] According to the modification of the first embodiment, even if a medical professional accidentally enters a patient's personal information into a prompt when using an external LLM service, the monitoring service sends an alert to the medical professional, thereby reducing the risk of information leakage. This allows medical professionals to use the LLM service with peace of mind.
[0060] Second Embodiment Next, a second embodiment will be described. The information leakage prevention system of the second embodiment can select the LLM to use depending on the content of the prompt. Note that the hardware configuration is the same as that of the first embodiment, so a description thereof will be omitted.
[0061] [Overall Configuration] Fig. 6 shows the overall configuration of the information leakage prevention system according to the second embodiment. The information leakage prevention system 1a according to the second embodiment includes a terminal device 5, an information processing device 10a, an external server 20, and an internal server 30. The internal server 30 is implemented by an in-house server device or the like, and provides an LLM created for in-house use. Note that the internal server 30 can only be accessed by specific individuals, such as employees.
[0062] The information processing device 10a controls the destination of a prompt received from the terminal device 5. Specifically, if the prompt received from the terminal device 5 contains confidential information or sensitive information, the information processing device 10a sends the prompt to the internal server 30. On the other hand, if the prompt received from the terminal device 5 does not contain confidential information or sensitive information, the information processing device 10a sends the prompt to the external server 20.
[0063] When the external server 20 or the internal server 30 receives the prompt from the information processing device 10a, it generates a response to the prompt and transmits the response to the information processing device 10a. The information processing device 10a then transmits the response received from the external server 20 or the internal server 30 to the terminal device 5.
[0064] In this way, if the prompt contains confidential or sensitive information, the LLM provided by the in-house server can be used, allowing the user to use the LLM without modifying the content of the prompt.
[0065] 7 is a block diagram showing the functional configuration of an information processing device 10a according to the second embodiment. Functionally, the information processing device 10a includes a related document search unit 111, a storage unit 112, a document disclosure range estimation unit 113, a disclosure permission determination unit 114, and an LLM selection unit 115a. The related document search unit 111, the storage unit 112, the document disclosure range estimation unit 113, and the disclosure permission determination unit 114 have the same configurations and operate in the same manner as those of the information processing device 10 according to the first embodiment, and therefore, a description thereof will be omitted.
[0066] A prompt is input to the LLM selection unit 115a. The LLM selection unit 115a also receives a determination result on whether the prompt can be made public from the disclosure determination unit 114. The LLM selection unit 115a selects the most appropriate LLM based on the determination result. Specifically, if the determination result is "disclosure possible," the LLM selection unit 115a transmits a prompt to the external server 20. On the other hand, if the determination result is "disclosure not possible," the LLM selection unit 115a transmits a prompt to the internal server 30.
[0067] [Disclosure Determination Process] Next, the disclosure determination process will be described. Fig. 8 is a flowchart of the disclosure determination process performed by the information processing device 10a. This process is realized by the processor 11 shown in Fig. 2 executing a program prepared in advance and operating as each element shown in Fig. 7.
[0068] Note that steps S21 to S24 are similar to steps S11 to S14 of the first embodiment shown in FIG. 5, and therefore a description thereof will be omitted.
[0069] The disclosure determination unit 114 outputs the determination result of whether the prompt can be disclosed to the LLM selection unit 115a. If the determination result is "disclosure not permitted" (step S25: No), the LLM selection unit 115a transmits the prompt to the internal server 30 (step S26). On the other hand, if the determination result is "disclosure permitted" (step S25: Yes), the LLM selection unit 115a transmits the prompt to the external server 20 (step S27). Then, the process ends.
[0070] [Modification] Next, a modification of the second embodiment will be described. The information processing device of the second embodiment may be applied to the medical or healthcare field. As in the first embodiment, by providing a monitoring service that implements the information processing device of the second embodiment, it is possible to reduce the risk of information leakage that occurs when using the LLM service.
[0071] For example, a medical professional sends a prompt to an information processing device via a terminal device. The information processing device determines whether the prompt corresponds to confidential information or sensitive information. The information processing device then selects a destination to send the prompt to based on the determination result. Specifically, if the prompt corresponds to confidential information or sensitive information, the information processing device sends the prompt to an internal LLM service. On the other hand, if the prompt does not correspond to confidential information or sensitive information, the information processing device sends the prompt to an external LLM service. Note that the internal LLM service is, for example, an LLM service provided by a server device within a hospital, and is an LLM service that is available only to specific individuals, such as medical professionals at the hospital.
[0072] 9A and 9B are examples of displays on the terminal device 5. Fig. 9A shows an example of a prompt input screen. The input screen 200 includes a prompt input area 201. The prompt input area 201 is an area where a medical professional inputs a prompt.
[0073] FIG. 9B shows an example of a display of an answer to a prompt. The answer screen 210 includes an LLM selection result 211, an answer area 212, and an evaluation button 213. The LLM selection result 211 is an area showing the LLM selected by the information processing device. In FIG. 9B, "Internal LLM selected" is displayed. This display allows the healthcare professional to understand that the prompt contains confidential or sensitive information, that the prompt was sent to an internal LLM service, and that an answer was generated by the internal LLM. The answer area 212 is an area displaying the answer generated by the LLM. The evaluation button 213 is a button that allows the healthcare professional to input an evaluation of the LLM selection result. The evaluation button 213 allows the information processing device to obtain feedback on the LLM selection result. Such feedback can be used as feedback for the online optimization described above.
[0074] Third Embodiment Next, a third embodiment will be described. The information leakage prevention system of the third embodiment can mask confidential information or sensitive information contained in a prompt and transmit the masked information to an external LLM service. Note that the overall configuration and hardware configuration are the same as those of the first embodiment, and therefore will not be described here.
[0075] 10 is a block diagram showing the functional configuration of an information processing device 10b according to the third embodiment. Functionally, the information processing device 10b includes a related document search unit 111, a storage unit 112, a document disclosure range estimation unit 113, a disclosure permission determination unit 114b, an output unit 115b, and an anonymization unit 116b. The related document search unit 111, the storage unit 112, and the document disclosure range estimation unit 113 have the same configurations and operate in the same manner as those of the information processing device 10 according to the first embodiment, and therefore, a description thereof will be omitted.
[0076] A prompt is input to the disclosure possibility determination unit 114b. In addition, an estimation result is input to the disclosure possibility determination unit 114b from the document disclosure range estimation unit 113. The disclosure possibility determination unit 114b determines whether the prompt can be made public based on the estimation result. If the determination result is "disclosure possible," the disclosure possibility determination unit 114b outputs the prompt to the output unit 115b. On the other hand, if the determination result is "disclosure not possible," the disclosure possibility determination unit 114b outputs the prompt to the anonymization unit 116b.
[0077] The anonymization unit 116b receives a prompt from the disclosure determination unit 114b. The anonymization unit 116b masks confidential information or sensitive information included in the prompt. For example, the anonymization unit 116b uses natural language processing technology or a pre-trained machine learning model to detect parts of the prompt that correspond to confidential information or sensitive information, and masks the parts. Note that the masking methods described above are merely examples and are not limited to these. The anonymization unit 116b outputs the masked prompt to the output unit 115b.
[0078] The anonymization unit 116b may repeat the process of determining whether the prompt can be made public by outputting the masked prompt to the related document search unit 111. For example, the anonymization unit 116b may repeat the masking of the prompt and the process of determining whether the prompt can be made public until it is determined that the prompt is "public." Alternatively, the anonymization unit 116b may repeat the masking of the prompt and the process of determining whether the prompt can be made public a predetermined number of times, and if the prompt is still "not public" when the masking and the process of determining whether the prompt can be made public have been repeated the predetermined number of times, the anonymization unit 116b may request the user to re-input the prompt.
[0079] The output unit 115b receives a prompt from the disclosure determination unit 114b or the anonymization unit 116b. The output unit 115b transmits the prompt to the external server 20.
[0080] [Disclosure Determination Process] Next, the disclosure determination process will be described. Fig. 11 is a flowchart of the disclosure determination process performed by the information processing device 10b. This process is realized by the processor 11 shown in Fig. 2 executing a program prepared in advance and operating as each element shown in Fig. 10.
[0081] Note that steps S31 to S34 are similar to the processes of steps S11 to S14 in the first embodiment shown in FIG. 5, and therefore a description thereof will be omitted.
[0082] The disclosure possibility determination unit 114b determines whether the prompt can be made public based on the estimation result. If the determination result is "disclosure possible" (step S35: Yes), the disclosure possibility determination unit 114b outputs the prompt to the output unit 115b. On the other hand, if the determination result is "disclosure not possible" (step S35: No), the disclosure possibility determination unit 114b outputs the prompt to the anonymization unit 116b.
[0083] The anonymization unit 116b masks confidential or sensitive information included in the prompt (step S36). The anonymization unit 116b outputs the masked prompt to the output unit 115b. The output unit 115b transmits the prompt input from the disclosure permission determination unit 114b or the anonymization unit 116b to the external server 20 (step S37). Then, the processing ends.
[0084] [Modification] Next, a modification of the third embodiment will be described. The information processing device of the third embodiment may be applied to the medical or healthcare field. As in the first embodiment, by providing a monitoring service that implements the information processing device of the third embodiment, it is possible to reduce the risk of information leakage that occurs when using the LLM service.
[0085] For example, a medical professional may send a prompt to an information processing device via a terminal device. The information processing device may determine whether the prompt contains confidential or sensitive information. If the prompt contains confidential or sensitive information, the information processing device may mask the confidential or sensitive information contained in the prompt before transmitting the prompt to an external LLM service.
[0086] 12A and 12B show examples of displays on the terminal device 5. Fig. 12A shows an example of a prompt input screen. The input screen 300 includes a prompt input area 301. The prompt input area 301 is an area where a medical professional inputs a prompt.
[0087] FIG. 12(B) shows an example of a masked prompt. The masking result screen 310 includes a masking result 311, a send button 312, and a re-enter button 313. The masking result 311 is an area for displaying a prompt masked by the information processing device. In FIG. 12(B), the patient's name and the name of the nearest station are masked. The send button 312 is a button for sending the prompt. For example, if the medical professional wants to send the masked prompt to an external LLM service, the medical professional presses "Send." The re-enter button 313 is a button for re-entering the prompt. If the medical professional wants to re-enter the prompt without sending the masked prompt to the external LLM service, the medical professional presses "re-enter."
[0088] 13 is a block diagram showing the functional configuration of an information processing apparatus according to Embodiment 4. The information processing apparatus 400 includes a search unit 401, an estimation unit 402, and a disclosure permission / non-permission determination unit 403.
[0089] 14 is a flowchart of processing by the information processing apparatus of the fourth embodiment. The search unit 401 searches for documents related to a prompt, which is an input sentence of a machine learning model (step S401). The estimation unit 402 estimates the disclosure range of each related document (step S402). The disclosure availability determination unit 403 determines whether the prompt should be disclosed based on the estimated disclosure range (step S403).
[0090] According to the information processing device 400 of the fourth embodiment, it is possible to prevent information leakage when using an AI service, and also to encourage the user to make appropriate decisions regarding information leakage.
[0091] Some or all of the above-described embodiments can be described as, but are not limited to, the following supplementary notes.
[0092] (Supplementary Note 1) An information processing device comprising: a search means for searching for documents related to a prompt, which is an input sentence of a machine learning model; an estimation means for estimating the disclosure range of each of the related documents; and a disclosure determination means for determining whether the prompt can be made public based on the estimated disclosure range.
[0093] (Appendix 2) An information processing device according to Appendix 1, comprising an output means for outputting the prompt, wherein the output means outputs the prompt to a first server if the means for determining whether or not the information is publicly available determines that the information is publicly available, and outputs the prompt to a second server if the means for determining whether or not the information is publicly available determines that the information is not publicly available.
[0094] (Appendix 3) An information processing device according to Appendix 1, comprising: an output means for outputting the prompt; and a document replacement means for replacing a part of a character string included in the prompt with another character string, wherein the document replacement means replaces personal information or confidential information included in the prompt with an abstract character string if the disclosure eligibility determination means determines that the prompt is not to be disclosed, and the output means outputs the prompt after the replacement to a first server.
[0095] (Appendix 4) An information processing device according to Appendix 1, further comprising an output means for outputting the prompt, wherein the output means outputs the prompt to a first server when the means for determining whether or not the information is publicly available determines that the information is publicly available, and outputs an alert to a user terminal when the means for determining whether or not the information is publicly available determines that the information is not publicly available.
[0096] (Supplementary Note 5) The information processing device of Supplementary Note 1, further comprising: a storage means for storing in-house document information; and the search means for searching for a document related to a prompt from the in-house document information.
[0097] (Supplementary Note 6) In the information processing device of Supplementary Note 1, when there are multiple documents related to a prompt that mention the same matter, the disclosure determination means determines whether the prompt should be made public for the document with the most recent update date and time among the multiple documents.
[0098] (Supplementary Note 7) An information processing method that searches for documents related to a prompt, which is an input sentence of a machine learning model, estimates the disclosure range of each related document, and determines whether to disclose the prompt based on the estimated disclosure range.
[0099] (Appendix 8) A recording medium having recorded thereon a program that causes a computer to execute a process of searching for documents related to a prompt, which is an input sentence of a machine learning model, estimating the disclosure range of each of the related documents, and determining whether or not to disclose the prompt based on the estimated disclosure range.
[0100] Although the present disclosure has been described above with reference to the embodiments and examples, the present disclosure is not limited to the above-described embodiments and examples. Various modifications that can be understood by a person skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure.
[0101] REFERENCE SIGNS LIST 1, 1a Information leakage prevention system 5 Terminal device 10, 10a, 10b Information processing device 20 External server 30 Internal server 111 Related document search unit 112 Storage unit 113 Document disclosure range estimation unit 114, 114b Disclosure possibility determination unit 115, 115b Output unit 115a LLM selection unit 116b Anonymization unit
Claims
1. A search method for searching for documents related to the prompt, which is the input sentence of a machine learning model, For each of the aforementioned related documents, there is an estimation means for estimating the scope of disclosure of the said document, A means for determining whether the prompt can be made public based on the estimated scope of public disclosure, An information processing device equipped with the following features.
2. The system includes output means for outputting the aforementioned prompt, The information processing apparatus according to claim 1, wherein the output means outputs the prompt to a first server when the disclosure approval / disclosure determination means determines that disclosure is possible, and outputs the prompt to a second server when the disclosure approval / disclosure determination means determines that disclosure is not possible.
3. Output means for outputting the aforementioned prompt, A document replacement means that replaces a part of the string contained in the prompt with another string, Equipped with, If the disclosure approval / disclosure determination means determines that disclosure is not permitted, the document replacement means replaces the personal information and confidential information contained in the prompt with an abstract string. The information processing apparatus according to claim 1, wherein the output means outputs the replaced prompt to the first server.
4. The system includes output means for outputting the aforementioned prompt, The information processing apparatus according to claim 1, wherein the output means outputs the prompt to the first server when the disclosure approval / disclosure determination means determines that disclosure is possible, and outputs an alert to the user terminal when the disclosure approval / disclosure determination means determines that disclosure is not possible.
5. Equipped with a storage means for storing internal document information, The information processing device according to claim 1, wherein the search means searches for documents related to a prompt from the internal document information.
6. The information processing device according to claim 1, wherein, if there are multiple documents relating to a prompt that refer to the same matter, the means for determining whether or not to disclose the prompt determines whether or not to disclose the prompt based on the document with the most recent update date and time among the multiple documents.
7. An information processing method performed by a computer, Search for documents related to the prompt, which is the input sentence for the machine learning model. For each of the aforementioned related documents, the scope of public access for that document is estimated. An information processing method for determining whether or not the prompt can be made public based on the estimated scope of public disclosure.
8. Search for documents related to the prompt, which is the input sentence for the machine learning model. For each of the aforementioned related documents, the scope of public access for that document is estimated. A program that causes a computer to perform a process to determine whether or not the prompt can be made public based on the estimated scope of public access.