Information leakage suppression device, information leakage suppression method, and program

Abstract

A terminal device (1) includes: a tokenizer (112) that converts to-be-evaluated text into a token string; a to-be-warned subject probability estimation unit (116) that estimates a probability of to-be-warned subject start and a probability of to-be-warned subject end of each token included in the token string by using a to-be-warned subject probability estimation model; a to-be-warned token identification unit (117) that identifies a range of to-be-warned tokens on the basis of the probability of to-be-warned subject start and the probability of to-be-warned subject end; and an evaluation result text generation unit (118) that generates evaluation result text information indicating an evaluation result of an information leakage possibility on the basis of text information indicating the text and the identified range of to-be-warned tokens.

Description

Information Leakage Suppression Device, Information Leakage Suppression Method, and Program

[0001] The present invention relates to an information leakage suppression device, an information leakage suppression method, and a program.

[0002] An information processing device has been proposed that accepts questions from a user, identifies at least one predetermined consultation response data from a plurality of consultation response data consisting of questions or answers to past legal consultations based on the accepted question, and generates a prompt that is an input sentence for an interactive response AI based on the accepted question and the identified predetermined consultation response data (see, for example, Patent Document 1).

[0003] Japanese Unexamined Patent Application Publication No. 2024 - 162162

[0004] By the way, the interactive response AI used by the information processing device as described in Patent Document 1 is generally provided as an external so - called response AI service. And the information processing device acquires response data by transmitting the generated prompt to an external server via a wide - area network. In this case, if the prompt contains text information indicating so - called confidential information such as personal names, personal addresses, ideas for new products, etc., there is a risk that the confidential information will leak outside in a form unintended by the user.

[0005] The present invention has been made in view of the above reasons, and an object thereof is to provide an information leakage suppression device, an information leakage suppression method, and a program that can suppress the leakage of confidential information when using an information processing service provided externally.

[0006] To achieve the above objective, the information leakage suppression device according to the present invention comprises: a tokenizer that converts text shown in text information subject to evaluation of the possibility of information leakage into a sequence of tokens; a warning target probability estimation unit that estimates the warning target start probability, which is the probability that the token is the first token in a range of warning target tokens subject to information leakage warning, and the warning target end probability, which is the probability that the token is the last token in the range of warning target tokens, using a warning target probability estimation model to estimate the warning target start probability and the warning target end probability for each of the tokens included in the sequence of tokens; a warning target token identification unit that identifies the range of warning target tokens and the location of the range of warning target tokens based on the warning target start probability and the warning target end probability; an evaluation result text generation unit that generates evaluation result text information showing an information leakage possibility evaluation result in which the warning target string is displayed in a different display manner from other strings, based on the text information and the identified range of warning target tokens and the location of the range of warning target tokens; a display unit; and a display control unit that causes the generated evaluation result text information to be displayed on the display unit.

[0007] According to the present invention, the warning target probability estimation unit estimates the warning start probability and warning end probability for each token included in the token sequence corresponding to the text indicated by the text information to be evaluated, using the aforementioned warning target probability estimation model. The warning target token identification unit identifies the warning target token range and the location of the warning target token range based on the warning target start probability and warning target end probability. The evaluation result text generation unit then generates evaluation result text information indicating the information leakage possibility evaluation result, in which the warning target string is displayed in a different display manner from other strings, based on the text information and the identified warning target token range and location of the warning target token range. As a result, users can quickly understand whether or not the text to be evaluated contains a target string that they want to prevent from being leaked to the outside, simply by referring to the evaluation result text information. Therefore, for example, when using an information processing service provided externally with the text to be evaluated, it is possible to prevent the text containing the aforementioned target string from being provided to the information processing service as is, thereby preventing the leakage of confidential information managed by the user.

[0008] This is a schematic diagram of the information leakage suppression system according to Embodiment 1 of the present invention. This is a block diagram showing the hardware configuration of the terminal device according to Embodiment 1. This is a block diagram showing the functional configuration of the terminal device according to Embodiment 1. This is a sequence diagram for explaining the operation of the information leakage suppression system according to Embodiment 1. This is a diagram showing an example of an evaluation result notification image displayed on the display unit of the terminal device according to Embodiment 1. This is a flowchart showing an example of the flow of information leakage suppression processing performed by the terminal device according to Embodiment 1. This is a flowchart showing an example of the flow of information leakage suppression processing performed by the terminal device according to Embodiment 1. This is a flowchart showing an example of the flow of teacher information generation processing according to Embodiment 1. This is a block diagram showing the functional configuration of the terminal device according to Embodiment 2. This is a sequence diagram for explaining the operation of the information leakage suppression system according to Embodiment 2. This is a diagram showing an example of query information before conversion according to Embodiment 2. This is a diagram showing an example of query information after conversion according to Embodiment 2. This is a diagram showing an example of query information to be sent to the large-scale language model management server according to Embodiment 2. This is a diagram showing an example of answer information before conversion according to Embodiment 2. This is a diagram showing an example of answer information after conversion according to Embodiment 2. This is a flowchart showing an example of the flow of information leakage suppression processing performed by the terminal device according to Embodiment 2. This is a flowchart showing an example of the flow of information leakage suppression processing performed by the terminal device according to Embodiment 2. This is a flowchart showing an example of the flow of teacher information generation processing according to Embodiment 2.

[0009] (Embodiment 1) Hereinafter, an information leakage suppression device according to one embodiment of the present invention will be described in detail with reference to the drawings. The information leakage suppression device according to this embodiment includes: a target text storage unit that stores text information to be evaluated for the possibility of information leakage; a tokenizer that converts the text indicated by the text information into a token sequence; a warning target probability estimation unit that estimates the warning target start probability and the warning target end probability for each token included in the token sequence using a warning target probability estimation model for estimating the warning target start probability, which is the probability that the token is the first token in a warning target token range that is subject to information leakage warning, and the warning target end probability, which is the probability that the token is the last token in a warning target token range; a warning target token identification unit that identifies a warning target token range based on the warning target start probability and the warning target end probability; an evaluation result text generation unit that generates text information and evaluation result text information indicating an information leakage possibility evaluation result in which the warning target string is displayed in a different display manner from other strings based on the identified warning target token range; a display unit; and a display control unit that causes the generated evaluation result text information to be displayed on the display unit.

[0010] The information leakage suppression system according to this embodiment includes, for example, as shown in Figure 1, an LLM management server 2 that manages a large-scale language model (hereinafter referred to as "LLM"), and a terminal device 1 that can communicate with the LLM management server 2 via a network NW1.

[0011] The LLM management server 2 manages an LLM based on the Transformer architecture and includes a query notification acquisition unit (not shown) that acquires query notification information transmitted from the terminal device 1 (described later), a query storage unit (not shown), a tokenizer (not shown), an answer generation unit that generates a token sequence corresponding to the answer information using the LLM, a detoxifier (not shown), and an answer notification unit (not shown) that generates the answer notification information described later and transmits it to the terminal device 1. Examples of LLMs include BERT (Bidirectional Encoder Representations from Transformers), GPT-4 (Generative Pre-trained Transformer 4), and PaLM2 (Pathways Language Model 2). The query storage unit stores the query information in association with source identification information that identifies the terminal device 1 that sent the query information. The query notification acquisition unit acquires query notification information transmitted from terminal device 1, extracts the query information contained in the acquired query notification information, associates the extracted query information with source identification information that identifies the source of the query information, and stores it in the query storage unit. The tokenizer is, for example, Sentencepiece, which converts the text indicated by the query information stored in the query storage unit into a token sequence and inputs it to the answer generation unit. The answer generation unit uses LLM to generate a token sequence corresponding to the answer information and outputs it to the detokenizer. The detokenizer generates the answer information by converting the input token sequence into text. The answer notification unit generates answer notification information including the generated answer information and the source identification information stored in the query storage unit and sends it to terminal device 1, the source of the query information.

[0012] Terminal device 1 is, for example, a personal computer, and as shown in Figure 2, comprises a CPU (Central Processing Unit) 101, a main memory unit 102, an auxiliary memory unit 103, a display unit 104, an input unit 105, a communication unit 106, and a bus 109 connecting the units, and functions as an information leakage suppression device that evaluates the possibility of information leakage. In this embodiment, an example in which terminal device 1 is equipped with a CPU 101 is described, but terminal device 1 is not limited to one equipped with a CPU 101, and may be equipped with a GPU (Graphics Processing Unit) or an NPU (Neural Processing Unit). The main memory unit 102 is composed of volatile memory such as RAM (Random Access Memory) and is used as a working area for the CPU 101. The auxiliary memory unit 103 is composed of non-volatile memory such as a magnetic disk or semiconductor memory and stores programs for realizing various functions of terminal device 1. The display unit 104 is a display device such as a liquid crystal display. The input unit 105 is an input device such as a keyboard. The communication unit 106 has a modem and a gateway and communicates with the LLM management server 2 via the network NW1.

[0013] In terminal device 1, the CPU 101 reads the program stored in auxiliary storage unit 103 into the main storage unit 102 and executes it, so that, as shown in Figure 3, it functions as a query generation unit 111, a tokenizer 112, a warning target probability estimation unit 116, a warning target token identification unit 117, an evaluation result text generation unit 118, a display control unit 119, an LLM response acquisition unit 120, a teacher information generation unit 121, and a model generation unit 122. Also, the auxiliary storage unit 103 shown in Figure 2 has, as shown in Figure 3, a target text storage unit 131, a warning target probability estimation model storage unit 132, a warning target token storage unit 133, an evaluation result storage unit 134, a leakage prevention list storage unit 135, a learning text storage unit 136, a teacher information storage unit 137, and a response storage unit 139. The target text storage unit 131 stores text information indicating the query content entered by the user of terminal device 1 via the input unit 105.

[0014] The warning target probability estimation model storage unit 132 stores warning target probability estimation model information that indicates a warning target probability estimation model for estimating the warning target probability, which is the starting token in the warning target token range that is subject to information leakage warnings, and the warning target end probability, which is the ending token in the warning target token range. Here, the warning target probability estimation model is a model such as an encoder based on BERT or a similar Transformer architecture, and when a token sequence is input to the input layer, the output layer outputs the warning target start probability and warning target end probability for each token that makes up the token sequence.

[0015] The warning target token storage unit 133 stores warning target token range information indicating a range of warning target tokens that are identified based on the aforementioned warning target start probability and warning target end probability. This warning target token range information includes information indicating at least one token included in the warning target token range and information indicating the position of the warning target token range in the token sequence corresponding to the text to be evaluated. The evaluation result storage unit 134 stores evaluation result text information indicating the information leakage possibility evaluation result in which the warning target string is displayed in a different display manner from other strings.

[0016] The leakage prevention list storage unit 135 stores warning target list information, which indicates strings that the user has designated in advance as text that should be prevented from being leaked to the outside (warning target strings). The learning text storage unit 136 stores learning text information, which indicates text that contains warning target strings that have been prepared in advance.

[0017] The teacher information storage unit 137 stores information indicating each token included in the token sequence corresponding to the text indicated by the learning text information, warning target start probability information indicating the warning target start probability corresponding to that token, and warning target end probability information indicating the warning target end probability, in a manner that associates these with each other.

[0018] The answer storage unit 139 stores answer information that indicates the answer corresponding to the query indicated by the aforementioned query information sent from the LLM management server 2.

[0019] When a user of the terminal device 1 performs a query input operation to input a query for LLM to the input unit 105, the query generation unit 111 receives the query input operation and generates query information based on the content of the received query input operation. The query generation unit 111 then stores the generated query information in the target text storage unit 131.

[0020] The tokenizer 112 is, for example, SentencePiece. Upon receiving the aforementioned query input operation, it converts the text indicated by the query information stored in the target text storage unit 131 into a token sequence and notifies the warning target probability estimation unit 116. Furthermore, when a user performs a model learning operation to train the aforementioned warning target probability estimation model, the tokenizer 112 receives the model learning operation and converts each string indicated by the list information stored in the leakage prevention list storage unit 135 into a token and notifies the teacher information generation unit 121. It also converts the text indicated by the learning text information stored in the learning text storage unit 136 into a token sequence and notifies the teacher information generation unit 121.

[0021] When the tokenizer 112 notifies the warning target probability estimation unit 116 of a token sequence corresponding to the text indicated by the query information, the warning target estimation model indicated by the warning target estimation model information stored in the warning target probability estimation model storage unit 132 estimates the aforementioned warning target start probability and warning target end probability corresponding to each token that makes up the notified token sequence. The warning target probability estimation unit 116 then notifies the warning target token identification unit 117 of the estimated warning target start probability and warning target end probability for each token, along with information indicating each token.

[0022] The Warning Target Token Identification Unit 117 identifies a range of warning-target tokens if one exists, based on the warning target start probability and warning target end probability for each token notified by the Warning Target Probability Estimation Unit 116 and the information indicating each token. If such a range of warning-target tokens exists, it determines that such a range does not exist. Here, the Warning Target Token Identification Unit 117 first identifies the token with the highest warning target start probability based on the warning target start probability of all tokens, and identifies that token as the warning target start token of the warning-target tokens. At this time, if the Warning Target Token Identification Unit 117 finds that the warning target start probability is highest for a pre-set special token not included in the aforementioned token sequence notified by the tokenizer 112 to the Warning Target Probability Estimation Unit 116, it determines that the warning-target token range is not included in that token sequence. Next, the Warning Target Token Identification Unit 117 identifies the token with the highest warning target end probability based on the warning target end probability of each token that constitutes the portion of the token sequence after the identified warning target start token, and identifies that token as the warning target end token. The warning target token identification unit 117 then causes the warning target token storage unit 133 to store warning target token range information, which includes information indicating at least one token existing between the identified warning target start token and warning target end token, and information indicating the location of the warning target token range, which is the location of the warning target start token and the location of the warning target end token.

[0023] The evaluation result text generation unit 118 generates evaluation result text information indicating an information leakage possibility evaluation result in which the strings subject to warning are displayed in a different display manner from other strings, based on the query information stored in the target text storage unit 131 and the warning target token information and location information stored in the warning target token storage unit 133. The evaluation result text generation unit 118 then stores the generated evaluation result text information in the evaluation result storage unit 134.

[0024] When a user performs a query transmission operation to send query information to the LLM management server 2 via the input unit 105, the LLM response acquisition unit 120 receives the query transmission operation and generates query notification information containing the query information stored in the target text storage unit 131. The LLM response acquisition unit 120 then transmits the generated query notification information to the LLM management server 2, thereby acquiring the aforementioned response notification information sent from the LLM management server 2. The LLM response acquisition unit 120 then extracts the response information contained in the acquired response notification information and stores the extracted response information in the response storage unit 139.

[0025] The display control unit 119 forms an evaluation result notification image on the display unit 104 to notify the user of the evaluation result text information stored in the evaluation result storage unit 134, and displays it on the display unit 104. In addition, when the answer storage unit 139 stores new answer information, the display control unit 119 forms an answer notification image on the display unit 104 to notify the user of the answer content to the query based on the newly stored answer information.

[0026] When the teacher information generation unit 121 receives notification from the tokenizer 112 of a token sequence corresponding to the text indicated by the learning text information, it generates teacher information based on the learning token sequence corresponding to the notified learning text information and the warning target token range. Here, the teacher information generation unit 121 generates teacher information while changing the range in which it selects candidate token ranges to be included in the teacher information from the learning token sequence corresponding to the learning text information. Specifically, the teacher information generation unit 121 searches for a token range that matches the warning target token range from the beginning of the notified token sequence, and if a match is found, it generates teacher information consisting of information indicating the first token of the token range and warning target start probability information indicating a probability higher than a preset standard warning target start probability, and stores the generated teacher information in the teacher information storage unit 137. Next, the teacher information generation unit 121 changes the search range of the token range to a range after the position that matches the warning target token range in the learning token sequence, and then searches for the token range that matches the warning target token range again. The teacher information generation unit 121 then repeats the search for the warning target token range, the generation of teacher information and its storage in the teacher information storage unit 137, and the change of the search range. The teacher information generation unit 121 may also generate teacher information using each of the token sequences corresponding to multiple learning text information.

[0027] The model generation unit 122 generates a new warning target probability model using the teacher information stored in the teacher information storage unit 137. Then, the model generation unit 122 updates the warning target probability estimation model information stored in the warning target probability estimation model storage unit 132 with the warning target probability estimation model information that represents the newly generated warning target probability estimation model.

[0028] Next, the operation of the information leakage suppression system according to this embodiment will be explained with reference to Figure 4. First, suppose a user of terminal device 1 performs a query input operation to input a query to LLM to the input unit 105. In this case, terminal device 1 receives the query input operation, generates query information based on the content of the received query input operation, and stores the generated query information in the target text storage unit 131 (step S1).

[0029] Next, the terminal device 1 converts the text indicated by the query information stored in the target text storage unit 131 into a token sequence (step S2).

[0030] Next, the terminal device 1 uses the warning target estimation model information stored in the warning target estimation model storage unit 132 to estimate the warning target start probability and warning target end probability for each token that makes up the token sequence (step S3).

[0031] Subsequently, the terminal device 1 determines that there is a range of tokens that are subject to warning, based on the estimated start and end probabilities of each token being subject to warning (step S4). In this case, the terminal device 1 identifies the range of tokens subject to warning and its location. Then, the terminal device 1 stores the information of the range of tokens subject to warning, which indicates the identified range of tokens subject to warning, in the token storage unit 133, associating it with the location information indicating the identified location (step S5).

[0032] Next, terminal device 1 changes the search range for the warning target token range to the token sequence after the identified warning target token range in the token sequence (step S6). Then, terminal device 1 repeats the process in steps S3 to S5 again. Thereafter, terminal device 1 repeats the process in steps S3 to S6 as long as it determines that there is a warning target token range.

[0033] Then, the terminal device 1 determines that there is no warning target token range based on the estimated warning target start probability and warning target end probability for each token (step S7). In this case, the terminal device 1 generates evaluation result text information indicating the information leakage possibility evaluation result in which the warning target string is displayed in a different display manner from other strings, based on the query information stored in the target text storage unit 131 and the warning target token range information and location information stored in the warning target token storage unit 133. Then, the terminal device 1 stores the generated evaluation result text information in the evaluation result storage unit 134 (step S8).

[0034] Next, the terminal device 1 forms an evaluation result notification image on the display unit 104 to notify the user of the evaluation result text information stored in the evaluation result storage unit 134, and displays it (step S9). At this time, the terminal device 1 displays an evaluation result notification image GA1 on the display unit 104, for example, as shown in Figure 5. In this evaluation result notification image GA1, the string WO1 that is the subject of the warning is displayed in a different display manner from other strings.

[0035] Returning to Figure 4, suppose the user determines that, for example, the display unit 104 of terminal device 1 does not display the string of characters to be warned, unlike the evaluation result notification image GA1 shown in Figure 5, and that it is safe to send the created query information to LLM management server 2, and performs a query transmission operation to send the query information to LLM management server 2. In this case, terminal device 1 receives the query transmission operation and generates query notification information including the query information stored in the target text storage unit 131 (step S10). Then, the generated query notification information is sent from terminal device 1 to LLM management server 2 (step S11). Meanwhile, when LLM management server 2 receives the query notification information sent from terminal device 1, it extracts the query information contained in the acquired query notification information and stores the extracted query information in the query storage unit (step S12). After that, LLM management server 2 generates answer information corresponding to the query information and generates answer notification information including the generated answer information (step S13). Next, the generated answer notification information is sent from LLM management server 2 to terminal device 1, the source of the query information (step S14).

[0036] Meanwhile, when terminal device 1 receives the aforementioned response notification information transmitted from LLM management server 2, it extracts the response information contained in the acquired response notification information and stores the extracted response information in response storage unit 139 (step S15). Subsequently, terminal device 1 forms a response notification image that notifies the user of the content of the response to the query based on the response information newly stored in response storage unit 139 and displays it on display unit 104 (step S16).

[0037] Furthermore, when a user performs a model learning operation on the input unit 105 of the terminal device 1 to train the aforementioned warning target probability estimation model, the terminal device 1 accepts the model learning operation and converts each string indicated by the list information stored in the leakage prevention list storage unit 135 into a token. The terminal device 1 also converts the text indicated by the learning text information stored in the learning text storage unit 136 into a token sequence. The terminal device 1 then generates teacher information consisting of warning target token range information indicating the warning target token range included in the learning token sequence corresponding to the text indicated by the learning text information, and a token sequence included in the search range corresponding to the warning target token range in the learning token sequence. Here, the terminal device 1 generates teacher information so as to identify the first warning token range that appears in the learning token sequence, then generates a new learning token sequence by deleting the learning token sequence prior to the warning target token range, and generates the next teacher information so as to identify the first warning target token range that appears in the new learning token sequence. Then, by repeating these processes, terminal device 1 generates teacher information to determine that there are no warning target token ranges when all of the training token sequences have been deleted. Terminal device 1 then stores the generated teacher information in the teacher information storage unit 137 (step S17).

[0038] Next, terminal device 1 generates a new warning target probability model using the teacher information stored in the teacher information storage unit 137 (step S18). Subsequently, terminal device 1 updates the warning target probability estimation model information stored in the warning target probability estimation model storage unit 132 with the warning target probability estimation model information indicating the newly generated warning target probability estimation model (step S19).

[0039] Next, the information leakage suppression process performed by the terminal device 1 according to this embodiment will be described with reference to Figures 6 to 8. This information leakage suppression process is started, for example, when a program for generating query information to be sent to the LLM management server 2 is started in the terminal device 1. First, as shown in Figure 6, the query generation unit 111 determines whether or not it has received the aforementioned query input operation performed by the user to the input unit 105 (step S101). If the query generation unit 111 determines that it has not received the query input operation (step S101: No), the process in step S115, which will be described later, is executed. On the other hand, if the query generation unit 111 determines that it has received the query input operation (step S101: Yes), it generates query information based on the content of the query input operation and stores the generated query information in the target text storage unit 131 (step S102).

[0040] Next, the tokenizer 112 converts the text indicated by the query information stored in the target text storage unit 131 into a sequence of tokens (step S103).

[0041] Next, the warning target probability estimation unit 116 estimates the aforementioned warning target start probability and warning target end probability corresponding to each token constituting the aforementioned token sequence using the warning target estimation model indicated by the warning target estimation model information stored in the warning target probability estimation model storage unit 132 (step S104). Subsequently, the warning target token identification unit 117 determines whether or not there is a warning target token range based on the estimated warning target start probability and warning target end probability for each token and the information indicating each token (step S105). If the warning target token identification unit 117 determines that there is a warning target token range (step S105: No), the warning target token identification unit 117 identifies the warning target token range that is subject to warning based on the estimated warning target start probability and warning target end probability for each token and the information indicating each token. Then, the warning target token identification unit 117 stores the warning target token range information indicating the identified warning target token range in the warning target token storage unit 133 (step S106). Next, the warning target probability estimation unit 116 changes the search range of the warning target token range to a sequence of tokens later than the identified warning target token range (step S107). Then, the warning target probability estimation unit 116 estimates the aforementioned warning target start probability and warning target end probability corresponding to each token included in the changed search range (step S104), and the processing from step S105 onwards is executed again.

[0042] Furthermore, the warning target token identification unit 117 determines in step S105 that there is no warning target token range (step S105: Yes). In this case, the evaluation result text generation unit 118 generates evaluation result text information indicating an information leakage possibility evaluation result in which the warning target string is displayed in a different display manner from other strings, based on the query information stored in the target text storage unit 131 and the warning target token range information stored in the warning target token storage unit 133. The evaluation result text generation unit 118 then stores the generated evaluation result text information in the evaluation result storage unit 134 (step S108).

[0043] Next, the display control unit 119 forms an evaluation result notification image for notifying the user of the result of evaluating the possibility of information leakage based on the evaluation result text information stored in the evaluation result storage unit 134, and causes the display unit 104 to display it (step S109).

[0044] Subsequently, the LLM answer acquisition unit 120 determines whether or not it has received the above-described query transmission operation performed by the user on the input unit 105 (step S110). Here, if the LLM answer acquisition unit 120 determines that it has not received the query transmission operation (step S110: No), the process of step S115 described later is executed. On the other hand, if the LLM answer acquisition unit 120 determines that it has received the query transmission operation (step S110: Yes), it generates query notification information including the query information stored in the target text storage unit 131. Then, the LLM answer acquisition unit 120 transmits the generated query notification information to the LLM management server 2 (step S111), and acquires the above-described answer notification information transmitted from the LLM management server 2 (step S112). Thereafter, the LLM answer acquisition unit 120 extracts the answer information included in the acquired answer notification information, and stores the extracted answer information in the answer storage unit 139 (step S113).

[0045] Next, when the answer storage unit 139 newly stores answer information, the display control unit 119 forms an answer notification image for notifying the user of the answer content to the query based on the newly stored answer information, and causes the display unit 104 to display it (step S114).

[0046] Subsequently, as shown in FIG. 7, the tokenizer 112 determines whether or not it has received the above-described model learning operation performed by the user on the input unit 105 (step S115). Here, if the tokenizer 112 determines that it has not received the model learning operation (step S115: No), the process of step S101 is executed again. On the other hand, if the tokenizer 所112 determines that it has received the model learning operation (step S115: Yes), teacher information generation processing is executed (step S116).

[0047] Here, the teacher information generation process will be explained in detail with reference to Figure 8. First, the tokenizer 112 converts each string indicated by the list information stored in the leakage prevention list storage unit 135 into a token (step S201). Next, the tokenizer 112 converts the text indicated by the learning text information stored in the learning text storage unit 136 into a learning token sequence (step S202). Subsequently, the teacher information generation unit 121 selects one token range included in the learning token sequence corresponding to the learning text information (step S203). After that, the teacher information generation unit 121 determines whether the selected token range matches any of the warning target token ranges corresponding to the aforementioned list information (step S204). Here, if the teacher information generation unit 121 determines that the selected token range does not match any of the warning target token ranges corresponding to the aforementioned list information (step S204: No), it determines whether all token ranges included in the learning token sequence corresponding to the text indicated by the learning text information have been selected (step S205). Here, if the teacher information generation unit 121 determines that it has selected all token ranges (step S205: Yes), it executes the process in step S209, which will be described later. On the other hand, if the teacher information generation unit 121 determines that there is a token sequence that has not yet been selected (step S205: No), it selects one other token range (step S203) and then executes the process in step S204 again. Here, the teacher information generation unit 121 selects token ranges in order from the beginning of the token sequence corresponding to the text indicated by the learning text information.

[0048] Also, assume that in step S204, the teacher information generation unit 121 determines that any one of the selected token range and the warning target token range corresponding to the above list information matches (step S204: Yes). In this case, if the teacher information generation unit 121 has not yet executed the change of the search range in step S208 to be described later, the teacher information generation unit 121 generates teacher information composed of warning target token range information indicating the matched warning target token range and information indicating the learning token sequence, and stores it in the teacher information storage unit 137. On the other hand, when the teacher information generation unit 121 has executed the change of the search range in step S208, the teacher information generation unit 121 generates teacher information composed of warning target token range information indicating the matched warning target token range and information indicating a new learning token sequence obtained by deleting the learning token sequence in front of the position matching the warning target token range in the learning token sequence, and stores it in the teacher information storage unit 137 (step S206). Thereafter, the teacher information generation unit 121 determines whether all token ranges included in the learning token sequence have been selected (step S207). Here, if the teacher information generation unit 121 determines that there is a token sequence that has not yet been selected (step S207: No), the search range of the warning target token range is changed to a range behind the position matching the warning target token range in the learning token sequence (step S208), and the process of step S203 is executed again. At this time, the token range is selected from the changed search range.

[0049] On the other hand, if the teacher information generation unit 121 determines that it has selected all the token ranges included in the learning token sequence (step S207: Yes), the tokenizer 112 determines whether or not there is other learning text information (step S209). If the tokenizer 112 determines that there is other learning text information that has not been converted into a token sequence (step S209: Yes), it selects other learning text information stored in the learning text storage unit 136 and executes the process in step S202 again. On the other hand, if the tokenizer 112 determines that it has converted all the learning text information stored in the learning text storage unit 136 into a token sequence (step S209: No), it terminates the teacher information generation process and executes the process in step S117 shown in Figure 7.

[0050] Next, as shown in Figure 7, the model generation unit 122 generates a new warning target probability model using the teacher information stored in the teacher information storage unit 137 (step S117). Subsequently, the model generation unit 122 updates the warning target probability estimation model information stored in the warning target probability estimation model storage unit 132 with the warning target probability estimation model information indicating the newly generated warning target probability estimation model (step S118). After that, the process of step S101 is executed again.

[0051] As described above, according to the terminal device 1 of this embodiment, the warning target probability estimation unit 116 estimates the warning target probability of each token included in the token sequence corresponding to the text indicated by the text information to be evaluated, using the aforementioned warning target probability estimation model. The warning target token identification unit 117 identifies the content and location of the warning target token based on the estimated warning target probability. The evaluation result text generation unit 118 then generates evaluation result text information indicating the information leakage possibility evaluation result, in which the warning target string is displayed in a different display manner from other strings, based on the text information and the content and location of the identified warning target token. As a result, users can quickly understand whether or not the text to be evaluated contains a target string that they want to prevent from being leaked to the outside, simply by referring to the evaluation result text information. Therefore, for example, when using an information processing service provided externally with the text to be evaluated, it is possible to suppress the provision of the text containing the aforementioned target string to the information processing service as is, thereby suppressing the leakage of confidential information managed by the user.

[0052] (Embodiment 2) Hereinafter, an information leakage suppression device according to one embodiment of the present invention will be described in detail with reference to the drawings. The information leakage suppression device according to this embodiment includes: a tokenizer that converts the text indicated by the text information to be processed into a sequence of tokens; a mask target probability estimation unit that estimates the mask target start probability and the mask target end probability for each token included in the sequence using a mask target probability estimation model for estimating the mask target start probability, which is the probability that the token is the first token in the range of mask target tokens to be masked, and the mask target end probability, which is the probability that the token is the last token in the range of mask target tokens to be masked; a mask target token identification unit that identifies the range of mask target tokens to be masked based on the mask target start probability and the mask target end probability; and a target text conversion unit that identifies the mask target string to be masked based on the text information and the identified mask target token range, and generates converted text information by converting the identified mask target string into a pre-set dummy string.

[0053] The information leakage suppression system according to this embodiment, as shown in Figure 9, comprises an LLM management server 2 described in Embodiment 1 and a terminal device 2001 that can communicate with the LLM management server 2 via a network NW1. In Figure 9, components similar to those in Embodiment 1 are denoted by the same reference numerals as in Figure 3. The LLM management server 2 comprises a query notification acquisition unit (not shown) that acquires query notification information transmitted from the terminal device 2001, a query storage unit (not shown), a tokenizer (not shown), a response generation unit that generates a token sequence corresponding to the response information using LLM, a detoxifier (not shown), and a response notification unit (not shown) that generates response notification information, which will be described later, and transmits it to the terminal device 2001. The query storage unit stores query information in association with source identification information that identifies the terminal device 2001 that sent the query information. The query notification acquisition unit acquires query notification information transmitted from the terminal device 2001, extracts the query information contained in the acquired query notification information, associates the extracted query information with source identification information that identifies the source of the query information, and stores it in the query storage unit. The response notification unit generates response notification information including the generated response information and the source identification information stored in the query storage unit, and sends it to the terminal device 2001 that sent the query information.

[0054] The terminal device 2001 is, for example, a personal computer and has the same hardware configuration as the terminal device 1 described in Embodiment 1. Therefore, the hardware configuration of the terminal device 2001 will be described using the same reference numerals as those shown in Figure 2. In the terminal device 2001, the CPU 101 reads the program stored in the auxiliary storage unit 103 into the main storage unit 102 and executes it, thereby functioning as a query generation unit 2111, a tokenizer 2112, a mask target probability estimation unit 2116, a mask target token identification unit 2117, a target text conversion unit 2118, a display control unit 2119, an LLM response acquisition unit 2120, a teacher information generation unit 2121, a model generation unit 2122, and a response conversion unit 2123. Furthermore, the auxiliary storage unit 103 includes a target text storage unit 2131, a mask target probability estimation model storage unit 2132, a mask target token storage unit 2133, a converted target text storage unit 2134, a mask target list storage unit 2135, a learning text storage unit 2136, a teacher information storage unit 2137, a dummy string storage unit 2138, an answer storage unit 2139, and a converted answer storage unit 2140. The target text storage unit 2131 stores text information indicating the query content entered by the user of the terminal device 2001 via the input unit 105.

[0055] The mask target probability estimation model storage unit 2132 stores mask target probability estimation model information that indicates a mask target probability estimation model for estimating the mask target start probability, which is the start token in the mask target token range to be masked, and the mask target end probability, which is the end token in the mask target token range. Here, the mask target probability estimation model is a model such as an encoder based on BERT or a similar Transformer architecture, and when a token sequence is input to the input layer, the output layer outputs the mask target start probability and the mask target end probability for each token constituting the token sequence.

[0056] The Masked Token Storage Unit 2133 stores Masked Token Range Information, which indicates the range of Masked Tokens to be Masked, identified based on the aforementioned Masked Start Probability and Masked End Probability. This Masked Token Range Information includes information indicating at least one token included in the Masked Token Range, and information indicating the position of the Masked Token Range in the token sequence corresponding to the text information stored in the Target Text Storage Unit 2131. The Converted Target Text Storage Unit 2134 stores Converted Target Text Information, which represents the converted text obtained by converting the strings to be masked included in the text indicated by the text information stored in the Target Text Storage Unit 2131 into dummy strings.

[0057] The mask target list storage unit 2135 stores mask target list information indicating strings that the user has specified in advance as text to be prevented from being leaked to the outside (mask target strings). The learning text storage unit 2136 stores learning text information indicating text that contains pre-prepared mask target strings.

[0058] The teacher information storage unit 2137 stores, in association with each other, information indicating each token included in the token sequence corresponding to the text indicated by the learning text information, mask target start probability information indicating the mask target start probability corresponding to that token, and mask target end probability information indicating the mask target end probability.

[0059] The dummy string storage unit 2138 stores dummy string information representing a dummy string in association with the string to be masked. Here, a dummy string is a string that does not evoke the string representing the specific personal information to be masked, and is, for example, a string consisting of letters, numbers, symbols, etc. The dummy string storage unit 2138 stores, for example, dummy string information representing the dummy string "xxxxxx" in association with string information representing the name of the individual identified as the target of masking, "Taro Yamada".

[0060] The answer storage unit 2139 stores answer information that represents answer text containing a dummy string corresponding to the query indicated by the aforementioned query information sent from the LLM management server 2. The converted answer storage unit 2140 stores converted answer information that represents the converted answer text after the dummy string contained in the answer text has been converted to the corresponding mask target string.

[0061] When a user of the terminal device 2001 performs a query input operation to input a query to LLM into the input unit 105, the query generation unit 2111 receives the query input operation and generates query information based on the content of the received query input operation. The query generation unit 2111 then stores the generated query information in the target text storage unit 2131.

[0062] The tokenizer 2112 is, for example, SentencePiece. After receiving the aforementioned query input operation, when the user performs a query transmission operation to the input unit 105 to send the aforementioned query information to the LLM management server 2, the tokenizer 2112 receives the query transmission operation, converts the text indicated by the query information stored in the target text storage unit 2131 into a token sequence, and notifies the mask target probability estimation unit 2116. Furthermore, when the user performs a model training operation to train the aforementioned mask target probability estimation model, the tokenizer 2112 receives the model training operation, converts each string indicated by the list information stored in the mask target list storage unit 2135 into a token, and notifies the teacher information generation unit 2121. It also converts the text indicated by the learning text information stored in the learning text storage unit 2136 into a token sequence and notifies the teacher information generation unit 2121.

[0063] When the Mask Target Probability Estimation Unit 2116 receives a token sequence corresponding to the text indicated by the query information from the tokenizer 2112, it uses the Mask Target Probability Estimation Model stored in the Mask Target Probability Estimation Model Storage Unit 2132 to estimate the aforementioned Mask Target Start Probability and Mask Target End Probability for each token in the notified token sequence. The Mask Target Probability Estimation Unit 2116 then notifies the Mask Target Token Identification Unit 2117 of the estimated Mask Target Start Probability and Mask Target End Probability for each token, along with information indicating each token.

[0064] The Masked Token Identification Unit 2117 identifies the aforementioned Masked Token Range if it exists, based on the Masked Token Start Probability and Masked Token End Probability for each token notified by the Masked Token Probability Estimation Unit 2116 and the information indicating each token, and determines that the Masked Token Range does not exist if it does not exist. Here, the Masked Token Identification Unit 2117 first identifies the token with the highest Masked Token Start Probability based on the Masked Token Start Probability of all tokens and identifies that token as the Masked Token Start Token of the Masked Token Range. At this time, if the Masked Token Identification Unit 2117 has the highest Masked Token Start Probability of a pre-set special token that is not included in the aforementioned token sequence notified by the Tokenizer 2112 to the Masked Token Probability Estimation Unit 2116, it determines that the Masked Token Range is not included in that token sequence. Next, the Masked Token Identification Unit 2117 identifies the token with the highest Masked Token End Probability based on the Masked Token End Probability of each token that constitutes the portion of the token sequence after the identified Masked Token Start Token, and identifies that token as the Masked Token End Token. The mask target token identification unit 2117 then causes the mask target token storage unit 2133 to store mask target token range information, which includes information indicating at least one token existing between the identified mask target start token and mask target end token, and information indicating the location of the mask target token range, which is the location of the mask target start token and the location of the mask target end token.

[0065] The target text conversion unit 2118 identifies the strings to be masked based on the query information stored in the target text storage unit 2131 and the range of masked tokens identified by the masked token identification unit 2117, and generates converted text information by converting the identified masked strings into pre-set dummy strings. Specifically, the target text conversion unit 2118 identifies the strings to be masked based on the query information stored in the target text storage unit 2131, information indicating at least one token included in the range of masked tokens indicated by the masked token range information stored in the masked token storage unit 2133, and information indicating the position of the range of masked tokens in the token sequence corresponding to the text to be processed. Next, the target text conversion unit 2118 generates converted text information by converting each of the identified masked strings into pre-set dummy strings. Here, if the text to be processed contains multiple types of masked strings, the target text conversion unit 2118 converts each of the multiple types of masked strings into a different dummy string. The target text conversion unit 2118 then stores the generated converted text information in the converted target text storage unit 2134, and stores dummy string information representing dummy strings contained in the text indicated by the converted text information in the dummy string storage unit 2138, associating it with mask target string information representing the corresponding mask target string.

[0066] When the converted text information is stored in the converted target text storage unit 2134, the LLM response acquisition unit 2120 generates query notification information that includes the converted text information stored in the converted target text storage unit 2134, query information for specifying the answer content that instructs the LLM management server 2 to respond as is for the dummy string, and query notification information. The LLM response acquisition unit 2120 then sends the generated query notification information to the LLM management server 2 to acquire the aforementioned response notification information sent from the LLM management server 2. The LLM response acquisition unit 2120 then extracts the answer information contained in the acquired response notification information and stores the extracted answer information in the response storage unit 2139.

[0067] The answer conversion unit 2123 identifies the mask target string corresponding to the dummy string information that indicates the dummy string contained in the answer text indicated by the answer information stored in the answer storage unit 2139, which is stored in the dummy string storage unit 2138. Next, the answer conversion unit 2123 generates converted answer information that indicates the converted answer text in which the dummy string contained in the answer text has been converted into the identified mask target string. Then, the answer conversion unit 2123 stores the generated converted answer information in the converted answer storage unit 2140.

[0068] When the converted response storage unit 2140 stores new converted response information, the display control unit 2119, based on the converted response information, forms a response notification image showing the content of the response from the LLM management server 2 to the query indicated by the query information entered by the user, and displays it on the display unit 104.

[0069] When the teacher information generation unit 2121 receives notification from the tokenizer 2112 of a token sequence corresponding to the text indicated by the learning text information, it generates teacher information based on the learning token sequence and the mask target token range corresponding to the notified learning text information. Here, the teacher information generation unit 2121 generates teacher information while changing the range in which it selects candidate token ranges to be included in the teacher information from the learning token sequence corresponding to the learning text information. Specifically, the teacher information generation unit 2121 searches for a token range that matches the mask target token range from the beginning of the notified token sequence, and if a match is found, it generates teacher information consisting of information indicating the first token of the token range and mask target start probability information indicating a probability higher than a preset standard mask target start probability, and stores the generated teacher information in the teacher information storage unit 2137. Next, the teacher information generation unit 2121 changes the search range of the token range to a range after the position that matches the masked token range in the learning token sequence, and then searches for the token range that matches the masked token range again. The teacher information generation unit 2121 then repeats the search for the masked token range, the generation of teacher information and its storage in the teacher information storage unit 2137, and the change of the search range. The teacher information generation unit 2121 may also generate teacher information using each of the token sequences corresponding to multiple learning text information.

[0070] The model generation unit 2122 generates a new mask target probability estimation model using the teacher information stored in the teacher information storage unit 2137. Then, the model generation unit 2122 updates the mask target probability estimation model information stored in the mask target probability estimation model storage unit 2132 with the mask target probability estimation model information representing the newly generated mask target probability estimation model.

[0071] Next, the operation of the information leakage suppression system according to this embodiment will be described with reference to Figures 10 to 12B. First, suppose a user of the terminal device 2001 performs a query input operation to input a query to the LLM to the input unit 105. In this case, the terminal device 2001 receives the query input operation, generates query information based on the content of the received query input operation, and stores the generated query information in the target text storage unit 131 (step S2001).

[0072] Now, suppose the user performs a query transmission operation to send the aforementioned query information to the LLM management server 2. In this case, the terminal device 2001 converts the text indicated by the query information stored in the target text storage unit 2131 into a token sequence (step S2002).

[0073] Next, the terminal device 2001 uses the mask target probability estimation model indicated by the mask target probability estimation model information stored in the mask target probability estimation model storage unit 2132 to estimate the mask target start probability and mask target end probability for the tokens constituting the token sequence (step S2003).

[0074] Next, the terminal device 2001 determines that there is a mask target token range based on the estimated mask target start probability and mask target end probability for each token (step S2004). In this case, the terminal device 2001 identifies the mask target token range and stores the mask target token range information indicating the identified mask target token range in the mask target token storage unit 2133 (step S2005).

[0075] Subsequently, the terminal device 2001 changes the search range for the masked token range to the token sequence after the identified masked token range in the token sequence (step S2006). Then, the terminal device 2001 repeats the processing in steps S3 to S5. Thereafter, as long as the terminal device 2001 determines that a masked token range exists, it repeats the processing in steps S2003 to S2006.

[0076] Then, the terminal device 2001 determines that there is no range of tokens to be masked, based on the estimated start and end probabilities of each token to be masked (step S2007). In this case, the terminal device 2001 identifies the string to be masked based on the text information stored in the target text storage unit 2131 and the range of tokens to be masked stored in the target token storage unit 2133. The terminal device 2001 also generates converted text information by converting each identified string to be masked into a pre-set dummy string. The terminal device 2001 then stores the generated converted text information in the converted target text storage unit 2134, and stores the dummy string information representing the dummy strings contained in the text indicated by the converted text information in the dummy string storage unit 2138, associating it with the corresponding string to be masked information (step S2008).

[0077] Suppose the user inputs a query that includes the masked strings WO11 and WO12, as shown in Figure 11A. In this case, the terminal device 2001 generates converted text information for the text representing the query, showing the converted text in which the masked strings have been converted to dummy strings WO21 and WO22, as shown in Figure 11B.

[0078] Returning to Figure 10, the terminal device 2001 then receives the query transmission operation and generates query notification information that includes the converted text information stored in the converted target text storage unit 134 (step S2009). Here, the query notification information includes the converted text information mentioned above, as well as the aforementioned query information for specifying the answer content, as shown in Figure 11C. Then, as shown in Figure 10, the generated query notification information is transmitted from the terminal device 2001 to the LLM management server 2 (step S2010). Meanwhile, when the LLM management server 2 receives the query notification information transmitted from the terminal device 2001, it extracts the converted text information and the aforementioned query information for specifying the answer content contained in the acquired query notification information and stores these extracted pieces of information in the query storage unit (step S2011). Subsequently, the LLM management server 2 generates answer information corresponding to the converted text information based on the converted text information and the query information for specifying the answer content stored in the query storage unit, and generates answer notification information that includes the generated answer information (step S2012). Next, the generated response notification information is sent from the LLM management server 2 to the terminal device 2001 that sent the query notification information (step S2013).

[0079] Meanwhile, when the terminal device 2001 receives the aforementioned response notification information transmitted from the LLM management server 2, it extracts the response information contained in the received response notification information and stores the extracted response information in the response storage unit 139 (step S2014). Here, the terminal device 2001 receives response information that represents a response text containing dummy strings WO23 and WO24, for example, as shown in Figure 12A. Then, as shown in Figure 10, the terminal device 2001 identifies a mask target string that corresponds to the dummy string information stored in the dummy string storage unit 2138, which indicates the dummy strings contained in the response text indicated by the response information stored in the response storage unit 2139. The terminal device 2001 also generates converted response information that indicates a converted response text in which the dummy strings contained in the response text have been converted to the identified mask target string. Then, the terminal device 2001 stores the generated converted response information in the converted response storage unit 2140 (step S2015). Here, the terminal device 2001 generates converted response information that shows the converted response text, in which the dummy strings of the response text indicated by the response information are converted into masked strings WO13 and WO14, for example, as shown in Figure 12B.

[0080] Returning to Figure 10, the terminal device 2001 then forms an answer notification image on the display unit 104 to notify the user of the answer content to the query based on the converted answer information newly stored in the converted answer storage unit 2140 (step S2016).

[0081] Furthermore, when a user performs a model learning operation on the input unit 105 of the terminal device 2001 to train the aforementioned mask target probability estimation model, the terminal device 2001 accepts the model learning operation and converts each string indicated by the list information stored in the mask target list storage unit 2135 into a token. The terminal device 2001 also converts the text indicated by the learning text information stored in the learning text storage unit 2136 into a token sequence. The terminal device 2001 then generates teacher information consisting of mask target token range information indicating the mask target token range included in the learning token sequence corresponding to the text indicated by the learning text information, and a token sequence included in the search range corresponding to the mask target token range in the learning token sequence. Here, the terminal device 2001 generates teacher information so that the first mask target token range that appears in the learning token sequence can be identified, then generates a new learning token sequence by deleting the learning token sequence preceding the mask target token range, and generates the next teacher information so that the first mask target token range that appears in the new learning token sequence can be identified. Then, by repeating these processes, the terminal device 2001 generates teacher information to determine that there is no range of tokens to be masked once all of the training token sequences have been deleted. The terminal device 2001 then stores the generated teacher information in the teacher information storage unit 137 (step S2017).

[0082] Next, the terminal device 2001 generates a new mask target probability estimation model using the teacher information stored in the teacher information storage unit 2137 (step S2018). Subsequently, the terminal device 2001 updates the mask target probability estimation model information stored in the mask target probability estimation model storage unit 2132 with the mask target probability estimation model information representing the newly generated mask target probability estimation model (step S2019).

[0083] Next, the information leakage suppression process performed by the terminal device 2001 according to this embodiment will be described with reference to Figures 13 to 15. This information leakage suppression process is started, for example, when a program for generating query information to be sent to the LLM management server 2 is started in the terminal device 2001. First, as shown in Figure 13, the query generation unit 2111 determines whether or not it has received the aforementioned query input operation performed by the user to the input unit 105 (step S2101). If the query generation unit 2111 determines that it has not received the query input operation (step S2101: No), the process in step S2115, which will be described later, is executed. On the other hand, if the query generation unit 2111 determines that it has received the query input operation (step S2101: Yes), it generates query information based on the content of the query input operation and stores the generated query information in the target text storage unit 2131 (step S2102).

[0084] Next, the tokenizer 2112 determines whether or not it has received the query transmission operation performed by the user to the input unit 105 (step S2103). If the tokenizer 2112 determines that it has not received the query transmission operation (step S2103: No), the process in step S2115, described later, is executed. On the other hand, if the tokenizer 2112 determines that it has received the query transmission operation (step S2103: Yes), it converts the text indicated by the query information stored in the target text storage unit 2131 into a token sequence (step S2104).

[0085] Next, the mask target probability estimation unit 2116 estimates the aforementioned mask target start probability and mask target end probability corresponding to each token constituting the aforementioned token sequence using the mask target probability estimation model indicated by the mask target probability estimation model information stored in the mask target probability estimation model storage unit 2132 (step S2105). Subsequently, the mask target token identification unit 2117 determines whether or not there is a mask target token range based on the estimated mask target start probability and mask target end probability for each token and the information indicating each token (step S2106). If the mask target token identification unit 2117 determines that there is a mask target token range (step S2106: No), the mask target token identification unit 2117 identifies the mask target token range to be masked based on the estimated mask target start probability and mask target end probability for each token and the information indicating each token. Then, the mask target token identification unit 2117 stores the mask target token range information indicating the identified mask target token range in the mask target token storage unit 2133 (step S2107). Next, the mask target probability estimation unit 2116 changes the search range of the mask target token range to a sequence of tokens later than the identified mask target token range (step S2108). Then, the mask target probability estimation unit 2116 estimates the aforementioned mask target start probability and mask target end probability corresponding to each token included in the changed search range (step S2105), and the processing from step S2106 onwards is executed again.

[0086] Furthermore, the Mask Target Token Identification Unit 2117 determines in step S2106 that there is no Mask Target Token Range (Step S2106: Yes). In this case, the Target Text Conversion Unit 2118 identifies the Mask Target String to be masked based on the query information stored in the Target Text Storage Unit 2131 and the Mask Target Token Range identified by the Mask Target Token Identification Unit 2117, and generates converted text information by converting the identified Mask Target String to a pre-set dummy string. The Target Text Conversion Unit 2118 then stores the generated converted text information in the Converted Target Text Storage Unit 2134, and stores the dummy string information representing the dummy string included in the text indicated by the converted text information in the dummy string storage unit 2138, associating it with the Mask Target String information representing the corresponding Mask Target String (Step S2109).

[0087] Subsequently, the LLM response acquisition unit 2120 generates query notification information that includes the converted text information stored in the converted target text storage unit 2134 and the aforementioned query information for specifying the response content. Then, the LLM response acquisition unit 2120 sends the generated query notification information to the LLM management server 2 (step S2110), thereby acquiring the aforementioned response notification information sent from the LLM management server 2 (step S2111). Subsequently, the LLM response acquisition unit 2120 extracts the response information contained in the acquired response notification information and stores the extracted response information in the response storage unit 2139 (step S2112).

[0088] Next, the answer conversion unit 2123 identifies a mask target string corresponding to the dummy string information stored in the answer information stored in the answer storage unit 2139, which is stored in the dummy string storage unit 2138. The answer conversion unit 2123 also generates converted answer information for the answer text, which shows the converted answer text in which the dummy string contained in the answer text has been converted to the identified mask target string. The answer conversion unit 2123 then stores the generated converted answer information in the converted answer storage unit 2140 (step S2113). Subsequently, when the answer storage unit 2139 stores new answer information, the display control unit 2119 forms an answer notification image based on the newly stored answer information to notify the user of the answer content to the query and displays it on the display unit 104 (step S2114).

[0089] Next, as shown in Figure 14, the tokenizer 2112 determines whether or not it has received the aforementioned model learning operation performed by the user on the input unit 105 (step S2115). If the tokenizer 2112 determines that it has not received the model learning operation (step S2115: No), the process in step S2101 is executed again. On the other hand, if the tokenizer 2112 determines that it has received the model learning operation (step S2115: Yes), the teacher information generation process is executed (step S2116).

[0090] Here, the teacher information generation process will be explained in detail with reference to Figure 15. First, the tokenizer 2112 converts each string indicated by the list information stored in the mask target list storage unit 2135 into a token (step S2201). Next, the tokenizer 2112 converts the text indicated by the learning text information stored in the learning text storage unit 2136 into a learning token sequence (step S2202). Subsequently, the teacher information generation unit 2121 selects one token range included in the learning token sequence corresponding to the learning text information (step S2203). After that, the teacher information generation unit 2121 determines whether the selected token range matches any of the mask target token ranges corresponding to the aforementioned list information (step S2204). Here, if the teacher information generation unit 2121 determines that the selected token range does not match either of the masked token ranges corresponding to the aforementioned list information (step S2204: No), it determines whether or not it has selected all the token ranges included in the learning token sequence corresponding to the text indicated by the learning text information (step S2205). Here, if the teacher information generation unit 2121 determines that it has selected all the token ranges (step S2205: Yes), it executes the process in step S2209 described below. On the other hand, if the teacher information generation unit 2121 determines that there is a token sequence that has not yet been selected (step S2205: No), it selects one more token range (step S2203) and then executes the process in step S2204 again. Here, the teacher information generation unit 2121 selects token ranges in order from the beginning of the token sequence corresponding to the text indicated by the learning text information.

[0091] Furthermore, the teacher information generation unit 2121 determines in step S2204 that the selected token range matches one of the mask target token ranges corresponding to the aforementioned list information (step S2204: Yes). In this case, if the teacher information generation unit 2121 has not yet performed the search range change in step S2208 described below, it generates teacher information consisting of mask target token range information indicating the matched mask target token range and information indicating the learning token sequence, and stores it in the teacher information storage unit 2137. On the other hand, if the teacher information generation unit 2121 has performed the search range change in step S2208, it generates teacher information consisting of mask target token range information indicating the matched mask target token range and information indicating a new learning token sequence obtained by deleting the learning token sequence prior to the position matching the mask target token range in the learning token sequence, and stores it in the teacher information storage unit 137 (step S2206). Subsequently, the teacher information generation unit 2121 determines whether or not all token ranges included in the learning token sequence have been selected (step S2207). If the teacher information generation unit 2121 determines that there are still token sequences that have not yet been selected (step S2207: No), it changes the search range for the mask target token range to a range after the position that matches the mask target token range in the learning token sequence (step S2208), and the process in step S2203 is executed again. At this time, the token range is selected from the changed search range.

[0092] On the other hand, if the teacher information generation unit 2121 determines that it has selected all the token ranges included in the learning token sequence (step S2207: Yes), the tokenizer 2112 determines whether or not there is other learning text information (step S2209). If the tokenizer 2112 determines that there is other learning text information that has not been converted into a token sequence (step S2209: Yes), it selects other learning text information stored in the learning text storage unit 2136 and executes the process in step S2202 again. On the other hand, if the tokenizer 2112 determines that it has converted all the learning text information stored in the learning text storage unit 2136 into a token sequence (step S2209: No), it terminates the teacher information generation process and executes the process in step S2117 shown in Figure 14.

[0093] Next, as shown in Figure 14, the model generation unit 2122 generates a new mask target probability estimation model using the teacher information stored in the teacher information storage unit 2137 (step S2117). Subsequently, the model generation unit 2122 updates the mask target probability estimation model information stored in the mask target probability estimation model storage unit 2132 with the mask target probability estimation model information indicating the newly generated mask target probability estimation model (step S2118). After that, the process of step S2101 is executed again.

[0094] As described above, according to the terminal device 2001 of this embodiment, the mask target probability estimation unit 2116 estimates the mask target start probability and mask target end probability for each token included in the token sequence corresponding to the text indicated by the text information to be sent to the LLM management server 2, using the mask target probability estimation model described above. The mask target token identification unit 2117 identifies the mask target token range based on the mask target start probability and mask target end probability. The target text conversion unit 2118 then identifies the mask target string to be masked based on the text information described above and the identified mask target token range, and generates converted text information by converting the identified mask target string to the aforementioned dummy string. As a result, when sending text information to the LLM management server 2, for example, in order to use the information processing service provided by the LLM management server 2, if the text information contains a string that you want to prevent from being leaked to the information processing service provider via the LLM management server 2, you can replace that string with a dummy string and provide it to the information processing service provider in a masked state. Therefore, it is possible to prevent text containing strings that you want to keep hidden from information processing service providers from being directly provided to those service providers, thereby preventing the leakage of confidential information managed by the user.

[0095] Although embodiments of the present invention have been described above, the present invention is not limited to the configuration of the embodiments described above. For example, the text information stored in the target text storage unit 131 described in Embodiment 1 is not limited to query information to be sent to the LLM management server 2, but may be text information for other purposes. For example, the text information may be query information to be sent to a server managed by a business operator that provides services that do not use external LLM, such as external search services. Alternatively, the text information may not be query information for using external services. The text information may be, for example, a file containing a large amount of text information stored inside the PC of a user's bereaved family member. In this case, the leakage prevention list storage unit 135 may store text information representing information about the bereaved family member's bank account as a warning target.

[0096] In the embodiment described, an example was described in which the terminal device 1 includes a teacher information generation unit 121 and a model generation unit 122. However, the embodiment is not limited to this, and for example, a model management server that manages the warning target probability estimation model may be separate from the terminal device 1 and have the functions of the aforementioned teacher information generation unit 121 and model generation unit 122. Here, the model management server may be equipped with a GPU, NPU, etc. In this case, the model management server should perform the generation of teacher information and the updating of the warning target probability estimation model information, and transmit the updated warning target probability estimation model information to the terminal device 1. The terminal device 1 should then acquire the warning target probability estimation model information transmitted from the model management server and store it in the warning target probability estimation model storage unit 132. In this case, the warning target probability estimation unit 116 should estimate the warning target start probability and the warning target end probability using the trained warning target probability estimation model indicated by the warning target probability estimation model information acquired from the model management server.

[0097] In Embodiment 2, for example, the text information stored in the target text storage unit 2131 is not limited to text information representing queries to be sent to the LLM management server 2, but may be text information for other purposes. For example, the text information may be query information to be sent to a server managed by a service provider that does not use an external LLM, such as an external search service. Alternatively, the text information may not be query information for using an external service. For example, the text information may be a file containing a large amount of text information stored on the PC of a bereaved family member of the user. In this case, the mask target list storage unit 2135 should store text information representing information about the bereaved family member's bank account as the target of the mask.

[0098] In Embodiment 2, an example was described in which the terminal device 2001 includes a teacher information generation unit 2121 and a model generation unit 2122. However, the invention is not limited to this, and for example, a model management server that manages the mask target probability estimation model may be separate from the terminal device 2001 and have the functions of the aforementioned teacher information generation unit 2121 and model generation unit 2122. Here, the model management server may be equipped with, for example, a GPU, NPU, etc. In this case, the model management server should perform the generation of teacher information and the updating of the mask target probability estimation model information, and transmit the updated mask target probability estimation model information to the terminal device 1. The terminal device 2001 should then acquire the mask target probability estimation model information transmitted from the model management server and store it in the mask target probability estimation model storage unit 2132. In this case, the mask target probability estimation unit 2116 should use the trained mask target probability estimation model indicated by the mask target probability estimation model information acquired from the model management server to estimate the mask target start probability and the mask target end probability.

[0099] In this embodiment, the various functions of the terminal device 1,2001 may be implemented at the so-called OS level, before the text information, which represents text entered via an input device such as a keyboard, is processed by other applications.

[0100] Furthermore, the various functions of the terminal device 1,2001 according to the present invention can be realized using a normal computer system, not a dedicated system. For example, a program for performing the above operations may be stored on a non-temporary recording medium (such as a CD-ROM (Compact Disc Read Only Memory)) that can be read by the computer system and distributed to a computer connected to a network, and the terminal device 1 that performs the above processing may be configured by installing the program on the computer system.

[0101] Furthermore, the method of providing the program to the computer is arbitrary. For example, the program may be uploaded to a bulletin board system (BBS) on a communication line and distributed to the computer via the communication line. The computer then launches this program and executes it under the control of the operating system (OS), just like any other application. In this way, the computer functions as terminal device 1, 2001 that performs the above-mentioned processing.

[0102] This invention allows for various embodiments and modifications without departing from the broad spirit and scope of the invention. Furthermore, the embodiments described above are for illustrative purposes only and do not limit the scope of the invention. In other words, the scope of the invention is indicated by the claims, not by the embodiments. Various modifications made within the scope of the claims and the equivalent meaning of the invention are considered to be within the scope of the invention.

[0103] This application is based on Japanese Patent Application No. 2025-015696 filed on 14 January 2025, Japanese Patent Application No. 2025-040734 filed on 25 February 2025, and Japanese Patent Application No. 2025-125943 filed on 9 July 2025. The entire specifications, claims, and drawings of Japanese Patent Application Nos. 2025-015696, 2025-040734, and 2025-125943 are incorporated herein by reference.

[0104] This invention is suitable as a system for suppressing information leakage when providing text to external services.

[0105] 1, 2001: Terminal device, 2: LLM management server, 101: CPU, 102: Main memory unit, 103: Auxiliary memory unit, 104: Display unit, 105: Input unit, 106: Communication unit, 109: Bus, 111, 2111: Query generation unit, 112, 2112: Tokenizer, 116: Warning target probability estimation unit, 117: Warning target token identification unit, 118: Evaluation result text generation unit, 119, 2119: Display control unit, 120, 2120: LLM response acquisition unit, 121, 2121: Teacher information generation unit, 122, 2122: Model generation unit, 131, 2131: Target text storage unit, 132: Warning target probability estimation model storage unit, 133: Warning target token storage unit, 134: Evaluation result storage unit, 135: Leakage prevention list storage unit, 136, 2136: Learning text storage unit, 137, 2137: Teacher information storage unit, 139, 2139: Answer storage unit, 2116: Mask target probability estimation unit, 2117: Mask target token identification unit, 2118: Target text conversion unit, 2123: Answer conversion unit, 2132: Mask target probability estimation model storage unit, 2133: Mask target token storage unit, 2134: Converted target text storage unit, 2135: Mask target list storage unit, 2138: Dummy string storage unit, 2140: Converted answer storage unit, NW1: Network, WO11, WO12, WO13, WO14: String, WO21, WO22, WO23, WO24: Dummy string

Claims

1. Information leakage suppression device comprising: a tokenizer that converts text information to be processed into a sequence of tokens; a mask target probability estimation unit that estimates the mask target start probability and the mask target end probability for each of the tokens included in the sequence of tokens using a mask target probability estimation model for estimating the mask target start probability, which is the probability that the token is the first token in a range of mask target tokens to be masked, and the mask target end probability, which is the probability that the token is the last token in the range of mask target tokens to be masked; a mask target token identification unit that identifies the range of mask target tokens to be masked based on the mask target start probability and the mask target end probability; and a target text conversion unit that identifies the mask target string to be masked based on the text information and the identified mask target token range, and generates converted text information by converting the identified mask target string into a pre-set dummy string.

2. The information leakage suppression device according to claim 1, further comprising: a dummy string storage unit that stores dummy string information indicating the dummy string in association with the mask target string; a large-scale language model response acquisition unit that acquires the response information from the large-scale language model management server by transmitting the converted text information to the large-scale language model management server that manages a large-scale language model that generates response information indicating response text including the dummy string based on the converted text information; and a response conversion unit that identifies the mask target string corresponding to the dummy string information indicating the dummy string included in the response text stored in the dummy string storage unit, and generates converted response information indicating converted response text in which the dummy string has been converted to the identified mask target string.

3. The information leakage suppression device according to claim 2, wherein the large-scale language model response acquisition unit acquires the response information by sending a query information for specifying the response content to the large-scale language model management server, which specifies to the large-scale language model management server that the response should be given as is for the dummy string, along with the converted text information.

4. The information leakage suppression device according to any one of claims 1 to 3, wherein the mask target token identification unit identifies the mask target token range if at least one mask target token range exists, while changing the range of the search for the mask target token range in the token sequence, and determines that the mask target token range does not exist if no mask target token range exists.

5. The information leakage suppression device according to claim 4, further comprising: a teacher information generation unit that generates teacher information based on a list of strings to be masked and learning text information for learning the mask target probability estimation model; and a model generation unit that generates a new mask target probability estimation model using the teacher information, wherein the teacher information generation unit generates the teacher information while changing the range for selecting candidate token ranges to be included in the teacher information in a learning token sequence corresponding to the learning text information.

6. Information leakage suppression device comprising: a tokenizer that converts text information subject to evaluation of the possibility of information leakage into a token sequence; a warning target probability estimation unit that estimates the warning target start probability and the warning target end probability for each token included in the token sequence using a warning target probability estimation model for estimating the probability that the token is the first token in a warning target token range subject to information leakage warning and the warning target end probability of the token is the last token in the warning target token range; a warning target token identification unit that identifies the warning target token range and the location of the warning target token range based on the warning target start probability and the warning target end probability; an evaluation result text generation unit that generates evaluation result text information indicating an information leakage possibility evaluation result in which the warning target string subject to warning is displayed in a different display manner from other strings based on the text information and the identified warning target token range and the location of the warning target token range; a display unit; and a display control unit that causes the generated evaluation result text information to be displayed on the display unit.

7. The information leakage suppression device according to claim 6, wherein the warning target token identification unit identifies the warning target token range if at least one warning target token range exists, while changing the range of searching the warning target token range in the token sequence, and determines that the warning target token range does not exist if no warning target token range exists.

8. The information leakage suppression device according to claim 7, further comprising: a teacher information generation unit that generates teacher information based on a list of warning targets indicating strings to be warned about and learning text information for learning the warning target probability estimation model; and a model generation unit that generates a new warning target probability estimation model using the teacher information, wherein the teacher information generation unit generates the teacher information while changing the range for selecting candidate token ranges to be included in the teacher information in a learning token sequence corresponding to the learning text information.

9. An information leakage suppression method comprising: a step of converting text information to be processed into a token sequence using an information leakage suppression device; a step of using a mask target probability estimation model to estimate the mask target start probability, which is the probability that a token is the first token in a range of mask target tokens to be masked, and the mask target end probability, which is the probability that a token is the last token in the range of mask target tokens to be masked, to estimate the mask target start probability and the mask target end probability for each token included in the token sequence; a step of identifying the range of mask target tokens to be masked based on the mask target start probability and the mask target end probability; and a step of using an information leakage suppression device to identify a mask target string to be masked based on the text information and the identified mask target token range, and generating converted text information by converting the identified mask target string into a pre-set dummy string.

10. An information leakage suppression method comprising: a step of converting text information to be evaluated for the possibility of information leakage into a token sequence; a step of the information leakage suppression device estimating the start probability of a warning target, which is the probability that the token is the first token in a range of warning target tokens subject to information leakage warning, and the end probability of a warning target, which is the probability that the token is the last token in the range of warning target tokens, using a warning target probability estimation model to estimate the start probability of a warning target and the end probability of a warning target for each token included in the token sequence; a step of the information leakage suppression device identifying the range of warning target tokens and the location of the range of warning target tokens based on the start probability of a warning target and the end probability of a warning target; a step of the information leakage suppression device generating evaluation result text information indicating an information leakage possibility evaluation result in which the warning target string is displayed in a different display manner from other strings, based on the text information and the identified range of warning target tokens and the location of the range of warning target tokens; and a step of the information leakage suppression device displaying the generated evaluation result text information on a display unit.

11. A program to cause a computer to function as: a tokenizer that converts text information to be processed into a sequence of tokens; a mask target probability estimation unit that estimates the mask target start probability and the mask target end probability for each of the tokens included in the sequence of tokens, using a mask target probability estimation model for estimating the mask target start probability, which is the probability that the token is the first token in a range of mask target tokens to be masked, and the mask target end probability, which is the probability that the token is the last token in the range of mask target tokens to be masked; a mask target token identification unit that identifies the range of mask target tokens to be masked based on the text information and the identified mask target token range, and generates converted text information by identifying the mask target string to be masked and converting the identified mask target string into a pre-set dummy string.

12. A program to cause a computer to function as: a tokenizer that converts text represented by text information subject to evaluation of the possibility of information leakage into a sequence of tokens; a warning target probability estimation unit that estimates the warning target start probability, which is the probability that the token is the first token in a range of warning target tokens subject to information leakage warnings, and the warning target end probability, which is the probability that the token is the last token in the range of warning target tokens, using a warning target probability estimation model to estimate the warning target start probability and the warning target end probability for each of the tokens included in the sequence of tokens; a warning target token identification unit that identifies the range of warning target tokens and the location of the range of warning target tokens based on the warning target start probability and the warning target end probability; an evaluation result text generation unit that generates evaluation result text information indicating an information leakage possibility evaluation result in which the warning target string is displayed in a different display manner from other strings, based on the text information and the identified range of warning target tokens and the location of the range of warning target tokens; and a display control unit that displays the generated evaluation result text information on a display unit.

Images