Information leakage possibility evaluation device, information leakage possibility evaluation method, and program

Abstract

An information leakage possibility evaluation device, an information leakage possibility evaluation method, and a program are provided that can prevent leakage of confidential information when using an externally provided information processing service. [Solution] The terminal device 1 includes a tokenizer 112 that converts the text to be evaluated into a token sequence, a warning target probability estimation unit 116 that uses a warning target probability estimation model to estimate the warning target start probability and warning target end probability for each token included in the token sequence, a warning target token identification unit 117 that identifies the warning target token range based on the warning target start probability and the warning target end probability, and an evaluation result text generation unit 118 that generates evaluation result text information that indicates the information leakage possibility evaluation result based on text information indicating the above-mentioned text and the identified warning target token range.

Description

[Technical Field]

[0001] The present invention relates to an information leakage possibility assessment device, an information leakage possibility assessment method, and a program. [Background technology]

[0002] An information processing device has been proposed that accepts a question from a user, identifies at least one predetermined consultation answer data from a plurality of consultation answer data consisting of questions or answers to past legal consultations based on the accepted question, and generates a prompt, which is an input sentence for a dialogue response AI, based on the accepted question and the identified predetermined consultation answer data (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2024-162162 Summary of the Invention [Problem to be solved by the invention]

[0004] The dialogue response AI used by the information processing device described in Patent Document 1 is generally provided as an external so-called dialogue response AI service. The information processing device then obtains response data by transmitting the generated prompt to an external server via a wide area network. In this case, if the prompt contains text information indicating so-called confidential information such as a person's name, a person's address, or a new product idea, there is a risk that the confidential information may be leaked to the outside without the user's intention.

[0005] The present invention has been made in consideration of the above-mentioned reasons, and aims to provide an information leakage possibility assessment device that can suppress the leakage of confidential information when using information processing services provided externally. [Means for solving the problem]

[0006] In order to achieve the above object, an information leakage possibility assessment device according to the present invention comprises: a tokenizer that converts text indicated by text information that is the subject of evaluation for the possibility of information leakage into a token sequence; a warning target probability estimation unit that estimates a warning target start probability, which is the probability that a token is the first token in a warning target token range that is subject to an information leakage warning, and a warning target end probability, which is the probability that a token is the last token in the warning target token range, using a warning target probability estimation model for estimating the warning target start probability and the warning target end probability for each token included in the token sequence; a warning target token specifying unit that specifies the warning target token range based on the warning target start probability and the warning target end probability; an evaluation result text generation unit that generates evaluation result text information indicating an information leakage possibility evaluation result in which a warning target character string that is a warning target is displayed in a display mode different from other character strings, based on the text information and the specified warning target token range; A display unit; and a display control unit that causes the generated evaluation result text information to be displayed on the display unit. [Effects of the Invention]

[0007] According to the present invention, the warning target probability estimation unit uses the warning target probability estimation model to estimate the warning target start probability and the warning target end probability for each token included in a token sequence corresponding to text indicated by the text information to be evaluated. The warning target token identification unit identifies the warning target token range and the location of the warning target token range based on the warning target start probability and the warning target end probability. The evaluation result text generation unit then generates evaluation result text information indicating an information leakage possibility evaluation result in which a warning target string is displayed in a different display mode from other strings based on the text information and the identified warning target token range. This allows a user to quickly determine whether the text to be evaluated contains a target string that the user wants to prevent from being leaked to the outside, simply by referring to the evaluation result text information. Therefore, for example, when using an external information processing service to evaluate the text to be evaluated, it is possible to prevent text containing the target string from being provided to the information processing service as is, thereby preventing the leakage of confidential information managed by the user. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a schematic configuration diagram of an information leakage possibility assessment system according to an embodiment of the present invention; [Figure 2] FIG. 2 is a block diagram showing a hardware configuration of a terminal device according to an embodiment. [Figure 3] FIG. 2 is a block diagram showing a functional configuration of a terminal device according to an embodiment. [Figure 4] FIG. 2 is a sequence diagram for explaining the operation of the information leakage possibility assessment system according to the embodiment. [Figure 5] FIG. 10 is a diagram showing an example of an evaluation result notification image displayed on a display unit of the terminal device according to the embodiment. [Figure 6] 10 is a flowchart showing an example of the flow of an information leakage possibility evaluation process executed by a terminal device according to an embodiment. [Figure 7]10 is a flowchart showing an example of the flow of an information leakage possibility evaluation process executed by a terminal device according to an embodiment. [Figure 8] 10 is a flowchart showing an example of the flow of a teacher information generation process according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] An information leakage possibility evaluation device according to an embodiment of the present invention will be described in detail below with reference to the drawings. The information leakage possibility evaluation device according to the present embodiment includes: a target text storage unit that stores text information to be evaluated for the possibility of information leakage; a tokenizer that converts text indicated by the text information into a token sequence; a warning target probability estimation unit that uses a warning target probability estimation model to estimate a warning target start probability, which is the probability that a token is the first token in a warning target token range that is a target for information leakage warning, and a warning target end probability, which is the probability that a token is the last token in the warning target token range, to estimate a warning target start probability and a warning target end probability for each token included in the token sequence; a warning target token identification unit that identifies a warning target token range based on the warning target start probability and the warning target end probability; an evaluation result text generation unit that generates evaluation result text information indicating an information leakage possibility evaluation result, in which a warning target string that is a target for information leakage is displayed in a different display mode from other strings, based on the text information and the identified warning target token range; a display unit; and a display control unit that displays the generated evaluation result text information on the display unit.

[0010] The information leakage possibility assessment system of this embodiment includes, for example, as shown in FIG. 1, an LLM management server 2 that manages a large-scale language model (hereinafter referred to as "LLM"), and a terminal device 1 that can communicate with the LLM management server 2 via a network NW1.

[0011] The LLM management server 2 manages an LLM based on, for example, the Transformer architecture, and includes a query notification acquisition unit (not shown) that acquires query notification information (described later) transmitted from the terminal device 1, a query storage unit (not shown), a tokenizer (not shown), an answer generation unit that generates a token string corresponding to answer information using the LLM, a detokenizer (not shown), and an answer notification unit (not shown) that generates answer notification information (described later) and transmits it to the terminal device 1. Examples of LLMs include Bidirectional Encoder Representations from Transformers (BERT), Generative Pre-trained Transformer 4 (GPT-4), and Pathways Language Model 2 (PaLM2). The query storage unit stores the query information in association with source identification information that identifies the terminal device 1 that transmitted the query information. Upon acquiring query notification information transmitted from the terminal device 1, the query notification acquisition unit extracts the query information included in the acquired query notification information and stores the extracted query information in the query storage unit in association with source identification information that identifies the source of the query information. The tokenizer is, for example, a sentencepiece, and converts text indicated by query information stored in the query storage unit into a token string and inputs the token string to the answer generation unit. The answer generation unit uses LLM to generate a token string corresponding to the answer information and outputs it to the detokenizer. The detokenizer generates answer information by converting the input token string into text. The answer notification unit generates answer notification information including the generated answer information and sender identification information stored in the query storage unit, and transmits the answer notification information to the terminal device 1 that is the sender of the query information.

[0012] The terminal device 1 is, for example, a personal computer. As shown in FIG. 2, the terminal device 1 includes a CPU (Central Processing Unit) 101, a main memory 102, an auxiliary memory 103, a display 104, an input unit 105, a communication unit 106, and a bus 109 connecting the various units. The terminal device 1 functions as an information leakage possibility assessment device that assesses the possibility of information leakage. While the present embodiment describes an example in which the terminal device 1 includes the CPU 101, the terminal device 1 is not limited to a device that includes the CPU 101 and may also include a GPU (Graphics Processing Unit) and an NPU (Neural Processing Unit). The main memory 102 is configured with a volatile memory such as a RAM (Random Access Memory) and is used as a work area for the CPU 101. The auxiliary memory 103 is configured with a nonvolatile memory such as a magnetic disk or semiconductor memory and stores programs for implementing various functions of the terminal device 1. The display 104 is a display device such as a liquid crystal display. The input unit 105 is an input device such as a keyboard. The communication unit 106 has a modem and a gateway, and communicates with the LLM management server 2 via the network NW1.

[0013] In the terminal device 1, the CPU 101 loads the program stored in the auxiliary storage unit 103 into the main storage unit 102 and executes it, thereby functioning as a query generation unit 111, a tokenizer 112, a warning target probability estimation unit 116, a warning target token identification unit 117, an evaluation result text generation unit 118, a display control unit 119, an LLM answer acquisition unit 120, a teacher information generation unit 121, and a model generation unit 122, as shown in Fig. 3. The auxiliary storage unit 103 shown in Fig. 2 also includes a target text storage unit 131, a warning target probability estimation model storage unit 132, a warning target token storage unit 133, an evaluation result storage unit 134, a leak prevention list storage unit 135, a training text storage unit 136, a teacher information storage unit 137, and an answer storage unit 139, as shown in Fig. 3. The target text storage unit 131 stores text information indicating the content of a query input by a user of the terminal device 1 via the input unit 105.

[0014] The warning target probability estimation model storage unit 132 stores warning target probability estimation model information indicating a warning target probability estimation model for estimating a warning target start probability, which is a start token in a warning target token range that is a target for an information leakage warning, and a warning target end probability, which is an end token in the warning target token range. Here, the warning target probability estimation model is a model such as an encoder based on a BERT or a similar Transformer architecture, and when a token sequence is input to the input layer, a warning target start probability and a warning target end probability for each token constituting the token sequence are output from the output layer.

[0015] The warning target token storage unit 133 stores warning target token range information indicating a warning target token range identified based on the warning target start probability and the warning target end probability. This warning target token range information includes information indicating at least one token included in the warning target token range and information indicating the position of the warning target token range in the token string corresponding to the evaluation target text. The evaluation result storage unit 134 stores evaluation result text information indicating an information leakage possibility evaluation result in which a warning target string that is a warning target is displayed in a different display mode from other strings.

[0016] The leak prevention list storage unit 135 stores warning target list information indicating character strings that are to be warned and that have been designated in advance by the user as text that should be prevented from leaking to the outside (warning target character strings). The learning text storage unit 136 stores learning text information indicating texts that include previously prepared warning target character strings.

[0017] The teacher information storage unit 137 stores information indicating each token included in the token sequence corresponding to the text indicated by the learning text information, warning target start probability information indicating the warning target start probability corresponding to the token, and warning target end probability information indicating the warning target end probability, in correspondence with each other.

[0018] The response storage unit 139 stores response information indicating a response corresponding to a query indicated by the above-mentioned query information transmitted from the LLM management server 2.

[0019] When a user of the terminal device 1 performs a query input operation to input a query for an LLM into the input unit 105, the query generation unit 111 accepts the query input operation and generates query information based on the operation content of the accepted query input operation. Then, the query generation unit 111 stores the generated query information in the target text storage unit 131.

[0020] The tokenizer 112 is, for example, a SentencePiece, and upon receiving the above-mentioned query input operation, converts the text indicated by the query information stored in the target text storage unit 131 into a token string and notifies the warning target probability estimation unit 116. Furthermore, when the user performs a model learning operation to train the above-mentioned warning target probability estimation model, the tokenizer 112 receives the model learning operation, converts each character string indicated by the list information stored in the leakage prevention list storage unit 135 into a token, and notifies the training information generation unit 121, and also converts the text indicated by the training text information stored in the training text storage unit 136 into a token string and notifies the training information generation unit 121.

[0021] When the warning target probability estimation unit 116 is notified of a token sequence corresponding to text indicated by the query information from the tokenizer 112, it estimates the above-mentioned warning target start probability and warning target end probability corresponding to each token constituting the notified token sequence, using the warning target estimation model indicated by the warning target estimation model information stored in the warning target probability estimation model storage unit 132. Then, the warning target probability estimation unit 116 notifies the warning target token identification unit 117 of the estimated warning target start probability and warning target end probability of each token and information indicating each token.

[0022] The warning-target token identification unit 117 identifies a warning-target token range if one exists, based on the warning-target start probability and warning-target end probability of each token notified by the warning-target probability estimation unit 116 and information indicating each token. If no warning-target token range exists, the warning-target token identification unit 117 determines that no warning-target token range exists. Here, the warning-target token identification unit 117 first identifies a token with the highest warning-target start probability based on the warning-target start probabilities of all tokens, and identifies the token as a warning-target start token among the warning-target tokens. If a preset special token not included in the token sequence notified by the tokenizer 112 to the warning-target probability estimation unit 116 has the highest warning-target start probability, the warning-target token identification unit 117 determines that the token sequence does not include a warning-target token range. Next, the warning-target token identification unit 117 identifies a token with the highest warning-target end probability based on the warning-target end probabilities of each token that constitutes the portion of the token sequence after the identified warning-target start token, and identifies the token as a warning-target end token. Then, the warning target token identification unit 117 stores warning target token range information in the warning target token storage unit 133, the warning target token range information including information indicating at least one token existing between the identified warning target start token and warning target end token, and information indicating the position of the warning target token range, which is the position of the warning target start token and the position of the warning target end token.

[0023] The evaluation result text generation unit 118 generates evaluation result text information indicating an information leakage possibility evaluation result in which a warning target character string that is a warning target is displayed in a display mode different from other character strings, based on the query information stored in the target text storage unit 131 and the warning target token information and position information stored in the warning target token storage unit 133. Then, the evaluation result text generation unit 118 stores the generated evaluation result text information in the evaluation result storage unit 134.

[0024] When the user performs a query sending operation on the input unit 105 to send query information to the LLM management server 2, the LLM answer acquisition unit 120 accepts the query sending operation and generates query notification information including the query information stored in the target text storage unit 131. The LLM answer acquisition unit 120 then transmits the generated query notification information to the LLM management server 2, thereby acquiring the above-mentioned answer notification information sent from the LLM management server 2. The LLM answer acquisition unit 120 then extracts answer information included in the acquired answer notification information, and stores the extracted answer information in the answer storage unit 139.

[0025] The display control unit 119 forms an evaluation result notification image that notifies the user of the result of the evaluation of the possibility of information leakage based on the evaluation result text information stored in the evaluation result storage unit 134, and displays the image on the display unit 104. Furthermore, when the answer storage unit 139 stores new answer information, the display control unit 119 forms an answer notification image that notifies the user of the content of the answer to the query based on the newly stored answer information, and displays the image on the display unit 104.

[0026] When the tokenizer 112 notifies the training information generation unit 121 of a token sequence corresponding to text indicated by the training text information, the training information generation unit 121 generates training information based on the training token sequence corresponding to the notified training text information and the warning target token range. Here, the training information generation unit 121 generates training information while changing the range from which candidate token ranges to be included in the training information are selected in the training token sequence corresponding to the training text information. Specifically, the training information generation unit 121 searches for a token range matching the warning target token range from the beginning of the notified token sequence. If a match is found, the training information generation unit 121 generates training information consisting of information indicating the first token in the token range and warning target start probability information indicating a probability higher than a preset standard warning target start probability, and information indicating the last token in the token range and warning target end probability information indicating a probability higher than a preset standard warning target end probability, and stores the generated training information in the training information storage unit 137. Next, the training information generation unit 121 changes the search range of the token range to a range after the position in the training token sequence that matches the warning target token range, and again searches for a token range that matches the warning target token range. The training information generation unit 121 then repeats the search for the warning target token range, the generation and storage of training information in the training information storage unit 137, and the change of the search range. The training information generation unit 121 may also generate training information using each of the token sequences corresponding to multiple pieces of training text information.

[0027] Model generation unit 122 generates a new warning target probability model using the teacher information stored in teacher information storage unit 137. Then, model generation unit 122 updates the warning target probability estimation model information stored in warning target probability estimation model storage unit 132 with warning target probability estimation model information indicating the newly generated warning target probability estimation model.

[0028] Next, the operation of the information leakage possibility assessment system according to this embodiment will be described with reference to Fig. 4. First, it is assumed that a user of the terminal device 1 performs a query input operation to input a query for an LLM into the input unit 105. In this case, the terminal device 1 accepts the query input operation, generates query information based on the operation content of the accepted query input operation, and stores the generated query information in the target text storage unit 131 (step S1).

[0029] Next, the terminal device 1 converts the text indicated by the query information stored in the target text storage unit 131 into a token string (step S2).

[0030] Next, the terminal device 1 estimates the warning target start probability and the warning target end probability for the tokens that make up the token sequence using the warning target estimation model indicated by the warning target estimation model information stored in the warning target probability estimation model memory unit 132 (step S3).

[0031] Thereafter, the terminal device 1 determines that there is a warning target token range that is a warning target, based on the estimated warning target start probability and warning target end probability for each token (step S4). In this case, the terminal device 1 identifies the warning target token range and its position. Then, the terminal device 1 stores warning target token range information indicating the identified warning target token range in the warning target token storage unit 133 in association with position information indicating the identified position (step S5).

[0032] Next, the terminal device 1 changes the search range for the warning target token range to a token sequence that follows the identified warning target token range in the token sequence (step S6).Then, the terminal device 1 again repeats the processes of steps S3 to S5. Thereafter, the terminal device 1 repeats the processes of steps S3 to S6 as long as it determines that a warning target token range exists.

[0033] Then, the terminal device 1 determines that there is no warning target token range that is subject to a warning based on the estimated warning target start probability and warning target end probability for each token (step S7). In this case, the terminal device 1 generates evaluation result text information indicating an information leakage possibility evaluation result in which warning target strings that are subject to a warning are displayed in a different display mode from other strings, based on the query information stored in the target text storage unit 131 and the warning target token range information and position information stored in the warning target token storage unit 133. Then, the terminal device 1 stores the generated evaluation result text information in the evaluation result storage unit 134 (step S8).

[0034] Next, the terminal device 1 forms an evaluation result notification image that notifies the user of the result of the evaluation of the possibility of information leakage based on the evaluation result text information stored in the evaluation result storage unit 134, and displays the image on the display unit 104 (step S9). At this time, the terminal device 1 displays an evaluation result notification image GA1, for example, as shown in Fig. 5, on the display unit 104. In this evaluation result notification image GA1, the character string WO1 that is the target of the warning is displayed in a different display mode from the other character strings.

[0035] Returning to FIG. 4, suppose that the user determines that the display unit 104 of the terminal device 1 does not display a character string that is subject to a warning, unlike the evaluation result notification image GA1 shown in FIG. 5, and therefore there is no problem in sending the created query information to the LLM management server 2, and performs a query sending operation to send the query information to the LLM management server 2. In this case, the terminal device 1 accepts the query sending operation and generates query notification information including the query information stored in the target text storage unit 131 (step S10). The generated query notification information is then transmitted from the terminal device 1 to the LLM management server 2 (step S11). Meanwhile, upon receiving the query notification information transmitted from the terminal device 1, the LLM management server 2 extracts the query information included in the acquired query notification information and stores the extracted query information in the query storage unit (step S12). Thereafter, the LLM management server 2 generates answer information corresponding to the query information and generates answer notification information including the generated answer information (step S13). Next, the generated answer notification information is transmitted from the LLM management server 2 to the terminal device 1 that transmitted the query information (step S14).

[0036] Meanwhile, when the terminal device 1 acquires the above-mentioned answer notification information transmitted from the LLM management server 2, it extracts the answer information included in the acquired answer notification information and stores the extracted answer information in the answer storage unit 139 (step S15). Next, the terminal device 1 forms an answer notification image that notifies the user of the content of the answer to the query based on the answer information newly stored in the answer storage unit 139, and displays the image on the display unit 104 (step S16).

[0037] Furthermore, when a user performs a model learning operation on the input unit 105 of the terminal device 1 to train the warning target probability estimation model, the terminal device 1 accepts the model learning operation and converts each character string indicated by the list information stored in the leakage prevention list storage unit 135 into a token. The terminal device 1 also converts text indicated by the training text information stored in the training text storage unit 136 into a token sequence. The terminal device 1 then generates training information consisting of warning target token range information indicating a warning target token range included in the training token sequence corresponding to the text indicated by the training text information, and a token sequence included in a search range corresponding to the warning target token range in the training token sequence. The terminal device 1 generates training information so as to identify the warning target token range that first appears in the training token sequence, then generates a new training token sequence by deleting the training token sequence preceding the warning target token range, and generates next training information so as to identify the warning target token range that first appears in the new training token sequence. The terminal device 1 repeats these processes until all of the learning token strings are deleted, and then generates training information for determining that there is no token range that is subject to a warning. The terminal device 1 then stores the generated training information in the training information storage unit 137 (step S17).

[0038] Next, the terminal device 1 generates a new warning target probability model using the teacher information stored in the teacher information storage unit 137 (step S18). Subsequently, the terminal device 1 updates the warning target probability estimation model information stored in the warning target probability estimation model storage unit 132 with warning target probability estimation model information indicating the newly generated warning target probability estimation model (step S19).

[0039] Next, the information leakage possibility evaluation process executed by the terminal device 1 according to this embodiment will be described with reference to FIGS. 6 to 8. This information leakage possibility evaluation process is started, for example, when a program for generating query information to be transmitted to the LLM management server 2 is started in the terminal device 1. First, as shown in FIG. 6, the query generation unit 111 determines whether or not the above-described query input operation performed by the user on the input unit 105 has been accepted (step S101). Here, if the query generation unit 111 determines that the query input operation has not been accepted (step S101: No), the process of step S114 described below is executed. On the other hand, if the query generation unit 111 determines that the query input operation has been accepted (step S101: Yes), the query generation unit 111 generates query information based on the operation content of the query input operation and stores the generated query information in the target text storage unit 131 (step S102).

[0040] Next, the tokenizer 112 converts the text indicated by the query information stored in the target text storage unit 131 into a token string (step S103).

[0041] Next, the warning target probability estimation unit 116 estimates the warning target start probability and warning target end probability corresponding to each token constituting the token sequence using the warning target estimation model indicated by the warning target estimation model information stored in the warning target probability estimation model storage unit 132 (step S104). Thereafter, the warning target token identification unit 117 determines whether or not the warning target token range exists based on the estimated warning target start probability and warning target end probability for each token and information indicating each token (step S105). Here, if the warning target token identification unit 117 determines that the warning target token range exists (step S105: No), the warning target token identification unit 117 identifies the warning target token range that will be subject to a warning based on the estimated warning target start probability and warning target end probability for each token and information indicating each token. Then, the warning target token identification unit 117 stores warning target token range information indicating the identified warning target token range in the warning target token storage unit 133 (step S106). Next, the warning target probability estimation unit 116 changes the search range of the warning target token range to a token sequence that comes after the identified warning target token range (step S107).The warning target probability estimation unit 116 then estimates the warning target start probability and warning target end probability corresponding to each token included in the changed search range (step S104), and the processing from step S105 onwards is executed again.

[0042] Also, assume that the warning target token identification unit 117 determines in step S105 that there is no warning target token range (step S105: Yes). In this case, the evaluation result text generation unit 118 generates evaluation result text information indicating the information leakage possibility evaluation result in which warning target strings that are the target of a warning are displayed in a different display mode from other strings, based on the query information stored in the target text storage unit 131 and the warning target token range information stored in the warning target token storage unit 133. Then, the evaluation result text generation unit 118 stores the generated evaluation result text information in the evaluation result storage unit 134 (step S108).

[0043] Next, the display control unit 119 forms an evaluation result notification image that notifies the user of the results of the evaluation of the possibility of information leakage based on the evaluation result text information stored in the evaluation result memory unit 134, and displays the image on the display unit 104 (step S109).

[0044] Next, the LLM answer acquisition unit 120 determines whether or not the above-mentioned query transmission operation performed by the user on the input unit 105 has been accepted (step S110). If the LLM answer acquisition unit 120 determines that the query transmission operation has not been accepted (step S110: No), the processing of step S114, which will be described later, is executed. On the other hand, if the LLM answer acquisition unit 120 determines that the query transmission operation has been accepted (step S110: Yes), it generates query notification information including the query information stored in the target text storage unit 131. Then, the LLM answer acquisition unit 120 transmits the generated query notification information to the LLM management server 2 (step S111) to acquire the above-mentioned answer notification information transmitted from the LLM management server 2 (step S112). Thereafter, the LLM answer acquisition unit 120 extracts answer information included in the acquired answer notification information and stores the extracted answer information in the answer storage unit 139 (step S113).

[0045] Next, when the answer storage unit 139 stores new answer information, the display control unit 119 forms an answer notification image based on the newly stored answer information to notify the user of the answer to the query, and displays the image on the display unit 104 (step S114).

[0046] 7, the tokenizer 112 determines whether or not the above-described model learning operation performed by the user on the input unit 105 has been accepted (step S115). If the tokenizer 112 determines that the model learning operation has not been accepted (step S115: No), the process of step S101 is executed again. On the other hand, if the tokenizer 112 determines that the model learning operation has been accepted (step S115: Yes), the training information generation process is executed (step S116).

[0047] The training information generation process will now be described in detail with reference to FIG. 8. First, the tokenizer 112 converts each character string indicated by the list information stored in the leak prevention list storage unit 135 into a token (step S201). Next, the tokenizer 112 converts the text indicated by the training text information stored in the training text storage unit 136 into a training token sequence (step S202). Next, the training information generation unit 121 selects one token range included in the training token sequence corresponding to the training text information (step S203). Thereafter, the training information generation unit 121 determines whether the selected token range matches any one of the warning target token ranges corresponding to the list information (step S204). Here, if the training information generation unit 121 determines that the selected token range does not match any of the warning target token ranges corresponding to the list information (step S204: No), it determines whether all token ranges included in the training token sequence corresponding to the text indicated by the training text information have been selected (step S205). If the teacher information generation unit 121 determines that all token ranges have been selected (step S205: Yes), the processing of step S209, which will be described later, is executed. On the other hand, if the teacher information generation unit 121 determines that there is a token sequence that has not yet been selected (step S205: No), the teacher information generation unit 121 selects another token range (step S203) and then executes the processing of step S204 again. Here, the teacher information generation unit 121 selects token ranges in order from the beginning of the token sequence corresponding to the text indicated by the learning text information.

[0048] Furthermore, it is assumed that the teacher information generation unit 121 determines in step S204 that the selected token range matches one of the warning target token ranges corresponding to the list information (step S204: Yes). In this case, if the search range has not yet been changed in step S208 (described later), the teacher information generation unit 121 generates teacher information consisting of warning target token range information indicating the matching warning target token range and information indicating the training token sequence, and stores the generated teacher information in the teacher information storage unit 137. On the other hand, if the search range has been changed in step S208, the teacher information generation unit 121 generates teacher information consisting of warning target token range information indicating the matching warning target token range and information indicating a new training token sequence obtained by deleting the training token sequence that precedes the position in the training token sequence that matches the warning target token range, and stores the generated teacher information in the teacher information storage unit 137 (step S206). Thereafter, the teacher information generation unit 121 determines whether all token ranges included in the training token sequence have been selected (step S207). If the teacher information generation unit 121 determines that there is a token string that has not yet been selected (step S207: No), it changes the search range for the warning target token range to a range after the position that matches the warning target token range in the training token string (step S208), and the process of step S203 is executed again. At this time, the token range is selected from the changed search range.

[0049] On the other hand, if the training information generation unit 121 determines that all token ranges included in the training token sequence have been selected (step S207: Yes), the tokenizer 112 determines whether there is other training text information (step S209). If the tokenizer 112 determines that there is other training text information that has not been converted into a token sequence (step S209: Yes), it selects other training text information stored in the training text storage unit 136 and executes the process of step S102 again. On the other hand, if the tokenizer 112 determines that all training text information stored in the training text storage unit 136 has been converted into token sequences (step S209: Yes), it ends the training information generation process and executes the process of step S117 shown in FIG. 6.

[0050] 7, model generation unit 122 generates a new warning target probability model using the teacher information stored in teacher information storage unit 137 (step S117). Subsequently, model generation unit 122 updates the warning target probability estimation model information stored in warning target probability estimation model storage unit 132 with warning target probability estimation model information indicating the newly generated warning target probability estimation model (step S118). Thereafter, the process of step S101 is executed again.

[0051] As described above, in the terminal device 1 according to the present embodiment, the warning target probability estimation unit 116 uses the warning target probability estimation model to estimate the warning target probability for each token included in the token sequence corresponding to the text indicated by the text information to be evaluated. The warning target token identification unit 117 identifies the content and position of the warning target token based on the estimated warning target probability. The evaluation result text generation unit 118 then generates evaluation result text information indicating an information leakage possibility evaluation result in which the warning target string is displayed in a different display mode from other strings, based on the text information and the content and position of the identified warning target token. This allows a user to quickly determine whether the text to be evaluated contains a target string that the user wants to prevent from being leaked to the outside, simply by referring to the evaluation result text information. Therefore, for example, when using an external information processing service to evaluate the text to be evaluated, it is possible to prevent text containing the target string from being provided to the information processing service as is, thereby preventing the leakage of confidential information managed by the user.

[0052] Although the embodiments of the present invention have been described above, the present invention is not limited to the configurations of the above-described embodiments. For example, the text information stored in the target text storage unit 131 is not limited to query information to be sent to the LLM management server 2, but may be text information for other purposes. For example, the text information may be query information to be sent to a server managed by a provider of a service that does not use an external LLM, such as an external search service. Alternatively, the text information may not be query information for using an external service. For example, the text information may be a file containing a large amount of text information stored in a PC of a user's surviving family member. In this case, the leak prevention list storage unit 135 may store text information representing information about the surviving family member's bank account as a warning target.

[0053] In the embodiment, an example has been described in which the terminal device 1 includes the teacher information generation unit 121 and the model generation unit 122. However, this is not limiting. For example, a model management server that manages a warning target probability estimation model, separate from the terminal device 1, may include the functions of the teacher information generation unit 121 and the model generation unit 122. Here, the model management server may include, for example, a GPU, an NPU, or the like. In this case, the model management server generates the teacher information and updates the warning target probability estimation model information, and transmits the updated warning target probability estimation model information to the terminal device 1. Then, the terminal device 1 acquires the warning target probability estimation model information transmitted from the model management server and stores it in the warning target probability estimation model storage unit 132. In this case, the warning target probability estimation unit 116 estimates the warning target start probability and the warning target end probability using the trained warning target probability estimation model indicated in the warning target probability estimation model information acquired from the model management server.

[0054] In an embodiment, various functions of the terminal device 1 may be realized at the so-called OS level, which is a stage before text information indicating text input via an input device such as a keyboard is processed by other applications.

[0055] Furthermore, the various functions of the terminal device 1 according to the present invention can be realized by using a normal computer system, not a dedicated system. For example, a program for executing the above operations may be stored on a non-transitory recording medium (such as a CD-ROM (Compact Disc Read Only Memory)) that can be read by a computer system and distributed to a computer connected to a network, and the program may be installed in the computer system to configure the terminal device 1 that executes the above processes.

[0056] The method of providing the program to the computer is arbitrary. For example, the program may be uploaded to a bulletin board system (BBS) on a communication line and distributed to the computer via the communication line. The computer then launches the program and executes it under the control of an operating system (OS) in the same way as other applications. In this way, the computer functions as a terminal device 1 that executes the above-mentioned processes.

[0057] Although the embodiments and modifications of the present invention have been described above, the present invention is not limited to these. The present invention includes any combination of the embodiments and modifications, and any combination to which appropriate modifications have been made. [Industrial Applicability]

[0058] The present invention is suitable as a system for suppressing information leakage when providing text to an external service. [Explanation of symbols]

[0059] 1: terminal device, 2: LLM management server, 101: CPU, 102: main memory, 103: auxiliary memory, 104: display, 105: input, 106: communication unit, 109: bus, 111: query generation unit, 112: tokenizer, 116: warning target probability estimation unit, 117: warning target token identification unit, 118: evaluation result text generation unit, 119: display control unit, 120: LLM answer acquisition unit, 121: teacher information generation unit, 122: model generation unit, 131: target text storage unit, 132: warning target probability estimation model storage unit, 133: warning target token storage unit, 134: evaluation result storage unit, 135: leak prevention list storage unit, 136: learning text storage unit, 137: teacher information storage unit, 139: answer storage unit, NW1: network

Claims

1. a tokenizer that converts text indicated by text information that is the subject of evaluation for the possibility of information leakage into a token sequence; a warning target probability estimation unit that estimates a warning target start probability, which is the probability that a token is the first token in a warning target token range that is subject to an information leakage warning, and a warning target end probability, which is the probability that a token is the last token in the warning target token range, using a warning target probability estimation model for estimating the warning target start probability and the warning target end probability for each token included in the token sequence; a warning target token identifying unit that identifies the warning target token range and a position of the warning target token range based on the warning target start probability and the warning target end probability; an evaluation result text generation unit that generates evaluation result text information indicating an information leakage possibility evaluation result in which a warning target character string that is a warning target is displayed in a display mode different from other character strings, based on the text information, the specified warning target token range and a position of the warning target token range; A display unit; a display control unit that displays the generated evaluation result text information on the display unit, Information leakage possibility evaluation device.

2. the warning target token identification unit, while changing a range in which to search for the warning target token range in the token sequence, identifies the warning target token range if at least one warning target token range exists, and determines that the warning target token range does not exist if no warning target token range exists. The information leakage possibility assessment device according to claim 1 .

3. a training information generating unit that generates training information based on warning target list information indicating character strings that are to be warned and learning text information for learning the warning target probability estimation model; a model generation unit that generates a new warning object probability estimation model using the teacher information, the training information generation unit generates the training information while changing a range for selecting a range of tokens that are candidates for inclusion in the training information in a training token sequence corresponding to the training text information. The information leakage possibility assessment device according to claim 2 .

4. A step in which an information leakage possibility assessment device converts text indicated by text information to be evaluated for the possibility of information leakage into a token string; the information leakage possibility evaluation device uses a warning target probability estimation model for estimating a warning target start probability, which is the probability that a token is the first token in a warning target token range that is the target of an information leakage warning, and a warning target end probability, which is the probability that a token is the last token in the warning target token range, to estimate the warning target start probability and the warning target end probability for each of the tokens included in the token sequence; a step in which the information leakage possibility evaluation device specifies the warning target token range based on the warning target start probability and the warning target end probability; a step in which the information leakage possibility evaluation device generates evaluation result text information indicating the text information, the specified warning target token range, and an information leakage possibility evaluation result in which a warning target character string that is a warning target is displayed in a display mode different from other character strings; and a step of causing the information leakage possibility assessment device to display the generated assessment result text information on a display unit. Information leakage possibility assessment method.

5. Computer, a tokenizer that converts text indicated by text information to be evaluated for the possibility of information leakage into a token sequence; a warning target probability estimation unit that estimates a warning target start probability, which is the probability that a token is the first token in a warning target token range that is the target of an information leakage warning, and a warning target end probability, which is the probability that a token is the last token in the warning target token range, using a warning target probability estimation model for estimating the warning target start probability and the warning target end probability for each of the tokens included in the token sequence; a warning target token identifying unit that identifies the warning target token range based on the warning target start probability and the warning target end probability; an evaluation result text generation unit that generates evaluation result text information indicating an information leakage possibility evaluation result, in which a warning target character string that is a warning target is displayed in a display mode different from other character strings, based on the text information and the specified warning target token range; a display control unit that displays the generated evaluation result text information on a display unit; A program to function as a

Images