Information leakage suppression device, information leakage suppression method, and program

Abstract

This invention provides an information leakage prevention device, an information leakage prevention method, and a program that can suppress the leakage of confidential information when using externally provided information processing services. [Solution] The terminal device 1 includes a tokenizer 112 that converts the text to be evaluated into a sequence of tokens, a mask target probability estimation unit 116 that estimates the mask target start probability and mask target end probability of each token included in the token sequence using a mask target probability estimation model, a mask target token identification unit 117 that identifies a range of mask target tokens based on the mask target start probability and mask target end probability, and a target text conversion unit 118 that identifies a mask target string to be masked based on the text information representing the aforementioned text and the identified mask target token range, and generates converted text information by converting the identified mask target string into a dummy string.

Description

Technical Field

[0001] The present invention relates to an information leakage suppression device, an information leakage suppression method, and a program.

Background Art

[0002] There has been proposed an information processing apparatus that receives a question from a user, identifies at least one predetermined consultation response data from a plurality of consultation response data consisting of questions or answers to past legal consultations based on the received question, and generates a prompt that is an input sentence for a dialogue response AI based on the received question and the identified predetermined consultation response data (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, the dialogue response AI used by the information processing apparatus as described in Patent Document 1 is generally provided as an external so-called response response AI service. Then, the information processing apparatus acquires response data by transmitting the generated prompt to an external server via a wide area network. In this case, if the prompt includes text information indicating so-called confidential information such as personal names, personal addresses, ideas for new products, etc., there is a risk that the confidential information will leak outside in a form unintended by the user.

[0005] The present invention has been made in view of the above circumstances, and an object thereof is to provide an information leakage suppression device, an information leakage suppression method, and a program capable of suppressing leakage of confidential information when using an information processing service provided externally.

Means for Solving the Problems

[0006] To achieve the above objective, the information leakage suppression device according to the present invention is A tokenizer that converts the text information to be processed into a sequence of tokens, A mask target probability estimation unit estimates the mask target start probability and the mask target end probability for each token in the token sequence using a mask target probability estimation model for estimating the mask target start probability, which is the probability that the token is the first token in the range of tokens to be masked, and the mask target end probability, which is the probability that the token is the last token in the range of tokens to be masked. A mask target token identification unit identifies the range of mask target tokens based on the mask target start probability and the mask target end probability, The system includes a target text conversion unit that identifies a target string to be masked based on the text information and the identified range of tokens to be masked, and generates converted text information by converting the identified target string to a pre-set dummy string. [Effects of the Invention]

[0007] According to the present invention, the mask target probability estimation unit estimates the mask target start probability and mask target end probability for each token included in the token sequence corresponding to the text indicated by the text information to be processed, using the aforementioned mask target probability estimation model. The mask target token identification unit identifies the mask target token range based on the mask target start probability and mask target end probability. The target text conversion unit then identifies the mask target string to be masked based on the text information and the identified mask target token range, and generates converted text information by converting the identified mask target string to a pre-set dummy string. As a result, when providing text information to an information processing service provider, for example, in order to use an external information processing service, if the text contains a string that you want to prevent from being leaked to the information processing service provider, you can provide the service provider with that string masked.Therefore, it is possible to prevent text containing a string that you want to prevent from being leaked to the information processing service provider from being provided to the information processing service provider as is, thereby preventing the leakage of confidential information managed by the user. [Brief explanation of the drawing]

[0008] [Figure 1] This is a schematic diagram of an information leakage suppression system according to an embodiment of the present invention. [Figure 2] This is a block diagram showing the hardware configuration of a terminal device according to an embodiment. [Figure 3] This is a block diagram showing the functional configuration of a terminal device according to an embodiment. [Figure 4] This is a sequence diagram illustrating the operation of the information leakage suppression system according to the embodiment. [Figure 5] (A) is a diagram showing an example of query information before conversion according to the embodiment, (B) is a diagram showing an example of query information after conversion according to the embodiment, and (C) is a diagram showing an example of query information to be sent to the large-scale language model management server along with the query information. [Figure 6](A) is a diagram showing an example of the response information before conversion according to the embodiment, and (B) is a diagram showing an example of the response information after conversion according to the embodiment. [Figure 7] This flowchart shows an example of the flow of information leakage prevention processing performed by the terminal device according to the embodiment. [Figure 8] This flowchart shows an example of the flow of information leakage prevention processing performed by the terminal device according to the embodiment. [Figure 9] This flowchart shows an example of the flow of the teacher information generation process according to the embodiment. [Modes for carrying out the invention]

[0009] Hereinafter, an information leakage suppression device according to one embodiment of the present invention will be described in detail with reference to the drawings. The information leakage suppression device according to this embodiment includes: a target text storage unit that stores text information to be evaluated for the possibility of information leakage; a tokenizer that converts the text indicated by the text information into a token sequence; a mask target probability estimation unit that estimates the mask target start probability and the mask target end probability for each of the tokens included in the token sequence using a mask target probability estimation model for estimating the mask target start probability, which is the probability that the token is the first token in the mask target token range to be masked, and the mask target end probability, which is the probability that the token is the last token in the mask target token range; a mask target token identification unit that identifies a mask target token range based on the mask target start probability and the mask target end probability; and a target text conversion unit that identifies a mask target string to be masked based on the aforementioned text information and the identified mask target token range, and generates converted text information by converting the identified mask target string into a pre-set dummy string.

[0010] The information leakage suppression system according to this embodiment includes, for example, as shown in Figure 1, an LLM management server 2 that manages a large-scale language model (hereinafter referred to as "LLM"), and a terminal device 1 that can communicate with the LLM management server 2 via a network NW1.

[0011] The LLM management server 2 manages an LLM based on the Transformer architecture and includes a query notification acquisition unit (not shown) that acquires query notification information transmitted from the terminal device 1 (described later), a query storage unit (not shown), a tokenizer (not shown), a response generation unit that generates a token sequence corresponding to the response information using the LLM, a detoxifier (not shown), and a response notification unit (not shown) that generates the response notification information described later and sends it to the terminal device 1. Examples of LLMs include BERT (Bidirectional Encoder Representations from Transformers), GPT-4 (Generative Pre-trained Transformer 4), and PaLM2 (Pathways Language Model 2). The query storage unit stores query information in association with source identification information that identifies the terminal device 1 that sent the query information. When the query notification acquisition unit acquires query notification information transmitted from the terminal device 1, it extracts the query information contained in the acquired query notification information and stores the extracted query information in the query storage unit in association with source identification information that identifies the source of the query information. The tokenizer, for example, is Sentencepiece, which converts the text indicated by the query information stored in the query storage unit into a token sequence and inputs it to the response generation unit. The response generation unit uses LLM to generate a token sequence corresponding to the response information and outputs it to the detokenizer. The detokenizer generates the response information by converting the input token sequence into text. The response notification unit generates response notification information including the generated response information and the source identification information stored in the query storage unit, and sends it to the terminal device 1 that sent the query information.

[0012] Terminal device 1 is, for example, a personal computer, and as shown in Figure 2, it comprises a CPU (Central Processing Unit) 101, a main memory unit 102, an auxiliary memory unit 103, a display unit 104, an input unit 105, a communication unit 106, and a bus 109 connecting the units, and functions as an information leakage suppression device that evaluates the possibility of information leakage. In this embodiment, an example in which terminal device 1 is equipped with a CPU 101 is described, but terminal device 1 is not limited to one equipped with a CPU 101, and may be equipped with a GPU (Graphics Processing Unit) or an NPU (Neural Processing Unit). The main memory unit 102 is composed of volatile memory such as RAM (Random Access Memory) and is used as a working area for the CPU 101. The auxiliary memory unit 103 is composed of non-volatile memory such as a magnetic disk or semiconductor memory and stores programs for realizing various functions of terminal device 1. The display unit 104 is a display device such as a liquid crystal display. The input unit 105 is an input device such as a keyboard. The communication unit 106 has a modem and a gateway and communicates with the LLM management server 2 via the network NW1.

[0013] In terminal device 1, the CPU 101 reads the program stored in auxiliary storage unit 103 into main storage unit 102 and executes it, so that, as shown in Figure 3, it functions as a query generation unit 111, a tokenizer 112, a mask target probability estimation unit 116, a mask target token identification unit 117, a target text conversion unit 118, a display control unit 119, an LLM answer acquisition unit 120, a teacher information generation unit 121, a model generation unit 122, and an answer conversion unit 123. Furthermore, the auxiliary storage unit 103 shown in Figure 2 has, as shown in Figure 3, a target text storage unit 131, a mask target probability estimation model storage unit 132, a mask target token storage unit 133, a converted target text storage unit 134, a mask target list storage unit 135, a learning text storage unit 136, a teacher information storage unit 137, a dummy string storage unit 138, an answer storage unit 139, and a converted answer storage unit 140. The target text storage unit 131 stores text information indicating the query content entered by the user of the terminal device 1 via the input unit 105.

[0014] The mask target probability estimation model storage unit 132 stores information about a mask target probability estimation model for estimating the mask target probability estimation model, which is the starting token in the range of mask target tokens to be masked, and the mask target end probability, which is the ending token in the range of mask target tokens. Here, the mask target probability estimation model is a model such as BERT or an encoder based on a similar Transformer architecture, and when a token sequence is input to the input layer, the output layer outputs the mask target start probability and the mask target end probability for each token constituting the token sequence.

[0015] The masked token storage unit 133 stores masked token range information indicating a masked token range that is a masked target specified based on the aforementioned masked target start probability and masked target end probability. This masked token range information includes information indicating at least one token included in the masked token range and information indicating the position of the masked token range in the token sequence corresponding to the text information stored in the target text storage unit 131. The post-conversion target text storage unit 134 stores post-conversion target text information representing the text after converting the text string to be masked included in the text indicated by the text information stored in the target text storage unit 131 into a dummy string.

[0016] The masked list storage unit 135 stores masked list information indicating a string specified in advance by the user as text (masked string) to be prevented from leaking to the outside. The learning text storage unit 136 stores learning text information indicating text including a masked string prepared in advance.

[0017] The teacher information storage unit 137 stores information indicating each token included in the token sequence corresponding to the text indicated by the learning text information, masked target start probability information indicating the masked target start probability corresponding to the token, and masked target end probability information indicating the masked target end probability, in association with each other.

[0018] The dummy string storage unit 138 stores dummy string information indicating a dummy string in association with the masked string. Here, the dummy string is a string that does not remind of a string representing specific personal information or the like that is the masked target, and is, for example, a string composed of alphabet characters, numbers, symbols, etc. The dummy string storage unit 138 stores, for example, dummy string information representing the dummy string "×××××" in association with the string information representing the name "Taro Yamada" of the individual specified as the masked target.

[0019] The answer storage unit 139 stores answer information that represents answer text containing a dummy string corresponding to the query indicated by the aforementioned query information sent from the LLM management server 2. The converted answer storage unit 140 stores converted answer information that represents the converted answer text after the dummy string contained in the answer text has been converted to the corresponding mask target string.

[0020] When a user of terminal device 1 performs a query input operation to input a query to LLM into input unit 105, the query generation unit 111 receives the query input operation and generates query information based on the content of the received query input operation. The query generation unit 111 then stores the generated query information in target text storage unit 131.

[0021] The tokenizer 112 is, for example, a SentencePiece. After receiving the aforementioned query input operation, when the user performs a query transmission operation to the input unit 105 to send the aforementioned query information to the LLM management server 2, the tokenizer 112 receives the query transmission operation, converts the text indicated by the query information stored in the target text storage unit 131 into a token sequence, and notifies the mask target probability estimation unit 116. Furthermore, when the user performs a model training operation to train the aforementioned mask target probability estimation model, the tokenizer 112 receives the model training operation, converts each string indicated by the list information stored in the mask target list storage unit 135 into a token, and notifies the teacher information generation unit 121. It also converts the text indicated by the learning text information stored in the learning text storage unit 136 into a token sequence and notifies the teacher information generation unit 121.

[0022] When the Mask Target Probability Estimation Unit 116 receives a token sequence corresponding to the text indicated by the query information from the tokenizer 112, it uses the Mask Target Probability Estimation Model stored in the Mask Target Probability Estimation Model Storage Unit 132 to estimate the aforementioned Mask Target Start Probability and Mask Target End Probability for each token that makes up the notified token sequence. The Mask Target Probability Estimation Unit 116 then notifies the Mask Target Token Identification Unit 117 of the estimated Mask Target Start Probability and Mask Target End Probability for each token, along with information indicating each token.

[0023] The Masked Token Identification Unit 117 identifies the aforementioned Masked Token Range if it exists, based on the Masked Token Start Probability and Masked Token End Probability for each token notified by the Masked Token Probability Estimation Unit 116 and the information indicating each token, and determines that the Masked Token Range does not exist if it does not exist. Here, the Masked Token Identification Unit 117 first identifies the token with the highest Masked Token Start Probability based on the Masked Token Start Probability of all tokens, and identifies that token as the Masked Token Start Token of the Masked Token Range. At this time, if the Masked Token Identification Unit 117 has the highest Masked Token Start Probability of a pre-set special token that is not included in the aforementioned token sequence notified by the Tokenizer 112 to the Masked Token Probability Estimation Unit 116, it determines that the Masked Token Range is not included in that token sequence. Next, the Masked Token Identification Unit 117 identifies the token with the highest Masked Token End Probability based on the Masked Token End Probability of each token that constitutes the portion of the token sequence after the identified Masked Token Start Token, and identifies that token as the Masked Token End Token. The mask target token identification unit 117 then causes the mask target token storage unit 133 to store mask target token range information, which includes information indicating at least one token existing between the identified mask target start token and mask target end token, and information indicating the location of the mask target token range, which is the location of the mask target start token and the location of the mask target end token.

[0024] The target text conversion unit 118 identifies the strings to be masked based on the query information stored in the target text storage unit 131 and the range of masked tokens identified by the masked token identification unit 117, and generates converted text information by converting the identified masked strings into pre-set dummy strings. Specifically, the target text conversion unit 118 identifies the strings to be masked based on the query information stored in the target text storage unit 131, information indicating at least one token included in the range of masked tokens indicated by the masked token range information stored in the masked token storage unit 133, and information indicating the position of the masked token range in the token sequence corresponding to the text to be processed. Next, the target text conversion unit 118 generates converted text information by converting each of the identified masked strings into pre-set dummy strings. Here, if the text to be processed contains multiple types of masked strings, the target text conversion unit 118 converts each of the multiple types of masked strings into a different dummy string. The target text conversion unit 118 then stores the generated converted text information in the converted target text storage unit 134, and stores dummy string information representing dummy strings included in the text indicated by the converted text information in the dummy string storage unit 138, associating it with mask target string information representing the corresponding mask target string.

[0025] When the converted text information is stored in the converted target text storage unit 134, the LLM response acquisition unit 120 generates query information for specifying the response content, which instructs the LLM management server 2 to respond as is for the dummy string, along with query notification information, using the converted text information stored in the converted target text storage unit 134. The LLM response acquisition unit 120 then sends the generated query notification information to the LLM management server 2, thereby acquiring the aforementioned response notification information sent from the LLM management server 2. The LLM response acquisition unit 120 then extracts the response information contained in the acquired response notification information and stores the extracted response information in the response storage unit 139.

[0026] The answer conversion unit 123 identifies the mask target string corresponding to the dummy string information stored in the answer information stored in the answer storage unit 139, which is stored in the dummy string storage unit 138. Next, the answer conversion unit 123 generates converted answer information for the answer text, which shows the converted answer text in which the dummy string contained in the answer text has been converted to the identified mask target string. Then, the answer conversion unit 123 stores the generated converted answer information in the converted answer storage unit 140.

[0027] When the converted response storage unit 140 stores new converted response information, the display control unit 119, based on the converted response information, forms a response notification image showing the content of the response from the LLM management server 2 to the query indicated by the query information entered by the user, and displays it on the display unit 104.

[0028] When the teacher information generation unit 121 receives notification from the tokenizer 112 of a token sequence corresponding to the text indicated by the learning text information, it generates teacher information based on the learning token sequence and the mask target token range corresponding to the notified learning text information. Here, the teacher information generation unit 121 generates teacher information while changing the range in which it selects candidate token ranges to be included in the teacher information from the learning token sequence corresponding to the learning text information. Specifically, the teacher information generation unit 121 searches for a token range that matches the mask target token range from the beginning of the notified token sequence, and if a match is found, it generates teacher information consisting of information indicating the first token of the token range and mask target start probability information indicating a probability higher than a preset standard mask target start probability, and stores the generated teacher information in the teacher information storage unit 137. Next, the teacher information generation unit 121 changes the search range of the token range to a range after the position that matches the masked token range in the learning token sequence, and then searches for the token range that matches the masked token range again. The teacher information generation unit 121 then repeats the search for the masked token range, the generation of teacher information and its storage in the teacher information storage unit 137, and the change of the search range. The teacher information generation unit 121 may also generate teacher information using each of the token sequences corresponding to multiple learning text information.

[0029] The model generation unit 122 generates a new mask target probability estimation model using the teacher information stored in the teacher information storage unit 137. Then, the model generation unit 122 updates the mask target probability estimation model information stored in the mask target probability estimation model storage unit 132 with the mask target probability estimation model information representing the newly generated mask target probability estimation model.

[0030] Next, the operation of the information leakage suppression system according to this embodiment will be explained with reference to Figure 4. First, suppose a user of terminal device 1 performs a query input operation to input a query to LLM to the input unit 105. In this case, terminal device 1 receives the query input operation, generates query information based on the content of the received query input operation, and stores the generated query information in the target text storage unit 131 (step S1).

[0031] Now, suppose the user performs a query transmission operation to send the aforementioned query information to the LLM management server 2. In this case, terminal device 1 converts the text indicated by the query information stored in target text storage unit 131 into a token sequence (step S2).

[0032] Next, terminal device 1 uses the mask target probability estimation model indicated by the mask target probability estimation model information stored in the mask target probability estimation model storage unit 132 to estimate the mask target start probability and mask target end probability for the tokens constituting the token sequence (step S3).

[0033] Next, terminal device 1 determines that there is a range of tokens to be masked, based on the estimated start and end probabilities of each token being masked (step S4). In this case, terminal device 1 identifies the range of tokens to be masked and stores the information of the range of tokens to be masked, which indicates the identified range of tokens to be masked, in the token storage unit 133 (step S5).

[0034] Subsequently, terminal device 1 changes the search range for the masked token range to the token sequence that is after the identified masked token range in the token sequence (step S6). Then, terminal device 1 repeats the process in steps S3 to S5 again. Thereafter, terminal device 1 repeats the process in steps S3 to S6 as long as it determines that a masked token range exists.

[0035] Then, terminal device 1 determines that there is no range of tokens to be masked, based on the estimated start and end probabilities of each token to be masked (step S7). In this case, terminal device 1 identifies the string to be masked based on the text information stored in the target text storage unit 131 and the range of tokens to be masked stored in the target token storage unit 133. Terminal device 1 also generates converted text information by converting each identified string to be masked into a pre-set dummy string. Then, terminal device 1 stores the generated converted text information in the converted target text storage unit 134, and stores dummy string information representing the dummy strings contained in the text indicated by the converted text information in the dummy string storage unit 138, associating it with the corresponding string to be masked information (step S8).

[0036] Suppose the user inputs a query containing the masked strings WO11 and WO12, as shown in Figure 5(A). In this case, terminal device 1 generates converted text information for the text representing the query, showing the converted text in which the masked strings have been converted to dummy strings WO21 and WO22, as shown in Figure 5(B).

[0037] Returning to Figure 4, the terminal device 1 then receives the query transmission operation and generates query notification information including the converted text information stored in the converted target text storage unit 134 (step S9). Here, the query notification information includes the converted text information mentioned above, as well as the aforementioned query information for specifying the answer content, as shown in Figure 5(C), for example. Then, as shown in Figure 4, the generated query notification information is sent from the terminal device 1 to the LLM management server 2 (step S10). Meanwhile, when the LLM management server 2 receives the query notification information sent from the terminal device 1, it extracts the converted text information and the aforementioned query information for specifying the answer content contained in the acquired query notification information and stores these extracted pieces of information in the query storage unit (step S11). Subsequently, the LLM management server 2 generates answer information corresponding to the converted text information based on the converted text information and the query information for specifying the answer content stored in the query storage unit, and generates answer notification information including the generated answer information (step S12). Next, the generated response notification information is sent from the LLM management server 2 to the terminal device 1 that sent the query notification information (step S13).

[0038] Meanwhile, when terminal device 1 receives the aforementioned response notification information transmitted from LLM management server 2, it extracts the response information contained in the received response notification information and stores the extracted response information in the response storage unit 139 (step S14). Here, terminal device 1 receives response information representing the response text which includes dummy strings WO23 and WO24, for example, as shown in Figure 6(A). Subsequently, as shown in Figure 4, terminal device 1 identifies the mask target string corresponding to the dummy string information stored in the dummy string storage unit 138, which indicates the dummy string contained in the response text indicated by the response information stored in the response storage unit 139. Terminal device 1 also generates converted response information for the response text which indicates the converted response text in which the dummy string contained in the response text has been converted into the identified mask target string. Then, the response conversion unit 123 stores the generated converted response information in the converted response storage unit 140 (step S15). Here, terminal device 1 generates converted response information that shows the converted response text, in which the dummy strings of the response text indicated by the response information are converted into masked strings WO13 and WO14, for example, as shown in Figure 6(B).

[0039] Returning to Figure 4, the terminal device 1 then forms an answer notification image on the display unit 104 to notify the user of the answer content to the query based on the converted answer information newly stored in the converted answer storage unit 140 (step S16).

[0040] Furthermore, when a user performs a model learning operation on the input unit 105 of the terminal device 1 to train the aforementioned mask target probability estimation model, the terminal device 1 accepts the model learning operation and converts each string indicated by the list information stored in the mask target list storage unit 135 into a token. The terminal device 1 also converts the text indicated by the learning text information stored in the learning text storage unit 136 into a token sequence. Then, the terminal device 1 generates teacher information consisting of mask target token range information indicating the mask target token range included in the learning token sequence corresponding to the text indicated by the learning text information, and a token sequence included in the search range corresponding to the mask target token range in the learning token sequence. Here, the terminal device 1 generates teacher information so as to identify the first mask target token range that appears in the learning token sequence, then generates a new learning token sequence by deleting the learning token sequence preceding the mask target token range, and generates the next teacher information so as to identify the first mask target token range that appears in the new learning token sequence. Then, by repeating these processes, terminal device 1 generates training information to determine if there are no tokens in the mask target range once all of the training token sequence has been deleted. Terminal device 1 then stores the generated training information in the training information storage unit 137 (step S17).

[0041] Next, terminal device 1 generates a new mask target probability estimation model using the teacher information stored in the teacher information storage unit 137 (step S18). Subsequently, terminal device 1 updates the mask target probability estimation model information stored in the mask target probability estimation model storage unit 132 with the mask target probability estimation model information representing the newly generated mask target probability estimation model (step S19).

[0042] Next, the information leakage suppression process performed by the terminal device 1 according to this embodiment will be explained with reference to Figures 7 to 9. This information leakage suppression process is started, for example, when a program for generating query information to be sent to the LLM management server 2 is started in the terminal device 1. First, as shown in Figure 7, the query generation unit 111 determines whether or not it has received the aforementioned query input operation performed by the user to the input unit 105 (step S101). If the query generation unit 111 determines that it has not received the query input operation (step S101: No), the process in step S115, which will be described later, is executed. On the other hand, if the query generation unit 111 determines that it has received the query input operation (step S101: Yes), it generates query information based on the content of the query input operation and stores the generated query information in the target text storage unit 131 (step S102).

[0043] Next, the tokenizer 112 determines whether or not it has received the query transmission operation performed by the user to the input unit 105 (step S103). If the tokenizer 112 determines that it has not received the query transmission operation (step S103: No), the process in step S115, described later, is executed. On the other hand, if the tokenizer 112 determines that it has received the query transmission operation (step S103: Yes), it converts the text indicated by the query information stored in the target text storage unit 131 into a token sequence (step S104).

[0044] Next, the mask target probability estimation unit 116 estimates the aforementioned mask target start probability and mask target end probability corresponding to each token constituting the aforementioned token sequence using the mask target probability estimation model indicated by the mask target probability estimation model information stored in the mask target probability estimation model storage unit 132 (step S105). Subsequently, the mask target token identification unit 117 determines whether or not there is a mask target token range based on the estimated mask target start probability and mask target end probability for each token and the information indicating each token (step S106). If the mask target token identification unit 117 determines that there is a mask target token range (step S106: No), the mask target token identification unit 117 identifies the mask target token range to be masked based on the estimated mask target start probability and mask target end probability for each token and the information indicating each token. Then, the mask target token identification unit 117 stores the mask target token range information indicating the identified mask target token range in the mask target token storage unit 133 (step S107). Next, the mask target probability estimation unit 116 changes the search range of the mask target token range to a sequence of tokens later than the identified mask target token range (step S108). Then, the mask target probability estimation unit 116 estimates the aforementioned mask target start probability and mask target end probability corresponding to each token included in the changed search range (step S105), and the processing from step S106 onwards is executed again.

[0045] Furthermore, the Mask Target Token Identification Unit 117 determines in step S106 that there is no Mask Target Token Range (Step S106: Yes). In this case, the Target Text Conversion Unit 118 identifies the Mask Target String to be masked based on the query information stored in the Target Text Storage Unit 131 and the Mask Target Token Range identified by the Mask Target Token Identification Unit 117, and generates converted text information by converting the identified Mask Target String to a pre-set dummy string. The Target Text Conversion Unit 118 then stores the generated converted text information in the Converted Target Text Storage Unit 134, and stores dummy string information representing the dummy string included in the text indicated by the converted text information in the dummy string storage unit 138, associating it with the Mask Target String information representing the corresponding Mask Target String (Step S109).

[0046] Subsequently, the LLM response acquisition unit 120 generates query notification information that includes the converted text information stored in the converted target text storage unit 134 and the aforementioned query information for specifying the response content. Then, the LLM response acquisition unit 120 sends the generated query notification information to the LLM management server 2 (step S110) and acquires the aforementioned response notification information sent from the LLM management server 2 (step S111). Subsequently, the LLM response acquisition unit 120 extracts the response information contained in the acquired response notification information and stores the extracted response information in the response storage unit 139 (step S112).

[0047] Next, the answer conversion unit 123 identifies a mask target string corresponding to the dummy string information stored in the answer information stored in the answer storage unit 139, which is stored in the dummy string storage unit 138. The answer conversion unit 123 also generates converted answer information for the answer text, which shows the converted answer text in which the dummy string contained in the answer text has been converted to the identified mask target string. The answer conversion unit 123 then stores the generated converted answer information in the converted answer storage unit 140 (step S113). Subsequently, when the answer storage unit 139 stores new answer information, the display control unit 119 forms an answer notification image based on the newly stored answer information to notify the user of the answer content to the query and displays it on the display unit 104 (step S114).

[0048] Next, as shown in Figure 8, the tokenizer 112 determines whether or not it has received the aforementioned model learning operation performed by the user on the input unit 105 (step S115). If the tokenizer 112 determines that it has not received the model learning operation (step S115: No), the process in step S101 is executed again. On the other hand, if the tokenizer 112 determines that it has received the model learning operation (step S115: Yes), the teacher information generation process is executed (step S116).

[0049] Here, the teacher information generation process will be explained in detail with reference to Figure 9. First, the tokenizer 112 converts each string indicated by the list information stored in the mask target list storage unit 135 into a token (step S201). Next, the tokenizer 112 converts the text indicated by the learning text information stored in the learning text storage unit 136 into a learning token sequence (step S202). Subsequently, the teacher information generation unit 121 selects one token range included in the learning token sequence corresponding to the learning text information (step S203). After that, the teacher information generation unit 121 determines whether the selected token range matches any of the mask target token ranges corresponding to the aforementioned list information (step S204). Here, if the teacher information generation unit 121 determines that the selected token range does not match any of the mask target token ranges corresponding to the aforementioned list information (step S204: No), it determines whether all token ranges included in the learning token sequence corresponding to the text indicated by the learning text information have been selected (step S205). Here, if the teacher information generation unit 121 determines that it has selected all token ranges (step S205: Yes), it executes the process in step S209, which will be described later. On the other hand, if the teacher information generation unit 121 determines that there is a token sequence that has not yet been selected (step S205: No), it selects one other token range (step S203) and then executes the process in step S204 again. Here, the teacher information generation unit 121 selects token ranges in order from the beginning of the token sequence corresponding to the text indicated by the learning text information.

[0050] Furthermore, the teacher information generation unit 121 determines in step S204 that the selected token range matches one of the mask target token ranges corresponding to the aforementioned list information (step S204: Yes). In this case, if the teacher information generation unit 121 has not yet performed the search range change in step S208 described below, it generates teacher information consisting of mask target token range information indicating the matched mask target token range and information indicating the learning token sequence, and stores it in the teacher information storage unit 137. On the other hand, if the teacher information generation unit 121 has performed the search range change in step S208, it generates teacher information consisting of mask target token range information indicating the matched mask target token range and information indicating a new learning token sequence obtained by deleting the learning token sequence prior to the position matching the mask target token range in the learning token sequence, and stores it in the teacher information storage unit 137 (step S206). Subsequently, the teacher information generation unit 121 determines whether or not it has selected all the token ranges included in the learning token sequence (step S207). Here, if the teacher information generation unit 121 determines that there is a token sequence that has not yet been selected (step S207: No), it changes the search range for the masked token range to a range after the position that matches the masked token range in the training token sequence (step S208), and the process in step S203 is executed again. At this time, the token range is selected from the changed search range.

[0051] On the other hand, when the teacher information generation unit 121 determines that it has selected all the token ranges included in the learning token sequence (step S207: Yes), the tokenizer 112 determines whether or not there is other learning text information (step S209). Here, if the tokenizer 112 determines that there is other learning text information that has not been converted into a token sequence (step S209: Yes), it selects other learning text information stored in the learning text storage unit 136 and executes the process in step S102 again. On the other hand, when the tokenizer 112 determines that it has converted all the learning text information stored in the learning text storage unit 136 into a token sequence (step S209: Yes), the teacher information generation process ends and the process in step S117 shown in Figure 8 is executed.

[0052] Next, as shown in Figure 8, the model generation unit 122 generates a new mask target probability estimation model using the teacher information stored in the teacher information storage unit 137 (step S117). Subsequently, the model generation unit 122 updates the mask target probability estimation model information stored in the mask target probability estimation model storage unit 132 with the mask target probability estimation model information indicating the newly generated mask target probability estimation model (step S118). After that, the process of step S101 is executed again.

[0053] As described above, according to the terminal device 1 of this embodiment, the mask target probability estimation unit 116 estimates the mask target start probability and mask target end probability for each token included in the token sequence corresponding to the text indicated by the text information to be sent to the LLM management server 2, using the mask target probability estimation model described above. The mask target token identification unit 117 identifies the mask target token range based on the mask target start probability and mask target end probability. The target text conversion unit 118 then identifies the mask target string to be masked based on the text information described above and the identified mask target token range, and generates converted text information by converting the identified mask target string to the dummy string described above. As a result, when sending text information to the LLM management server 2, for example, in order to use the information processing service provided by the LLM management server 2, if the text information contains a string that you want to prevent from being leaked to the information processing service provider via the LLM management server 2, that string can be replaced with a dummy string and provided to the information processing service provider in a masked state. Therefore, it is possible to prevent text containing strings that you want to keep hidden from information processing service providers from being directly provided to those service providers, thus preventing the leakage of confidential information managed by the user.

[0054] Although embodiments of the present invention have been described above, the present invention is not limited to the configuration of the embodiments described above. For example, the text information stored in the target text storage unit 131 is not limited to text information representing queries to be sent to the LLM management server 2, but may be text information for other purposes. For example, the text information may be query information to be sent to a server managed by a business operator that provides services that do not use external LLM, such as external search services. Alternatively, the text information may not be query information for using external services. For example, the text information may be stored on the PC of a user's bereaved family member. The file may contain a large amount of text information. In this case, the mask target list storage unit 135 should store text information representing information about the bereaved family's bank accounts as the target of masking.

[0055] In the embodiment described, an example was given in which the terminal device 1 includes a teacher information generation unit 121 and a model generation unit 122. However, the embodiment is not limited to this, and for example, a model management server that manages the mask target probability estimation model may be separate from the terminal device 1 and have the functions of the aforementioned teacher information generation unit 121 and model generation unit 122. Here, the model management server may be equipped with a GPU, NPU, etc. In this case, the model management server should perform the generation of teacher information and the updating of the mask target probability estimation model information, and send the updated mask target probability estimation model information to the terminal device 1. The terminal device 1 should then acquire the mask target probability estimation model information sent from the model management server and store it in the mask target probability estimation model storage unit 132. In this case, the mask target probability estimation unit 116 should use the trained mask target probability estimation model indicated by the mask target probability estimation model information acquired from the model management server to estimate the mask target start probability and the mask target end probability.

[0056] In this embodiment, the various functions of the terminal device 1 may be implemented at the so-called OS level, before the text information, which represents text entered via an input device such as a keyboard, is processed by other applications.

[0057] Furthermore, the various functions of the terminal device 1 according to the present invention can be realized using a normal computer system, not a dedicated system. For example, a program for performing the above operations may be stored on a non-temporary recording medium (such as a CD-ROM (Compact Disc Read Only Memory)) that can be read by the computer system and distributed to computers connected to a network, and the terminal device 1 that performs the above operations may be configured by installing the program on the computer system.

[0058] Furthermore, the method of providing the program to the computer is arbitrary. For example, the program may be uploaded to a bulletin board system (BBS) on a communication line and distributed to the computer via the communication line. The computer then launches this program and executes it under the control of the operating system (OS), just like any other application. In this way, the computer functions as terminal device 1, performing the processes described above.

[0059] Although embodiments and variations of the present invention have been described above, the present invention is not limited thereto. The present invention includes embodiments and variations that are appropriately combined, and those that are appropriately modified thereto. [Industrial applicability]

[0060] This invention is suitable as a system for suppressing information leakage when providing text to external services. [Explanation of symbols]

[0061] 1: Terminal device, 2: LLM management server, 101: CPU, 102: Main memory unit, 103: Auxiliary memory unit, 104: Display unit, 105: Input unit, 106: Communication unit, 109: Bus, 111: Query generation unit, 112: Tokenizer, 116: Mask target probability estimation unit, 117: Mask target token identification unit, 118: Target text conversion unit, 119: Display control unit, 120: LLM response acquisition unit, 121: Training information generation unit, 122: Model generation unit, 123: Response conversion unit, 131: 132: Target text memory, 133: Masked target probability estimation model memory, 134: Masked target token memory, 135: Converted target text memory, 136: Masked target list memory, 137: Training text memory, 138: Teacher information memory, 139: Dummy string memory, 140: Converted answer memory, NW1: Network, WO11, WO12, WO13, WO14: Strings, WO21, WO22, WO23, WO24: Dummy strings

Claims

1. A tokenizer that converts the text information to be processed into a sequence of tokens, A mask target probability estimation unit estimates the mask target start probability and the mask target end probability for each token in the token sequence using a mask target probability estimation model for estimating the mask target start probability, which is the probability that the token is the first token in the range of tokens to be masked, and the mask target end probability, which is the probability that the token is the last token in the range of tokens to be masked. A mask target token identification unit identifies the range of mask target tokens based on the mask target start probability and the mask target end probability, The system includes a target text conversion unit that identifies a target string to be masked based on the text information and the identified range of tokens to be masked, and generates converted text information by converting the identified target string to a pre-set dummy string. Information leakage prevention device.

2. A dummy string storage unit stores dummy string information indicating the dummy string in association with the mask target string, A large-scale language model response acquisition unit obtains the response information from the large-scale language model management server by transmitting the converted text information to the large-scale language model management server, which manages a large-scale language model that generates response information indicating the response text including the dummy string based on the converted text information. The dummy string storage unit stores a mask target string corresponding to the dummy string information that indicates the dummy string contained in the answer text, and the answer conversion unit generates converted answer information that indicates the converted answer text in which the dummy string has been converted to the identified mask target string. The information leakage suppression device according to claim 1.

3. The large-scale language model response acquisition unit acquires the response information by sending a query information for specifying the response content to the large-scale language model management server, along with the converted text information, specifying to the large-scale language model management server that the response should be given as is for the dummy string. The information leakage suppression device according to claim 2.

4. The mask target token identification unit changes the range in which it searches for the mask target token range in the token sequence, and identifies the mask target token range if at least one mask target token range exists, and determines that the mask target token range does not exist if no such range exists. An information leakage suppression device according to any one of claims 1 to 3.

5. A teacher information generation unit generates teacher information based on the mask target list information indicating the strings to be masked and the training text information for training the mask target probability estimation model. The system further comprises a model generation unit that generates a new mask target probability estimation model using the aforementioned training information, The teacher information generation unit generates the teacher information while changing the range for selecting candidate token ranges to be included in the teacher information in the learning token sequence corresponding to the learning text information. The information leakage suppression device according to claim 4.

6. The information leakage prevention device performs the steps of converting the text represented by the text information to be processed into a sequence of tokens, The information leakage suppression device uses a mask target probability estimation model to estimate the mask target start probability, which is the probability that the token is the first token in the range of tokens to be masked, and the mask target end probability, which is the probability that the token is the last token in the range of tokens to be masked, to estimate the mask target start probability and the mask target end probability for each of the tokens included in the token sequence. The information leakage suppression device includes the step of identifying the range of tokens to be masked based on the mask target start probability and the mask target end probability, The information leakage suppression device includes the steps of: identifying the string to be masked based on the text information and the identified range of the token to be masked; and generating converted text information by converting the identified string to be masked into a pre-set dummy string. Methods for preventing information leaks.

7. Computers, A tokenizer that converts the text information to be processed into a sequence of tokens. A mask target probability estimation unit estimates the mask target start probability and the mask target end probability for each token in the token sequence using a mask target probability estimation model for estimating the mask target start probability, which is the probability that the token is the first token in the range of tokens to be masked, and the mask target end probability, which is the probability that the token is the last token in the range of tokens to be masked. A mask target token identification unit identifies the range of mask target tokens based on the mask target start probability and the mask target end probability. A target text conversion unit identifies the string to be masked based on the text information and the identified range of tokens to be masked, and generates converted text information by converting the identified string to be masked into a pre-set dummy string. A program designed to function as such.

Images