Machine learning device, machine learning method, and machine learning program

Abstract

An inference unit (113) performs, for each model parameter set, inference on input data using a machine learning model to which the model parameter set is set, and obtains output data indicating the result of inference by the machine learning model to which the model parameter set is set. A comparison unit (114) compares features of the inference results between sets of output data to obtain a comparison result. An output unit (115) determines, on the basis of the comparison result, information leakage-related information in the information included in the inference result indicated by one of the sets of output data, applies a change for the information leakage-related information to the output data, and outputs the changed output data.

Description

Machine learning device, machine learning method, and machine learning program

[0001] The present disclosure relates to measures to prevent information leakage from machine learning models.

[0002] Generative AI is expected to be used in a variety of industrial fields. AI is an abbreviation for artificial intelligence. In particular, it is expected to be used in machine learning models with large model parameters, such as large-scale language models.

[0003] Here, we will discuss large-scale language models as a representative example of machine learning models with large model parameters. Note that what is discussed here also applies to machine learning models for other tasks, such as image generation models. Building and training a large-scale language model from scratch requires enormous computational costs and a huge dataset. Therefore, it is considered to perform additional training (fine-tuning) on ​​existing trained large-scale language models using new training data. For example, additional training can be used to augment a pre-trained large-scale language model with a company's unique knowledge.

[0004] Tuning all model parameters (full fine-tuning) for a large-scale language model still requires high computational costs due to the parameter scale of the large-scale language model. Therefore, Parameter Efficient Fine-Tuning (PEFT), a parameter-efficient fine-tuning method, has attracted attention. PEFT tunes only a small number of additional model parameters without updating most of the model parameters of a pre-trained model. This significantly reduces computational costs and storage. PEFT replaces a small number of tuned parameters without replacing the entire model. This also makes it possible to use a single pre-trained model to perform multiple tasks.

[0005] Large-scale language models have enormous model structures. In other words, they have a huge number of model parameters. For this reason, it has been pointed out that large-scale language models tend to store training data internally. It has also been pointed out that they are vulnerable to the leakage of private information related to training data from the model. It has also been pointed out that they are vulnerable to the leakage of sensitive information.

[0006] Patent Literature 1 discloses the configuration of a system for evaluating the risk of information leakage from a learning model. This vulnerability exists even in fine-tuned large-scale language models.

[0007] Japanese Patent Application Laid-Open No. 2022-007311

[0008] Fine-tuned machine learning models (large-scale language models) have the problem of information leakage.

[0009] The present disclosure aims to suppress information leakage from machine learning models.

[0010] The machine learning device of the present disclosure includes an inference unit that, for each model parameter set included in a plurality of mutually different model parameter sets, performs inference on input data using a machine learning model to which the model parameter set is set, and obtains output data indicating the inference result by the machine learning model when the model parameter set is set; a comparison unit that compares features of the inference results between multiple output data corresponding to the plurality of model parameter sets and obtains a comparison result; and an output unit that determines, based on the comparison result, information regarding information leakage from the information included in the inference result indicated in any of the output data, makes changes to the output data regarding the information leakage, and outputs the changed output data.

[0011] According to the present disclosure, it is possible to suppress information leakage from machine learning models.

[0012] FIG. 1 is a configuration diagram of a machine learning device 100 according to a first embodiment. FIG. 2 is a flowchart of a machine learning method according to the first embodiment. FIG. 3 is a flowchart of an example according to the first embodiment. FIG. 4 is a configuration diagram of a machine learning device 100 according to a second embodiment. FIG. 5 is a flowchart of a machine learning method according to the second embodiment. FIG. 6 is a flowchart of an example according to the second embodiment. FIG. 7 is a hardware configuration diagram of a machine learning device 100 according to an embodiment.

[0013] In the embodiments and drawings, the same or corresponding elements are denoted by the same reference numerals. The description of elements denoted by the same reference numerals as those already described will be omitted or simplified as appropriate. Arrows in the drawings primarily indicate the flow of data or the flow of processing.

[0014] First Embodiment A countermeasure against information leakage from a machine learning model will be described with reference to FIGS.

[0015] ***Description of Configuration*** The configuration of the machine learning device 100 will be described with reference to Figure 1. The machine learning device 100 is a computer equipped with hardware such as a processor 101, a memory 102, an auxiliary storage device 103, and an input / output interface 104. These pieces of hardware are connected to one another via signal lines.

[0016] The processor 101 is an IC that performs arithmetic processing and controls other hardware. For example, the processor 101 is a CPU, a DSP, or a GPU. IC is an abbreviation for Integrated Circuit. CPU is an abbreviation for Central Processing Unit. DSP is an abbreviation for Digital Signal Processor. GPU is an abbreviation for Graphics Processing Unit.

[0017] The memory 102 is a volatile or non-volatile storage device. The memory 102 is also called a primary storage device or a main memory. For example, the memory 102 is a RAM. Data stored in the memory 102 is saved in the secondary storage device 103 as needed. RAM is an abbreviation for Random Access Memory.

[0018] The auxiliary storage device 103 is a non-volatile storage device. For example, the auxiliary storage device 103 is a ROM, a HDD, a flash memory, or a combination of these. Data stored in the auxiliary storage device 103 is loaded into the memory 102 as needed. ROM is an abbreviation for Read Only Memory. HDD is an abbreviation for Hard Disk Drive.

[0019] The input / output interface 104 is a port to which an input device and an output device are connected. For example, the input / output interface 104 is a USB terminal, and the input devices are a keyboard, a mouse, and a communication device (receiver), and the output device is a display and a communication device (transmitter). Input and output to and from the machine learning device 100 are performed using the input / output interface 104. USB is an abbreviation for Universal Serial Bus.

[0020] The machine learning device 100 includes elements such as an input unit 111, a switching unit 112, an inference unit 113, a comparison unit 114, and an output unit 115. These elements are realized by software.

[0021] The auxiliary storage device 103 stores a machine learning program for causing the computer to function as an input unit 111, a switching unit 112, an inference unit 113, a comparison unit 114, and an output unit 115. The machine learning program is loaded into the memory 102 and executed by the processor 101. The auxiliary storage device 103 also stores an OS. At least a portion of the OS is loaded into the memory 102 and executed by the processor 101. The processor 101 executes the machine learning program while running the OS. OS is an abbreviation for Operating System.

[0022] Input and output data of the machine learning program is stored in the storage unit 120. The memory 102 functions as the storage unit 120. However, a storage device such as the auxiliary storage device 103, a register in the processor 101, or a cache memory in the processor 101 may function as the storage unit 120 instead of or together with the memory 102.

[0023] The machine learning program can be recorded (stored) in a computer-readable manner on a non-volatile recording medium such as an optical disk or flash memory.

[0024] ***Description of Operation*** The operational procedure of the machine learning device 100 corresponds to a machine learning method. Also, the operational procedure of the machine learning device 100 corresponds to a processing procedure by a machine learning program.

[0025] The machine learning method will be described with reference to FIG. 2. In step S101, the input unit 111 acquires input data. The input data is data that serves as input to the machine learning model. For example, the input data is stored in advance in the storage unit 190. The machine learning model is a model (trained model) generated by machine learning. The machine learning model is stored in advance in the storage unit 190. An example of the machine learning model is a large-scale language model. When the machine learning model is a large-scale language model, text data serves as input data.

[0026] Steps S102 to S104 are repeatedly executed. Steps S102 to S104 are executed by the switching unit 112 and the inference unit 113. The switching unit 112 switches between model parameter sets to be set in the machine learning model using multiple model parameter sets. The model parameter set is one or more parameters (model parameters) used in the machine learning model. For example, the switching unit 112 generates a new model parameter set by adding a new model parameter to the current model parameter set of the machine learning model. The switching unit 112 then uses the current model parameter set and the new model parameter set as the multiple parameter sets. For example, the switching unit 112 generates a new model parameter set by adding a new model parameter to the current model parameter set of the machine learning model for each new model parameter included in the two or more new model parameters. The switching unit 112 then uses two or more new model parameter sets corresponding to the two or more new model parameters as the multiple parameter sets. For each model parameter set, the inference unit 113 performs inference on input data using the machine learning model in which the model parameter set is set. As a result, output data is obtained. The output data indicates the inference results of the machine learning model when the model parameters are set.

[0027] The procedure from step S102 to step S104 will be described. In step S102, the switching unit 112 sets a model parameter set to the machine learning model. Each time step S102 is executed, a part of the model parameter set is different.

[0028] In step S103, the inference unit 113 performs inference on the input data using a machine learning model. This results in output data. If the machine learning model is a large-scale language model, text data becomes the output data. In this case, the output text (output data) indicates a token string corresponding to the input text (input data). The token string consists of one or more tokens. A token corresponds to, for example, a word.

[0029] In step S104, the switching unit 112 determines whether to switch the model parameter set set in the machine learning model. For example, if the number of times the model parameter set has been switched has not reached a predetermined number, the switching unit 112 determines to switch the model parameter set. If the model parameter set set in the machine learning model is to be switched, the process proceeds to step S102. If the model parameter set set in the machine learning model is not to be switched, the process proceeds to step S105.

[0030] Steps S102 to S104 provide a plurality of output data corresponding to a plurality of model parameter sets.

[0031] In step S105, the comparison unit 114 compares the features of the inference results between the multiple pieces of output data. This results in a comparison. Examples of the features to be compared include the probability transition, probability distribution, or cosine similarity of the inference results. These are features related to information leakage. When the machine learning model is a large-scale language model, for example, the probability transitions between tokens in the token sequences shown in the output texts are compared between the output texts. Then, abnormal probability transitions are detected. For example, an abnormal probability transition is a state in which the transition is unlikely to occur under normal operation. An abnormal probability transition can be detected by evaluating whether the magnitude (probability) of the transition is lower than a predetermined threshold.

[0032] In step S106, the output unit 115 selects one of the plurality of output data. For example, if the plurality of output data is two pieces of output data corresponding to the current model parameter set and a new model parameter set, the output unit 115 selects the output data corresponding to the new model parameter set.

[0033] Next, the output unit 115 determines, based on the comparison result, information related to information leakage from among the information included in the inference result shown in the selected output data. For example, the output unit 115 determines, as information related to information leakage, a portion of the token sequence shown in the output text in which an abnormal probability transition is detected.

[0034] Next, the output unit 115 applies the changes to the information related to the information leakage to the selected output data.

[0035] For example, the output unit 115 modifies the output data in one of the following ways: The output unit 115 eliminates (deletes) information about information leakage. The output unit 115 replaces information about information leakage with information whose features are suppressed. The output unit 115 integrates (adds) information about information leakage of other output data into information about information leakage of the selected output data.

[0036] When the machine learning model is a large-scale language model, for example, a token in a portion where an abnormal probability transition is detected is replaced with a similar token that suppresses the characteristic. The token in a portion where an abnormal probability transition is detected is sensitive information in additional learning (additional parameters). For example, if the sensitive information is an individual's name, it is replaced with a similar token such as a fictitious name that does not actually exist. In this way, the characteristics of the sensitive information can be removed from the output data, thereby preventing information leakage from the machine learning model.

[0037] The output unit 115 then outputs the output data after the change has been made. For example, the output unit 115 stores the output data after the change in the storage unit 190.

[0038] ***Description of the Example*** An example of comparing output features with and without PEFT parameters set for a large-scale language model will be described with reference to FIG. 3 . The large-scale language model is a pre-trained machine learning model. In PEFT, the parameters of the pre-trained machine learning model themselves are not updated. Therefore, the model parameters of the large-scale language model can be switched depending on whether or not parameters are added. In step S101, the input unit 111 acquires input text 121. In step S102, the switching unit 112 adds PEFT parameters and sets a model parameter set to the large-scale language model. The large-scale language model with the PEFT parameters added is referred to as the large-scale language model 122. In step S103, the inference unit 113 performs inference on the input text 121 using the large-scale language model 122. As a result, output text is obtained. In step S102, the switching unit 112 sets a model parameter set to the large-scale language model without adding PEFT parameters. The large-scale language model without adding PEFT parameters is referred to as the large-scale language model 123. In step S103, the inference unit 113 performs inference on the input text 121 using the large-scale language model 123. This results in an output text. In step S105, the comparison unit 114 compares the predicted probabilities of word strings (token strings) between the two output texts. In step S106, if the transitions in the predicted probabilities of the word strings differ between the output texts, the output unit 115 replaces the target token in the output text of the large-scale language model 122 with a similar token to reduce the difference in the transitions in the predicted probabilities. The output text of the large-scale language model 122 after the replacement is referred to as output text 124. The output unit 115 outputs the output text 124.

[0039] ***Effects of First Embodiment*** The first embodiment aims to make it easier to apply information leakage countermeasures for fine-tuned machine learning models (large-scale language models) to existing fine-tuning methods without degrading model performance. The first embodiment can realize a machine learning device 100, a machine learning method, and a machine learning program that prevent information leakage about training data when inferring certain input data. The first embodiment compares output data when switching additional parameters for a fine-tuned (PEFT) machine learning model, eliminating or suppressing features related to information leakage. The main effect of the first embodiment is to prevent information leakage from machine learning models. The first embodiment is configured by switching additional parameters and comparing inference results. Therefore, the first embodiment can be introduced without manipulating model learning, and the first embodiment causes less degradation in model performance compared to existing methods. Adding the configuration of parameter switching and output comparison can be easily applied to existing fine-tuning methods.

[0040] Second Embodiment A second embodiment in which a machine learning model is retrained using output data obtained in the first embodiment will be described below, focusing mainly on the differences from the first embodiment, with reference to FIGS.

[0041] ***Description of Configuration*** The configuration of the machine learning device 100 will be described with reference to Fig. 4. The machine learning device 100 further includes a relearning unit 116. The machine learning program further causes a computer to function as the relearning unit 116.

[0042] ***Description of Operation*** The machine learning method will be described with reference to Fig. 5. Steps S101 to S106 are the same as those described in the first embodiment.

[0043] In step S107, the re-learning unit 116 uses the input data obtained in step S101 as learning data and the modified output data obtained in step S106 as correct answer data to re-learn the machine learning model.

[0044] For example, the retraining unit 116 performs fine tuning by using the training data set (training data and correct answer data) to update additional parameters (parameters that are part of the machine learning model).

[0045] ***Description of Example*** An example of comparing output features when different PEFT parameter settings are made for a large-scale language model will be described with reference to FIG. 6 . In step S101, the input unit 111 acquires input text 125. The input text 125 is text data collected for training. In step S102, the switching unit 112 adds a PEFT parameter (1) to set a model parameter set to the large-scale language model. The large-scale language model to which the PEFT parameter (1) has been added is referred to as the large-scale language model 126. In step S103, the inference unit 113 performs inference on the input text 125 using the large-scale language model 126. As a result, output text is obtained. In step S102, the switching unit 112 adds a PEFT parameter (2) to set a model parameter set to the large-scale language model. The large-scale language model to which the PEFT parameter (2) has been added is referred to as the large-scale language model 127. In step S103, the inference unit 113 performs inference on the input text 125 using the large-scale language model 127. This results in an output text. In step S105, the comparison unit 114 compares the predicted probabilities of word strings (token strings) between the two output texts. In step S106, the output unit 115 modifies the target tokens of the output text of the large-scale language model 126 or the large-scale language model 127 if the transitions in the predicted probabilities of the word strings differ between the output texts. For example, the output unit 115 integrates the target tokens of the output text of the large-scale language model 127 with the target tokens of the output text of the large-scale language model 126. Alternatively, the output unit 115 replaces the target tokens of the output text of the large-scale language model 126 with the target tokens of the output text of the large-scale language model 127. The modified output text is referred to as output text 128. In step S107, the retraining unit 116 uses the input text 125 as training data and the output text 128 as correct answer data to create a training dataset (training data and correct answer data). Then, the retraining unit 116 retrains the large-scale language model by updating the PEFT parameters (1) and PEFT (2) using the training data set.

[0046] ***Effects of Second Embodiment*** The second embodiment aims to generate a machine learning model with high output data accuracy and no information leakage. The second embodiment has a configuration for switching multiple additional parameters and a configuration for retraining the machine learning model using output data with compared features. The second embodiment can realize a machine learning device 100, a machine learning method, and a machine learning program for training a machine learning model that does not leak information about training data when inferring certain input data. The second embodiment compares output data when switching additional parameters for a fine-tuned (PEFT) machine learning model, and uses data in which features related to information leakage have been eliminated, suppressed, or integrated as new ground truth data. The main effect of the second embodiment is to generate a machine learning model that does not leak information by retraining a fine-tuned (PEFT) machine learning model. The second embodiment is configured by adding a configuration for parameter switching and output comparison. Therefore, the second embodiment can be easily applied to existing fine-tuning methods. Furthermore, the second embodiment generates a machine learning model that does not leak information. Therefore, when applying a machine learning model to a product, it is not necessary to implement additional functions to prevent information leakage.

[0047] *** Supplementary Notes on the Embodiments *** The type of machine learning operation is not limited to deep learning, but may be an operation such as regression, decision tree learning, Bayesian method, or clustering.

[0048] The hardware configuration of the machine learning device 100 will be described with reference to Fig. 7. The machine learning device 100 includes a processing circuit 109. The processing circuit 109 is hardware that implements an input unit 111, a switching unit 112, an inference unit 113, a comparison unit 114, an output unit 115, and a relearning unit 116. The processing circuit 109 may be dedicated hardware, or may be a processor 101 that executes a program stored in a memory 102.

[0049] When the processing circuit 109 is dedicated hardware, the processing circuit 109 may be, for example, a single circuit, a multiple circuit, a programmed processor, a parallel programmed processor, an ASIC, an FPGA, or a combination thereof. ASIC is an abbreviation for Application Specific Integrated Circuit. FPGA is an abbreviation for Field Programmable Gate Array.

[0050] The machine learning device 100 may include multiple processing circuits that replace the processing circuit 109.

[0051] In the processing circuit 109, some functions may be realized by dedicated hardware, and the remaining functions may be realized by software or firmware.

[0052] In this way, the functions of the machine learning device 100 can be realized by hardware, software, firmware, or a combination of these.

[0053] Each embodiment is an example of a preferred embodiment and is not intended to limit the technical scope of the present disclosure. Each embodiment may be implemented in part or in combination with other embodiments. Procedures described using flowcharts, etc. may be modified as appropriate.

[0054] The "part" of each element of the machine learning device 100 may be read as a "process," a "step," a "circuit," or a "circuitry."

[0055] 100 Machine learning device, 101 Processor, 102 Memory, 103 Auxiliary storage device, 104 Input / output interface, 109 Processing circuit, 111 Input unit, 112 Switching unit, 113 Inference unit, 114 Comparison unit, 115 Output unit, 116 Re-learning unit, 121 Input text, 122 Large-scale language model, 123 Large-scale language model, 124 Output text, 125 Input text, 126 Large-scale language model, 127 Large-scale language model, 128 Output text, 190 Storage unit.

Claims

1. For each model parameter set included in a plurality of mutually different model parameter sets, perform an inference on input data using a machine learning model set with the model parameter set, and obtain output data indicating an inference result by the machine learning model when the model parameter set is set; a comparison unit that compares the characteristics of the inference results between the output data for the plurality of output data corresponding to the plurality of model parameter sets and obtains a comparison result; and an output unit that determines information related to information leakage among the information included in the inference result indicated by any of the output data based on the comparison result, makes a change to the output data for the information related to information leakage, and outputs the changed output data. A machine learning device comprising:

2. The machine learning device according to claim 1, wherein the comparison unit compares the probability transition of the inference results between the output data as the characteristics.

3. The machine learning device according to claim 1, wherein the comparison unit compares the probability distribution of the inference results between the output data as the characteristics.

4. The machine learning device according to claim 1, wherein the comparison unit compares the cosine similarity of the inference results between the output data as the characteristics.

5. The machine learning device according to any one of claims 1 to 4, wherein the output unit performs any one of exclusion, suppression, and integration as a change to the information related to information leakage on the output data.

6. The machine learning device according to any one of claims 1 to 5, further comprising a switching unit that adds new model parameters to the current model parameter set of the machine learning model to generate a new model parameter set, and switches the model parameter set set in the machine learning model using the current model parameter set and the new model parameter set as the plurality of model parameter sets.

7. A machine learning apparatus according to any one of claims 1 to 5, comprising a switching unit that generates a new model parameter set by adding the new model parameter to the current model parameter set of the machine learning model for each new model parameter included in two or more new model parameters, and switches the model parameter set set in the machine learning model using two or more new model parameter sets corresponding to the two or more new model parameters as the plurality of model parameter sets.

8. A machine learning apparatus according to any one of claims 1 to 7, comprising a relearning unit that executes relearning of the machine learning model using the input data as learning data and using the output data after the change as correct data.

9. For each model parameter set included in a plurality of different model parameter sets, an inference is performed on input data using a machine learning model in which the model parameter set is set, and output data indicating an inference result by the machine learning model when the model parameter set is set is obtained. For a plurality of output data corresponding to the plurality of model parameter sets, the characteristics of the inference results between the output data are compared to obtain a comparison result. Information regarding information leakage among the information included in the inference result indicated by any of the output data is determined based on the comparison result, and the output data is modified with respect to the information regarding the information leakage, and the modified output data is output. Machine learning method.

10. For each model parameter set included in a plurality of mutually different model parameter sets, perform an inference on input data using a machine learning model in which the model parameter set is set, and obtain output data indicating an inference result by the machine learning model when the model parameter set is set; an inference process; a comparison process of comparing features of the inference results between the output data for the plurality of output data corresponding to the plurality of model parameter sets and obtaining a comparison result; and determining information related to information leakage among the information included in the inference result indicated in any of the output data based on the comparison result, and applying a change to the output data for the information related to the information leakage, and outputting the changed output data; A machine learning program for causing a computer to execute output processing.

Images