Information processing apparatus and information processing method

Abstract

This information processing apparatus comprises: a synthesis source generation unit that is configured to train a plurality of synthesis source models using a data set which contains certain data and train a plurality of synthesis source models using a data set which does not contain said certain data; a selection unit that is configured to select a plurality of sets of two synthesis source models from the synthesis source models; a synthesis unit that is configured to synthesize the two synthesis source models of each of the plurality of sets to generate a synthesis model; and a distribution calculation unit that is configured to calculate a first distribution regarding outputs produced upon the input of said certain data into the plurality of synthesis models which contain, as synthesis sources, the synthesis source models having learned said certain data, and a second distribution regarding outputs produced upon the input of said certain data into the plurality of synthesis models which do not contain, as the synthesis sources, the synthesis source models having learned said certain data. This makes it possible to evaluate, from synthesized models, whether certain data is used in training synthesis source models.

Description

Information processing device and information processing method

[0001] The present invention relates to an information processing device and an information processing method.

[0002] The utility of machine learning models, including LLMs (Large Language Models), is increasing. To improve the performance of machine learning models, a technique called model synthesis, which aggregates multiple models into a single model by averaging the model parameters, is attracting attention.

[0003] On the other hand, it is also important to consider privacy, such as whether information about the training data used to train a machine learning model is leaked from the model's output or internal state.

[0004] Membership inference technology, which uses information obtained from machine learning models to infer whether or not information has been used in training data as a privacy assessment, is attracting attention, and LiRA has been noted for its good performance in membership assessment (Non-Patent Document 1).

[0005] Nicholas Carlini et al., "Membership Inference Attacks From First Principles", [online], 2022, Internet <URL: https: / / arxiv.org / abs / 2112.03570>, 2022

[0006] In LiRA, a privacy auditor creates a number of models (hereinafter referred to as "shadow models") similar to a model to be evaluated (hereinafter referred to as "target model").

[0007] At this time, the distribution of output information (IN distribution) when a shadow model (IN model) that has learned the dataset that is to be the subject of privacy evaluation (hereinafter referred to as the "evaluation dataset") inputs each data of the evaluation dataset, and the distribution of output information (OUT distribution) when a shadow model (OUT model) that has not learned the evaluation dataset inputs each data of the evaluation dataset are calculated.

[0008] When each data item in the evaluation dataset is input to the target model, a likelihood ratio test is used to determine whether the output information (e.g., the entropy or gradient of the model output) obtained from the output of the target model is closer to the IN distribution or the OUT distribution. This makes it possible to infer whether the evaluation dataset is included in the training data for the target model or not.

[0009] Although LiRA is a useful membership evaluation method for the training data used in the target model, models created by model synthesis are created using a method different from normal training, making it difficult to evaluate the membership of the data used in the model before synthesis from the model after synthesis.

[0010] The present invention has been made in view of the above points, and has an object to make it possible to evaluate from a synthesized model whether or not certain data was used in learning the original model.

[0011] Therefore, in order to solve the above problem, an information processing device has a synthesis source generation unit configured to train a plurality of synthesis source models using a dataset including certain data and to train a plurality of synthesis source models using a dataset not including the certain data; a selection unit configured to select a plurality of combinations of two synthesis source models from the synthesis source models; a synthesis unit configured to generate a synthesis model by synthesizing, for each of the plurality of combinations, the two synthesis source models related to the combination; and a distribution calculation unit configured to calculate a first distribution regarding outputs when the certain data is input to a plurality of synthesis models that include the synthesis source model that has learned the certain data as a synthesis source, and a second distribution regarding outputs when the certain data is input to a plurality of synthesis models that do not include the synthesis source model that has learned the certain data as a synthesis source.

[0012] It is possible to evaluate from the synthesized model whether or not certain data was used in learning the original model.

[0013] FIG. 1 is a diagram illustrating an example of a hardware configuration of an information processing device 10 according to a first embodiment. FIG. 2 is a diagram illustrating an example of a functional configuration of the information processing device 10 according to the first embodiment. FIG. 3 is a flowchart illustrating an example of a processing procedure for generating a shadow model according to the first embodiment. FIG. 4 is a flowchart illustrating an example of a processing procedure for membership estimation according to the first embodiment. FIG. 5 is a flowchart illustrating an example of a processing procedure for generating a shadow model according to a second embodiment.

[0014]

[0023] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. Fig. 1 is a diagram showing an example of the hardware configuration of an information processing device 10 according to a first embodiment. The information processing device 10 in Fig. 1 includes a drive device 100, an auxiliary storage device 102, a memory device 103, a processor 104, and an interface device 105, all of which are interconnected via a bus B.

[0015] A program that realizes processing in the information processing device 10 is provided by a recording medium 101 such as a CD-ROM. When the recording medium 101 storing the program is set in the drive device 100, the program is installed from the recording medium 101 to the auxiliary storage device 102 via the drive device 100. However, the program does not necessarily have to be installed from the recording medium 101, but may be downloaded from another computer via a network. The auxiliary storage device 102 stores the installed program as well as necessary files, data, etc.

[0016] When an instruction to start a program is received, the memory device 103 reads the program from the auxiliary storage device 102 and stores it. The processor 104 is a CPU or a GPU (Graphics Processing Unit), or a CPU and a GPU, and executes functions related to the information processing device 10 in accordance with the program stored in the memory device 103. The interface device 105 is used as an interface for connecting to a network.

[0017] 2 is a diagram showing an example of the functional configuration of the information processing device 10 according to the first embodiment. As shown in FIG. 2, the information processing device 10 includes a shadow model generation unit 11, a distribution calculation unit 12, and a determination unit 13. Each of these units is realized by a process in which one or more programs installed in the information processing device 10 are executed by a processor 104.

[0018] The shadow model generation unit 11 generates, by model synthesis, multiple models (hereinafter referred to as "shadow models") similar to a machine learning model (hereinafter referred to as a "target model") that is to be the subject of privacy evaluation. At this time, for each piece of data in the evaluation dataset D, the shadow model generation unit 11 generates a shadow model (IN model) that has learned the data and a shadow model (OUT model) that has not learned the data. Note that each piece of data in the evaluation dataset D is a pair of input data for the target model and a correct label for the input data.

[0019] In this embodiment, it is assumed that the target model is also generated by model synthesis. Here, model synthesis refers to separately training multiple models that have the same input / output structure and internal structure and perform the same task, and aggregating the model parameters into one type by averaging the model parameters of each model. Furthermore, the evaluation dataset D is a dataset that belongs to the same distribution as the true training dataset (the training dataset of the target model), and is a collection of multiple evaluation data to be evaluated for use in training the target model. For example, if the target model is a disease prediction model, both the evaluation data and the training data will be data from some patient.

[0020] In FIG. 2, the shadow model generation unit 11 includes a synthesis source generation unit 111 , a selection unit 112 , and a synthesis unit 113 .

[0021] The synthesis source generation unit 111 trains a plurality of synthesis source models using a dataset that includes certain data, and trains a plurality of synthesis source models using a dataset that does not include the certain data. In the first embodiment, for each of a plurality of pieces of data (each piece of data in the evaluation target dataset) that are to be evaluated as to whether or not they were used in training a certain machine learning model (target model), the synthesis source generation unit 111 trains a plurality of the synthesis source models using a dataset that includes the data, and trains a plurality of the synthesis source models using a dataset that does not include the data.

[0022] The selection unit 112 selects a plurality of pairs of two source models from the source models. For each of the plurality of data (each data of the dataset to be evaluated), the selection unit 112 in the first embodiment randomly selects a pair of two source models that have learned the data, a pair of the source model that has learned the data and the source model that has not learned the data, and a pair of two source models that have not learned the data.

[0023] The synthesis unit 113 generates a synthesized model (shadow model) by synthesizing the two synthesis source models related to each of the plurality of pairs.

[0024] The distribution calculation unit 12 calculates, for each data piece in the evaluation dataset D, a distribution of output information (IN distribution) when an IN model group that has learned the data inputs the data, and a distribution of output information (OUT distribution) when an OUT model group that has not learned the data inputs the data. That is, the distribution calculation unit 12 calculates a first distribution (IN distribution) regarding outputs when a plurality of synthesized models that include the source synthesis model that has learned the certain data as their synthesis sources input the certain data, and a second distribution (OUT distribution) regarding outputs when a plurality of synthesized models that do not include the source synthesis model that has learned the certain data input the certain data. More specifically, for each of the plurality of data pieces, the distribution calculation unit 12 calculates the first distribution regarding outputs when a plurality of synthesized models that include the source synthesis model that has learned the certain data as their synthesis sources input the certain data, and the second distribution regarding outputs when a plurality of synthesized models that do not include the source synthesis model that has learned the certain data input the certain data.

[0025] The determination unit 13 performs membership estimation for each data in the evaluation dataset D by determining whether the information (output and gradient information) output by the target model when the target model inputs the data is closer to the IN distribution or the OUT distribution.

[0026] The processing procedure executed by the information processing device 10 will be described below.

[0027] 3 is a flowchart for explaining an example of a processing procedure for generating a shadow model in the first embodiment. In the first embodiment, an IN model and an OUT model are generated sequentially for each data of the evaluation data set D.

[0028] 3, a learning dataset (hereinafter referred to as "dataset D'") having the same configuration as the evaluation dataset D is created in advance. In dataset D' and evaluation dataset D, input data groups to which the same correct answer labels are assigned follow the same distribution (probability distribution).

[0029] The shadow model generating unit 11 executes a loop process L1 including a loop process L2 and steps S103 and S104 for each piece of data belonging to the evaluation data set D. The data being processed in the loop process L1 is referred to as "data x."

[0030] In the loop process L1, the synthesis element generation unit 111 first executes a loop process L2 including steps S101 and S102 for each of a plurality of data sets D'. The data set D' to be processed in the loop process L2 is hereinafter referred to as the "target set D'."

[0031] In step S101, the synthesis source generation unit 111 obtains a synthesis source model X by training an untrained model using a data set X obtained by adding data x to a target set D'. Here, the untrained model refers to a machine learning model that has the same architecture as the target model and has not been trained. Furthermore, training a model refers to updating the parameters of the model so that the output from the model when input data from the training data is input to the model approaches the correct label of the training data.

[0032] Next, the synthesis source generation unit 111 obtains a synthesis source model Y by training an untrained model using a data set Y to which data x is not added to the target set D' (S102).

[0033] Therefore, when loop process L2 ends, a synthesis source model X and a synthesis source model Y are generated for each data set D'. In this way, the number of data sets D' affects the number of synthesis source models X and Y. The number of synthesis source models X and Y affects the number of shadow models obtained by synthesizing the synthesis source models. Therefore, it is sufficient to prepare a number of data sets D' that will provide a sufficient number of shadow models to obtain IN distributions and OUT distributions.

[0034] Following loop processing L2, the selection unit 112 selects multiple pairs of two models from the group of synthesis source models X and the group of synthesis source models Y (S103). Here, if a pair of synthesis source models that are the synthesis sources of a shadow model that is the IN model for data x is defined as an "IN pair," and a pair of synthesis source models that are the synthesis sources of a shadow model that is the OUT model for data x is defined as an "OUT pair," the selection unit 112 selects the IN pair and the OUT pair as follows:

[0035] <Method of Selecting an IN Pair> An IN pair is selected by either of the following methods (1) and (2).

[0036] (1) One source model is randomly selected from each of the source model group X and the source model group Y. Therefore, one source model X and one source model Y are selected.

[0037] (2) Two source models X are randomly selected from the source model X group only.

[0038] <Method of Selecting OUT Pairs> Two source models X are randomly selected from the source model Y group only.

[0039] The selection unit 112 selects a plurality of IN pairs and a plurality of OUT pairs.

[0040] Next, the synthesis unit 113 generates a shadow model for the target model by synthesizing the two synthesis source models belonging to each IN pair and OUT pair selected by the selection unit 112 (S104). At this time, the shadow model generated based on the IN pair becomes the IN model, and the shadow model generated based on the OUT pair becomes the OUT model. Representative methods for synthesizing models include References 1 and 2, but there are many other methods. The performance of membership evaluation can be improved by using a shadow model synthesis method similar to the target model synthesis method.

[0041] 3 ends when loop process L1 is completed for all data belonging to evaluation dataset D. As a result, multiple IN models and multiple OUT models are generated for each data belonging to evaluation dataset D.

[0042] 4 is a flowchart illustrating an example of a membership estimation process according to the first embodiment. In FIG. 4, a loop process L3 is executed for each piece of data in the evaluation data set D. The data being processed in the loop process L1 is referred to as "data x."

[0043] In step S201, the distribution calculation unit 12 inputs data x to all the IN models generated by the shadow model generation unit 11, thereby obtaining outputs from all the IN models.

[0044] Subsequently, the distribution calculation unit 12 inputs the data x to all the OUT models generated by the shadow model generation unit 11, thereby obtaining outputs from all the OUT models.

[0045] Next, the distribution calculation unit 12 calculates an IN distribution, which is a distribution of outputs from the IN model, and an OUT distribution, which is a distribution of outputs from the OUT model (S203). Note that the calculation method of the IN distribution and the OUT distribution may be the same as that of Non-Patent Document 1.

[0046] Next, the determination unit 13 determines whether the output Z from the target model to which the data x is input is closer to the IN distribution or the OUT distribution by a likelihood ratio test (S204). That is, if the output Z is closer to the IN distribution, it is determined that the data x has been used in training the target model, and if the output Z is closer to the OUT distribution, it is determined that the data x has not been used in training the target model. The determination method in step S204 may be the same as that in Non-Patent Document 1.

[0047] Next, the determination unit 13 outputs the determination result (S205).

[0048] When the loop process L3 is completed, the results of membership estimation for all data belonging to the evaluation data set D are obtained.

[0049] As described above, according to the first embodiment, it is possible to evaluate, from the synthesized model, whether certain data was used in training the original model. That is, for each piece of data in the evaluation dataset D, multiple OUT models and multiple IN models are generated, and an IN distribution can be obtained by inputting each piece of data from the evaluation dataset into each IN model, and an OUT distribution can be obtained by inputting each piece of data from the evaluation dataset into each OUT model. Using the IN distribution and OUT distribution obtained in this way, membership estimation can be performed on the output obtained by inputting each piece of data from the evaluation dataset into the target model, thereby evaluating whether each piece of data was used in the original model.

[0050] Next, a second embodiment will be described. In the second embodiment, differences from the first embodiment will be described. Points not specifically mentioned in the second embodiment may be the same as those in the first embodiment.

[0051] The second embodiment discloses a method for collectively generating shadow models (IN model and OUT model) for each data of the evaluation dataset D. According to the second embodiment, it is possible to reduce the time required to generate the shadow models compared to the first embodiment.

[0052] In the second embodiment, the synthesis source generation unit 111 learns the synthesis source model using a portion of data (N pieces of data) randomly selected from a plurality of data (evaluation datasets) to be evaluated for their use in learning a certain machine learning model (target model).

[0053] In the second embodiment, the selection unit 112 randomly selects a plurality of pairs of two source models from among the plurality of source models.

[0054] In the second embodiment, the processing procedure shown in FIG. 5 is executed instead of the processing procedure shown in FIG.

[0055] FIG. 5 is a flowchart illustrating an example of a processing procedure for generating a shadow model according to the second embodiment.

[0056] First, the synthesis element generation unit 111 repeats a loop process L4 including steps S301 and S302 a number of times, as long as a sufficient number of shadow models are obtained.

[0057] In step S301, the synthesis source generation unit 111 randomly selects (samples) N pieces of data d from the evaluation dataset D. Then, the synthesis source generation unit 111 obtains a synthesis source model by training an untrained model using the N pieces of data d (S302). The untrained model may be the same as the untrained model used in steps S101 and S102 of FIG. 3. The N pieces of data d may be a portion of a plurality of pieces of data in the evaluation dataset D.

[0058] When loop process L4 ends, a set of synthesis source models (hereinafter referred to as "set M") equal to the number of loops has been generated. Note that the synthesis source generation unit 111 records information indicating which of the N pieces of data d were used in training each synthesis source model (hereinafter referred to as "synthesis source training information") in the memory device 103, the auxiliary storage device 102, or the like.

[0059] Subsequently, a loop process L5 including steps S303 to S305 is repeated a number of times, as long as a sufficient number of shadow models are obtained.

[0060] In step S303, the selection unit 112 randomly selects two source models from the set M.

[0061] Next, the composition unit 113 generates a shadow model by combining the two selected source models (S304). Representative methods for combining models include those described in References 1 and 2, but there are many other methods.

[0062] Next, the synthesis unit 113 labels the generated shadow model as either an IN model or an OUT model for each piece of data d included in the evaluation data set D (S305). If a certain piece of data d is included in the synthesis source learning information of at least one of the two synthesis source models that are the sources of synthesis of the shadow model, the shadow model is labeled as an IN model for the piece of data d; otherwise, the shadow model is labeled as an OUT model for the piece of data d.

[0063] When the loop process L5 is completed, a number of shadow models have been generated for the number of times the loop process L5 has been executed.

[0064] The membership estimation processing procedure in the second embodiment may be as described in Fig. 4. In this case, in steps S201 and S202, it is possible to identify which of the shadow models generated in step S304 in Fig. 4 is the IN model or OUT model for data x, based on the labels assigned to each shadow model in step S305 in Fig. 5.

[0065] As described above, according to the second embodiment, it is possible to obtain the same effects as those of the first embodiment.

[0066] Furthermore, in the second embodiment, a synthesis source model is generated by randomly sampling from the evaluation dataset D, so that a synthesis source for the IN model and a synthesis source for the OUT model are generated probabilistically for each data d. The selection unit 112 also randomly selects a synthesis source model, unlike the sequential type in the first embodiment. As a result, an IN model and an OUT model are obtained probabilistically for each data d. Since it is not necessary to repeatedly generate an IN model and an OUT model sequentially for each data d, the time required to generate a shadow model can be significantly reduced.

[0067] [References] [Reference 1] Samuel K. Ainsworth, Jonathan Hayase, Siddhartha Srinivasa, "Git Re-Basin: Merging Models modulo Permutation Symmetries", ICLR 2023, https: / / arxiv.org / abs / 2209.04836 [Reference 2] Fidel A. Guerrero Pena, Heitor Rapela Medeiros, Thomas Dubail, Masih Aminbeidokhti, Eric Granger, Marco Pedersoli, "Re-basin via implicit Sinkhorn differentiation", https: / / arxiv.org / abs / 2212.12042 Although the embodiments of the present invention have been described in detail above, the present invention is not limited to such specific embodiments, and various modifications and variations are possible within the scope of the gist of the present invention as defined in the claims.

[0068] REFERENCE SIGNS LIST 10 Information processing device 11 Shadow model generation unit 12 Distribution calculation unit 13 Determination unit 100 Drive device 101 Recording medium 102 Auxiliary storage device 103 Memory device 104 Processor 105 Interface device 111 Synthesis source generation unit 112 Selection unit 113 Synthesis unit B Bus

Claims

1. An information processing device comprising: a synthesis source generation unit configured to train a plurality of synthesis source models using a dataset including certain data, and to train a plurality of synthesis source models using a dataset not including the certain data; a selection unit configured to select a plurality of combinations of two synthesis source models from the synthesis source models; a synthesis unit configured to generate a synthesis model by synthesizing the two synthesis source models for each of the plurality of combinations; and a distribution calculation unit configured to calculate a first distribution regarding outputs when the certain data is input to a plurality of synthesis models that include the synthesis source model that has learned the certain data in their synthesis sources, and a second distribution regarding outputs when the certain data is input to a plurality of synthesis models that do not include the synthesis source model that has learned the certain data in their synthesis sources.

2. The information processing device according to claim 1, characterized in that the synthesis source generation unit is configured to train multiple synthesis source models using a dataset including multiple pieces of data to be evaluated for their use in training a certain machine learning model, and train multiple synthesis source models using a dataset not including the data, for each of the multiple pieces of data, the selection unit is configured to randomly select, for each of the multiple pieces of data, a set of two synthesis source models that have trained the data, a set of the synthesis source model that has trained the data and the synthesis source model that has not trained the data, and a set of two synthesis source models that have not trained the data, and the distribution calculation unit is configured to calculate, for each of the multiple pieces of data, the first distribution regarding outputs when the data is input to multiple synthesis models that include the synthesis source model that has trained the data in their synthesis sources, and the second distribution regarding outputs when the data is input to multiple synthesis models that do not include the synthesis source model that has trained the data in their synthesis sources.

3. The information processing device according to claim 1, characterized in that the synthesis source generation unit is configured to train the synthesis source model using a portion of data for each of a plurality of pieces of data randomly selected from a plurality of pieces of data to be evaluated for use in training a certain machine learning model, the selection unit is configured to randomly select a plurality of pairs of two synthesis source models from the plurality of synthesis source models, and the distribution calculation unit is configured to calculate, for each of the plurality of pieces of data, the first distribution regarding outputs when the data is input to a plurality of synthesis models that include the synthesis source model that learned the data in their synthesis sources, and the second distribution regarding outputs when the data is input to a plurality of synthesis models that do not include the synthesis source model that learned the data in their synthesis sources.

4. An information processing method characterized in that a computer executes the following steps: a synthesis source generation step of training a plurality of synthesis source models using a dataset including certain data, and training a plurality of synthesis source models using a dataset not including the certain data; a selection step of selecting a plurality of combinations of two synthesis source models from the synthesis source models; a synthesis step of generating a synthetic model by synthesizing the two synthesis source models for each of the plurality of combinations; and a distribution calculation step of calculating a first distribution related to the output when the certain data is input to a plurality of synthetic models that include the synthesis source model that has learned the certain data in their synthesis sources, and a second distribution related to the output when the certain data is input to a plurality of synthetic models that do not include the synthesis source model that has learned the certain data in their synthesis sources.

Images