Verification device, verfication method and program

Abstract

A verification device according to one embodiment of the present disclosure verifies identity between a first machine learning model and a second machine learning model, both models configured so that output changes stochastically with respect to input. The verification device comprises: an input unit that inputs the same input a predetermined number of times to the first machine learning model and the second machine learning model; and a verification unit that verifies identity between the first machine learning model and the second machine learning model, on the basis of a first model characteristic that represents the distribution of the outputs of the first machine learning model when the input is input the predetermined number of times, and a second model characteristic that represents the distribution of the outputs of the second machine learning model when the input is input the prescribed number of times.

Description

Verification device, verification method, and program

[0001] The present disclosure relates to a verification device, a verification method, and a program.

[0002] Among the machine learning models that realize AI (Artificial Intelligence), some continue to change through continuous machine learning in order to improve their sophistication and adapt to external environments. Such machine learning models have the problem that it is difficult to detect changes due to erroneous learning or unauthorized substitution of machine learning models by third parties. In response to this problem, a method has been proposed for verifying the identity of multiple machine learning models by utilizing the characteristics of the machine learning models (Non-Patent Document 1).

[0003] Naoto Kiribuchi, Yuya Sato, Ryohei Suzuki, Nami Ashizawa, Satoshi Oki, Hirofumi Mineno, Masakatsu Nishigaki, "A Study on the Use of Verifiable Machine Learning Models: Proposal of a Model Identity Verification Method Based on Robustness to Image Manipulation," Research Report Computer Security (CSEC), vol.2023-CSEC-101, no.16, pp.1-7, 2023.

[0004] However, the method proposed in Non-Patent Document 1 sometimes fails to verify the identity of machine learning models whose outputs are not deterministically determined for the same input.

[0005] The present disclosure has been made in consideration of the above points, and aims to verify the identity of a machine learning model whose output is not deterministically determined for the same input.

[0006] A verification device according to one aspect of the present disclosure is a verification device that verifies the identity of a first machine learning model and a second machine learning model whose output changes probabilistically in response to an input, and includes an input unit that inputs the same input to each of the first machine learning model and the second machine learning model a predetermined number of times, and a verification unit that verifies the identity of the first machine learning model and the second machine learning model based on a first model characteristic that represents the distribution of the output of the first machine learning model when the input is inputted the number of times, and a second model characteristic that represents the distribution of the output of the second machine learning model when the input is inputted the number of times.

[0007] It is possible to verify the identity of machine learning models whose output is not deterministically determined for the same input.

[0008] It is a diagram illustrating an example of a hardware configuration of a verification device according to the present embodiment. It is a diagram illustrating an example of a functional configuration of a verification device according to the present embodiment. It is a flowchart illustrating an example of a verification process according to the present embodiment.

[0009] An embodiment of the present invention will be described in detail below with reference to the drawings. In the following embodiment, a verification device 10 will be described that can verify the identity of two machine learning models whose output is not deterministically determined for each input. Hereinafter, the two machine learning models to be verified for identity will be referred to as "machine learning model T" and "machine learning model V." Machine learning model T is assumed to be, for example, a legitimate machine learning model that operates as intended by the model creator or the like. On the other hand, machine learning model V is a machine learning model whose identity with machine learning model T is to be verified, and is assumed to be, for example, a machine learning model that may have changed due to erroneous learning or may have been illegally replaced by a third party.

[0010] An example of a machine learning model in which the output is not deterministically determined for the same input is a machine learning model called a large language model (LLM) that realizes generative AI (or may be called generative AI). Such machine learning models do not necessarily obtain the same output for the same input, but rather the output changes probabilistically for the same input. However, a machine learning model that realizes generative AI is just one example, and the following embodiment can be similarly applied to verifying the identity of any two machine learning models in which the output is not deterministically determined for the input (in other words, any two machine learning models in which the output changes probabilistically for the input).

[0011] <Example of Hardware Configuration of Verification Device 10> An example of the hardware configuration of the verification device 10 according to this embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of the hardware configuration of the verification device 10 according to this embodiment.

[0012] 1, the verification device 10 according to this embodiment includes an input device 101, a display device 102, an external I / F 103, a communication I / F 104, a RAM (Random Access Memory) 105, a ROM (Read Only Memory) 106, an auxiliary storage device 107, and a processor 108. Each of these pieces of hardware is connected to each other via a bus 109 so as to be able to communicate with each other.

[0013] The input device 101 is, for example, a keyboard, a mouse, a touch panel, a physical button, etc. The display device 102 is, for example, a display, a display panel, etc. Note that the verification device 10 does not necessarily have to include at least one of the input device 101 and the display device 102, for example.

[0014] The external I / F 103 is an interface with an external device such as a recording medium 103a. Examples of the recording medium 103a include a CD (Compact Disc), a DVD (Digital Versatile Disk), an SD memory card (Secure Digital memory card), and a USB (Universal Serial Bus) memory card.

[0015] The communication I / F 104 is an interface for connecting to a communication network. The RAM 105 is a volatile semiconductor memory (storage device) that temporarily stores programs and data. The ROM 106 is a non-volatile semiconductor memory (storage device) that can store programs and data even when the power is turned off. The auxiliary storage device 107 is a non-volatile storage device such as a hard disk drive (HDD), a solid state drive (SSD), or a flash memory. The processor 108 is a variety of arithmetic devices such as a central processing unit (CPU) or a graphic processing unit (GPU).

[0016] 1 is an example, and the hardware configuration of the verification device 10 is not limited to this. For example, the verification device 10 may have multiple auxiliary storage devices 107 or multiple processors 108, may not have some of the hardware shown in the figure, or may have various hardware other than the hardware shown in the figure.

[0017] <Example of Functional Configuration of Verification Apparatus 10> An example of the functional configuration of the verification apparatus 10 according to this embodiment will be described with reference to Fig. 2. Fig. 2 is a diagram showing an example of the functional configuration of the verification apparatus 10 according to this embodiment.

[0018] 2, the verification device 10 according to this embodiment includes a model input unit 201, a model property acquisition unit 202, a verification unit 203, and a verification result output unit 204. Each of these units is implemented, for example, by a process in which one or more programs installed in the verification device 10 are executed by the processor 108 or the like. The verification device 10 according to this embodiment also includes an input storage unit 205 and a model property storage unit 206. Each of these storage units is implemented, for example, by a storage area of ​​the auxiliary storage device 107 or the like. Note that at least one of the input storage unit 205 and the model property storage unit 206 may be implemented by a storage area of ​​a storage device (e.g., a storage device included in a database server) or the like that is communicatively connected to the verification device 10.

[0019] The model input unit 201 inputs the input x stored in the input storage unit 205 to the machine learning model T k times. As a result, k outputs y Ｔ，１ , ..., y Ｔ，ｋ Here, k is a predetermined integer of 2 or more. Ｔ，ｉ is the output when the input x is input to the machine learning model T for the i-th time (where 1≦i≦k). In general, each time the i-th input x is input to the machine learning model T, the i-th output y Ｔ，ｉ is obtained.

[0020] Furthermore, the model input unit 201 inputs the input x stored in the input storage unit 205 to the machine learning model V k times. As a result, k outputs y Ｖ，１ , ..., y Ｖ，ｋ where y Ｖ，ｉ is the output when the input x is input to the machine learning model V for the i-th time (where 1≦i≦k). In general, each time the i-th input x is input to the machine learning model V, the i-th output y Ｖ，ｉ is obtained.

[0021] The format of the input x may differ depending on the machine learning models T and V. For example, when the machine learning models T and V are machine learning models such as large-scale language models that realize generative AI, the input x is generally in text format. Similarly, the output yＴ，ｉ and y Ｖ，ｉ The format of the output y may also differ depending on the machine learning models T and V. For example, when the machine learning models T and V are machine learning models such as large-scale language models that realize generative AI, the output y Ｔ，ｉ and y Ｖ，ｉ However, it is also possible to input and output data in various formats, such as still images, video, and audio.

[0022] The machine learning models T and V are stored in a storage area such as the auxiliary storage device 107. However, at least one of the machine learning models T and V may be stored in a storage area such as a storage device communicatively connected to the verification device 10 (e.g., a storage device provided in an API (Application Programming Interface) server).

[0023] The model characteristic acquisition unit 202 acquires k outputs y from the machine learning model T. Ｔ，１ , ..., y Ｔ，ｋ Let Y be the model characteristic of machine learning model T. Ｔ = (y Ｔ，１ , ..., y Ｔ，ｋ Similarly, the model characteristic acquisition unit 202 acquires k outputs y Ｖ，１ , ..., y Ｖ，ｋ Let V be the model characteristic Y of the machine learning model V. Ｖ = (y Ｖ，１ , ..., y Ｖ，ｋ ) is obtained.

[0024] Furthermore, the model characteristic acquisition unit 202 acquires the model characteristic Y Ｔ and Y Ｖ At least one of the above is stored in the model characteristic storage unit 206.

[0025] The verification unit 203 determines the model characteristic Y Ｔ and Y Ｖ The verification unit 203 verifies whether the machine learning model V can be considered to be the same as the machine learning model T by using the model characteristic Y Ｔ and Y Ｖ The identity of the machine learning model T and the machine learning model V is verified using

[0026] The verification result output unit 204 outputs the verification result by the verification unit 203 to a predetermined output destination. The predetermined output destination may be, for example, the display device 102 such as a display, a storage area such as the auxiliary storage device 107, or another device or equipment connected to the verification device 10 so as to be able to communicate with the verification device 10.

[0027] The input storage unit 205 stores an input x that is given as an input to the machine learning models T and V.

[0028] The model characteristic storage unit 206 stores the model characteristic Y Ｔ and Y Ｖ Remember.

[0029] <Verification Process> The verification process according to this embodiment will be described with reference to Fig. 3. Fig. 3 is a flowchart showing an example of the verification process according to this embodiment.

[0030] The model input unit 201 inputs the input x stored in the input storage unit 205 to the machine learning model T k times (step S101). As a result, k outputs y Ｔ，１ , ..., y Ｔ，ｋ is obtained.

[0031] The model characteristic acquisition unit 202 calculates the k outputs y obtained in step S101. Ｔ，１ , ..., y Ｔ，ｋ The model characteristic Y Ｔ = (y Ｔ，１ , ..., y Ｔ，ｋ ) (step S102). Ｔ is stored in the model characteristic storage unit 206 as needed. Ｔ is stored in the model characteristic storage unit 206, for example, Ｔ For example, after obtaining the ID, the identity verification in step S105 described later is performed later.

[0032] The model input unit 201 inputs the input x stored in the input storage unit 205 to the machine learning model V k times (step S103). As a result, k outputs y Ｖ，１ , ..., y Ｖ，ｋ is obtained.

[0033] The model characteristic acquisition unit 202 calculates the k outputs y obtained in step S103. Ｖ，１ , ..., y Ｖ，ｋ The model characteristic Y Ｖ (Step S104). Ｖ The model characteristic Y is stored in the model characteristic storage unit 206 as needed. Ｖ is stored in the model characteristic storage unit 206, for example, Ｖ For example, after obtaining the ID, the identity verification in step S105 described later is performed later.

[0034] The order of steps S101 to S102 and steps S103 to S104 is not particularly limited. That is, for example, steps S101 to S102 may be executed after steps S103 to S104.

[0035] The verification unit 203 determines the model characteristic Y Ｔ and Y Ｖ (Step S105). Here, the verification unit 203 can verify the identity of the machine learning model T and the machine learning model V using, for example, distribution distance or statistical hypothesis testing.

[0036] ・When using the distance between distributions, model characteristic Y Ｔ = (y Ｔ，１ , ..., y Ｔ，ｋ ) is k points y Ｔ，１ , ..., y Ｔ，ｋ Similarly, the model characteristic Y Ｖ = (y Ｖ，１ , ..., y Ｖ，ｋ ) also has k points y Ｖ，１ , ..., y Ｖ，ｋTherefore, the model characteristic Y Ｔ = (y Ｔ，１ , ..., y Ｔ，ｋ ) and model characteristic Y Ｖ = (y Ｖ，１ , ..., y Ｖ，ｋ ) and if the value is less than a predetermined threshold, the verification result is that "machine learning model V can be considered to be identical to machine learning model T." If not, the verification result is that "machine learning model V cannot be considered to be identical to machine learning model T." Ｔ and model characteristic Y Ｖ As a measure for measuring the distance between the distributions, for example, KL (Kullback-Leibler) divergence can be used.

[0037] ・When using statistical hypothesis testing, model characteristics Y Ｔ = (y Ｔ，１ , ..., y Ｔ，ｋ ) k outputs y Ｔ，１ , ..., y Ｔ，ｋ can be considered as the distribution of a sample statistically extracted from a certain population. Similarly, the model characteristic Y Ｖ = (y Ｖ，１ , ..., y Ｖ，ｋ ) k outputs y Ｖ，１ , ..., y Ｖ，ｋcan also be statistically considered to be the distribution of a sample extracted from a certain population. Therefore, whether the population probability distributions differ is tested by statistical hypothesis testing of two samples. Specifically, the null hypothesis is that "the population probability distributions are identical," and the null hypothesis is rejected if the p-value is smaller than a predetermined significance level. If the null hypothesis is rejected, the test result indicates that the alternative hypothesis "the population probability distributions do not match," is correct, and the verification result is that "machine learning model V cannot be considered identical to machine learning model T." On the other hand, if the null hypothesis is not rejected, the alternative hypothesis is not necessarily correct, but the alternative hypothesis can be considered correct, and the verification result of the identity verification is that "machine learning model V can be considered identical to machine learning model T." Note that, for example, the Kolmogorov-Smirnov test or the Mann-Whitney U test can be used as a statistical hypothesis test for two samples.

[0038] The verification result output unit 204 outputs the verification result obtained in step S105 to a predetermined output destination (step S106).

[0039] <Modification> In the above embodiment, the input x is input k times to the machine learning model T in step S101 of Fig. 3 , and the input x is input k times to the machine learning model V in step S103, but the number of inputs may differ between the machine learning model T and the machine learning model V. That is, for example, the input x may be input k times to the machine learning model T in step S101 of Fig. 3 , and the input x may be input k' (where k' ≠ k) times to the machine learning model V in step S103. In this case, in step S104 of Fig. 3 , the model characteristic Y Ｖ = (y Ｖ，１ , ..., y Ｖ，ｋ' ) is obtained. Even if k'≠k, in step S105 of FIG. 3, identity can be verified using a statistical hypothesis test, just as in the case where k'=k.

[0040] <Summary> As described above, the verification device 10 according to this embodiment verifies the identity of two machine learning models T and V by providing the same input x k times to each of the two machine learning models T and V and comparing the distributions of their outputs. This makes it possible for the verification device 10 according to this embodiment to verify the identity of the machine learning models T and V even when the outputs of the machine learning models T and V change probabilistically with respect to the input. This makes it possible to detect machine learning models that may have changed due to erroneous learning or may have been illicitly replaced by a third party, for example, targeting machine learning models that realize generative AI.

[0041] The present invention is not limited to the above-described specifically disclosed embodiments, and various modifications, changes, and combinations with known technologies are possible without departing from the scope of the claims.

[0042] REFERENCE SIGNS LIST 10 Verification device 101 Input device 102 Display device 103 External I / F 103a Recording medium 104 Communication I / F 105 RAM 106 ROM 107 Auxiliary storage device 108 Processor 109 Bus 201 Model input unit 202 Model characteristic acquisition unit 203 Verification unit 204 Verification result output unit 205 Input storage unit 206 Model characteristic storage unit

Claims

1. A verification device that verifies the identity of a first machine learning model and a second machine learning model whose output changes probabilistically in response to an input, comprising: an input unit that inputs the same input to each of the first machine learning model and the second machine learning model a predetermined number of times; and a verification unit that verifies the identity of the first machine learning model and the second machine learning model based on a first model characteristic that represents the distribution of the output of the first machine learning model when the input is inputted the number of times, and a second model characteristic that represents the distribution of the output of the second machine learning model when the input is inputted the number of times.

2. The verification device described in claim 1, wherein the verification unit verifies the identity of the first machine learning model and the second machine learning model based on the inter-distribution distance between the distribution represented by the first model characteristic and the distribution represented by the second model characteristic.

3. The verification device of claim 1, wherein the verification unit verifies the identity of the first machine learning model and the second machine learning model by testing, by statistical hypothesis testing, whether the population of distributions represented by the first model characteristics and the population of distributions represented by the second model characteristics match when the output of the first machine learning model and the output of the second machine learning model are each regarded as samples extracted from a certain population.

4. A verification device according to any one of claims 1 to 3, wherein the first machine learning model and the second machine learning model are machine learning models that realize generative AI.

5. A verification method in which a verification device verifies the identity of a first machine learning model and a second machine learning model whose output changes probabilistically with respect to an input, the verification device executing: an input procedure in which the same input is input to each of the first machine learning model and the second machine learning model a predetermined number of times; and a verification procedure in which the verification device verifies the identity of the first machine learning model and the second machine learning model based on a first model characteristic representing the distribution of the output of the first machine learning model when the input is input the number of times, and a second model characteristic representing the distribution of the output of the second machine learning model when the input is input the number of times.

6. A program that causes a verification device that verifies the identity of a first machine learning model and a second machine learning model whose output changes probabilistically in response to an input to execute: an input procedure that inputs the same input to each of the first machine learning model and the second machine learning model a predetermined number of times; and a verification procedure that verifies the identity of the first machine learning model and the second machine learning model based on a first model characteristic that represents the distribution of the output of the first machine learning model when the input is inputted the number of times, and a second model characteristic that represents the distribution of the output of the second machine learning model when the input is inputted the number of times.

Images