Verification device, verification method, and program

Abstract

A verification device according to one aspect of the present disclosure, which verifies the identity of a first machine learning model and a second machine learning model, the outputs of which vary stochastically in response to inputs, comprises: an input unit that enters a plurality of inputs into each of the first machine learning model and the second machine learning model, said plurality of inputs being such that discrete value outputs are produced; and a verification unit that verifies the identity of the first machine learning model and the second machine learning model on the basis of first model characteristics that represent the plurality of outputs of the first machine learning model when the plurality of inputs are each input thereto and second model characteristics that represent the plurality of outputs of the second machine learning model when the plurality of inputs are each input thereto.

Description

Verification device, verification method, and program

[0001] The present disclosure relates to a verification device, a verification method, and a program.

[0002] Among the machine learning models that realize AI (Artificial Intelligence), some continue to change through continuous machine learning in order to improve their sophistication and adapt to external environments. Such machine learning models have the problem that it is difficult to detect changes due to erroneous learning or unauthorized substitution of machine learning models by third parties. In response to this problem, a method has been proposed for verifying the identity of multiple machine learning models by utilizing the characteristics of the machine learning models (Non-Patent Document 1).

[0003] Naoto Kiribuchi, Yuya Sato, Ryohei Suzuki, Nami Ashizawa, Satoshi Oki, Hirofumi Mineno, Masakatsu Nishigaki, "A Study on the Use of Verifiable Machine Learning Models: Proposal of a Model Identity Verification Method Based on Robustness to Image Manipulation," Research Report Computer Security (CSEC), vol.2023-CSEC-101, no.16, pp.1-7, 2023.

[0004] However, the method proposed in Non-Patent Document 1 sometimes fails to verify the identity of machine learning models whose outputs are not deterministically determined for the same input.

[0005] The present disclosure has been made in consideration of the above points, and aims to verify the identity of a machine learning model whose output is not deterministically determined for the same input.

[0006] A verification device according to one aspect of the present disclosure is a verification device that verifies the identity of a first machine learning model and a second machine learning model whose output changes probabilistically in response to an input, and includes an input unit that inputs, to each of the first machine learning model and the second machine learning model, a plurality of inputs that result in discrete value outputs, and a verification unit that verifies the identity of the first machine learning model and the second machine learning model based on first model characteristics that represent a plurality of outputs when the plurality of inputs are each input to the first machine learning model, and second model characteristics that represent a plurality of outputs when the plurality of inputs are each input to the second machine learning model.

[0007] It is possible to verify the identity of machine learning models whose output is not deterministically determined for the same input.

[0008] FIG. 1 is a diagram illustrating an example of a hardware configuration of a verification device according to a first embodiment; FIG. 2 is a diagram illustrating an example of a functional configuration of a verification device according to the first embodiment; FIG. 3 is a flowchart illustrating an example of verification processing according to the first embodiment; FIG. 4 is a diagram illustrating an example of a functional configuration of a verification device according to a second embodiment; and FIG. 5 is a flowchart illustrating an example of verification processing according to the second embodiment.

[0009] First and second embodiments of the present invention will be described in detail below with reference to the drawings. In the following embodiments, a verification device 10 will be described that can verify the identity of two machine learning models whose output is not deterministically determined for each input. Hereinafter, the two machine learning models whose identity is to be verified will be referred to as "machine learning model T" and "machine learning model V." Machine learning model T is assumed to be, for example, a legitimate machine learning model that operates as intended by the model creator or the like. On the other hand, machine learning model V is a machine learning model whose identity with machine learning model T is to be verified, and is assumed to be, for example, a machine learning model that may have changed due to erroneous learning or may have been illegally replaced by a third party.

[0010] An example of a machine learning model in which the output is not deterministically determined for the same input is a machine learning model called a large language model (LLM) that realizes generative AI (or may be called generative AI). Such machine learning models do not necessarily obtain the same output for the same input, but rather the output changes probabilistically for the same input. However, a machine learning model that realizes generative AI is just one example, and the following embodiments can be similarly applied to verifying the identity of any two machine learning models in which the output is not deterministically determined for the input (in other words, any two machine learning models in which the output changes probabilistically for the input).

[0011] First Embodiment A first embodiment will be described below.

[0012] <Example of Hardware Configuration of Verification Device 10 According to First Embodiment> An example of the hardware configuration of the verification device 10 according to the first embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram illustrating an example of the hardware configuration of the verification device 10 according to the first embodiment.

[0013] 1, a verification device 10 according to the first embodiment includes an input device 101, a display device 102, an external I / F 103, a communication I / F 104, a RAM (Random Access Memory) 105, a ROM (Read Only Memory) 106, an auxiliary storage device 107, and a processor 108. Each of these pieces of hardware is connected to each other via a bus 109 so as to be able to communicate with each other.

[0014] The input device 101 is, for example, a keyboard, a mouse, a touch panel, a physical button, etc. The display device 102 is, for example, a display, a display panel, etc. Note that the verification device 10 does not necessarily have to include at least one of the input device 101 and the display device 102, for example.

[0015] The external I / F 103 is an interface with an external device such as a recording medium 103a. Examples of the recording medium 103a include a CD (Compact Disc), a DVD (Digital Versatile Disk), an SD memory card (Secure Digital memory card), and a USB (Universal Serial Bus) memory card.

[0016] The communication I / F 104 is an interface for connecting to a communication network. The RAM 105 is a volatile semiconductor memory (storage device) that temporarily stores programs and data. The ROM 106 is a non-volatile semiconductor memory (storage device) that can store programs and data even when the power is turned off. The auxiliary storage device 107 is a non-volatile storage device such as a hard disk drive (HDD), a solid state drive (SSD), or a flash memory. The processor 108 is a variety of arithmetic devices such as a central processing unit (CPU) or a graphic processing unit (GPU).

[0017] 1 is an example, and the hardware configuration of the verification device 10 is not limited to this. For example, the verification device 10 may have multiple auxiliary storage devices 107 or multiple processors 108, may not have some of the hardware shown in the figure, or may have various hardware other than the hardware shown in the figure.

[0018] <Example of Functional Configuration of Verification Device 10 According to First Embodiment> An example of the functional configuration of the verification device 10 according to the first embodiment will be described with reference to Fig. 2. Fig. 2 is a diagram illustrating an example of the functional configuration of the verification device 10 according to the first embodiment.

[0019] 2 , the verification device 10 according to the first embodiment includes a model input unit 201, a model property acquisition unit 202, a verification unit 203, and a verification result output unit 204. These units are implemented, for example, by a processor 108 or the like executing one or more programs installed in the verification device 10. The verification device 10 according to the first embodiment also includes an input storage unit 205 and a model property storage unit 206. These storage units are implemented, for example, by a storage area of ​​the auxiliary storage device 107 or the like. Note that at least one of the input storage unit 205 and the model property storage unit 206 may be implemented by a storage area of ​​a storage device (e.g., a storage device included in a database server) or the like communicatively connected to the verification device 10.

[0020] The model input unit 201 receives the input x stored in the input storage unit 205. １ , ..., x ｋ are input to the machine learning model T. As a result, k outputs y Ｔ，１ , ..., y Ｔ，ｋ Here, k is a predetermined integer of 2 or more. ｉ (where 1≦i≦k) is an input that, when input to the machine learning models T and V, results in a discrete value output. Ｔ，ｉ is the input x ｉ is the output when input to the machine learning model T, and takes a discrete value (for example, 0 or 1 representing "yes" or "no," an integer value representing some category, etc.). In general, for each of i = 1, ..., k, the input x ｉ Whenever input to the machine learning model T, the output y Ｔ，ｉ is obtained.

[0021] The model input unit 201 also receives the input x stored in the input storage unit 205. １ , ..., x ｋ are input to the machine learning model V. As a result, k outputs y Ｖ，１ , ..., y Ｖ，ｋ where y Ｖ，ｉ is the input x ｉis the output when input to the machine learning model V, and takes a discrete value. In general, for each of i = 1, ..., k, the input x ｉ Whenever V is input to the machine learning model, the output y Ｖ，ｉ is obtained.

[0022] Note that the input x １ , ..., x ｋ The format of may vary depending on the machine learning models T and V. For example, when the machine learning models T and V are machine learning models such as large-scale language models that realize generative AI, the input x １ , ..., x ｋ is generally in text format. Ｔ，ｉ and y Ｖ，ｉ The format is input x ｉ However, in the following, the output y Ｔ，ｉ and y Ｖ，ｉ is expressed as a scalar that takes an integer value or a vector, matrix, tensor, etc. that has integer values ​​as elements, and the output y Ｔ，ｉ and output y Ｖ，ｉ However, it is also possible to input data in various formats, such as still images, moving images, and audio.

[0023] The machine learning models T and V are stored in a storage area such as the auxiliary storage device 107. However, at least one of the machine learning models T and V may be stored in a storage area such as a storage device communicatively connected to the verification device 10 (e.g., a storage device provided in an API (Application Programming Interface) server).

[0024] The model characteristic acquisition unit 202 acquires k outputs y from the machine learning model T. Ｔ，１ , ..., y Ｔ，ｋ Let T be the model characteristic Y of the machine learning model T. Ｔ = (y Ｔ，１ , ..., y Ｔ，ｋ Similarly, the model characteristic acquisition unit 202 acquires k outputs y Ｖ，１ , ..., y Ｖ，ｋ Let Y be the model characteristic of machine learning model V.Ｖ = (y Ｖ，１ , ..., y Ｖ，ｋ ) is obtained.

[0025] Furthermore, the model characteristic acquisition unit 202 acquires the model characteristic Y Ｔ and Y Ｖ At least one of the above is stored in the model characteristic storage unit 206.

[0026] The verification unit 203 determines the model characteristic Y Ｔ and Y Ｖ The verification unit 203 verifies whether the machine learning model V can be considered to be the same as the machine learning model T by using the model characteristic Y Ｔ and Y Ｖ The identity of the machine learning model T and the machine learning model V is verified using

[0027] The verification result output unit 204 outputs the verification result by the verification unit 203 to a predetermined output destination. The predetermined output destination may be, for example, the display device 102 such as a display, a storage area such as the auxiliary storage device 107, or another device or equipment connected to the verification device 10 so as to be able to communicate with the verification device 10.

[0028] The input storage unit 205 stores k inputs x that, when input to the machine learning models T and V, produce discrete value outputs. １ , ..., x ｋ In addition, when the machine learning models T and V are machine learning models that realize generative AI, the input x that, when input to the machine learning models T and V, produces a discrete value output is stored. ｉ Examples include, "Is the weather sunny today? Please answer 'yes' or 'no'." and "Please rate your happiness level on a 5-point scale, with 1 being the lowest and 5 being the highest."

[0029] The model characteristic storage unit 206 stores the model characteristic Y Ｔ and Y Ｖ Remember.

[0030] <Verification Process According to First Embodiment> The verification process according to the first embodiment will be described with reference to Fig. 3. Fig. 3 is a flowchart showing an example of the verification process according to the first embodiment.

[0031] The model input unit 201 receives the input x stored in the input storage unit 205. １ , ..., x ｋ are input to the machine learning model T (step S101). As a result, k outputs y Ｔ，１ , ..., y Ｔ，ｋ It should be noted that each output y Ｔ，１ , ..., y Ｔ，ｋ is expressed as a scalar that takes an integer value, or a vector, matrix, tensor, etc. that has integer values ​​as elements.

[0032] The model characteristic acquisition unit 202 calculates the k outputs y obtained in step S101. Ｔ，１ , ..., y Ｔ，ｋ The model characteristic Y Ｔ = (y Ｔ，１ , ..., y Ｔ，ｋ ) (step S102). Ｔ is stored in the model characteristic storage unit 206 as needed. Ｔ is stored in the model characteristic storage unit 206, for example, Ｔ For example, after obtaining the ID, the identity verification in step S105 described later is performed later.

[0033] The model input unit 201 receives the input x stored in the input storage unit 205. １ , ..., x ｋ are input to the machine learning model V (step S103). As a result, k outputs y Ｖ，１ , ..., y Ｖ，ｋ It should be noted that each output y Ｖ，１ , ..., y Ｖ，ｋ is expressed as a scalar that takes an integer value, or a vector, matrix, tensor, etc. that has integer values ​​as elements.

[0034] The model characteristic acquisition unit 202 calculates the k outputs y obtained in step S103. Ｖ，１ , ..., y Ｖ，ｋ The model characteristic Y Ｖ (Step S104). Ｖ The model characteristic Y is stored in the model characteristic storage unit 206 as needed. Ｖ is stored in the model characteristic storage unit 206, for example, Ｖ For example, after obtaining the ID, the identity verification in step S105 described later is performed later.

[0035] The order of steps S101 to S102 and steps S103 to S104 is not particularly limited. That is, for example, steps S101 to S102 may be executed after steps S103 to S104.

[0036] The verification unit 203 determines the model characteristic Y Ｔ and Y Ｖ Here, the verification unit 203 verifies whether the machine learning model V can be considered to be the same as the machine learning model T using, for example, L ｐ Using the norm, it is possible to verify the identity of the machine learning model T and the machine learning model V. ｐ A norm is an example of a measure that represents distance.

[0037] For example, output y Ｔ，１ , ..., y Ｔ，ｋ , y Ｖ，１ , ..., y Ｖ，ｋ are scalars, the verification unit 203 ｐ (Y Ｔ , Y Ｖ ) : = (| y Ｔ，１ -y Ｖ，１ | ｐ +...+ |y Ｔ，ｋ -y Ｖ，ｋ | ｐ ) １／ｐ Calculate this L ｐ (Y Ｔ , Y ＶIf the value of L is less than a predetermined threshold, the verification result is that "machine learning model V can be considered to be identical to machine learning model T," and if not, the verification result is that "machine learning model V cannot be considered to be identical to machine learning model T." １ Norm or L ２ Besides the norm, for example, L ∞ The norm may be used to verify the identity of machine learning model T and machine learning model V.

[0038] Note that a certain output y Ｔ，ｉ and y Ｖ，ｉ is expressed as a vector, matrix or tensor, the output y Ｔ，ｉ and y Ｖ，ｉ Instead of ｐ Specifically, we can calculate the norm of a given output y Ｔ，ｉ and y Ｖ，ｉ is a vector and y Ｔ，ｉ = (y Ｔ，ｉ，ｊ ), y Ｖ，ｉ = (y Ｖ，ｉ，ｊ ), then |y Ｔ，ｉ -y Ｖ，ｉ | ｐ :=Σ ｊ |y Ｔ，ｉ，ｊ -y Ｖ，ｉ，ｊ | ｐ As the above L ｐ Norm L ｐ (Y Ｔ , Y Ｖ ) can be calculated. Ｔ，ｉ，ｊ is the vector y Ｔ，ｉ The jth element of y Ｖ，ｉ，ｊ is the vector y Ｔ，ｉ Similarly, for a given output y Ｔ，ｉ and y Ｖ，ｉ is a matrix and y Ｔ，ｉ = (y Ｔ，ｉ，ｊ，ｊ' ), y Ｖ，ｉ = (y Ｖ，ｉ，ｊ，ｊ' ), then |y Ｔ，ｉ -y Ｖ，ｉ | ｐ :=Σ ｊ，ｊ' |y Ｔ，ｉ，ｊ，ｊ' -y Ｖ，ｉ，ｊ，ｊ' | ｐ As the above L ｐ Norm Lｐ (Y Ｔ , Y Ｖ ) can be calculated. Ｔ，ｉ，ｊ，ｊ' is the matrix y Ｔ，ｉ the (j, j') element of y Ｖ，ｉ，ｊ，ｊ' is the matrix y Ｔ，ｉ The (j, j') element of a certain output y Ｔ，ｉ and y Ｖ，ｉ The same can be considered when is a tensor.

[0039] The verification result output unit 204 outputs the verification result obtained in step S105 to a predetermined output destination (step S106).

[0040] [Second Embodiment] The second embodiment will be described below. In the first embodiment, the output y Ｔ，１ , ..., y Ｔ，ｋ , y Ｖ，１ , ..., y Ｖ，ｋ Although both of the above are assumed to take discrete values, there are cases where the output itself is not a discrete value but can be converted into a discrete value. For example, if the output is the answer to a certain question, the output can be converted into 0 or 1, which represents a correct or incorrect answer. Therefore, in the second embodiment, a case will be described in which the outputs of the machine learning models T and V may include outputs that can be converted into discrete values.

[0041] In the second embodiment, differences from the first embodiment will be mainly described, and a description of components that may be the same as those in the first embodiment will be omitted.

[0042] <Example of Functional Configuration of Verification Device 10 According to Second Embodiment> An example of the functional configuration of the verification device 10 according to the second embodiment will be described with reference to Fig. 4. Fig. 4 is a diagram illustrating an example of the functional configuration of the verification device 10 according to the second embodiment.

[0043] 4, the verification device 10 according to the second embodiment includes a discretization unit 207 in addition to the units described in the first embodiment. The discretization unit 207 is realized, for example, by a process in which one or more programs installed in the verification device 10 are executed by the processor 108 or the like. Here, the input storage unit 205 stores k inputs x that, when input to the machine learning models T and V, produce discrete value outputs or outputs that can be converted into discrete values. １ , ..., x ｋ In addition, when the machine learning models T and V are machine learning models that realize generative AI, an input x that, when input to the machine learning models T and V, produces an output that can be converted into a discrete value is stored. ｉ An example of this would be "Please solve the above problem" after defining a problem with a uniquely determined answer.

[0044] The discretization unit 207 calculates the model characteristic Y Ｔ = (y Ｔ，１ , ..., y Ｔ，ｋ ) includes an output that is not a discrete value (i.e., an output that is not a discrete value output but can be converted into a discrete value), the discretization unit 207 converts the output into a discrete value. Ｖ = (y Ｖ，１ , ..., y Ｖ，ｋ ) contains an output that is not a discrete value (i.e., an output that is not a discrete value output but can be converted to a discrete value), convert that output to a discrete value output.

[0045] <Verification Process According to Second Embodiment> The verification process according to the second embodiment will be described with reference to Fig. 5. Fig. 5 is a flowchart showing an example of the verification process according to the second embodiment.

[0046] Steps S201 and S202 may be similar to steps S101 and S102 in FIG. 3, respectively, and therefore a description thereof will be omitted.

[0047] The discretization unit 207 calculates the model characteristic Y obtained in step S202. Ｔ = (y Ｔ，１ , ..., y Ｔ，ｋIf an output that is not a discrete value is included in the output y, the output is converted into a discrete value (step S203). For example, if an output y Ｔ，ｉ is not a discrete value, the discretization unit 207 Ｔ，ｉ is converted into a discrete value. Ｔ，ｉ If is a vector, matrix, or tensor, the output y Ｔ，ｉ Among the elements of , all elements that are not discrete values ​​are converted to discrete values. Ｔ，ｉ Also, "output y Ｔ，ｉ " will be expressed as ".

[0048] Output y Ｔ，ｉ There are various methods for converting the output y into a discrete value. For example, the discretization unit 207 converts the output y into a discrete value by a method such as a true / false judgment or clustering. Ｔ，ｉ Specifically, for example, when using a true / false judgment, the discretization unit 207 discretizes the output y Ｔ，ｉ is correct, it is converted to 0, and if not, it is converted to 1. Also, for example, when using clustering, the discretization unit 207 converts the output y Ｔ，ｉ By clustering the output y Ｔ，ｉ can be converted into a discrete value represented by the cluster to which it belongs.

[0049] Steps S204 and S205 may be similar to steps S103 and S104 in FIG. 3, respectively, and therefore a description thereof will be omitted.

[0050] The discretization unit 207 discretizes the model characteristic Y obtained in step S205 in the same manner as in step S203. Ｖ = (y Ｖ，１ , ..., y Ｖ，ｋ If an output that is not a discrete value is included in the output y, the output is converted into a discrete value (step S206). For example, if an output y Ｖ，ｉ is not a discrete value, the discretization unit 207 Ｖ，ｉ is converted into a discrete value. Ｖ，ｉ If is a vector, matrix, or tensor, the output y Ｖ，ｉ Among the elements of , all elements that are not discrete values ​​are converted to discrete values.Ｖ，ｉ Also, "output y Ｖ，ｉ " will be expressed as ".

[0051] Steps S207 and S208 may be similar to steps S105 and S106 in FIG. 3, respectively, and therefore a description thereof will be omitted.

[0052] [Modifications] Modification 1 In the first and second embodiments, the input x １ , ..., x ｋ By inputting these values ​​into the machine learning models T and V, respectively, the output y Ｔ，１ , ..., y Ｔ，ｋ and y Ｖ，１ , ..., y Ｖ，ｋ However, if an output that is not a discrete value and cannot be converted into a discrete value is obtained, that output may be excluded.

[0053] For example, given an input x ｉ The output y when input into the machine learning models T and V is Ｔ，ｉ and y Ｖ，ｉ If y is not a discrete value and cannot be converted to a discrete value, then the output y Ｔ，ｉ and y Ｖ，ｉ may be excluded (deleted) from the model characteristics.

[0054] Modification 2 In the first and second embodiments, the machine learning models T and V are given input x １ , ..., x ｋ For example, N is a predetermined integer of 2 or more, and x １ , ..., x ｋ are input N times, and N model characteristics Y Ｔ and N model characteristics Y Ｖ The identity may be verified from

[0055] Specifically, the nth (where 1≦n≦N) input x １ , ..., x ｋ x １ （ｎ） , ..., x ｋ （ｎ） Also, x １ （ｎ） , ..., xｋ （ｎ） The model characteristics when input into the machine learning model T are expressed as Y Ｔ （ｎ） , the model characteristics when input to the machine learning model V are Y Ｖ （ｎ） In addition, x ｉ （１） =...=x ｉ （Ｎ） is.

[0056] In this case, for n=1, . . . , N, x １ （ｎ） , ..., x ｋ （ｎ） are input to the machine learning models T and V, respectively, and the model characteristics Y Ｔ （ｎ） = (y Ｔ，１ （ｎ） , ..., y Ｔ，ｋ （ｎ） ) and model characteristic Y Ｖ （ｎ） = (y Ｖ，１ （ｎ） , ..., y Ｖ，ｋ （ｎ） ) and obtain the model characteristic Y Ｔ （ｎ） (n=1,...,N) and model characteristic Y Ｖ （ｎ） (n=1, . . . , N) to verify the identity of machine learning model T and machine learning model V.

[0057] When verifying the identity of machine learning model T and machine learning model V, for example, the model characteristic Y Ｔ （ｎ） and Y Ｖ （ｎ） 3, the identity is verified using the model characteristic Y Ｔ （ｎ） and Y Ｖ （ｎ）3, the identity may be verified using the model characteristics Y. Only when the number of n for which it has been verified that "machine learning model V can be considered identical to machine learning model T" is equal to or greater than a predetermined number, the final verification result may be "machine learning model V can be considered identical to machine learning model T." However, in addition to these, N model characteristics Y Ｔ （ｎ） and N model characteristics Y Ｖ （ｎ） Any other method may be used as long as it can verify the identity of machine learning model T and machine learning model V using the above.

[0058] [Summary] As described above, the verification device 10 according to the first and second embodiments uses k inputs x that produce discrete value outputs or outputs that can be converted into discrete values ​​for each of the two machine learning models T and V. １ , ..., x ｋ By comparing the outputs of the machine learning models T and V, the identity of the machine learning models T and V is verified. ｐ Because a norm is used, it is possible to verify the identity of two machine learning models T and V even if the outputs are discrete values. Therefore, for example, even if the two machine learning models T and V are machine learning models that output discrete values ​​regardless of the input x given (or machine learning models whose outputs are mostly discrete values), it is possible to verify the identity of the machine learning models T and V. Therefore, for example, it is possible to detect machine learning models that may have been changed due to erroneous learning or may have been fraudulently replaced by a third party, targeting machine learning models that realize generative AI.

[0059] The present invention is not limited to the above-described specifically disclosed embodiments, and various modifications, changes, and combinations with known technologies are possible without departing from the scope of the claims.

[0060] 10 Verification device 101 Input device 102 Display device 103 External I / F 103a Recording medium 104 Communication I / F 105 RAM 106 ROM 107 Auxiliary storage device 108 Processor 109 Bus 201 Model input unit 202 Model characteristic acquisition unit 203 Verification unit 204 Verification result output unit 205 Input storage unit 206 Model characteristic storage unit 207 Discretization unit

Claims

1. A verification device that verifies the identity of a first machine learning model and a second machine learning model whose output changes probabilistically with respect to an input, comprising: an input unit that inputs, to each of the first machine learning model and the second machine learning model, a plurality of inputs that result in discrete value outputs; and a verification unit that verifies the identity of the first machine learning model and the second machine learning model based on first model characteristics that represent a plurality of outputs when the plurality of inputs are each input to the first machine learning model, and second model characteristics that represent a plurality of outputs when the plurality of inputs are each input to the second machine learning model.

2. The verification device according to claim 1, wherein the plurality of inputs include one or more inputs that produce an output that can be converted into a discrete value, and wherein, when the output that can be converted into a discrete value is included in the first model characteristic and the second model characteristic, the verification device further comprises a conversion unit that converts the output that can be converted into a discrete value into an output of a discrete value.

3. A verification device as described in claim 1 or 2, wherein the verification unit verifies the identity of the first machine learning model and the second machine learning model based on an evaluation measure representing the distance between the first model characteristic and the second model characteristic.

4. The verification device according to claim 3, wherein the first machine learning model and the second machine learning model are machine learning models that realize generative AI.

5. A verification method in which a verification device verifies the identity of a first machine learning model and a second machine learning model whose output changes probabilistically with respect to an input, the verification device executing: an input procedure in which a plurality of inputs that result in discrete value outputs are input to each of the first machine learning model and the second machine learning model; and a verification procedure in which the verification device verifies the identity of the first machine learning model and the second machine learning model based on first model characteristics that represent a plurality of outputs when the plurality of inputs are input to the first machine learning model, and second model characteristics that represent a plurality of outputs when the plurality of inputs are input to the second machine learning model, respectively.

6. A program that causes a verification device that verifies the identity of a first machine learning model and a second machine learning model whose output changes probabilistically with respect to input to execute the following steps: an input step of inputting, to each of the first machine learning model and the second machine learning model, a plurality of inputs that result in discrete value outputs; and a verification step of verifying the identity of the first machine learning model and the second machine learning model based on first model characteristics that represent a plurality of outputs when the plurality of inputs are each input to the first machine learning model, and second model characteristics that represent a plurality of outputs when the plurality of inputs are each input to the second machine learning model.

Images