Information processing device, conversion method and program
The conversion of Integrated Gradient (IG) explanations into Vanilla Gradient (VG) explanations enables better understanding and evaluation of machine learning models by addressing the gap in existing conversion technologies, enhancing interpretability and model evaluation.
Patent Information
- Application Number
- JP2023515972
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2021-04-22
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2041-04-22
AI Technical Summary
There is a lack of technology to convert Integrated Gradient (IG) explanations, which have better overall explanatory ability but simpler information, into Vanilla Gradient (VG) explanations, which are useful for evaluating model extraction resistance and data-free knowledge distillation in machine learning models.
An estimation result acquisition unit obtains estimation results and first explanation data, and an explanatory data conversion unit converts the first explanation data into second explanation data, determining whether a bias term is present and applying perturbation as necessary to achieve the conversion.
First explanation data can be effectively converted into second explanation data, allowing users to understand and evaluate machine learning models more comprehensively by comparing different explanatory methods.
Smart Images

Figure 0007589804000007 
Figure 0007589804000008 
Figure 0007589804000009
Abstract
Description
[Technical field]
[0001] The present invention relates to an information processing device, a conversion method, and a program. [Background technology]
[0002] Machine learning such as deep learning is used in various fields, but the problem is that the computational process is complex and difficult for humans to understand. For this reason, there has been much work being done in recent years on the interpretability of machine learning.
[0003] To explain deep learning models that perform image classification, etc., a "gradient-based explanation" that highlights pixels that were important for the classification is often used. Techniques for gradient-based explanations include Vanilla Gradient (VG) (Non-Patent Document 1) and Integrated Gradient (IG) (Non-Patent Document 2). [Prior art documents] [Non-patent literature]
[0004] [Non-Patent Document 1] Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks:Visualising image classification models and saliency maps. arXiv preprint arXiv:1312.6034, 2013 [Non-Patent Document 2] Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In International Conference on Machine Learning, pp. 3319-3328. PMLR, 2017 Summary of the Invention [Problem to be solved by the invention]
[0005] Comparing IG (first explanatory data) and VG (second explanatory data), IG has better overall explanatory power, but VG is simpler information, so it is more useful for evaluating the resistance of trained models to model extraction and for data-free knowledge distillation (a type of model compression technology). Therefore, it would be convenient for model users if the explanation of a machine learning model explained by IG could be converted to VG, but such technology did not exist until now.
[0006] The disclosed technology aims to convert first explanation data provided as an explanation for a machine learning model into second explanation data. [Means for solving the problem]
[0007] The disclosed technology includes an estimation result acquisition unit that acquires estimation result data indicating an estimation result of a machine learning model and first explanatory data for explaining the estimation result, and an explanatory data conversion unit that converts the first explanatory data into second explanatory data based on the estimation result data, the explanatory data conversion unit determines whether the estimation result data includes a bias term, and if the estimation result data includes the bias term, converts the first explanatory data into the second explanatory data including a perturbation. It is an information processing device. Effect of the Invention
[0008] The first explanatory data provided as an explanation for the machine learning model can be converted into second explanatory data. [Brief description of the drawings]
[0009] [Figure 1] FIG. 2 is a functional configuration diagram of the explanation conversion system. [Diagram 2] FIG. 13 is a diagram illustrating an example of an algorithm for explanation conversion processing. [Diagram 3] 13 is a flowchart showing an example of the flow of a description conversion process. [Figure 4]FIG. 2 is a diagram illustrating an example of a hardware configuration of an information processing device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0010] Hereinafter, an embodiment of the present invention (the present embodiment) will be described with reference to the drawings. The embodiment described below is merely an example, and the embodiment to which the present invention is applied is not limited to the following embodiment.
[0011] The explanation conversion system 1 according to the present embodiment includes an information processing device 10 and a server device 20. The information processing device 10 and the server device 20 are communicatively connected via a communication line or the like.
[0012] The information processing device 10 transmits query data to the server device 20. The server device 20 executes an inference process based on the received query data, and transmits inference result data and first explanation data to the information processing device 10. The information processing device 10 converts the received first explanation data into second explanation data and outputs the second explanation data.
[0013] The first explanation data is explanation data assigned to the estimation result data by Integrated Gradient (IG) (Non-Patent Document 2). The second explanation data is explanation data assigned to the estimation result data by Vanilla Gradient (VG) (Non-Patent Document 1).
[0014] The information processing device 10 includes a query sending unit 11, an inference result acquiring unit 12, an explanation data converting unit 13, and an output unit 14.
[0015] The query sending unit 11 sends query data to the server device 20 in response to a user operation or the like. The inference result acquiring unit 12 acquires the inference result data and the first explanation data from the server device 20. The explanation data converting unit 13 converts the first explanation data into second explanation data.
[0016] The output unit 14 outputs the converted second explanatory data. Specifically, the output unit 14 displays an image showing the second explanatory data on a screen, or transmits the second explanatory data to another device or the like.
[0017] The server device 20 includes a query acquisition unit 21, an estimation unit 22, an estimation result output unit 23, and a deep learning model 24.
[0018] The query acquisition unit 21 acquires query data from the information processing device 10. The estimation unit 22 executes an estimation process on the query data using a deep learning model 24. Here, the estimation unit 22 assigns first explanatory data to the estimation result data by the IG. The estimation result output unit 23 outputs the estimation result data and the first explanatory data to the information processing device 10.
[0019] The deep learning model 24 is an example of a machine learning model. The deep learning model 24 is, for example, a neural network whose activation function is a Rectified Linear Unit (ReLU), but is not limited to this.
[0020] (Operation of information processing device) 2 is a diagram showing an example of an algorithm for the explanation conversion process. The information processing device 10 converts the first explanation data into the second explanation data in the explanation data conversion unit 13 using the algorithm shown in FIG.
[0021] FIG. 3 is a flowchart showing an example of the flow of the description conversion process.
[0022] Fig. 3 is a flowchart showing a process flow using the algorithm shown in Fig. 2. Upon receiving a user operation or the like, the query acquisition unit 21 of the information processing device 10 transmits query data (step S101).
[0023] The input query data is
[0024]
number
[0025] Next, the estimation result acquisition unit 12 acquires the estimation result data and the first explanation data (step S102). The estimation result data (f) and the first explanation data (IG) are
[0026]
number
[0027] Next, the explanation data conversion unit 13 determines whether or not the estimation result data (f) includes a bias term (step S103). If the explanation data conversion unit 13 determines that the estimation result data (f) includes a bias term (step S103: Yes), it calculates second explanation data (ans) as follows (step S104).
[0028] g1 = IG(x) x=x+ε g2 = IG(x) ans=(g2-g1) / ε
[0029] where ε is a sufficiently small perturbation
[0030]
number
[0031] Furthermore, when the explanation data conversion unit 13 determines that the estimation result data (f) does not include a bias term (step S103: No), it determines whether or not x includes a zero component (step S105).
[0032] When determining that x does not include a zero component (step S105: No), the explanation data conversion unit 13 calculates second explanation data (ans) as follows (step S106).
[0033] ans = IG(x) ans=ans / x
[0034] Furthermore, when the explanation data conversion unit 13 determines that x includes a zero component (step S105: Yes), it calculates the second explanation data (ans) as follows (step S107).
[0035] x=x+ε ans = IG(x) ans=ans / x
[0036] Following step S104, S106 or S107, the output unit 14 outputs the calculated ans (step S108).
[0037] When the deep learning model 24 is, for example, a neural network whose activation function is ReLU, it has local linearity. Therefore, when there is no bias term in f, the relationship between IG and VG is as follows:
[0038]
number
[0039] Also, if f has a bias term, the relationship between IG and VG is given by a sufficiently small vector ε as follows:
[0040]
number
[0041] The above-mentioned explanation conversion process is a process that utilizes such a feature. The second explanation data (VG) output as a result of the explanation conversion process is as shown in Table 1 below.
[0042] [Table 1]
[0043] (Example of Hardware Configuration According to the Present Embodiment) The information processing device 10 can be realized, for example, by making a computer execute a program in which the processing contents described in this embodiment are described. Note that this "computer" may be a physical machine or a virtual machine on the cloud. When a virtual machine is used, the "hardware" described here is virtual hardware.
[0044] The above program can be recorded on a computer-readable recording medium (such as a portable memory) and can be stored or distributed. The above program can also be provided via a network such as the Internet or e-mail.
[0045] Fig. 4 is a diagram showing an example of the hardware configuration of the computer. The computer in Fig. 4 has a drive device 1000, an auxiliary storage device 1002, a memory device 1003, a CPU 1004, an interface device 1005, a display device 1006, an input device 1007, an output device 1008, etc., which are all connected to each other via a bus B.
[0046] The program for realizing the processing on the computer is provided by a recording medium 1001 such as a CD-ROM or a memory card. When the recording medium 1001 storing the program is set in the drive device 1000, the program is installed from the recording medium 1001 to the auxiliary storage device 1002 via the drive device 1000. However, the program does not necessarily have to be installed from the recording medium 1001, but may be downloaded from another computer via a network. The auxiliary storage device 1002 stores the installed program as well as necessary files, data, etc.
[0047] The memory device 1003 reads out and stores the program from the auxiliary storage device 1002 when an instruction to start the program is received. The CPU 1004 realizes the functions related to the device in accordance with the program stored in the memory device 1003. The interface device 1005 is used as an interface for connecting to a network. The display device 1006 displays a GUI (Graphical User Interface) or the like according to a program. The input device 1007 is composed of a keyboard and mouse, buttons, a touch panel, or the like, and is used to input various operation instructions. The output device 1008 outputs the results of calculations.
[0048] According to the information processing device 10 of the present embodiment, the first explanatory data is converted into the second explanatory data, which allows the user to consider the behavior of the machine learning model while comparing the first explanatory data with the second explanatory data, or by using any appropriate explanatory data.
[0049] (Summary of the embodiment) This specification describes at least the information processing device, conversion method, and program described in the following sections. (Section 1) an estimation result acquisition unit that acquires estimation result data indicating an estimation result of a machine learning model and first explanation data for explaining the estimation result; and an explanation data conversion unit that converts the first explanation data into second explanation data based on the estimation result data. Information processing device. (Section 2) the explanatory data conversion unit determines whether the estimation result data includes a bias term, and if the estimation result data includes the bias term, converts the estimation result data into the second explanatory data including a perturbation. 2. The information processing device according to claim 1. (Section 3) When the bias term is not included, the explanatory data conversion unit determines whether or not a zero component is included in the query data input to the machine learning model, and when the zero component is included, converts the query data into the second explanatory data including the perturbation. 3. The information processing device according to claim 2. (Section 4) The machine learning model is a neural network whose activation function is ReLU. 4. An information processing device according to any one of claims 1 to 3. (Section 5) The first explanation data is explanation data by IG (Integrated Gradient), The second explanation data is explanation data by VG (Vanilla Gradient), 5. An information processing device according to any one of claims 1 to 4. (Section 6) 1. A computer-implemented method comprising: A step of acquiring estimation result data indicating an estimation result of a machine learning model and first explanation data for explaining the estimation result; and converting the first explanation data into second explanation data based on the estimation result data. How to convert. (Section 7) A program for causing a computer to function as each unit in the information processing device according to any one of claims 1 to 5.
[0050] Although the present embodiment has been described above, the present invention is not limited to such a specific embodiment, and various modifications and variations are possible within the scope of the gist of the present invention described in the claims. [Explanation of symbols]
[0051] 1. Description conversion system 10. Information processing device 11 Query sending section 12 Estimation result acquisition part 13 Explanation data conversion section 14 Output section 20 Server device 21 Query Acquisition Unit 22 Estimation part 23 Estimation result output section 24 Deep Learning Models 1000 Drive Device 1001 Recording media 1002 Auxiliary storage 1003 Memory device 1004 CPU 1005 Interface device 1006 Display device 1007 Input Device 1008 Output device
Claims
1. an estimation result acquisition unit that acquires estimation result data indicating an estimation result of a machine learning model and first explanation data for explaining the estimation result; an explanation data conversion unit that converts the first explanation data into second explanation data based on the estimation result data, the explanatory data conversion unit determines whether the estimation result data includes a bias term, and if the estimation result data includes the bias term, converts the first explanatory data into the second explanatory data including a perturbation. Information processing device.
2. When the bias term is not included, the explanatory data conversion unit determines whether or not a zero component is included in the query data input to the machine learning model, and when the zero component is included, converts the first explanatory data into the second explanatory data including the perturbation. The information processing device according to claim 1 .
3. The machine learning model is a neural network whose activation function is ReLU.
3. The information processing device according to claim 1 or 2.
4. The first explanation data is explanation data by IG (Integrated Gradient), The second explanation data is explanation data in VG (Vanilla Gradient), The information processing device according to claim 1 .
5. 1. A computer implemented method of converting, comprising: A step of acquiring estimation result data indicating an estimation result of a machine learning model and first explanation data for explaining the estimation result; and an explanation data conversion step of converting the first explanation data into second explanation data based on the estimation result data, in the explanatory data conversion step, determining whether the estimation result data includes a bias term, and if the estimation result data includes the bias term, converting the first explanatory data into the second explanatory data including a perturbation; How to convert.
6. A program for causing a computer to function as each unit in the information processing device according to any one of claims 1 to 4.
