Method for providing description of artificial intelligence model based on plug-and-play mode

Abstract

The method for providing description of an artificial intelligence model based on a plug-and-play mode according to the present invention comprises the following steps: a service providing company server sets the type of the artificial intelligence model; when the artificial intelligence model is input, the plug-and-play manager identifies the artificial intelligence model and constructs a list of available description modules based on an identification result; the service providing company server provides key points about the description module for the user terminal; the user terminal selects one description module in the description modules and provides the selected description module for the service providing company server; the plug-and-play manager obtains a description module selected from the description modules on the list; and the obtained description module provides description of the input artificial intelligence model for the user terminal.

Description

Technical Field

[0001] The present invention relates to a method for providing an explanation of an artificial intelligence model, and more particularly to a method for providing an explanation of an artificial intelligence model based on a plug-and-play mode, by which a module for explaining a given artificial intelligence model is automatically found in the plug-and-play mode, so as to provide an explanation in a manner that is easy for a user to understand. Background Art

[0002] These days, with the development of artificial intelligence, artificial intelligence technology is being actively applied to various fields such as image recognition, speech recognition, dialogue systems, and autonomous driving. Although artificial intelligence technology is being applied to many fields of real life and artificial intelligence technology is developing day by day, the structure of artificial intelligence models trained based on data is constantly becoming more complex, making it difficult to accurately understand the decision-making principle of artificial intelligence models.

[0003] If such a complex artificial intelligence model is applied to a critical task that has a significant impact on human life or property, consumers may be damaged by using an incomplete artificial intelligence model. To address this, the European Union has been mandating the interpretability of artificial intelligence models through the General Data Protection Regulation (GDPR) since 2018, and recently raised the standards to classify artificial intelligence technologies involved in human biometric signals, autonomous driving, personnel assessment, and credit assessment as high-risk artificial intelligence, thus more strongly demanding the reliability and transparency of artificial intelligence models.

[0004] Although various methods are being developed to explain the decisions of artificial intelligence models, it is necessary to provide an explanation by reflecting various characteristics of artificial intelligence models. Therefore, it is difficult to use one method to explain the decisions of all models, and thus an algorithm for explaining decisions is provided individually for each artificial intelligence model.

[0005] For example, the techniques for explaining artificial intelligence models with a decision tree structure, the techniques for explaining deep learning models, and the techniques for explaining Bayesian-based probability models are different, and there are techniques for providing a general explanation, but the accuracy is usually low.

[0006] Deep learning frameworks such as "TensorFlow" from Google and "Pytorch" from Meta also use a method called "Integrated Gradients" and a technique called "DeepLift" to explain deep learning. These techniques are for explaining deep learning decisions, but they only provide explanations for deep learning models developed in each platform.

[0007] Meanwhile, Korean Patent Publication No. 10-2020-0092447 discloses "Explainable Artificial Intelligence Modeling and Simulation System and Method", which includes designing an artificial intelligence workflow model when an algorithm suitable for a workflow for creating and editing a subject area is selected from stored algorithms, and when input information is input, performing a simulation on the artificial intelligence workflow model for the input information.

[0008] The above patent document has the following advantages: visualization of the connection of workflow-based algorithms, automatic performance verification through integrated simulation, simultaneous simulation of multiple artificial intelligence algorithms based on workflows for mutual performance comparison, and explaining the cause / basis by using images and standardized features from classification analysis results to suggest directions for improving algorithm performance. However, this technology is a mechanism for designing and performing a simulation of a corresponding artificial intelligence workflow model when an algorithm suitable for a workflow for creating / editing a subject area such as semiconductor and display manufacturing processes is selected, and thus this technology also only provides an explanation or simulation of an artificial intelligence model designed under specific conditions, and thus has the problem of not being able to generally explain various artificial intelligence models. Summary of the Invention

[0009] Technical Problem

[0010] Accordingly, the present invention has been made in view of the above problems, and an object of the present invention is to provide a method for providing an explanation of an artificial intelligence model based on a plug-and-play mode, by which a general explanation of various artificial intelligence models can be provided, and an explanation module for explaining a given artificial intelligence model can be automatically found in a plug-and-play mode, thereby providing an explanation in a manner that is easy for a user to understand.

[0011] Another object of the present invention is to provide a method for providing explainability for an artificial intelligence model based on a plug-and-play mode, by which when an explanation of an artificial intelligence model is required, an algorithm capable of identifying various features of the artificial intelligence model and explaining the artificial intelligence model can be automatically found, and a user can be enabled to select a format and content that can be explained to be provided with a customized explanation.

[0012] Technical Solution

[0013] According to an aspect of the present invention, the above and other objects can be achieved by providing a method for providing an explanation of an artificial intelligence model based on a plug-and-play mode.

[0014] The method is based on a system for providing explanations for artificial intelligence models in a plug-and-play mode. The system includes: a service provider server equipped with an explanation module for providing explanations for artificial intelligence models and a plug-and-play manager for connecting the artificial intelligence model and the explanation module, and the service provider server provides explanations for artificial intelligence models; and a user terminal configured to communicate with the service provider server via the Internet and receive an explanation service for the artificial intelligence model from the service provider server. The method includes:

[0015] a) A step in which a controller of the service provider server sets the type of artificial intelligence model that can be interpreted by each explanation module for communication between the artificial intelligence model and the explanation module;

[0016] b) A step in which the plug-and-play manager identifies the manufacturer, framework, and type of the artificial intelligence model when the artificial intelligence model to be explained is input;

[0017] c) A step in which the plug-and-play manager creates a list of available explanation modules based on the identification result considering the constraints on the explanation modules;

[0018] d) A step in which the service provider server provides the user terminal with key information about the explanation modules in the list of available explanation modules;

[0019] e) A step in which the user terminal selects one of the explanation modules in the list of explanation modules with reference to the key information about the explanation modules in the list and notifies the service provider server of the selected explanation module;

[0020] f) A step in which the plug-and-play manager obtains the explanation module selected by the user terminal from the list of explanation modules; and

[0021] g) A step in which the obtained explanation module provides the user terminal with an explanation of the input artificial intelligence model.

[0022] Here, the plug-and-play manager may have a list of explanation modules pre-provided by developers of interpretable artificial intelligence, and the explanation modules may have information about the types of interpretable artificial intelligence models.

[0023] In addition, in step a), for communication between the artificial intelligence model and the explanation module, the artificial intelligence model can be divided into interpretable artificial intelligence explanation targets, explanation techniques (explanation kernels), explanation methods, and explanation modalities.

[0024] Here, the explanation technique (explanation kernel) can be divided into an explanation module for explaining the input contribution of the artificial intelligence model and an explanation module for explaining the inside of the artificial intelligence model.

[0025] Here, the interpretation target can be an AI model including at least one of a convolutional neural network (CNN / DNN), a recurrent neural network (RNN / LSTM), a transformer, a decision tree, a Bayesian, or a finite state machine (FSM), or can be an AI model that is a combination including at least two of a CNN / DNN, an RNN / LSTM, a transformer, a decision tree, a Bayesian, or an FSM.

[0026] Here, the interpretation modality can be an interpretation interface suitable for different data formats for vision, language, speech time series, and behavior.

[0027] In addition, step b) can include: configuring an interface between the artificial intelligence model and an interpretation module for interpreting the input of the artificial intelligence model, and configuring an interface between the artificial intelligence model and an interpretation module for interpreting the interior of the artificial intelligence model.

[0028] Here, the interpretation module can provide global interpretation (considering the statistical contribution of the entire data) values and local interpretation (contribution to specific input data) values of the input variables and internal variables of the artificial intelligence model, in order to interpret the decisions of the artificial intelligence model to the interface.

[0029] In addition, in step d), the key information about the interpretation modules in the list of available interpretation modules can include the characteristics of each interpretation module and the interpretation methods and interfaces supported by each interpretation module.

[0030] In addition, in step g), when the obtained interpretation module provides an interpretation of the input artificial intelligence model, the obtained interpretation module can provide a local interpretation based on individual data and a global interpretation based on validation data.

[0031] Here, the local interpretation can include input contribution interpretation and counterfactual example interpretation, and the global interpretation can include feature importance interpretation.

[0032] Advantageous Effects

[0033] According to the present invention, various artificial intelligence models are generally interpreted, and an interpretation module for interpreting a given artificial intelligence model is automatically found in a plug-and-play mode for interpretation, and thus an interpretation can be provided in a manner that is easy for a user to understand.

[0034] In addition, by confirming various characteristics of the artificial intelligence model, an algorithm capable of interpreting the artificial intelligence model is automatically found, and the user can be enabled to select the format and content that can be interpreted, so that a customized interpretation can be provided to the user. Brief Description of the Drawings

[0035] Figure 1FIG. 0 is a diagram schematically showing a configuration of a system for providing an explanation of an artificial intelligence model based on a plug-and-play mode, the system being configured to implement a method for providing an explanation according to the present invention;

[0036] Figure 2 FIG. 4 is a flowchart showing a process of executing a method for providing an explanation of an artificial intelligence model based on a plug-and-play mode according to an embodiment of the present invention;

[0037] Figure 3 FIG. 8 is a diagram showing classification of an artificial intelligence model into an explainable artificial intelligence explanation target (XAI target), an explanation technique (explanation kernel), an explanation method, and an explanation modality;

[0038] Figures 4a to 4d FIG. 12 is a diagram schematically showing a representative algorithm of explainable artificial intelligence for implementing the method of the present invention;

[0039] Figure 5 FIG. 16 is a diagram showing a sequence operation between a user terminal and a plug-and-play manager related to a method for providing an explanation of an artificial intelligence model based on a plug-and-play mode according to the present invention; and

[0040] Figure 6 FIG. 20 is a diagram showing a mechanism for selecting a user-customized plug-and-play artificial intelligence model explanation interface for implementing the method of the present invention. DETAILED DESCRIPTION

[0041] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0042] Figure 1 FIG. 29 is a diagram schematically showing a configuration of a system for providing an explanation of an artificial intelligence model based on a plug-and-play mode, the system being configured to implement a method for providing an explanation of an artificial intelligence model based on a plug-and-play mode according to the present invention.

[0043] Referring to Figure 1 , a system 100 for providing an explanation of an artificial intelligence model based on a plug-and-play mode, which is configured to implement a method for providing an explanation of an artificial intelligence model based on a plug-and-play mode according to the present invention, includes a service provider server 110 and a user terminal 120.

[0044] The service provider server 110 basically includes a controller and a memory, in which an interpretation module (which is an application as a software program) for providing an interpretation of an artificial intelligence model and a plug-and-play manager (as an application of a software program) for connecting the artificial intelligence model and the interpretation module are stored (installed), and the service provider server 110 provides an interpretation of the artificial intelligence model. The service provider server 110 can be configured as a general desktop PC and can include a database (DB) in which data and information related to various services are stored.

[0045] The user terminal 120 communicates with the service provider server 110 via the Internet and receives an interpretation service for the artificial intelligence model from the service provider server 110. The user terminal 120 can include a mobile phone (smartphone) 120a, a laptop computer 120b, a desktop computer 120c, etc.

[0046] Hereinafter, a method for providing an interpretation of an artificial intelligence model based on the plug-and-play mode will be described below. This method is based on a system for providing an interpretation of an artificial intelligence model based on the plug-and-play mode having the above configuration.

[0047] Figure 2 is a flowchart showing the processing of executing a method for providing an interpretation of an artificial intelligence model based on the plug-and-play mode according to an embodiment of the present invention.

[0048] Referring to Figure 2 , the method for providing an interpretation of an artificial intelligence model based on the plug-and-play mode according to the present invention is a method for providing an interpretation of an artificial intelligence model based on the above system 100 for providing an interpretation of an artificial intelligence model based on the plug-and-play mode. The system 100 includes: a service provider server 110, which is equipped with an interpretation module for providing an interpretation of an artificial intelligence model and a plug-and-play manager for connecting the artificial intelligence model and the interpretation module, and the service provider server 110 provides an interpretation of the artificial intelligence model; and a user terminal 120, which receives an interpretation service for the artificial intelligence model from the service provider server 110. In this method, first, a controller (not shown) of the service provider server 110 sets the type of artificial intelligence model that can be interpreted by each interpretation module for communication between the artificial intelligence model and the interpretation module (step S201). Here, the plug-and-play manager can have a list of interpretation modules pre-provided by developers of explainable artificial intelligence, and the interpretation module can have information about the type (manufacturer, format, language, etc.) of the explainable artificial intelligence model.

[0049] In addition, for the communication between the artificial intelligence model and the explanation module, the artificial intelligence model can be classified into an explainable artificial intelligence explanation target (XAI target) 310, an explanation technique (explanation kernel) 320, an explanation method 330, and an explanation modality 340, as Figure 3 shown. Here, the explanation technique (explanation kernel) 320 can be divided into an explanation module 320a for explaining the input contribution of the artificial intelligence model and an explanation module 320b for explaining the interior of the artificial intelligence model.

[0050] In addition, the explanation target 310 can be an AI model including at least one of a convolutional neural network (CNN / DNN), a recurrent neural network (RNN / LSTM), a Transformer, a decision tree, a Bayesian, or a finite state machine (FSM), or can be an AI model including a combination of at least two or more of them.

[0051] In addition, the explanation technique (explanation kernel) 320 can be an algorithmic technique for explaining the input contribution (importance propagation explanation and perturbation-based explanation) of the artificial intelligence model or for explaining the internal operating principle (adversarial boundary explanation, behavior explanation, and example-based explanation).

[0052] In addition, the explanation method 330 is a method for providing an explanation, and can be a method for explaining the global operating principle of the artificial intelligence model or the local operating principle of the input data. That is, the explanation method 330 is a method for providing an explanation of the more important inputs of the AI model or for calculating and explaining the local contributions of each input around a specific input.

[0053] In addition, the explanation modality 340 can provide an explanation interface suitable for different data formats for vision, language, speech, time series, and behavior.

[0054] The explanation target (XAI target) 310, the explanation technique (explanation kernel) 320, the explanation method 330, and the explanation modality 340 as described above can provide explanations related to application fields such as medicine, law, manufacturing, transportation, and finance.

[0055] Meanwhile, when the artificial intelligence model to be explained is input, the plug-and-play manager identifies the manufacturer, framework, and type of the input artificial intelligence model (step S202). The identification of the manufacturer, framework, and type of the input artificial intelligence model by the plug-and-play manager will be described later. Here, an interface can be configured between the artificial intelligence model and the explanation module for explaining the input of the artificial intelligence model, and an interface can be configured between the artificial intelligence model and the explanation module for explaining the interior of the artificial intelligence model.

[0056] At this time, the explanation module can provide global explanation (considering the statistical contribution of the entire data) values and local explanation (contribution to specific input data) values of the input variables and internal variables of the artificial intelligence model, so as to explain the decision of the artificial intelligence model to the interface.

[0057] As described above, after identifying the manufacturer, framework, and type of the input artificial intelligence model, the plug-and-play manager considers the constraints on the explanation module to create a list of available explanation modules based on the identification result (step S203).

[0058] Then, the service provider server 110 provides the user terminal 120 with key information about the explanation modules in the list of available explanation modules (step S204). Here, the key information about the explanation modules in the list of available explanation modules may include the characteristics of each explanation module and the explanation methods and interfaces supported by each explanation module.

[0059] When the key information about the explanation modules in the list of available explanation modules is provided from the service provider server 110 to the user terminal 120, the user terminal 110 selects one explanation module from the explanation modules in the list with reference to the key information and notifies the service provider server 110 of the selected explanation module (step S205).

[0060] Therefore, the plug-and-play manager in the service provider server 110 obtains the explanation module selected by the user terminal 120 from the explanation modules in the list (step S206).

[0061] Then, the obtained explanation module provides an explanation of the input artificial intelligence model to the user terminal (step S207). Here, when the obtained explanation module provides an explanation of the input artificial intelligence model, the explanation module can provide a local explanation based on individual data and a global explanation based on verification data.

[0062] Here, the local explanation may include input contribution explanation and counterfactual example explanation, and the global explanation may include feature importance explanation.

[0063] Figure 4a 、 Figure 4b 、 Figure 4c and Figure 4d are diagrams schematically showing representative algorithms of explainable artificial intelligence for implementing the method of the present invention.

[0064] Referring to Figure 4a the algorithm for explaining the input contribution explains which part of the dog image is used to determine a dog when the dog image is input.

[0065] Referring to Figure 4b, the interpretation algorithm of decision trees widely used in financial credit rating models, etc., simplifies the decision model to interpret it as a single decision model, or extracts and interprets the changed decision through counterfactual inference for changing several variables.

[0066] Refer to Figure 4c , if it is decided to buy stocks / cryptocurrencies by looking at the trend of time series data, the example-based interpretation algorithm provides representative trend patterns for the nodes inside the deep learning model to make that decision, thus helping users understand the decision-making process.

[0067] Refer to Figure 4d , the behavior interpretation algorithm widely used in reinforcement learning extracts program code or state machines that can be understood by humans from the reinforcement learning model that has learned the behavior and interprets it.

[0068] Figure 5 FIG. is a diagram showing a sequence operation between a user terminal and a plug-and-play manager related to a method of providing an interpretation of an artificial intelligence model based on a plug-and-play mode according to the present invention.

[0069] Figure 5 Shows a process of providing an interpretation of a user-customized plug-and-play AI model. Refer to Figure 5 , when an interpretation of an artificial intelligence model (AI model) (e.g., Open AI GPT-3) input from a user terminal is requested, the plug-and-play manager (PnP XAI manager) identifies the AI model. For example, the plug-and-play manager identifies that the input AI model is a model using the GPT-3 structure in language generation models.

[0070] Thereafter, the plug-and-play manager queries the manufacturer, language, and type of the AI model. Then, the user terminal provides the features of the artificial intelligence model (AI model) as an answer to the query. For example, the user terminal provides the features of the artificial intelligence model (AI model) by responding "This is a model developed by Open AI based on transformers in Python, and the model structure is..."

[0071] Therefore, the plug-and-play manager matches the input artificial intelligence model (AI model) with an interpretation module (XAI module), creates a list of interpretation modules (XAI modules) that can interpret the input artificial intelligence model (AI model), and provides the list to the user terminal, and the user terminal selects an interpretation module (XAI module) from the list of interpretation modules (XAI modules) and notifies the plug-and-play manager of the selected interpretation module.

[0072] Then, the Plug and Play manager obtains the interpretation module selected by the user terminal from the list of interpretation modules, and the obtained interpretation module provides an interpretation of the input artificial intelligence model. Here, when the obtained interpretation module provides an interpretation of the input artificial intelligence model, the interpretation module can provide a global interpretation and a local interpretation. In other words, the obtained interpretation module can provide a global feature importance interpretation of the input artificial intelligence model (GPT-3), and provide a local input contribution interpretation and a counterfactual example interpretation.

[0073] Figure 6 FIG. is a diagram showing a mechanism of a user-customized Plug and Play artificial intelligence model interpretation interface selected for implementing the method of the present invention.

[0074] Referring to Figure 6 , the explainable artificial intelligence (XAI) interface is divided into a decision interpretation method area and an AI model modality area, and the decision interpretation method area is further divided into a global interpretation area and a local interpretation area.

[0075] As Figure 5 described above, an interpretation module that can interpret the input artificial intelligence model is obtained, and the obtained interpretation module provides an interpretation of the input artificial intelligence model. Here, if the user (user terminal) selects "feature importance interpretation" in the global interpretation area, "input contribution interpretation" and "counterfactual example interpretation" in the local interpretation area, and provides the selected interpretations to the service provider server, the Plug and Play manager in the service provider server will provide the selected items to the interpretation module (the interpretation module obtained for interpreting the artificial intelligence model), and the interpretation module provides a global feature importance interpretation, a local input contribution interpretation, and a counterfactual example interpretation of the input artificial intelligence model (GPT-3) based on the selected items.

[0076] As described above, the method for providing an interpretation of an artificial intelligence model based on the Plug and Play mode according to the present invention provides a general interpretation of various artificial intelligence models, automatically finds an interpretation module for interpreting a given artificial intelligence model in the Plug and Play mode, and thus can provide an interpretation in a manner that is easy for users to understand.

[0077] In addition, by identifying the characteristics of various artificial intelligence models, automatically finding an algorithm capable of interpreting the artificial intelligence model, and enabling the user to select the format and content that can be interpreted, a customized interpretation can be provided to the user.

Claims

1. A method for providing an explanation of an artificial intelligence model based on a plug-and-play mode, the method being based on a system for providing an explanation of an artificial intelligence model based on a plug-and-play mode, the system comprising: A service provider server, which is equipped with an explanation module for providing an explanation of an artificial intelligence model and a plug-and-play manager for connecting the artificial intelligence model and the explanation module, and the service provider server provides an explanation of the artificial intelligence model; And a user terminal, which is configured to communicate with the service provider server via the Internet and receive an explanation service for the artificial intelligence model from the service provider server. The method includes: a) A step in which a controller of the service provider server sets the type of artificial intelligence model that can be explained by each explanation module for communication between the artificial intelligence model and the explanation module; b) A step in which the plug-and-play manager identifies the manufacturer, framework, and type of the artificial intelligence model when the artificial intelligence model to be explained is input; c) A step in which the plug-and-play manager creates a list of available explanation modules based on the identification result considering the constraints on the explanation modules; d) A step in which the service provider server provides the user terminal with key information about the explanation modules in the list of available explanation modules; e) A step in which the user terminal selects one of the explanation modules in the list of explanation modules with reference to the key information about the explanation modules in the list and notifies the service provider server of the selected explanation module; f) A step in which the plug-and-play manager obtains the explanation module selected by the user terminal from the list of explanation modules; and g) A step in which the obtained explanation module provides the user terminal with an explanation of the input artificial intelligence model.

2. The method according to claim 1, wherein The plug-and-play manager has a list of explanation modules pre-provided by developers of explainable artificial intelligence, and the explanation module has information about the type of explainable artificial intelligence model.

3. The method according to claim 1, wherein, In step a), for communication between the artificial intelligence model and the explanation module, the artificial intelligence model is divided into an explainable artificial intelligence explanation target, an explanation technique (explanation kernel), an explanation method, and an explanation modality.

4. The method according to claim 3, wherein The explanation technique (explanation kernel) is divided into an explanation module for explaining the input contribution of the artificial intelligence model and an explanation module for explaining the inside of the artificial intelligence model.

5. The method according to claim 3, wherein, The explanation target is an AI model including at least one of a convolutional neural network (CNN / DNN), a recurrent neural network (RNN / LSTM), a transformer, a decision tree, a Bayesian, or a finite state machine (FSM), or an AI model including a combination of at least two of CNN / DNN, RNN / LSTM, a transformer, a decision tree, a Bayesian, or FSM.

6. The method according to claim 3, wherein The explanation modality is an explanation interface suitable for different data formats for vision, language, speech time series, and behavior.

7. The method according to claim 1, wherein Step b) includes: configuring an interface between the artificial intelligence model and an explanation module for explaining the input of the artificial intelligence model, and configuring an interface between the artificial intelligence model and an explanation module for explaining the inside of the artificial intelligence model.

8. The method according to claim 7, wherein The explained module provides global interpretation (considering the statistical contributions of the entire data) values and local interpretation (contributions to specific input data) values of the input variables and internal variables of the artificial intelligence model, so as to explain the decisions of the artificial intelligence model to the interface.

9. The method according to claim 1, wherein In step d), the key information about the explained modules in the list of available explained modules includes the features of each explained module and the interpretation methods and interfaces supported by each explained module.

10. The method according to claim 1, wherein, In step g), when the obtained explained module provides an interpretation of the input artificial intelligence model, the obtained explained module provides local interpretations based on individual data and global interpretations based on validation data.

11. The method according to claim 10, wherein, The local interpretations include input contribution interpretations and counterfactual example interpretations, while the global interpretations include feature importance interpretations.

Images