A plug-and-play based explanation method for artificial intelligence models

Abstract

The method for providing an explanation for an artificial intelligence model based on the plug-and-play method according to the present invention includes a step in which a service provider server sets the type of artificial intelligence model; when an artificial intelligence model is input, a step in which a plug-and-play manager recognizes the artificial intelligence model and creates a list of usable explanation modules based on the recognition results; a step in which the service provider server provides key information for each explanation module to a user terminal; a step in which the user terminal selects one of the explanation modules and provides it to the service provider server; a step in which the plug-and-play manager derives the selected explanation module from each explanation module in the list; and a step in which the derived explanation module provides an explanation for the input artificial intelligence model to the user terminal.

Description

[Technical Field]

[0001] The present invention relates to a method for providing an explanation for an artificial intelligence model, and more particularly to a method for providing an explanation for an artificial intelligence model based on a plug-and-play method, which can automatically search for a module that explains a given artificial intelligence model in a plug-and-play manner and provide an explanation in a manner that is easy for a user to understand. [Background technology]

[0002] In recent years, with the development of artificial intelligence, it has been actively applied to many fields, such as image recognition, speech understanding, dialogue systems, and autonomous driving. As such, artificial intelligence technology is being applied to many aspects of real life and is developing every day, but the structure of artificial intelligence models trained with data continues to become more complex, making it difficult to accurately understand their decision-making principles.

[0003] As mentioned above, when complex AI models are applied to mission-critical tasks that have a significant impact on human life and property, consumers may suffer damage as a result of using an imperfect AI model. To address this, the EU has been mandating the explainability of AI models through the General Data Protection Regulation (GDPR) since 2018. In recent years, the EU has strengthened its standards, classifying AI technologies related to human biometric signals, autonomous driving, personnel evaluations, and credit assessments as high-risk AI, and is further focusing on requiring the reliability and transparency of AI models.

[0004] Various methods have been developed to explain the decision-making of AI models. However, since the explanation must reflect the diverse characteristics of the AI ​​model, it is difficult for a single method to effectively explain the decision-making of all models. Therefore, algorithms that effectively explain the decision-making of each AI model are differentiated.

[0005] For example, there are different techniques for explaining artificial intelligence models with decision tree structures, deep learning models, and Bayesian-based probability models, and while there are techniques that provide general-purpose explanations, their accuracy is often low.

[0006] Deep learning frameworks such as Google's "TensorFlow" and Meta's "Pytorch" also explain deep learning using the "Integrated Gradient" method and "DeepLift" technology, which explain deep learning decision-making, but they only provide explanations for the deep learning models developed on each platform.

[0007] Meanwhile, Korean Patent Application Publication No. 10-2020-0092447 discloses an "Explainable AI Modeling and Simulation System and Method," which is characterized by including a step of designing an AI workflow model when an algorithm suitable for a workflow created and edited to suit a subject area is selected from among already stored algorithms; and a step of performing a simulation of the AI ​​workflow model based on the input information when input information is input.

[0008] The patent document has the advantages of being able to visualize the connection of algorithms on a workflow basis and automatically verify performance through integrated simulation, being able to simultaneously simulate multiple workflow-based AI algorithms and compare their performance, and being able to explain the cause / basis of the results of classification analysis using images and standardized shape data (Features) and suggest directions for improving algorithm performance. However, this is primarily a mechanism for selecting an algorithm suitable for a workflow created / edited to suit a subject area such as semiconductor and display manufacturing processes, and then designing and simulating a corresponding AI workflow model. However, this also only provides an explanation or simulation of an AI model designed under specific conditions, and therefore has the problem of not being able to provide a general explanation for various AI models. Summary of the Invention [Problem to be solved by the invention]

[0009] The present invention has been created by comprehensively taking into consideration the above points, and aims to provide a method for providing an explanation for an AI model based on a plug-and-play method, which can provide a general explanation for various AI models, automatically search for an explanation module that explains a given AI model in a plug-and-play manner, and provide an explanation in a manner that is easy for the user to understand.

[0010] Another object of the present invention is to provide a method for providing explainability for an AI model based on a plug-and-play method, which, when an explanation for an AI model is required, grasps the characteristics of various AI models, automatically finds an algorithm that can explain the AI ​​model, allows the user to select the format and content of the explanation, and provides a customized explanation. [Means for solving the problem]

[0011] In order to achieve the above object, the method for providing explanations for an artificial intelligence model based on a plug-and-play method according to the present invention includes:

[0012] A method for providing an explanation for an artificial intelligence model based on a system for providing an explanation for an artificial intelligence model based on a plug-and-play method, the system including: an explanation module for providing an explanation for the artificial intelligence model; a service provider server that is equipped with a plug-and-play manager that connects the artificial intelligence model and the explanation module and provides an explanation for the artificial intelligence model; and a user terminal that communicates with the service provider server via the Internet and receives an explanation service for the artificial intelligence model from the service provider server,

[0013] a) a step in which the control unit of the service provider server sets the type of artificial intelligence model that each explanation module can successfully explain for communication between the artificial intelligence model and the explanation module;

[0014] b) when an AI model requiring explanation is input, the plug and play manager recognizes the manufacturer, framework, and type of the AI ​​model;

[0015] c) the plug-and-play manager creating a list of available explanation modules based on the recognition result and taking into account constraints of the explanation modules;

[0016] d) the service provider server providing the user terminal with key points for each instruction module in the list of available instruction modules;

[0017] e) the user terminal refers to the main points of each instruction module in the list, selects one of the instruction modules in the list, and provides the selected one to the service provider server;

[0018] f) the plug and play manager deriving a description module selected by the user terminal from each description module in the list; and

[0019] g) the derived explanation module provides an explanation for the input artificial intelligence model to the user terminal.

[0020] Here, the plug and play manager may have a list of explanation modules provided in advance by developers of explainable artificial intelligence, and the explanation modules may have information on the types of explainable artificial intelligence models.

[0021] In addition, in step a), for communication between the artificial intelligence model and the explanation module, the artificial intelligence model can be classified into explainable artificial intelligence explanation objects, explanation techniques (explanation kernels), explanation methods, and explanation modals.

[0022] In this case, the explanation technique (explanation kernel) can be divided into an explanation module that explains the input contribution of the AI ​​model and an explanation module that explains the inside of the AI ​​model.

[0023] In this case, the object of explanation may be an AI model including at least one of a convolutional neural network (CNN / DNN), a recurrent neural network (RNN / LSTM), a transformer, a decision tree, a Bayesian, and a finite state machine (FSM), or an AI model consisting of a combination of at least two of these.

[0024] In this case, the explanation modal may be an explanation interface that is suitable for different data formats such as visual, linguistic, audio time series, and behavioral.

[0025] In addition, in step b), an interface can be configured between the artificial intelligence model and an explanation module to explain the input of the artificial intelligence model, and an interface can be configured between the artificial intelligence model and an explanation module to explain the inside of the artificial intelligence model.

[0026] In this case, the explanation module can provide the interface with global explanation (statistical contribution taking into account the entire data) and local explanation (contribution to specific input data) values ​​for the input variables and internal variables of the artificial intelligence model to explain the decision-making of the artificial intelligence model.

[0027] Also, in step d), the main details for each instruction module in the list of available instruction modules may include the features of each instruction module, the instruction methods and interfaces that the instruction module supports.

[0028] In addition, in step g), when the derived explanation module provides an explanation for the input artificial intelligence model, it can provide a local explanation based on individual data and a global explanation based on verification data.

[0029] In this case, the local explanation may include an input contribution explanation and a counterfactual example explanation, and the global explanation may include a feature importance explanation. [Effects of the Invention]

[0030] According to the present invention, various artificial intelligence models are generally explained, and an explanation module that explains a given artificial intelligence model is automatically searched for and explained in a plug-and-play manner, which has the advantage of providing an explanation in a manner that is easy for users to understand.

[0031] Another advantage is that it can grasp the characteristics of various AI models, automatically find algorithms that can explain the AI ​​models, and allow users to select the format and content of the explanation, allowing users to receive a customized explanation. [Brief explanation of the drawings]

[0032] [Figure 1] 1 is a diagram illustrating the configuration of an explanation providing system for a plug-and-play based artificial intelligence model constructed to implement an explanation providing method according to the present invention; [Figure 2] 1 is a flowchart illustrating a process of performing a method for providing explanations for a plug-and-play-based artificial intelligence model according to an embodiment of the present invention. [Figure 3] This diagram shows that artificial intelligence models are classified into explainable artificial intelligence explanation targets (XAI targets), explanation techniques (explanation kernels), explanation methods, and explanation modals. [Figure 4A] FIG. 1 is a diagrammatic representation of an exemplary explainable artificial intelligence algorithm employed to implement the method of the present invention. [Figure 4B] FIG. 1 is a diagrammatic representation of an exemplary explainable artificial intelligence algorithm employed to implement the method of the present invention. [Figure 4C] FIG. 1 is a diagrammatic representation of an exemplary explainable artificial intelligence algorithm employed to implement the method of the present invention. [Figure 4D] FIG. 1 is a diagrammatic representation of an exemplary explainable artificial intelligence algorithm employed to implement the method of the present invention. [Figure 5] 1 is a diagram illustrating a sequence operation between a user terminal and a plug-and-play manager related to a method for providing explanations for a plug-and-play based artificial intelligence model according to the present invention. [Figure 6] FIG. 1 is a diagram illustrating a customized plug-and-play artificial intelligence model explanation interface selection mechanism employed to implement the method of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0033] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0034] FIG. 1 is a diagram illustrating the configuration of a system for providing explanations for a plug-and-play based artificial intelligence model, which is constructed to implement the method for providing explanations for a plug-and-play based artificial intelligence model according to the present invention.

[0035] Referring to FIG. 1, a system 100 for providing explanations for a plug-and-play based artificial intelligence model, which is constructed to implement the method for providing explanations for a plug-and-play based artificial intelligence model according to the present invention, comprises a service provider server 110 and a user terminal 120.

[0036] The service provider server 110 basically includes a control unit and a memory, and the memory stores (is equipped with) an explanation module (which is an application as a kind of software program) for providing an explanation for the AI ​​model, and a plug-and-play manager (which is an application as a kind of software program) for connecting the AI ​​model and the explanation module, thereby providing an explanation for the AI ​​model. The service provider server 110 can be configured as a general desktop PC, and can include a database (DB) for storing various service-related data and information.

[0037] The user terminal 120 communicates with the service provider server 110 via the Internet and receives an explanation service for the AI ​​model from the service provider server 110. The user terminal 120 may include a mobile phone (smartphone) 120a, a notebook PC 120b, a desktop PC 120c, etc.

[0038] Hereinafter, a method for providing an explanation for an AI model based on the plug-and-play based explanation providing system for an AI model having the above configuration will be described.

[0039] FIG. 2 is a flowchart illustrating a process of providing an explanation for an AI model based on a plug-and-play method according to an embodiment of the present invention.

[0040] 2, the method for providing an explanation for an AI model based on a plug-and-play method according to the present invention is based on a system 100 for providing an explanation for an AI model based on a plug-and-play method, including a service provider server 110 that is equipped with an explanation module for providing an explanation for the AI ​​model, a plug-and-play manager that connects the AI ​​model and the explanation module and provides an explanation for the AI ​​model, and a user terminal 120 that receives an explanation service for the AI ​​model from the service provider server 110. First, a control unit (not shown) of the service provider server 110 sets the type of AI model that can be well explained by each explanation module for communication between the AI ​​model and the explanation module (step S201). Here, the plug-and-play manager may have a list of explanation modules provided in advance by a developer of the explainable AI, and the explanation module may have information on the type (manufacturer, format, language, etc.) of the AI ​​model that can be explained.

[0041] 3, the AI ​​model can be classified into an explainable AI explanation target (XAI target) 310, an explanation technique (explanation kernel) 320, an explanation method 330, and an explanation modal 340. In this case, the explanation technique (explanation kernel) 320 can be divided into an explanation module 320a that explains the input contribution of the AI ​​model and an explanation module 320b that explains the inside of the AI ​​model.

[0042] In this case, the object of explanation 310 may be an AI model including at least one of a convolutional neural network (CNN / DNN), a recurrent neural network (RNN / LSTM), a transformer, a decision tree, a Bayesian, and a finite state machine (FSM), or an AI model consisting of a combination of at least two or more of these.

[0043] In this case, the explanation technique (explanation kernel) 320 may be an algorithmic technique that explains the input contribution of the artificial intelligence model (importance propagation explanation, perturbation-based explanation) or explains the internal operating principles (adversarial boundary explanation, behavioral explanation, example-based explanation).

[0044] In this case, the explanation method 330 is a method for providing an explanation, and may be a method for explaining the global operating principle of an AI model or the local operating principle around input data. That is, it is a method for providing an explanation of which input is more important overall for an AI model, or a method for calculating and explaining how each input contributes locally around a specific input.

[0045] In this case, the explanation modal 340 can provide an explanation interface suitable for different data formats such as visual, verbal, audio time series, and behavior.

[0046] The above-described explanation object (XAI object) 310, explanation technology (explanation kernel) 320, explanation method 330, and explanation modal 340 can provide explanations related to application fields such as medicine, law, manufacturing, transportation, and finance.

[0047] Meanwhile, when an AI model requiring explanation is input, the Plug and Play Manager recognizes the manufacturer, framework, and type of the input AI model (step S202). Here, the Plug and Play Manager's recognition of the manufacturer, framework, and type of the input AI model will be described later. Here, an interface between the AI ​​model and an explanation module can be configured to explain the input of the AI ​​model, and an interface between the AI ​​model and an explanation module can be configured to explain the inside of the AI ​​model.

[0048] In this case, the explanation module can provide global explanation (statistical contribution taking into account the entire data) and local explanation (contribution to specific input data) values ​​for the input variables and internal variables of the artificial intelligence model to explain the decision-making of the artificial intelligence model to the interface.

[0049] After recognizing the manufacturer, framework, and type of the input artificial intelligence model as described above, the plug and play manager creates a list of available explanation modules based on the recognition results and taking into account the constraints of the explanation modules (step S203).

[0050] Thereafter, the service provider server 110 provides main details for each instruction module in the list of available instruction modules to the user terminal 120 (step S204). Here, the main details for each instruction module in the list of available instruction modules may include the features of each instruction module, an instruction method and an interface supported by the instruction module.

[0051] In this manner, when the service provider server 110 provides the user terminal 120 with the main points for each instruction module in the list of available instruction modules, the user terminal 120 refers to the main points for each instruction module in the list, selects one of the instruction modules on the list, and provides it to the service provider server 110 (step S205).

[0052] Accordingly, the plug and play manager in the service provider server 110 derives the instruction module selected by the user terminal 120 from each instruction module in the list (step S206).

[0053] Thereafter, the derived explanation module provides an explanation for the input artificial intelligence model to the user terminal (step S207). Here, when the derived explanation module provides an explanation for the input artificial intelligence model, it can provide a local explanation based on individual data and a global explanation based on verification data.

[0054] In this case, the local explanation may include an input contribution explanation and a counterfactual example explanation, and the global explanation may include a feature importance explanation.

[0055] FIG. 4 is a diagrammatic representation of a representative explainable artificial intelligence algorithm employed to implement the method of the present invention.

[0056] Referring to Figure 4a, this is an algorithm that explains input contributions, and as shown, when an image of a dog is input, it explains which parts of the image were looked at to determine that it was a dog.

[0057] Referring to Figure 4b, this is an explanation algorithm for decision trees, which is often used in financial credit assessment models, etc. It simplifies the decision model and explains it as a single decision model, or extracts and explains decisions that change through counterfactual reasoning that changes several variables.

[0058] Referring to Figure 4c, this is an example-based explanation algorithm that helps users understand when they decide to buy a stock / coin based on the trend of time-series data by providing a representative pattern of which trend pattern the node in the deep learning model saw and made that decision.

[0059] Referring to Figure 4d, this is a behavior explanation algorithm, which is often used in reinforcement learning, and it extracts and explains human-understandable program code, state machines, etc. from a reinforcement learning model that has learned behavior.

[0060] FIG. 5 is a diagram illustrating a sequence operation between a user terminal and a plug-and-play manager in connection with a method for providing an explanation for an artificial intelligence model based on a plug-and-play method according to the present invention.

[0061] Referring to Figure 5, this shows a process of providing an explanation for a customized plug-and-play AI model. When an explanation for an artificial intelligence model (AI model) (e.g., Open AI GPT-3) input from a user terminal is requested, the plug-and-play manager (PnP XAI manager) recognizes the AI ​​model and, for example, recognizes that it is a model using the GPT-3 structure among language generation models.

[0062] The Plug and Play Manager then asks which company, which language, and which model it is made of. The user terminal then provides the characteristics of the AI ​​model as a response to the question. For example, the user terminal may provide the characteristics of the AI ​​model by answering, "This is a model developed by Open AI in Python based on Transformer, and the model structure is ____."

[0063] Accordingly, the Plug and Play Manager matches the input artificial intelligence model (AI model) with the explanation module (XAI module), creates a list of explanation modules (XAI modules) that can explain the input artificial intelligence model (AI model), and provides this to the user terminal. The user terminal selects one explanation module (XAI module) from the list of explanation modules (XAI modules), and provides it to the Plug and Play Manager.

[0064] The plug and play manager then derives an explanation module selected by the user terminal from the list of explanation modules, and the derived explanation module provides an explanation for the input artificial intelligence model. At this time, when the derived explanation module provides an explanation for the input artificial intelligence model, it may provide global and local explanations. That is, the derived explanation module may provide a global feature importance explanation for the input artificial intelligence model (GPT-3), and locally provide an input contribution explanation and a counterfactual example explanation.

[0065] FIG. 6 is a diagram illustrating a mechanism for selecting a custom-made plug-and-play AI model explanation interface employed to implement the method of the present invention.

[0066] Referring to Figure 6, the explainable artificial intelligence (XAI) interface is divided into a decision-making explanation method domain and an AI model modal domain, and the decision-making explanation method domain is again divided into a global explanation domain and a local explanation domain.

[0067] As described above in Figure 5, an explanation module that can explain the input AI model is derived, and the derived explanation module provides an explanation for the input AI model. At this time, when a user (user terminal) selects "feature importance explanation" in the global explanation area and "input contribution explanation" and "counterfactual example explanation" in the local explanation area and provides these to the service provider server, the plug and play manager in the service provider server provides the selections to the explanation module (explanation module derived to explain the AI ​​model), and the explanation module provides a global feature importance explanation for the input AI model (GPT-3) based on the selections, and a local input contribution explanation and counterfactual example explanation.

[0068] As described above, the method for providing an explanation for an artificial intelligence model based on a plug-and-play method according to the present invention has the advantage of being able to provide an explanation in a manner that is easy for the user to understand, since it provides a general explanation for various artificial intelligence models and automatically searches for and explains an explanation module that explains a given artificial intelligence model in a plug-and-play manner.

[0069] In addition, the present invention has the advantage of grasping the characteristics of various artificial intelligence models, automatically searching for algorithms that can explain the artificial intelligence models, and allowing users to select the format and content of the explanation, thereby allowing users to receive a customized explanation.

Claims

1. A method for providing an explanation for an artificial intelligence model based on a system for providing an explanation for an artificial intelligence model based on a plug-and-play method, the system including: an explanation module for providing an explanation for the artificial intelligence model; a service provider server that is equipped with a plug-and-play manager that connects the artificial intelligence model and the explanation module and provides an explanation for the artificial intelligence model; and a user terminal that communicates with the service provider server via the Internet and receives an explanation service for the artificial intelligence model from the service provider server, a) a step in which the control unit of the service provider server sets the type of artificial intelligence model that each explanation module can successfully explain for communication between the artificial intelligence model and the explanation module; b) when an AI model requiring explanation is input, the plug and play manager recognizes the manufacturer, framework, and type of the AI ​​model; c) the plug-and-play manager creating a list of available explanation modules based on the recognition result and taking into account constraints of the explanation modules; d) the service provider server providing the user terminal with key points for each instruction module in the list of available instruction modules; e) the user terminal refers to the main points of each explanation module in the list, selects one of the explanation modules in the list, and provides the selected one to the service provider server; f) the plug and play manager deriving a description module selected by the user terminal from each description module in the list; and g) the derived explanation module provides an explanation for the input artificial intelligence model to the user terminal.

2. 2. The method of claim 1, wherein the plug-and-play manager has a list of explanation modules provided in advance by a developer of the explainable artificial intelligence, and the explanation modules have information on the types of explainable artificial intelligence models.

3. 2. The method for providing explanations for an artificial intelligence model based on a plug-and-play method as described in claim 1, characterized in that in step a), the artificial intelligence model is classified into an explainable artificial intelligence explanation object, explanation technology (explanation kernel), explanation method, and explanation modal for communication between the artificial intelligence model and the explanation module.

4. 4. The method for providing an explanation for an artificial intelligence model based on a plug-and-play method, as described in claim 3, characterized in that the explanation technology (explanation kernel) is divided into an explanation module that explains the input contribution of the artificial intelligence model and an explanation module that explains the internal structure of the artificial intelligence model.

5. 4. The method for providing an explanation for an artificial intelligence model based on a plug-and-play method according to claim 3, wherein the explanation target is an AI model including at least one of a convolutional neural network (CNN / DNN), a recurrent neural network (RNN / LSTM), a transformer, a decision tree, a Bayesian, and a finite state machine (FSM), or an AI model consisting of a combination of at least two or more of these.

6. 4. The method of claim 3, wherein the explanation modal is an explanation interface suitable for different data formats for visual, linguistic, audio time series, and behavioral data.

7. 2. The method for providing explanations for an artificial intelligence model based on a plug-and-play method according to claim 1, wherein in step b), an interface is configured between the artificial intelligence model and an explanation module to explain the input of the artificial intelligence model, and an interface is configured between the artificial intelligence model and the explanation module to explain the inside of the artificial intelligence model.

8. 8. The method for providing an explanation for an artificial intelligence model based on a plug-and-play method according to claim 7, wherein the explanation module provides global explanation (statistical contribution taking into account all data) and local explanation (contribution to specific input data) values ​​for the input variables and internal variables of the artificial intelligence model to the interface to explain the decision-making of the artificial intelligence model.

9. 2. The method for providing explanations for a plug-and-play based artificial intelligence model according to claim 1, wherein in step d), the main details for each explanation module in the list of available explanation modules include the features of each explanation module, the explanation methods and interfaces supported by the explanation module.

10. 2. The method for providing an explanation for an artificial intelligence model based on a plug-and-play method according to claim 1, wherein in step g), when the derived explanation module provides an explanation for the input artificial intelligence model, it provides a local explanation based on individual data and a global explanation based on verification data.

11. 11. The method of claim 10, wherein the local explanations include input contribution explanations and counterfactual example explanations, and the global explanations include feature importance explanations.

Images