Method and apparatus for automatic generation of explanations for algorithmic predictions

Abstract

The present disclosure provides a method and apparatus for automatic generation of explanations for algorithmic predictions, automatically generating an explanation for a decision prediction from a machine learning algorithm includes running a machine learning algorithm using one or more input data using a first processor of a computing device; generating a decision prediction output based on the one or more input data; accessing the decision prediction output of the first processor using a second processor; generating additional information identifying one or more causal relationships between the prediction of the first algorithm and the one or more input data; and providing the additional information as an explanation in a user-understandable format on a display of the computing device.

Description

Technical Field

[0001] The aspects of the disclosed embodiments generally relate to machine learning systems, and more specifically to generating interpretations of predictions made by machine learning systems. Background Technology

[0002] As machine learning is increasingly used in fields ranging from security to medicine, the transparency and interpretability of algorithms used for decision prediction are crucial, as this directly impacts end-user trust in the algorithm and, consequently, trust in the algorithm itself. Currently, most systems and applications present decision predictions without any explanation.

[0003] Some systems and applications use explainable artificial intelligence (xAI) methods to explain the predictions of their corresponding algorithms. Recent xAI methods attempt to explain the decision-making reasoning process by using visualizations depicting the correlation between input pixels (or low-level features) and the final output. However, these methods have some key limitations.

[0004] First, the resulting explanations are limited to low-level relations and do not provide in-depth reasoning for model inference. Second, these methods lack a systematic process for verifying the reliability of the proposed model explanations. Finally, these methods do not provide guidance on how to correct errors caused by the original model. It would be advantageous to be able to receive human-understandable explanations of the causes or reasoning underlying decision predictions or other outputs generated by machine learning algorithms.

[0005] Therefore, it is desirable to provide methods and devices for solving at least some of the aforementioned problems. Summary of the Invention

[0006] The disclosed embodiments relate to methods, apparatus, and systems for automatically generating interpretations of decision predictions generated by machine learning algorithms. This and other advantages of the disclosed embodiments are provided substantially as shown in at least one drawing and / or described in conjunction with that drawing, as set forth in the independent claims. Further advantageous modifications may be found in the dependent claims.

[0007] According to a first aspect, the disclosed embodiments provide a method for generating an explanation of predictions generated by a machine learning algorithm, the method comprising: using a first hardware processor to run the machine learning algorithm using input data; generating a prediction output based on one or more input data; using a second hardware processor to access the prediction output of the first algorithm and the input data; and generating additional information revealing one or more causal relationships between the prediction output of the first algorithm and the input data. The aspects of the disclosed embodiments provide a human-understandable explanation of the reasoning behind decision predictions made by a machine learning process. In addition to providing a deep understanding of the reasoning processes used by the machine learning algorithm and precise causal relationships, the aspects of the disclosed embodiments can also help diagnose errors in the original model and improve the performance of the machine learning algorithm.

[0008] In possible implementations, generating additional information includes: identifying basic concepts in the input data; constructing a representation of the object of interest using the identified basic concepts and the relationship between the object and the object; calculating the correlation between the decision prediction output and the various components in the representation; converting the calculated correlation into a causal importance score; and presenting a visualization of the causal importance score on a user interface of a computing device. The aspects of the disclosed embodiments relate to identifying causal relationships between input data and prediction outputs of machine learning algorithms, and enable visualization of these relationships in a human-understandable manner.

[0009] In possible implementations, machine learning algorithms include one or more of mathematical formulas, statistical models, machine learning models, or neural networks.

[0010] In a possible implementation, the second processor can access the machine learning algorithms run by the first processor. Accessing the machine learning algorithms run by the first processor will improve the reliability of the machine learning algorithms run by the second processor.

[0011] In a possible implementation, the second processor cannot access the machine learning algorithms run by the first processor. Maintaining this separation protects the privacy of the machine learning algorithms run by the first processor.

[0012] In a possible implementation, the second processor may have access to one or more input data sets. Access to the input data of the machine learning algorithm run by the first processor allows the second processor to identify the correlations and relationships between the input data and the predicted output data.

[0013] In a possible implementation, the second processor cannot access one or more input data sets. This protects the privacy of data accessed by the first processor. The second processor can operate given intermediate representations output from the first processor and provide interpretations given these intermediate representations.

[0014] In a possible implementation, causality can be the spatial correlation between the output of a machine learning algorithm and its input data. The spatial correlation can be understood by relying on the pixel locations of visual concepts.

[0015] In a possible implementation, causality can be the temporal correlation between the output of a machine learning algorithm and its input data. The temporal correlation of visual concepts can be understood by relying on their different timestamps.

[0016] In a possible implementation, causality is a structural representation of different components that share causal relationships with the input data or output of a machine learning algorithm. Typically, the correlation between each component and the output can be calculated to determine the causal relationship.

[0017] In a possible implementation, a computer-aided medical diagnostic system generates diagnostic-related predictions and simultaneously automatically provides reasoning or evidence to support the generated predictions.

[0018] In possible implementations, the quality control system generates a quality assessment of the product and simultaneously automatically provides reasoning or evidence to support that assessment. This could include identifying which part of the product failed a test and the severity of the failure.

[0019] In possible implementations, the generated causal relationships can be used to evaluate the performance of machine learning algorithms in the first process.

[0020] In possible implementations, the generated causal relationships can be used to identify defects, limitations, or other kinds of shortcomings of machine learning algorithms in the first process.

[0021] In a possible implementation, the generated causal relationships can be automatically corrected by another algorithm or manually by the user. This correction can then be used to improve the performance of the machine learning algorithm.

[0022] In possible implementations, the generated causal relationships can be used as constraints, guidance, supervision, or auxiliary information during the training of other machine learning algorithms.

[0023] In a possible implementation, the causal relationships generated by the second processor are based on imitation training of another machine learning algorithm that accesses a machine learning algorithm run by the first processor.

[0024] In possible implementations, the causal explanations generated by the second processor can be used to extend the applications, capabilities, or functions of machine learning algorithms run by the first processor.

[0025] According to a second aspect, the disclosed embodiments provide an apparatus for generating an explanation of predictions generated by a machine learning algorithm. In one embodiment, the apparatus includes a first processor configured to run the machine learning algorithm using one or more input data and generate a predictive output based on the one or more input data. A second processor is configured to access the predictive output of the first algorithm and generate additional information revealing one or more causal relationships between the predictive output of the first algorithm and the input data. The aspects of the disclosed embodiments provide a human-understandable explanation of the reasoning behind decision predictions made by a machine learning process. In addition to providing a deep understanding of the reasoning processes used by the machine learning algorithm and precise causal relationships, the aspects of the disclosed embodiments can also help diagnose errors in the original model and improve the performance of the machine learning algorithm.

[0026] According to a third aspect, the disclosed embodiments relate to a computer program product having a non-transitory computer-readable medium on which machine-readable instructions are stored, which, when executed by a computer, cause the computer to: use a first processor to run a machine learning algorithm using one or more input data; generate a predictive output based on the one or more input data; use a second processor to access the predictive output of the first algorithm; and generate additional information revealing one or more causal relationships between the predictive output of the first algorithm and the one or more input data.

[0027] These and other aspects, embodiments, and advantages of the exemplary embodiments will become apparent from the embodiments described herein in conjunction with the accompanying drawings. However, it should be understood that the specification and drawings are for illustrative purposes only and not as definitions of limitations on the disclosed invention, the limitations of which should be referenced to the appended claims. Further aspects and advantages of the invention will be set forth in the following description and will be apparent in part from the description, or may be learned by practice of the invention. Moreover, aspects and advantages of the invention can be realized and obtained by means and combinations particularly pointed out in the appended claims. Attached Figure Description

[0028] In the following detailed sections of this disclosure, aspects of the disclosed embodiments will be described in more detail with reference to exemplary embodiments illustrated in the accompanying drawings, in which:

[0029] Figure 1 This is a block diagram of an exemplary device according to aspects of the disclosed embodiments.

[0030] Figure 2This is a diagram illustrating an exemplary workflow according to aspects of the disclosed embodiments.

[0031] Figure 3 This is a diagram illustrating an exemplary workflow that incorporates aspects of the disclosed embodiments.

[0032] Figure 4 This is a block diagram of exemplary components of a computing device according to aspects of the disclosed embodiments. Detailed Implementation

[0033] The following detailed description illustrates exemplary aspects of the disclosed embodiments and how they can be implemented. Although some modes of performing aspects of the disclosed embodiments have been disclosed, those skilled in the art will recognize that other embodiments for performing or practicing aspects of the disclosed embodiments are also possible.

[0034] See Figure 1 The illustration depicts an apparatus 100 for generating explanations of predictions produced by a machine learning model or algorithm. In one embodiment, apparatus 100 includes, for example, a computing device configured to run or execute one or more machine learning algorithms. For the purposes described herein, machine learning algorithms, models, or processes will generally be referred to as machine learning models. Aspects of the disclosed embodiments are intended to provide human-understandable explanations or visualizations of the reasoning behind decision predictions made by machine learning models. In addition to providing a deep understanding of the reasoning processes used by machine learning models and accurate causal relationships, aspects of the disclosed embodiments can also help diagnose errors in the original model and improve the model's performance.

[0035] like Figure 1 For example, device 100 includes a machine learning module 104. In one embodiment, the machine learning module 104, which includes a machine learning algorithm, is typically configured to receive input data 102, referred to in this example as input data 1 through input data n. The machine learning module 104 will compute an output 106 based on the input 102. In this example, the output 106 is referred to as a decision prediction, and... Figure 1 The outputs are referred to as outputs 1 through n. As will generally be understood, output 106 will typically include decision predictions from the algorithm running in machine learning module 104 based on input data 102.

[0036] Device 100 also includes a predictive explanation module 108. Predictive explanation module 108 is typically configured to access input 102, output 106, and generate additional information revealing one or more causal relationships between output 106 and input 102. Explanation output 110 is the additional information from predictive explanation module 108 and identifies causal or structural relationships in the input data 102 to explain the reasoning behind output 106. Explanation output 110 is presented in a human-understandable manner.

[0037] Figure 2 This is a block diagram of an exemplary device 200 for generating interpretations of algorithmic predictions according to aspects of the disclosed embodiments. Device 200 typically includes a first processor 202 and a second processor 204. Although the first processor 202 and the second processor 204 are described herein, aspects of the disclosed embodiments are not limited thereto. In alternative embodiments, the first processor 202 and the second processor 204 may include a single processor or processing device, or be part of the same computing device. In alternative embodiments, the first processor 202 and the second processor 204 may be on different computing devices. In one embodiment, the first processor 202 and the second processor 204 include hardware processors.

[0038] See also Figure 1 The machine learning module 104 will typically include or otherwise couple to the first processor 202. The first processor 202 is configured to run the machine learning algorithms of the machine learning module 104 using the input data 102. The first processor 202 will implement the machine learning algorithms of the machine learning module 104 and generate model inferences or outputs 106.

[0039] Moreover, see Figure 2 In one embodiment, the prediction interpretation module 108 includes or is coupled to a second processor 204. For example, in one embodiment, the second processor 204 is configured to run the algorithm of the prediction interpretation module 108 by at least accessing the output 106 and input 102 of the machine learning module 104, and to generate an interpretation output 110. The interpretation output 110 will identify or reveal one or more causal relationships between the prediction output 106 of the first machine learning module 104 and the input data 102.

[0040] Figure 3 An example of a workflow incorporating aspects of a disclosed embodiment is illustrated. In this example, by Figure 1 The machine learning module 104 has implemented a machine learning algorithm that has generated output 106. The prediction and interpretation module 108 receives this output, also referred to as "prediction" or "decision prediction", 302.

[0041] The comparison or analysis 304 is performed relative to the input data 102, or the predictive decision output 106 from the machine learning module 104 is analyzed. In one embodiment, the analysis or comparison 304 includes: generating gradients from the predictive decision output 106 with respect to the various visual concepts in the input data 102, and ranking the importance of these visual concepts by the values ​​of the gradients.

[0042] Based on comparison 304, the relationship between the input data and the output prediction data is identified 306. The relationship data can be presented to the user in a human-understandable manner 308. For example, in one embodiment, an explanation of the reasoning leading to the predictive decision 106 can be presented, such as on a user interface of a computing device.

[0043] Figure 4 A flowchart illustrating an exemplary method 400 incorporated into aspects of a disclosed embodiment is provided. In this example, method 400 aims to generate additional information to identify one or more causal relationships between a predicted output 106 of a machine learning algorithm 104 and input data 102. In one embodiment, identification 402 identifies fundamental concepts in the input data 102. Fundamental concepts can generally be considered as visual concepts extracted from the input data. For example, consider an image. In this example, visual concepts would typically include groups of pixels representing objects of interest. A first processor classifies an image with a Jeep (object of interest) in the foreground as "Jeep". Fundamental concepts can be groups of image pixels (parts / components of a foreground object in the input image) that include, for example, the Jeep logo on the car, wheels and / or windows, and other aspects of the car.

[0044] Representations of 404 objects of interest are constructed using basic concepts and relationships between them. As explained above, for an image labeled or classified into a category (e.g., a Jeep), the image itself may contain unrelated background objects (e.g., people, roads, etc.). In this example, the object of interest refers to the Jeep itself. The term "representation" here refers to a feature vector projected from a visual concept onto a learned feature space through inference performed by a machine / deep learning model. This feature vector can then be used to represent the visual concept in the learned feature space.

[0045] The correlation between the predicted output 106 (406) and the individual components in the representation is calculated, and the calculated correlation is converted into a causal importance score (408). For example, the correlation between the input concept representation and the predicted output can be mathematically quantified as certain values ​​normalized between 0 and 1. A correlation value of 0 means no correlation (the input concept is neither helpful nor relevant to the model's prediction). A correlation value of 1 indicates a strong correlation. This correlation value can be calculated based on the gradient generated from the predicted output 106.

[0046] In one embodiment, the causal importance score of the transformation is configured to be visualized in a human-understandable manner. For example, in one embodiment, the causal importance score of the transformation is presented on a user interface of an associated computing device. Representations and basic concepts will correspond to each other (i.e., one-to-one correspondence). After generating causal importance scores for the various concept representations, these basic concepts can be shown to the user via the user interface of the computing device, where the causal importance scores decrease / increase. An actual order of presentation is not required. Instead, in one embodiment, concepts with corresponding causal importance scores can be presented.

[0047] See you again Figure 1 and Figure 2 In one embodiment, either alone or in combination with any one or more embodiments described herein, the second processor 204 running the algorithm of the prediction interpretation module 108 may access the machine learning algorithm run by the first processor 202 associated with the machine learning module 104.

[0048] In one embodiment, either alone or in combination with any one or more embodiments described herein, the second processor 204 running the algorithm of the prediction interpretation module 108 cannot access the machine learning algorithm of the machine learning module 104 running by the first processor 202. Thus, the operation and workflow of the first processor 202 will not affect the performance and workflow of the second processor 204. The separation between the first processor 202 and the second processor 204 also protects the privacy of the machine learning algorithm running by the first processor 202.

[0049] In one embodiment, the second processor 202 may access the input data 102, either alone or in combination with any one or more embodiments described herein. Access to the input data 102 allows the second processor 202 to directly compare the input data 102 and the output data 106 for analysis. This allows the importance of visual concepts to be ranked based on the values ​​of the gradients determined as described above.

[0050] In one embodiment, either alone or in combination with any one or more embodiments described herein, the second processor 202 cannot access the input data 102. In this example, the second processor 202 can only access an intermediate representation of the input 102. Otherwise, the evaluation is performed in a manner similar to that described above. This separation maintains the privacy of the input data for the machine learning algorithm 104.

[0051] In one embodiment, either alone or in combination with any one or more embodiments described herein, the causal relationship identified by the prediction interpretation module 108 is the spatial correlation between visual concepts in the input data 102. Spatial correlation can be used to identify patterns or other important visual concepts or cues in the input data used to form a decision.

[0052] For example, a given number of images belong to different categories. The input images are segmented into groups of pixels, also referred to herein as "visual concepts." These pixel groups are fed into a learning machine / deep model, such as the model implemented by the prediction interpretation module 108. By ranking the scores corresponding to each visual concept, the top "n" visual concepts can be selected based on the output of the machine / deep model. This ranking is presented by the interpretation output 110.

[0053] For example, visual pattern recognition can be used to explain why an input image is a cat or not a dog. This can include identifying features associated with cats and features associated with dogs. The correlation between the visual aspects of each feature and the output data 106 is used to generate the explanatory output 110.

[0054] In one embodiment, either alone or in combination with any one or more embodiments described herein, the identified causal relationship may be a temporal correlation between output 106 and input data 102. For example, when identifying different activities such as basketball and tennis in video input, visual concepts such as hands and legs may be identified from video input data 102. Hands and legs in a basketball game will move differently over time than hands and legs in a tennis game. Thus, although hands and legs belong to the same set of visual concepts, different sets will move differently over time in different activities. Therefore, the temporal correlation or relationship will be different.

[0055] In one embodiment, either alone or in combination with any one or more embodiments described herein, the identified causal relationship can be a structural representation of different components sharing the causal relationship with input data 102 or output 106. As an example, when distinguishing between a fire truck and a car, a fire truck will have different components or parts than a car. These different parts or structural representations will provide different visual concepts. For example, a fire truck may typically be painted red. Furthermore, the windows, doors, and tires on a fire truck will have different sizes, shapes, and spatial relationships relative to similar parts on a car. The spatial relationships between these visual concepts can be used to generate interpretations.

[0056] As another example, the spatial distance between a car's side window and its front wheels will differ from the spatial distance between a fire truck's side window and its front wheels. While output 106 is a prediction about whether input 102 is a car or a fire truck, the interpretive output 110 can also include information about the spatial distance between the side window and the front wheels. For example, if it is determined that this spatial distance exceeds a predetermined distance, this determination can be associated with the fire truck prediction output 106. Thus, the interpretive output 110 can include information such as the determined color, the determined tire size, and the determined spatial distance to explain the underlying decision logic of the machine learning module 104 in generating output 106.

[0057] One possible implementation of device 100 is in a computer-aided medical diagnostic system. As an example, a computer-aided medical diagnostic system generates diagnostic-related predictions. When device 100 is implemented in such a system, it can provide reasoning or evidence to support the generated diagnostic-related predictions.

[0058] Another example of a possible implementation of device 100 is in a quality control system. For example, a quality control system can be configured to generate a quality assessment of a product. By implementing device 100 in such a system, reasoning or evidence supporting the generated assessment can also be provided. For example, this could also include identifying which part of the product failed the test and the severity of the failure.

[0059] In one embodiment, the identified causal relationships, either alone or in combination with any one or more embodiments described herein, can be used to evaluate the performance of a machine learning algorithm in a first process. By revealing or explaining the algorithm's reasoning logic, a user can visually verify whether the output is consistent with their understanding. If inconsistent, it is necessary to determine whether the inconsistency is due to misunderstandings or incorrect assumptions in the algorithm. The user can then rate the algorithm's performance.

[0060] In one embodiment, either alone or in combination with any one or more embodiments described herein, the identified causal relationship can be used to identify defects, limitations, or other kinds of shortcomings of the machine learning algorithm in a first process. For example, a top visual concept of a fire truck may be wheels. However, other types of trucks also have wheels, which may look similar to those on a fire truck. This potentially indicates that the machine learning module 104 may confuse different trucks with fire trucks. Thus, if the top identifier of a fire truck is wheels, and an image of a truck with similar wheels is input 102, then output 106 will most likely predict input image 102 as a fire truck, which is an incorrect or inaccurate prediction.

[0061] In one embodiment, the identified causal relationship, either alone or in combination with any one or more embodiments described herein, can be automatically corrected by another algorithm or manually by the user. Typically, this will occur when output 106 is an incorrect prediction based on input 104. This correction can then be used to improve the performance of the machine learning algorithm. For example, a user identifies a visual concept in the representation of an input image that should not be associated with a category of interest. This visual concept may potentially cause confusion when the algorithm of machine learning module 104 generates predictions using unseen images. In this case, the user can remove the visual concept from the representation and transfer the knowledge back to the algorithm, for example, through knowledge distillation.

[0062] In one embodiment, either alone or in combination with any one or more embodiments described herein, the identified causal relationships can be used as constraints, guidance, supervision, or auxiliary information in the training of other machine learning algorithms. After the second processor 204 learns the inference logic from or executed by the machine learning algorithm 104 running on the first processor 202, the knowledge and representation of the categories of interest are agnostic to the underlying algorithm. Thus, machine learning algorithms such as machine learning algorithm 104 can be transferred to or used to train other different algorithms with the same task objective.

[0063] In one embodiment, either alone or in combination with any one or more embodiments described herein, the causal relationships generated by the second processor 204 are based on imitation training of another machine learning algorithm that accesses the machine learning algorithm in the first processor 202. This means that, given the same input 102, the second processor 204 has learned to generate predictions that are the same as or very similar to those of the machine learning algorithm 104 running by the first processor 202.

[0064] In one embodiment, either alone or in combination with any one or more embodiments described herein, the causal explanations generated by the second processor 204 can be used to extend the application capabilities or functionality of the machine learning algorithm 104 run by the first processor 202. In the presence of multiple algorithms targeting different tasks, the second processor 204 can learn inference logic from multiple algorithms and transfer the accumulated knowledge back to the single algorithm 104 run by the first processor 202. This learning and training process will extend the capabilities of the machine learning algorithm 104. For example, initially, the machine learning algorithm 104 is trained and can recognize ten different object categories. After the second processor 204 has learned the inference logic, this information can be used by the machine learning algorithm 104, enabling it to recognize more object categories, such as twenty.

[0065] Since concept maps are used to represent different categories, existing concept maps can be adapted, for example, by removing or replacing certain concepts or removing or editing edges, to represent new objects. As an example, by editing a concept map for buses, a user can create a concept map for fire trucks. In this way, the user can define the representation of a new object, which can then be "refined" and transferred to machine learning algorithm 104, allowing the algorithm to learn to encode these objects.

[0066] See you again Figure 2Processors 202 and 204 (generally referred to herein as "processors" for ease of explanation) typically include appropriate logic, circuitry, interfaces, and / or code configured to process data provided as inputs (such as input 102 and output 106). The processors are configured to respond to and process instructions from the driving device 100. Examples of processors 202 and 204 may include, but are not limited to, microprocessors, microcontrollers, complex instruction set computing (CISC) microprocessors, reduced instruction set computing (RISC) microprocessors, very long instruction word (VLIW) microprocessors, or any other type of processing circuitry. Optionally, processors 202 and 204 may be one or more separate processors, processing devices, and various elements associated with the processing devices that can be shared by other processing devices. Additionally, one or more separate processors, processing devices, and elements are arranged in various architectures to respond to and process instructions from the driving device 100.

[0067] In one aspect, the disclosed embodiments include a training phase and an operation phase. During the training phase, the predictive interpretation module 108 is trained using training data so that it can perform specific, intended functions during the operation phase. In one embodiment, the second processor 204 is configured to perform unsupervised or semi-supervised training of the predictive interpretation module 108 using the training data to obtain a trained predictive interpretation module 108. In the unsupervised training of the predictive interpretation module 108, unlabeled training data is used to train the predictive interpretation module 108. Furthermore, in the semi-supervised training of the predictive interpretation module 108, a smaller amount of labeled training data and a larger amount of unlabeled training data are used to train the predictive interpretation module 108.

[0068] Based on the training of the prediction interpretation module 108, a trained prediction interpretation module 108 is obtained for use during the operation phase of the device 100.

[0069] See you again Figure 2In one embodiment, network interface 208 may be configured to include or contain a medium through which machine learning module 104, prediction and interpretation module 108, and other connected devices can communicate with each other. The communication network, not shown, may be a wired or wireless communication network. Examples of suitable communication networks may include, but are not limited to, Wi-Fi networks, local area networks (LANs), wireless personal area networks (WPANs), wireless local area networks (WLANs), wireless wide area networks (WWANs), cloud networks, Long Term Evolution (LTE) networks, Common Old-Style Telephone Service (POTS), metropolitan area networks (MANs), and / or the Internet. The devices of system or device 100 may be configured to connect to a communication network according to various wired and wireless communication protocols. Examples of such wired and wireless communication protocols may include, but are not limited to, Transmission Control Protocol and Internet Protocol (TCP / IP), User Datagram Protocol (UDP), Hypertext Transfer Protocol (HTTP), File Transfer Protocol (FTP), ZigBee, EDGE, Infrared (IR), IEEE 802.11, 802.16, Long Term Evolution (LTE), Li-Fi, and / or other cellular communication protocols or Bluetooth (BT) communication protocols, including their variants.

[0070] See also Figure 2 In operation, processor 204 is configured to obtain output 106 from machine learning module 104. In one embodiment, processor 204 receives output 106 via communication network 508 or any other suitable communication connection. In one embodiment, processor 204 is configured to store the received output 106 in suitable memory 206 or other storage device of device 100.

[0071] Memory 206 may include appropriate logic, circuitry, interfaces, and / or code, and may be configured to store instructions executable by processors 202 and 204. Memory 206 is also configured to store the operating system and associated applications of device 100, including predictive interpretation module 108. Examples of implementations of memory 206 may include, but are not limited to, random access memory (RAM), read-only memory (ROM), hard disk drive (HDD), flash memory, and / or secure digital storage (SD) cards. Computer-readable storage media for providing non-transitory memory may include, but are not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof.

[0072] Network interface 208 includes appropriate logic, circuitry, and / or interfaces configured to communicate with one or more external devices, such as electronic devices (e.g., smartphones). Examples of network interface 208 may include, but are not limited to, radio frequency (RF) transceivers, antennas, telematics units, one or more amplifiers, one or more oscillators, digital signal processors, encoder-decoder (CODEC) chipsets, and / or user identity module (SIM) cards. Optionally, network interface 204 can communicate using various wired or wireless communication protocols.

[0073] The various embodiments and variations of the aforementioned device or system 100, with necessary modifications, are applicable to this method. The method described herein is computationally efficient and does not impose a processing burden on processors 202 and 204.

[0074] Modifications to embodiments of the disclosed embodiments described above are possible without departing from the scope of the aspects of the disclosed embodiments as defined by the appended claims. Expressions such as “comprising,” “incorporating,” “having,” and “are” are used to describe and claim aspects of the disclosed embodiments and are intended to be interpreted in a non-exclusive manner, allowing for the presence of additional items, components, or elements not explicitly described. References to the singular are also interpreted to refer to the plural.

[0075] Therefore, while the essential novel features of the invention applied to exemplary embodiments thereof have been shown, described, and pointed out, it should be understood that those skilled in the art may make various omissions, substitutions, and changes in the form and details of the illustrated devices and methods, as well as in their operation, without departing from the spirit and scope of the currently disclosed invention. Furthermore, it is clearly contemplated that all combinations of those elements that perform substantially the same function in substantially the same manner to achieve the same result are within the scope of the invention. Moreover, it should be recognized that structures and / or elements shown and / or described in connection with any disclosed form or embodiment of the invention may be incorporated into any other disclosed or described or suggested form or embodiment as a general matter of design choice. Therefore, the invention is intended to be limited only by the scope indicated by the appended claims.

Claims

1. A method for automatically generating explanations for decision predictions from machine learning algorithms, the method comprising: The machine learning algorithm is run using a first hardware processor of a computing device with one or more input data. Generate a decision prediction output based on the one or more input data; The decision prediction output of the first hardware processor is accessed using a second hardware processor. Generate additional information to identify one or more causal relationships between the decision prediction output of the machine learning algorithm and the one or more input data; as well as Identify basic concepts in the input data, which are visual concepts extracted from the input data; use the identified basic concepts and the relationship between the objects of interest to establish a representation of the objects of interest; Calculate the correlation between the decision prediction output and the various components in the representation; The calculated relevance is converted into a causal importance score; And a visualization of the causal importance score is presented on the user interface of the computing device; The additional information is provided on the display of the computing device in a user-understandable format as an explanation.

2. The method according to claim 1, wherein, The second hardware processor may have access to the machine learning algorithm run by the first hardware processor, or the second hardware processor may not have access to the machine learning algorithm run by the first hardware processor.

3. The method according to claim 1, wherein, The second hardware processor may access the one or more input data, or the second hardware processor may not access the one or more input data.

4. The method according to claim 1, wherein, The causal relationship includes the spatial correlation between the decision prediction output of the machine learning algorithm from the first hardware processor and the input data.

5. The method according to claim 1, wherein, The causal relationship includes the temporal correlation between the decision prediction output of the machine learning algorithm from the first hardware processor and the input data.

6. The method according to claim 1, wherein, The causal relationship includes a structural representation of different components that share the causal relationship with the input data of the machine learning algorithm or the decision prediction output.

7. The method according to claim 1, further comprising: The identified causal relationships are used to evaluate the performance of the machine learning algorithm in the first process.

8. An apparatus for automatically generating explanations for decision predictions from a machine learning algorithm, the apparatus comprising: A first hardware processor of a computing device is configured to run the machine learning algorithm using one or more input data and to generate a decision prediction output based on the one or more input data; A second hardware processor is configured to access the decision prediction output of the first hardware processor and generate additional information that identifies one or more causal relationships between the decision prediction output of the machine learning algorithm and the one or more input data. And identify basic concepts in the input data, the basic concepts being visual concepts extracted from the input data; use the identified basic concepts and the relationship between the objects of interest to establish a representation of the objects of interest; Calculate the correlation between the decision prediction output and the various components in the representation; The calculated relevance is converted into a causal importance score; The user interface is configured to provide the additional information as an explanation in a user-understandable format, and to present a visualization of the causal importance score.

9. A computer program product comprising a non-transitory computer-readable medium having machine-readable instructions stored thereon, the machine-readable instructions, when executed by a computer, causing the computer to generate an interpretation of decision predictions from a machine learning algorithm: The machine learning algorithm is run using a first hardware processor of a computing device with one or more input data. Generate a decision prediction output based on the one or more input data; The decision prediction output of the first hardware processor is accessed using a second hardware processor. Generate additional information to identify one or more causal relationships between the decision prediction output of the machine learning algorithm and the one or more input data; as well as Identify basic concepts in the input data, which are visual concepts extracted from the input data; use the identified basic concepts and the relationship between the objects of interest to establish a representation of the objects of interest; Calculate the correlation between the decision prediction output and the various components in the representation; The calculated relevance is converted into a causal importance score; And a visualization of the causal importance score is presented on the user interface of the computing device; The additional information is provided on the display of the computing device in a user-understandable format as an explanation.

Images