Counterfactual-based attribution activation map guided explainable method system

Abstract

The present application relates to a kind of based on counterfactual attribution activation map guided explainable method system, for the image target detection deep learning model that has been trained, using the model to obtain the prediction result of image prediction, according to the prediction result using Grad-CAM method obtains the attribution heat activation map of detection result on original image to show which area is important, combined with counterfactual method answers these different important degree area specifically what kind of role is played.By counterfactual explanation module, the key area feature is inactivated again input model prediction result, according to the IoU threshold between the prediction frame of the prediction result generated again and the original prediction frame is best matched and compared with the change of two explain selection area specifically what kind of role is played, enhance the credibility of model, can further assist doctor to judge illness and help model researcher to optimize iteration model.

Description

Technical Field

[0001] This invention relates to the field of interpretable artificial intelligence, and in particular to an interpretable method system based on counterfactual attribution activation graphs for image object detection models. Background Technology

[0002] Deep learning-based image object detection technology has been widely used in applications such as facial recognition, autonomous driving, smart healthcare, smart homes, and smart security. However, most existing convolutional deep learning-based image object detection methods focus solely on model performance metrics without recognizing that a deep learning model is a black box, neglecting the features upon which the model arrives at its predictions. Therefore, research on model interpretability is crucial. This is significant for users to better trust the model's predictions and for developers to continuously optimize the model based on its weaknesses. Especially in the highly sensitive field of medical image object detection, providing reasonable explanations for the model's predictions is essential. This not only provides doctors and patients with important diagnostic criteria but also significantly enhances the model's credibility. Most deep learning object detection methods simply provide the results without explaining why or what criteria were used to arrive at those results.

[0003] Counterfactual interpretation explains the changes that occur after removing certain features using controlled variables, thus revealing the specific role those features played. In image object detection, automatic segmentation algorithms are often used to divide the image into several small parts, which are then masked one by one and the results are judged. This process is undoubtedly inefficient, and the selected regions lack theoretical support. In particular, for image object detection tasks, there are often multiple targets in the image, making traversal-based counterfactual interpretation methods even more time-consuming. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide an attribution activation graph-guided counterfactual interpretable method for image object detection models, which can quickly generate corresponding interpretation results based on user input requirements and is applicable to object detection models for any task.

[0005] A counterfactual interpretability method guided by attribution activation graphs for image object detection models, the method comprising:

[0006] The model prediction module obtains the model's prediction results for the image;

[0007] The attribution heatmap mapping module iterates through the prediction results and obtains the attribution heatmap corresponding to the prediction category using the Grad-CAM method, and forms a heatmap mask that is mapped to the current prediction box area.

[0008] The key region selection module independently assigns a weight percentile to each pixel in each predicted box region, and obtains the key region based on the upper and lower bounds of the percentiles input by the user.

[0009] The counterfactual explanation module deactivates the key region features and re-inputs them into the model prediction results. It then performs the best match based on the IoU threshold between the regenerated prediction results and the original prediction results, and compares the changes between the two to explain the specific role played by the selected region.

[0010] The specific operation process of the model prediction module is as follows:

[0011] S11 Input the image to be predicted;

[0012] S12 transforms the size of the input image to the input size of the prediction model;

[0013] S13 uses the prediction results of a fully trained image object detection model to obtain the detection box and its classification result.

[0014] The specific operation process of the attribution heat activation map mapping module is as follows:

[0015] S21 Input all predicted bounding boxes and their categories for the predicted image;

[0016] S22 iterates through these prediction boxes and uses the Grad-CAM method to obtain the heat map of the current prediction box category, and then extracts a portion of the prediction box to obtain the activation map of the prediction box.

[0017] S23 merges the activation maps on these prediction boxes and maps them onto the original image to obtain an image corresponding to the thermal activation map mapping of the current model.

[0018] The specific operation process of the key area selection module is as follows:

[0019] S31 Input the percentile interval [L,R] of the region to be interpreted;

[0020] Based on the feature map mapping result obtained in S23, S32 assigns a percentile to each pixel in each prediction box independently within the prediction box.

[0021] S33 obtains the set of pixels corresponding to the percentiles in the interval [L, R];

[0022] S34 first extracts the content of these pixels in the original image to obtain a sub-image of the selected region, and assigns the RGB values ​​of the pixels in the obtained pixel set to 0, thus rendering these pixels ineffective, to obtain a counterfactual interpretation image.

[0023] The specific operation process of the counterfactual interpretation module is as follows:

[0024] S41 uses the model prediction module to predict the counterfactual interpretation map generated by S34 to obtain a new prediction box and its prediction category;

[0025] S42 compares the IoU results to match the original image prediction box and the counterfactual interpretation image prediction box;

[0026] S43 explains the effect of the selected region on the changes in the matching results through three aspects: changes in confidence, changes in classification results, and changes in the predicted bounding box.

[0027] The attribution heatmap represented by the Grad-CAM method in this invention can backpropagate the gradient from the last layer of the network to calculate the feature attribution value of each feature point on the original image through the network output, so as to measure the contribution of each feature to the model prediction result.

[0028] This invention targets a pre-trained deep learning model for image object detection. Using this model, predictions are obtained from images. Based on the predictions, the Grad-CAM method is used to generate an attribution heatmap of the detection results on the original image, demonstrating which regions are important. A counterfactual approach is then used to answer the specific roles played by these regions of varying importance. Taking the cervical cancer TCT image detection task as an example, the model prediction module obtains the model's prediction results for cervical cancer TCT images. The attribution heatmap mapping module iterates through the prediction results and obtains the attribution heatmap corresponding to the prediction category using the Grad-CAM method, forming a heatmap mask that is mapped onto the current prediction box region. The key region selection module independently assigns a weight percentile to each pixel in each prediction box region, obtaining key regions based on the upper and lower bounds of the percentiles input by the user. The counterfactual interpretation module deactivates the key region features and re-inputs them into the model's prediction results. It then performs optimal matching based on the IoU threshold between the regenerated prediction box and the original prediction box, comparing the changes between the two to explain the specific role of the selected region, enhancing the model's credibility. This further assists doctors in diagnosing conditions and helps model researchers optimize and iterate the model. Attached Figure Description

[0029] Figure 1 This is the overall process framework of the present invention;

[0030] Figure 2 This is a schematic diagram of the model prediction process in this invention;

[0031] Figure 3 This is a schematic diagram of the attribution thermal activation map mapping module of the present invention;

[0032] Figure 4 This is a schematic diagram of the key area selection module process of the present invention;

[0033] Figure 5 This is a flowchart illustrating the counterfactual explanation module of the present invention; Detailed Implementation

[0034] The present invention will be further illustrated below with specific embodiments—interpretation of cervical cancer TCT pathological images based on YOLOv5. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. Furthermore, it should be understood that after reading the teachings of this invention, those skilled in the art can make various alterations or modifications to the invention, and these equivalent forms also fall within the scope defined by the appended claims.

[0035] like Figure 1 As shown, embodiments of the present invention relate to an interpretable method system guided by counterfactual attribution activation graphs, the method comprising:

[0036] The model prediction module obtains prediction results for cervical cancer TCT pathology images using a fully trained YOLOv5 model.

[0037] The attribution heatmap mapping module traverses the prediction results and obtains the feature heatmap corresponding to the prediction category through the Grad-CAM method, and forms a heatmap mask to map onto the current prediction box area;

[0038] The key region selection module independently assigns a weight percentile to each pixel in each predicted box region, and obtains the key region based on the upper and lower bounds of the percentiles input by the user.

[0039] The counterfactual explanation module deactivates the key region features and re-inputs them into the model prediction results. It then performs the best match based on the IoU threshold between the regenerated prediction results and the original prediction results, and compares the changes between the two to explain the specific role played by the selected region.

[0040] Combination Figure 2 The specific operation process of the model prediction module is as follows:

[0041] S11 Input the TCT pathology image of the cervical cancer to be predicted;

[0042] S12 transforms the size of the input cervical cancer TCT pathology image to the input size of the YOLOv5 model, which is 1024*1024.

[0043] S13 uses the fully trained YOLOv5 model to predict the bounding boxes and their classification results.

[0044] Combination Figure 3 The specific operation process of the attribution thermal activation map mapping module is as follows:

[0045] S21 Input all the predicted bounding boxes and their categories for the already predicted cervical cancer TCT pathology image;

[0046] S22 iterates through these prediction boxes and uses the Grad-CAM method to obtain the heat map of the current prediction box category, and then extracts a portion of the prediction box to obtain the heat map of the prediction box.

[0047] S23 merges the thermal activation maps on these prediction boxes and maps them to the size of the original image to obtain a thermal activation map mapping of the cervical cancer TCT pathology image corresponding to the current model.

[0048] Combination Figure 4 The specific operation process of the key area selection module is as follows:

[0049] S31 Input the percentile interval [L,R] of the region to be interpreted;

[0050] Based on the feature map mapping results obtained in S23, S32 assigns a percentile to each pixel in each prediction box independently within the prediction box, i.e., obtains the percentile position by sorting according to the weights of the heat map.

[0051] S33 obtains the set of pixels corresponding to the percentiles in the interval [L, R] based on the result of S32;

[0052] S34 first extracts the content of these pixels in the original cervical cancer TCT pathology image to obtain a sub-image of the selected region and assigns the RGB value of the pixels in the obtained pixel set to 0, which has the effect of rendering these pixels ineffective and obtaining a counterfactual interpretation image.

[0053] Combination Figure 5 The specific operation process of the counterfactual explanation module is as follows:

[0054] S41 uses the model prediction module to predict the counterfactual interpretation map generated by S34 to obtain a new prediction box and its prediction category;

[0055] S42 compares the IoU results to match the original image prediction boxes and the counterfactual interpretation image prediction boxes. It calculates the intersection-union ratio (IoU) between each pair of the four coordinate points of each prediction box and matches the prediction boxes one by one according to the maximum IoU matching principle.

[0056] S43 interprets the effect of the selected region on changes in the matching results through three aspects: changes in confidence, changes in classification results, and changes in predicted bounding boxes. If the confidence changes, it indicates that the feature has a positive or negative effect on the current classification result; if the category changes, it indicates that the feature is the decisive feature that determines the model's judgment; if the predicted bounding box disappears, it indicates that the feature is the decisive feature that determines whether the model detects the region.

Claims

1. A system of interpretable methods guided by counterfactual attribution activation graphs, characterized in that, The system includes: The model prediction module is used to obtain the model's prediction results for the image; The attribution heatmap mapping module iterates through the prediction results and obtains the attribution heatmap corresponding to the prediction category using the Grad-CAM method, and forms a heatmap mask that is mapped to the current prediction box area. The key region selection module independently assigns a weighted percentile to each pixel within each predicted bounding box region, and obtains the key region based on the upper and lower bounds of the percentiles input by the user; the specific operation process is as follows: S31: Input the percentile interval [L,R] of the region to be interpreted; S32: Based on the feature map mapping results obtained from the attribution heat activation map mapping module, assign percentiles independently to each pixel in each prediction box within the scope of the prediction box; S33: Obtain the set of pixels corresponding to the percentiles in the interval [L, R]; S34: First, extract the content of these pixels in the original image to obtain a sub-image of the selected region, and then assign the RGB values ​​of the pixels in the obtained pixel set to 0 to make these pixels inactive, thus obtaining a counterfactual interpretation image; The counterfactual explanation module deactivates the key region features and re-inputs them into the model prediction results. It then performs the best match between the predicted bounding box of the regenerated prediction results and the original predicted bounding box based on the intersection-union ratio threshold, and compares the changes between the two to explain the specific role played by the selected region.

2. The interpretable method system guided by counterfactual attribution activation graphs according to claim 1, characterized in that, The specific operation process of the model prediction module is as follows: S11: Input the image to be predicted; S12: Transform the size of the input image to the input size of the prediction model; S13: Use the prediction results of the fully trained image object detection model to obtain the detection box and its classification result.

3. The interpretable method system guided by counterfactual attribution activation graphs according to claim 1, characterized in that, The specific operation process of the attribution heat activation map mapping module is as follows: S21: Input all predicted bounding boxes and their categories for the predicted image; S22: Traverse these prediction boxes and use the Grad-CAM method to obtain the attribution heatmap of the current prediction box category, and extract the prediction box portion to obtain the prediction box activation map; S23: Merge the activation maps on these prediction boxes and map them onto the original image to obtain the thermal activation map mapping corresponding to the current model.

4. The interpretable method system guided by counterfactual attribution activation graphs according to claim 1, characterized in that, The specific operation process of the counterfactual interpretation module is as follows: S41: Use the model prediction module to predict the counterfactual interpretation map generated in S34 to obtain a new prediction box and its prediction category; S42: Compare the cross-union comparison results to match the original image prediction box and the counterfactual interpretation image prediction box; S43: Based on the changes in the matching results, the effects of the selected region are explained in three aspects: changes in confidence, changes in classification results, and changes in the predicted bounding box.

Images