Class activation mapping interpretability method fusing spatial perturbation mechanism

Abstract

The invention relates to the technical field of artificial intelligence, in particular to a class activation mapping interpretability method fusing a spatial perturbation mechanism, which comprises the following steps: a first stage: capturing contribution of each channel to prediction, and obtaining a channel attention weight of a deep learning model through a gradient method; in the second stage, disturbance analysis is introduced, and space importance weights given to image samples by the model are obtained; and a third stage: fusing the interpretation information of the two view angles through weighted integration, and generating an interpretation saliency map of the region-level granularity. According to the invention, the dual positioning capability of an interpretable technology in a channel domain and a space domain can be improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a class activation mapping interpretability method integrating a spatial perturbation mechanism. Background Art

[0002] Region-level interpretability technology aims to generate saliency maps of continuous regions to show the regional importance of model decisions after feature attribution. The model structure dependency explanation method based on single-channel feature maps is closest to the present invention, namely the class activation mapping method.

[0003] The Class Activation Map (CAM) method is a typical single-channel feature map interpretability technology. This method decomposes the weight assigned by the model classifier to a specific category and infers it to the feature map output by the last convolutional layer as the weighted weight of each feature map. The weighted sum is performed channel by channel to generate a saliency map. Among them, the weighted weight is obtained based on global average pooling, gradient and forward propagation, and interpretability algorithms such as GAP-CAM, Grad-CAM, Grad-CAM++ and Score-CAM are established respectively.

[0004] Representative technologies in the first category include the global average pooling-based GAP-CAM algorithm and the gradient-based Grad-CAM and Grad-CAM++ algorithms. The GAP-CAM algorithm performs global average pooling on the convolutional feature map to obtain a global response value for each channel, which is then used as a weight to weight the feature map, thereby generating a saliency map. The Grad-CAM algorithm introduces backpropagation, calculating the gradient of the loss function of a specified category with respect to the convolutional feature map to measure the impact of each channel on that category. The channel gradients are then globally average pooled and used as weights to fuse together to obtain an explanation map. Grad-CAM++ further optimizes the gradient weighting mechanism in Grad-CAM, introducing second-order derivative information to more finely characterize the saliency contributions of multiple targets or fine-grained regions. Overall, this type of method is computationally simple, but has a limited field of view, is insensitive to small key features, cannot explain interactions between categories, and may misjudge the model's decision logic due to its inability to distinguish between positive and negative contributions of feature regions.

[0005] The second type of representative technology is the Score-CAM algorithm based on forward propagation. Score-CAM abandons the reliance on gradient information and instead uses the response strength of the model's forward propagation to evaluate the importance of feature maps. This method first normalizes and upsamples the convolutional feature map of each channel into a saliency mask of the same size as the input image, and then applies the mask to the original image one by one to generate a series of occlusion-enhanced input images, which are sequentially fed into the model for forward reasoning, and the predicted scores for the target category are recorded. These predicted scores are regarded as the weights of the corresponding channels, and the weighted summation of the feature maps is performed to obtain a saliency map. This method can avoid the problems of gradient noise and gradient vanishing, and has stronger robustness and higher regional distinction capabilities, but it also increases the computational cost, the inference time is relatively longer, and it is difficult to achieve efficient real-time interpretability requirements.

[0006] In summary, existing region-level interpretability techniques based on single-channel feature maps generally focus on weighting the importance between feature map channels, but ignore the fine distribution of spatial information within the feature map. Such methods treat feature maps as holistic units for weighted processing, but lack further exploration of pixel-level response patterns within channels. As a result, the generated saliency maps often suffer from precision issues such as inaccurate positioning, blurred boundaries, and diffuse response areas. In practical application scenarios, image samples are highly complex and heterogeneous, with significant differences in content structure and background interference. The salient regions generated by existing methods struggle to maintain a stable focus on the true basis for model decisions, limiting their scalability and practicality in critical tasks. Summary of the Invention

[0007] The content of the present invention is to provide a class activation mapping interpretability method that integrates the spatial perturbation mechanism. It generates a saliency map with enhanced comprehensibility to visualize the prediction results of the interpretation model, effectively solving the problems of insufficient interpretation positioning accuracy and completeness in regional-level interpretability technology.

[0008] A class activation map interpretability method integrating a spatial perturbation mechanism according to the present invention comprises the following steps:

[0009] The first stage: Capture the contribution of each channel to the prediction and obtain the channel attention weight of the deep learning model through the gradient method;

[0010] The second stage: introduce perturbation analysis to obtain the spatial importance weights assigned to image samples by the model;

[0011] The third stage: Through weighted integration, the interpretation information of the two perspectives is fused to generate an interpretation saliency map at the regional level.

[0012] As a preference, in the first stage, the second-order gradient is used as the channel attention weight of each feature map As shown in formula (1):

[0013]

[0014] Where, is the gradient weight of the pixel at (i, j) in the feature map when predicting category d on channel k; d is the prediction score of category d; is the pixel at (i, j) of the feature map A on channel k.

[0015] As a preference, After the prediction score Y for category d d The relationship between the activation value and the activation value is modeled and derived for calculation, as shown in formula (2):

[0016]

[0017] (a, b) Pixel locations used to index the 2D activation map;

[0018] The mathematical transformation is used to simplify the calculation process as follows:

[0019]

[0020] Where, is the weight at (i, j), and ε is a minimum value used to avoid division by zero errors in calculations; the ReLU function ensures that only positive contributions are retained while suppressing negative or irrelevant activation responses.

[0021] As a preference, in the second stage, the top-k single-channel feature maps of the last layer are selected according to their weights for bilinear interpolation upsampling, as shown in formula (4):

[0022]

[0023] in, represents the pixel value at (x, y) of the activation map A on channel k after upsampling, p and q represent the relative distances of the horizontal and vertical offsets during the upsampling process, (i, j) is the integer coordinate of the upper left corner closest to the target position (x, y), and satisfies i≤x≤i+1,j≤y≤j+1, and the four nearest neighboring points in the feature map are weighted averaged, and the weight is determined by the relative distance between the target point and each neighboring point; thus, a smooth feature map with the same size as the original image is obtained.

[0024] As a preference, in the second stage, perturbations are applied to important regions of the feature map to measure the sensitivity of the model to the perturbations, which is divided into two steps: important region determination and progressive perturbation analysis;

[0025] The key areas are as follows:

[0026] First, for each selected feature map Setting thresholds It is used to divide the important and unimportant areas in the feature map. p' is the threshold percentage of the maximum value pixel in the feature map. Only the important areas suggested by the feature map are disturbed to perform spatial importance analysis.

[0027] According to the threshold τ k Define important area masks Formula (5):

[0028]

[0029] Therefore, the important area R of the feature map k Determined as formula (6):

[0030]

[0031] The progressive perturbation analysis is as follows:

[0032] Sort the pixels in the important area based on the size of the pixel values ​​in the feature map, starting from the pixel with the highest activation value and gradually expanding the perturbation range to the pixels with lower activation values; let N k =|R k | for R k The total number of pixels in, if the perturbation process is divided into m steps, each step processes a subset of pixels The calculation process is formally expressed as:

[0033]

[0034] Where, α1, α m is the cumulative proportion coefficient, satisfying 0<α1<α2<…α m =1;

[0035] For each subset Apply targeted noise perturbation to generate a perturbed image As shown in formula (8):

[0036]

[0037] Where, the mapping relationship from (x, y) to (i, j) is determined by the upsampling process, I(x, y) represents the original input image, β is a parameter that controls the perturbation intensity, and a neutral grayscale perturbation of v = 127 is used to offset the information in the original image without introducing too much interference. That is, when β = 1, the perturbed area is completely replaced by gray pixels. Combining the masking and perturbation processes, the perturbed image is expressed as:

[0038]

[0039] Where ⊙ represents the Hadamard product, For subset The corresponding binary mask.

[0040] Preferably, in the third stage, each perturbed image is Input the original model for forward propagation to obtain the model's prediction confidence for category d Compare the difference between the original confidence and the perturbed confidence to get the importance quantification result between feature regions

[0041]

[0042] The greater the decrease in confidence in formula (10), the more important the region where the perturbation is performed is to the model prediction;

[0043] Based on this, the spatial importance weight assigned to each pixel in the feature map is

[0044]

[0045] Where 1[·] is the indicator function, when the pixel (x, y) belongs to the subset The value is 1 when , otherwise it is 0; through normalization, the confidence reduction values ​​of all steps are converted into standardized weight distributions to ensure that the total weight is 1; thus, it is equivalent to a secondary test of the spatial preliminary weights of important areas of the feature map, and the test process is highly faithful to the model itself, meeting the basic credibility requirements of interpretability technology.

[0046] The feature map is reconstructed according to the spatial importance weight in formula (11) to strengthen the expression of important areas and suppress the interference of secondary areas, as shown in formula (12):

[0047]

[0048] Reconstructed feature map set Will replace the selected Top-k part in the original feature map to form the final feature map set A k , and the weight of the channel dimension Combined, the final explanation saliency map is generated, as shown in formula (13):

[0049]

[0050] L d represents the final explained saliency map.

[0051] The key technical points and beneficial effects of the present invention are as follows:

[0052] 1) Different from GAP-CAM based on global average pooling: GAP-CAM constructs an original framework that uses the highest feature map of the last convolutional layer of the neural network as the explanation intuition, and performs weighted fusion of single-channel feature maps based on this to generate saliency map explanations. For a specific category d, the fusion weights of its k channels are directly used by the GAP layer and the FC layer. The connection weight parameters are obtained, and the method of the present invention does not require a GAP layer structure and can be applied to any deep neural network.

[0053] 2) Difference from the gradient-based CAM method: For a specific category d, Grad-CAM or Grad-CAM++ calculates the partial derivative of the logits value of the softmax layer with respect to the global highest feature map. This method regards all spatial positions of the feature map as one, and obtains the gradient at each pixel by summing and normalizing it. The method of the present invention inherits this idea to obtain the relative importance between channels, but only as part of the final weight rather than directly fusing the feature maps.

[0054] 3) Difference from non-gradient CAM methods: For example, Score-CAM and its variants give up the ability of CAM framework to analyze the internal structure of the model, and instead obtain weights from the forward propagation outside the model instead of the back propagation inside the model. For a specific category d, use the model to Prediction score of feature map addition information As the fusion weight of the feature map. The method of the present invention does not simply add the feature map, but guides a more refined progressive perturbation; instead of using the full single-channel feature map, a small number of channels with higher relative importance are selected, reducing the amount of computation while achieving a more refined interpretation.

[0055] In general, the SP-CAM method of the present invention enhances the dual localization capabilities of interpretability techniques in both channel and spatial domains by improving the weight distribution in the class activation mapping (CAM) framework. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 Flowchart of a class activation mapping interpretability method integrating a spatial perturbation mechanism in an embodiment;

[0057] Figure 2 This is a diagram showing the effect of improving the category relevance of correctly classified samples in the embodiment;

[0058] Figure 3 This is a diagram showing the effect of improving the category relevance on misclassified samples in the embodiment;

[0059] Figure 4This is a diagram showing the accuracy and completeness improvement effects on a single target sample in the embodiment;

[0060] Figure 5 This is a diagram showing the accuracy and completeness improvement effects on multiple target samples in the embodiment. DETAILED DESCRIPTION

[0061] In order to further understand the content of the present invention, the present invention is described in detail with reference to the accompanying drawings and embodiments. It should be understood that the embodiments are merely for explaining the present invention and are not intended to limit the present invention.

[0062] Example

[0063] like Figure 1 As shown, this embodiment provides a class activation map interpretability method integrating a spatial perturbation mechanism, which includes the following steps:

[0064] The first stage: Capture the contribution of each channel to the prediction and obtain the channel attention weight of the deep learning model through the gradient method;

[0065] The second stage: introduce targeted perturbation analysis to obtain the spatial importance weights assigned to image samples by the model;

[0066] The third stage: Through weighted integration, the interpretation information of the two perspectives is fused to generate an interpretation saliency map at the regional level.

[0067] The method of this embodiment retains the channel-based computing framework of CAM. After obtaining the single-channel feature map of the model, it introduces the ideas of perturbation and forward propagation, supplements the optimization of the spatial distribution of the feature map, and then returns to perform channel-level weighted fusion to form a regional-level interpretability technology that has the ability to extract both channel and spatial features.

[0068] Phase 1

[0069] For a specific category d, let the model's prediction score for this category be Y d In this embodiment, different weight coefficients are assigned to each pixel position in a single-channel feature map, and the second-order gradient is used as the channel attention weight of each feature map. This is to further expand the capture of the model's attention area and ensure the integrity of the model's interpretability. As shown in formula (1):

[0070]

[0071] Where, is the gradient weight of the pixel at (i, j) in the feature map when predicting category d on channel k; d is the prediction score of category d; is the pixel at (i, j) of the feature map A on channel k.

[0072] After the prediction score Y for category d d The relationship between the activation value and the activation value is modeled and derived for calculation, as shown in formula (2):

[0073]

[0074] (a, b) are used to index the pixel position of the two-dimensional activation map, which is a letter representation proposed to distinguish it from i and j;

[0075] In order to avoid the large computational overhead introduced by directly calculating high-order derivatives in practical application calculations, appropriate mathematical transformations are adopted to simplify the calculation process as follows:

[0076]

[0077] Where, is the weight at (i, j), and ε is a minimum value used to avoid division by zero errors in calculations. The ReLU function ensures that only positive contributions are retained, which helps highlight channels that contribute positively to specific features while suppressing negative or irrelevant activation responses. This simplified calculation method still maintains the adaptability of the activation-based weight allocation mechanism, thus effectively explaining the network decision-making process in the absence of complete internal model information.

[0078] Phase II

[0079] The single-channel feature maps of the last layer of the model are selected according to the weight size and bilinear interpolation upsampling is performed, as shown in formula (4):

[0080]

[0081] in, represents the pixel value at (x, y) of the activation map A on channel k after upsampling, p and q represent the relative distances of the horizontal and vertical offsets during the upsampling process, (i, j) is the integer coordinate of the upper left corner closest to the target position (x, y), and satisfies i≤x≤i+1,j≤y≤j+1, and the four nearest neighboring points in the feature map are weighted averaged, and the weight is determined by the relative distance between the target point and each neighboring point; thus, a smooth feature map with the same size as the original image is obtained, which reduces the visual artifacts caused by the resolution change to a certain extent, and maintains the spatial continuity in the feature map by considering the contribution of multiple neighboring points, thereby generating a feature map that is more in line with visual perception.

[0082] To address the problem that common CAM methods directly use the raw activation values ​​of feature maps to generate visualization results, ignoring differences in spatial importance within feature maps, this example proposes a targeted perturbation method guided by feature maps. By applying meaningful perturbations to important regions of feature maps and measuring the model's sensitivity to perturbations, this method more accurately assesses the importance distribution of feature spaces. This process consists of two steps: identifying important regions and performing progressive perturbation analysis.

[0083] The key areas are as follows:

[0084] First, for each selected feature map Setting thresholds It is used to divide the important and unimportant areas in the feature map. p' is the threshold percentage of the maximum value pixel in the feature map. Only the important areas suggested by the feature map are disturbed to perform spatial importance analysis. This can improve the computational efficiency of this solution without losing key feature information.

[0085] According to the threshold τ k Define important area masks Formula (5):

[0086]

[0087] Therefore, the important area R of the feature map k Determined as formula (6):

[0088]

[0089] Based on this important area R k This embodiment introduces the idea of ​​perturbation to further refine the feature importance distribution within the region, and uses the original feature map information to guide the location and amplitude of the perturbation to maximize the efficiency of the perturbation. The progressive perturbation analysis is specifically as follows:

[0090] The pixels in the important area are sorted based on the size of the pixel values ​​in the feature map, starting from the pixel with the highest activation value and gradually expanding the perturbation range to the pixels with lower activation values; the progressive expansion strategy effectively captures the visual features that the neural network pays attention to, and analyzes them in order of importance to reduce the perturbation cost with less utility. Let N k =|R k | for R k The total number of pixels in, if the perturbation process is divided into m steps, each step processes a subset of pixels The calculation process is formally expressed as:

[0091]

[0092] Where, α1, α mis the cumulative proportion coefficient, satisfying 0<α1<α2<…α m =1;

[0093] For each subset Apply targeted noise perturbation to generate a perturbed image As shown in formula (8):

[0094]

[0095] Where, the mapping relationship from (x, y) to (i, j) is determined by the upsampling process, I(x, y) represents the original input image, β is the parameter that controls the perturbation intensity, and a neutral grayscale perturbation of v = 127 is used to effectively offset the information in the original image without introducing too much interference. That is, when β = 1, the perturbed area is completely replaced by gray pixels. Therefore, combining the masking and perturbation processes, the perturbed image is expressed as:

[0096]

[0097] Where ⊙ represents the Hadamard product, For subset The corresponding binary mask.

[0098] Phase 3

[0099] By transforming each perturbed image Input the original model for forward propagation to obtain the model's prediction confidence for category d Compare the difference between the original confidence and the perturbed confidence to get the importance quantification result between feature regions

[0100]

[0101] The greater the decrease in confidence in formula (10), the more important the region where the perturbation is performed is to the model prediction;

[0102] Based on this, the spatial importance weight assigned to each pixel in the feature map is

[0103]

[0104] Where 1[·] is the indicator function, when the pixel (x, y) belongs to the subset The value is 1 when , otherwise it is 0; through normalization, the confidence reduction values ​​of all steps are converted into standardized weight distributions to ensure that the total weight is 1; thus, it is equivalent to a secondary test of the spatial preliminary weights of important areas of the feature map, and the test process is highly faithful to the model itself, meeting the basic credibility requirements of interpretability technology.

[0105] In order to directly and effectively improve the spatial attention of the interpretability technology, the feature map is reconstructed according to the spatial importance weight in formula (11) to strengthen the expression of important areas and suppress the interference of secondary areas, as shown in formula (12):

[0106]

[0107] This reconstruction method retains the basic structure of the original feature map, corrects the incorrect attention allocation, and at the same time performs differentiated enhancements on different regions based on the perturbation analysis results, so that the feature map is more focused on the areas that are truly important to the model decision.

[0108] Reconstructed feature map set Will replace the selected Top-k part in the original feature map to form the final feature map set A k , and the weight of the channel dimension Combined, the final explanation saliency map is generated, as shown in formula (13):

[0109]

[0110] L d represents the final explained saliency map.

[0111] Through this spatial weight-based feature map reconstruction, the transition from uniform processing of feature space to differentiated processing is achieved, and the spatial importance information revealed by perturbation analysis is fully utilized.

[0112] The algorithm steps of this embodiment are as follows:

[0113]

[0114]

[0115] Experimental verification

[0116] To verify the effectiveness of the interpretability technology of this embodiment, the mainstream deep learning framework PyTorch was used to build an experimental platform, and a representative sample subset was selected from the validation set of the ImageNet dataset for experiments. The interpreted model uniformly uses the ResNet50 network pre-trained by ImageNet to avoid interference with the visualization effect due to differences in model structure and ensure fair comparison of interpretation performance. The method of this embodiment is compared and evaluated with the existing four typical class activation mapping methods of GAP-CAM, Grad-CAM, Grad-CAM++ and Score-CAM on the same model to ensure that the experimental results are comparable and of reference value, and can objectively reflect the improvement effect of the method of this embodiment in improving the relevance, accuracy and completeness of saliency map categories.

[0117] (1) Qualitative evaluation of effects

[0118] In the regional interpretability method, category relevance is a key indicator, which represents whether the explanation map can accurately correspond to the category area that the model focuses on. Figure 2 The SP-CAM method proposed in this example found that when the top-1 category prediction is correct, the generated explanation map is more spatially concentrated and has clearer boundaries, fully covering the target area of ​​the model's attention, significantly improving the category relevance and visual readability of the explanation map. Especially when processing fine-grained category samples, SP-CAM maintains regional integrity while maintaining higher discriminative power, effectively highlighting key semantic regions, and thus more accurately reflecting the model's internal decision-making basis.

[0119] In the case of Top-1 category prediction errors, the explanation map generated by this embodiment can still focus attention on the target area in the image that is most relevant to the true semantics, such as Figure 3 As shown, the proposed SP-CAM method demonstrates good robustness and generalization capabilities, demonstrating the higher semantic directivity and interpretation accuracy of this example. The excellent performance of this method in such complex scenarios validates its practical value in assisting model error diagnosis and improving model controllability and credibility in real-world applications.

[0120] In addition to the fact that the category explanation of the interpretable output must be consistent with the model conclusion, a good regional-level interpreter should accurately and completely locate the target object in the image. "Accurate" means that the image area annotated by the interpreter is consistent with the image area referred to by the classification, and can clearly provide the feature basis for interpretation and decision-making; "complete" means that the image area annotated by the interpreter covers comprehensive image features, and does not include irrelevant areas or omit relevant areas of the target object as much as possible. Figure 4 、 Figure 5 As shown, the SP-CAM method proposed in this embodiment has better accuracy and completeness, the edge of the positioning area is basically consistent with the outline of the target object, and has better visual effects on image samples of single or multiple target objects.

[0121] (2) Quantitative evaluation of effects

[0122] To further validate the practical interpretability advantages of the SP-CAM method in this embodiment, we systematically evaluated its regional directivity, causal consistency, and computational efficiency using mainstream quantitative evaluation metrics from current CAM-based interpretation method research. Specifically, these metrics included the deletion-insertion metric, the average decrease-average increase metric, and an energy-based pointing game, comprehensively and quantitatively demonstrating the improved accuracy and completeness of this embodiment in practical applications.

[0123] Table 1 Comparison of the effects of interpretability techniques under the deletion-insertion indicator

[0124]

[0125] As shown in Table 1, the SP-CAM method proposed in this example is significantly better than other methods. The lower the deletion index and the higher the insertion and comprehensive indexes, the better the consistency of the interpreter between input and output and the model causal reasoning. This is highly related to the addition of model cognitive capture based on targeted perturbations in the method.

[0126] Table 2 Interpreter evaluation table of mean decrease-mean increase indicator

[0127]

[0128] As shown in Table 2, the SP-CAM method proposed in this embodiment performs better than other methods in the overall performance on the batch sample set, with the average rise index reaching 39.17%, further demonstrating that the area located by the interpreter has a stronger influence on the category judgment of the model.

[0129] Table 3. Energy-based explainer evaluation of pointing game metrics

[0130]

[0131] As shown in Table 3, the SP-CAM method of this embodiment has more concentrated and accurate pointing energy than the GAP-CAM based on global average pooling, and its indicators are significantly improved; compared with the gradient-free ScoreCAM method, its performance is slightly improved.

[0132] In summary, the SP-CAM regional-level interpretability technology proposed in this embodiment has achieved effective improvements in positioning accuracy, completeness, and fidelity. The generated explanation saliency map has good accuracy and comprehensibility. Therefore, based on this feature, it can be suitable for application scenarios such as weakly supervised target positioning, auxiliary semantic segmentation, and image editing and processing, providing them with better usage performance and experience.

[0133] The above is a schematic description of the present invention and its embodiments, which is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. Therefore, if a person skilled in the art is inspired by this and, without departing from the purpose of the present invention, designs a structure and embodiment similar to this technical solution without inventiveness, they shall fall within the scope of protection of the present invention.

Claims

1. A class activation map interpretability method that integrates a spatial perturbation mechanism, characterized by: The following steps are involved: The first stage: Capture the contribution of each channel to the prediction and obtain the channel attention weight of the deep learning model through the gradient method; The second stage: introduce perturbation analysis to obtain the spatial importance weights assigned to image samples by the model; The third stage: Through weighted integration, the interpretation information of the two perspectives is fused to generate an interpretation saliency map at the regional level.

2. The class activation map interpretability method integrating spatial perturbation mechanism according to claim 1, characterized in that: In the first stage, the second-order gradient is used as the channel attention weight of each feature map As shown in formula (1): Where, Y is the gradient weight of the pixel at (i, j) in the feature map when predicting category d on channel k; d is the prediction score of category d; is the pixel at (i, j) of the feature map A on channel k.

3. The class activation map interpretability method integrating spatial perturbation mechanism according to claim 2, characterized in that: After the prediction score Y for category d d The relationship between the activation value and the activation value is modeled and derived for calculation, as shown in formula (2): (a, b) Pixel locations used to index the 2D activation map; The mathematical transformation is used to simplify the calculation process as follows: Where, is the weight at (i, j), and ε is a minimum value used to avoid division by zero errors in calculations; the ReLU function ensures that only positive contributions are retained while suppressing negative or irrelevant activation responses.

4. The class activation map interpretability method integrating spatial perturbation mechanism according to claim 3, characterized in that: In the second stage, the top-k single-channel feature maps of the last layer are selected according to their weights and then upsampled by bilinear interpolation, as shown in formula (4): in, represents the pixel value at (x, y) of the activation map A on channel k after upsampling, p and q represent the relative distances of the horizontal and vertical offsets during the upsampling process, (i, j) is the integer coordinate of the upper left corner closest to the target position (x, y), and satisfies i≤x≤i+1,j≤y≤j+1, and the four nearest neighboring points in the feature map are weighted averaged, and the weight is determined by the relative distance between the target point and each neighboring point; thus, a smooth feature map with the same size as the original image is obtained.

5. The class activation map interpretability method integrating spatial perturbation mechanism according to claim 4, characterized in that: In the second stage, perturbations are applied to important regions of the feature map to measure the model’s sensitivity to the perturbations. This is done in two steps: identifying important regions and performing progressive perturbation analysis. The key areas are as follows: First, for each selected feature map Setting thresholds It is used to divide the important and unimportant areas in the feature map. p' is the threshold percentage of the maximum value pixel in the feature map. Only the important areas suggested by the feature map are disturbed to perform spatial importance analysis. According to the threshold τ k Define important area masks Formula (5): Therefore, the important area R of the feature map k Determined as formula (6): The progressive perturbation analysis is as follows: Sort the pixels in the important area based on the size of the pixel values ​​in the feature map, starting from the pixel with the highest activation value and gradually expanding the perturbation range to the pixels with lower activation values; let N k =|R k | for R k The total number of pixels in, if the perturbation process is divided into m steps, each step processes a subset of pixels The calculation process is formally expressed as: Where, α1, α m is the cumulative proportion coefficient, satisfying 0<α1<α2<…α m =1; For each subset Apply targeted noise perturbation to generate a perturbed image As shown in formula (8): Where, the mapping relationship from (x, y) to (i, j) is determined by the upsampling process, I(x, y) represents the original input image, β is a parameter that controls the perturbation intensity, and a neutral grayscale perturbation of v = 127 is used to offset the information in the original image without introducing too much interference. That is, when β = 1, the perturbed area is completely replaced by gray pixels. Combining the masking and perturbation processes, the perturbed image is expressed as: Where ⊙ represents the Hadamard product, For subset The corresponding binary mask.

6. The class activation map interpretability method integrating spatial perturbation mechanism according to claim 5, characterized in that: In the third stage, each perturbed image is transformed into Input the original model for forward propagation to obtain the model's prediction confidence for category d Compare the difference between the original confidence and the perturbed confidence to get the importance quantification result between feature regions The greater the decrease in confidence in formula (10), the more important the region where the perturbation is performed is to the model prediction; Based on this, the spatial importance weight assigned to each pixel in the feature map is Where 1[·] is the indicator function, when the pixel (x, y) belongs to the subset The value is 1 when , otherwise it is 0; through normalization, the confidence reduction values ​​of all steps are converted into standardized weight distributions to ensure that the total weight is 1; Therefore, it is equivalent to a secondary test of the spatial preliminary weights of important areas of the feature map, and the test process is highly faithful to the model itself, meeting the basic credibility requirements of interpretability technology. The feature map is reconstructed according to the spatial importance weight in formula (11) to strengthen the expression of important areas and suppress the interference of secondary areas, as shown in formula (12): Reconstructed feature map set Will replace the selected Top-k part in the original feature map to form the final feature map set A k , and the weight of the channel dimension Combined, the final explanation saliency map is generated, as shown in formula (13): L d represents the final explained saliency map.

Images