Unsupervised explanation method based on adaptive semantic features

Abstract

The application proposes an unsupervised explanation method based on adaptive semantic features, and the constructed explainable model mainly includes three core modules: feature-level explainability module, adaptive feature expression module and feature importance calculation module. Firstly, the deep feature mapping of the deep neural network is regarded as the abstract expression of the high-level concept semantics learned by it, then the non-negative matrix decomposition technology is used to unsupervisedly extract key semantic information from it, and feature-level explanation or structured attribution is carried out; through the dimension scaling scheme, adaptive number of feature extraction is realized; in the feature importance calculation, the Shapley value algorithm is used for calculation. In addition, a saliency visual explanation is generated to highlight the key area of the model decision. Experiments show that the method has higher explanation accuracy, and in the environment of different data sets and explained models, the explanation accuracy is better than that of the existing method, and has the characteristics of accurate explanation, robustness and universality.

Description

Technical Field

[0001] This invention relates to the field of interpretability research of deep neural networks, and more particularly to an unsupervised interpretation method for adaptive semantic features. Background Technology

[0002] The study of interpretability of deep neural networks aims to unveil their black-box nature and provide a reasonable methodology for explaining their decision-making behavior. Previously, the basic framework for interpreting deep neural networks mainly consisted of defining input units and calculating their importance. However, defining input units and determining how many features (input units) to extract remain challenges for the basic framework of interpretability methods. The lack of semantic information in input units hinders our understanding of their specific meaning and structured attribution analysis; therefore, most existing methods do not provide feature-level interpretability. For tuning and optimizing feature representations, adaptively determining the number of input units and defining their semantics are equally important. This is to constrain feature representations, prevent entities from being infinitely subdivided, and improve computational efficiency.

[0003] The interpretability of deep neural networks has been extensively studied; however, interpretive methods rarely focus on the semantics of input units, often constructing them from isolated pixels with weak connections. For example, Recipro-CAM retains or removes some image regions to calculate importance, and its simple image segmentation operation severely weakens the readability of input units. Random Input Sampling Interpretation (RISE) uses Monte Carlo sampling to generate random masks, thus it doesn't need to focus on semantic information. However, apart from RISE, other methods use non-random masks for perturbation, and most cannot extract semantic input units. For example, Score-CAM utilizes the spatial correlation between deep feature images and input images, mapping the feature images onto the input images to generate non-random masks. Since the feature images contain a large amount of meaningless redundant information, they are essentially no different from random masks. Furthermore, the class activation mapping in Grad-CAM is generated based on gradients from backpropagation. It has some meaning because the gradient direction reflects the feature learning direction of the model during training; however, gradient-based methods are highly sensitive to noise, which can lead to inaccurate semantics of input units. Some CAM variants, such as Cluster-CAM and Group-CAM, use clustering or grouping methods to quickly aggregate CAMs with similar semantics in Grad-CAM and filter redundant semantic information to generate masks, thereby accelerating computation. However, they only consider semantic similarity and do not delve into the semantic content. Unsupervised superpixel segmentation algorithms effectively extract useful semantic information from the input image to construct input units by clustering similar pixel spaces, but it is not related to neural networks, so we cannot understand the network model's understanding of semantics. Concept Activation Vectors (CAVs) summarize important concepts by selecting a set of examples representing a concept or by finding a separate dataset that labels the concept; such work lacks the flexibility of unsupervised methods.

[0004] Furthermore, determining the number of features to extract is crucial for optimizing feature representation, as it modulates semantics. Early research used neurons, pixels, or superpixels to represent input units, such as CeCNN, LRP, and LIME, resulting in a relatively large number of features. Additionally, in some methods, feature maps under each channel can also be considered input units, such as Ablation-CAM. Some methods generate random masks to form input units, such as RISE and Score-CAM's non-random masked input units. These methods extract a very large and fixed number of features. Group-CAM and Cluster-CAM use grouping or clustering methods to reduce the number of features to speed up computation. Even in some methods, the number of features can be set as a hyperparameter, such as CAV. In short, none of these methods adaptively determine the number of features, thus failing to optimize feature representation.

[0005] Shortcomings of existing technology:

[0006] 1. Existing methods lack semantic analysis and refinement of input units, which hinders the structured attribution of models and the provision of feature-level interpretability.

[0007] Typical deep neural network visual interpretation methods consider the impact of the absence or preservation of individual input units on the final model output, thus calculating their importance—a process known as ablation. However, most methods rarely define the semantics of the input units, resulting in a lack of feature-level interpretation. Taking images as an example, pixels are the basic input units. Often, similar pixels, due to their high correlation, activate semantic features in clusters, significantly influencing the network model's decisions. Simply cutting the image into isolated regions to construct input units destroys its semantics and weakens its contextual connections.

[0008] 2. There is a lack of an adaptive scheme for determining the number of features to be extracted, thereby adjusting the semantics of the input units and optimizing the feature representation.

[0009] Existing interpretation methods mostly determine the number of input units or features as hyperparameters. Some methods strive to reduce the number of features to improve computational efficiency, but the deeper implications of the number of features have not been thoroughly explored. In single-sample local interpretation methods, the number of features should be adaptively determined when representing the features of entities (images or natural language). This affects the semantics of the input units and the final feature representation results. For example, setting too many features may lead to the repeated activation of similar features, resulting in multiple calculations of importance.

[0010] 3. The saliency of the interpretation method and its visual interpretation performance still need to be improved.

[0011] The visual interpretation of saliency reveals whether a neural network has correctly noticed the region that has the greatest impact on the model's decision, i.e., the salient region. Existing methods may still misinterpret the model's decision, possibly because excessive repetition or activation of redundant features in feature selection leads to the same feature's importance being calculated multiple times, or because the importance calculation algorithm lacks fairness by failing to consider the interactions between features. Summary of the Invention

[0012] To address the shortcomings of existing technologies, this invention proposes an unsupervised interpretation method based on adaptive semantic features. A visual interpretation method for classification tasks using deep neural networks is designed, providing feature-level interpretability and saliency visual explanation. The constructed interpretability method includes a feature-level interpretability module, an adaptive feature representation module, and a feature importance calculation module. The feature-level interpretability module unsupervisedly extracts high-level semantic features to provide feature-level interpretability. The adaptive feature representation module converts the number of input units extracted into a dimensionality scaling solution to address the problem of determining the target dimension for dimensionality reduction during nonnegative matrix factorization. The feature importance calculation module introduces the Shapley value method to calculate the importance of features, specifically including:

[0013] Step 1: Prepare image datasets, including the Mini-ImageNet 2012 dataset and the PASCAL VOC2012 dataset;

[0014] Step 2: Preprocess the two datasets separately. For the training set, randomly select 1000 image samples from the validation sets of both datasets. For the test set, randomly select 1 image from 1000 categories in the Mini-ImageNet2012 dataset and randomly select 1000 images from the PASCAL VOC2012 dataset.

[0015] Step 3: Extract a sample image X from the test set and input it into the feature-level interpretability module and the adaptive feature representation module to obtain the feature-level interpretability and feature mask image, specifically including:

[0016] Step 31: Process a sample image X from the test set into a three-channel RGB image with a resolution of 224×224, and standardize it according to the preset mean and standard deviation;

[0017] Step 32: Input the sample image X and its class label into the DNN network being interpreted, and then extract the feature map V in the last convolutional layer;

[0018] Step 321: Input the hyperparameter information entropy a′ into the dimension scaling solution module DSS. The input of the nonnegative matrix factorization module is the feature map V and the target dimension k, and the output is the coefficient matrix U.

[0019] Step 33: Treat the coefficient matrix U as the feature matrix of the target dimension k, and consider it to have spatial correlation with the sample image X. Upsample, normalize, and binarize the coefficient matrix U to obtain binary masks U of the target dimension k with the same resolution as the sample image X. mask ;

[0020] Step 34: Apply the binary mask Umask The region with a median value of 1 is used to retain the pixels of the sample image X, and the region with a median value of 0 is used to fog up the sample image X. At the same time, the boundary between 0 and 1 is visually enhanced. The final image shows the expression of the features in the sample image X, that is, the feature interpretability.

[0021] Step 35: Apply the binary mask U mask The region with a median value of 1 is used to retain the pixels of the sample image X, and the region with a median value of 0 is used to mask the pixels of the sample image X. The generated feature mask image is input into the feature importance calculation module.

[0022] Step 4: Using the feature mask image obtained in Step 35 as the feature to be used to calculate importance, the Shapley value method is introduced, specifically including:

[0023] Step 41: Treat the k feature mask images as k features and calculate the importance of the k features. The specific steps are as follows:

[0024] Step 411: Let N represent the set of k features, and j represent a feature in N, j = {1, 2, ..., k}. Taking feature j as an example, calculate the importance of feature j;

[0025] Step 412: Perform a multilinear sampling in N\{j}, using L as the sampling feature. i Let i = {1, 2, ..., QM} represent the multilinear sampling result L = {L1, L2, ..., L...} QM}, where Q and M are positive integers L i A Bernoulli sequence with a value of 0 or 1;

[0026] Step 413: Use combined features L i ∪{j} represents the result of multilinear sampling plus feature j, which is then fed back into the DNN network to obtain the output value Yc(L). i Similarly, we obtain Yc(L) by ∪{j}). i And obtain Yc(L) i ∪{j})-Yc(L i The result represents the marginal contribution of the combined features of this sampling result;

[0027] Step 414: According to the multilinear sampling requirement, QM sampling is required, that is, repeating steps 412 and 413 QM times and summing the marginal contributions of each sampling to obtain the Shapley value of feature j, that is, the importance of feature j.

[0028] Step 42: Repeat step 41 to calculate the importance of all k features and obtain the importance vector;

[0029] Step 5: Assign the importance vector as weights to the corresponding features, that is, linearly combine the importance vector and the coefficient matrix U to obtain the class activation map, then upsample the class activation map to the input size (m, n) of the sample image X, and normalize the maximum and minimum values ​​to [0,1], and fuse it with the sample image X to obtain the saliency map;

[0030] Step 6: Repeat steps 3 to 5 to generate a significance map for each sample in the test set;

[0031] Step 7: The importance of the corresponding pixel in the saliency map of the sample image X is represented by the element values.

[0032] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0033] 1. This invention designs an unsupervised interpretation method based on adaptive semantic features. The method provides feature-level interpretability and improves the performance of saliency visual interpretation on the basis of existing methods.

[0034] 2. The adaptive feature representation module (SFR) of the interpretation method proposed in this paper proposes a dimension scaling solution (DSS) to adaptively determine the number of features to be extracted in order to adjust the semantics of the input unit and optimize the feature representation, thereby providing feature-level interpretability.

[0035] 3. Extensive experiments have demonstrated that the unsupervised interpretation method based on adaptive semantic features proposed in this invention outperforms existing interpretation methods in quantitative experiments on different datasets and the network models being interpreted. This indicates that the method of this invention is more universal, reliable, and robust. Attached Figure Description

[0036] Figure 1 This is a schematic diagram of the algorithm proposed in this invention;

[0037] Figure 2 This is a schematic diagram of feature-level interpretability;

[0038] Figure 3 This is a specific implementation of the derivation diagram of the adaptive feature representation module;

[0039] Figure 4 This is a specific implementation of the derivation diagram of the adaptive feature representation module;

[0040] Figure 5 This is a graph showing the qualitative results of this invention on the ILSVRC 2012 validation set;

[0041] Figure 6 This is a schematic diagram of the method flow of the present invention. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments and the accompanying drawings. It should be understood that these descriptions are merely exemplary and not intended to limit the scope of the invention. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concept of the invention.

[0043] In this invention, FLI stands for Feature-level Interpretability.

[0044] In this invention, SFR stands for Self-adaptive Feature Representation, which is a part of the FLI module.

[0045] In this invention, FIC refers to Feature Importance Calculation.

[0046] The DSS in this invention refers to: Dimensionality Scaling Solution, which is applied in the SFR module.

[0047] The AD mentioned in this invention refers to Average Drop.

[0048] The AI ​​involved in this invention refers to Average Increase.

[0049] This invention relates to the field of interpretability research in deep neural networks, particularly to the interpretability analysis of various network models in image classification tasks. It primarily provides an interpretation method that offers accurate, robust, and universal interpretation results, and provides feature-level interpretability. Existing interpretation methods often lack semantic information in the input units, hindering our understanding of their specific meanings and enabling structured attribution analysis. Therefore, most existing methods do not provide feature-level interpretability. For adjusting and optimizing feature representations, adaptively determining the number of input units and defining their semantics is equally important. This is to constrain feature representations, prevent entities from being infinitely subdivided, and improve computational efficiency. Furthermore, input units containing redundant information may repeatedly activate features, leading to biased importance calculations and resulting in inaccurate interpretation results.

[0050] To address the shortcomings of existing solutions, this invention proposes an unsupervised interpretation method based on adaptive semantic features. Figure 1This is a schematic diagram of the overall algorithm flow of the interpretation method proposed in this invention. Figure 6 This is a schematic diagram of the method flow of the present invention. Its input data is a three-channel RGB original image from the test set, and the output has two parts: one part is feature-level interpretability, such as... Figure 2 As shown, it provides visualizations of the important components of the entity being explained, and another part is the visual explanation of salience. The experimental results are shown in the figure below. Figure 5 As shown, it provides the regions of greatest interest to the network model, presented in the form of a saliency map. The input image is represented by X, with a resolution of 3×m×n, where 3 represents the RGB three channels, m represents the image width, and n represents the image height. The output feature-level interpretation image and saliency interpretation image have the same resolution as the original image X. The algorithm of this invention mainly involves inputting the original image X into a pre-trained network model and extracting the feature map V from the last convolutional layer. Then, the c×h×w feature map V is reduced to a k×h×w feature matrix U. During this process, an information entropy a′ is used to adaptively determine the value of the target dimension k. The target dimension k, as the number of features to be extracted, affects the semantics of the extracted features and the final feature representation, which is addressed by the Adaptive Feature Representation (SFR) module and the Dimension Scaling Solution (DSS). Then, a series of operations, such as upsampling the feature matrix U, are performed to obtain the visualization effect of feature-level interpretability; this is the principle of the Feature-Level Interpretability (FLI) module. Finally, the feature mask image obtained through the matrix feature U is used, and the Shapley value method is used to calculate the importance of the features; this is the principle of the Feature Importance Calculation (FIC) module.

[0051] The core and main innovation of the deep neural network interpretability method of this invention lies in the adaptive feature representation module (SFR) and the dimension scaling solution (DSS). Figure 3 and Figure 4 This is a specific implementation of the derivation diagram of the proposed Dimensional Scaling Solution (DSS) of this invention.

[0052] The details of the Dimensional Scaling Solution (DSS) are as follows: Let P be the interpretability metric, which includes the comprehensive calculation of average decrease (AD), average increase (AI), deletion, and insertion. For the training set S, the impact of the target dimension k within a certain positive integer range on the interpretability metric P is calculated. It is found that the interpretability metric P exhibits a trend of first increasing and then decreasing, meaning that there exists a dimension k that allows the interpretability metric P to reach a local maximum. Let k′ represent the corresponding locally optimal dimension. However, for each sample s∈S, the optimal dimension k′ does not necessarily cause the interpretability metric P of individual sample s to also reach a local maximum. Figure 3As shown. Therefore, we find the k such that the interpretability index P reaches a local maximum for each sample s. i This allows for further optimization of the overall interpretability index P. To further determine k... i Information entropy is introduced, representing a measure of information content or uncertainty. It can be calculated before the interpretability index P, making it easier to replace P. The function for calculating information entropy is denoted by H, with independent variable k and output as a positive real number denoted by a. Ideally, there should be an a for each sample. i =H i (k i This causes the sample's interpretability index P to reach a local maximum. However, in reality, a i Solving this problem remains challenging, as it can be understood as the difficulty in determining the specific optimal feature information entropy. A simpler approach is to find a representative information entropy because H... i This represents different mapping relationships, i.e., in the case of a1 = a2 = ... = a |s| Even in this situation, there are still Established, among which The symbols are not completely equal, indicating that using a representative information entropy value still satisfies the basic condition for adaptively determining the target dimension k. The information entropy H(k′) corresponding to k′ that makes the overall sample interpretability P reach a local maximum can precisely satisfy representativeness; this value is denoted by a′. For example... Figure 4 As shown, a′ serves as a supervisory signal in the adaptive feature representation module (SFR) to perform adaptive semantic feature extraction by determining k.

[0053] Figure 3 Explanation: The diagram includes a single line representing the overall interpretability performance as a function of dimension k, and three lines representing the interpretability performance of three samples as a function of dimension k. Overall performance is optimal when k = 15, but for each sample, the best k values ​​are 6, 10, and 25.

[0054] Figure 4 Explanation: Step 1: The overall interpretability of the training set is maximized, corresponding to dimension k = 15; Step 2: The value of the overall information entropy curve corresponding to k = 15 is a′, which is the hyperparameter to be calculated; Step 3: An example is given, using a′ to inversely calculate the most suitable dimension of the sample.

[0055] To address the shortcomings of existing technologies, this invention proposes an unsupervised interpretation method based on adaptive semantic features. This method designs a visual interpretation approach for classification tasks using deep neural networks, providing feature-level interpretability and saliency visual explanations. The constructed interpretability network includes a Feature-Level Interpretability Module (FLI), an Adaptive Feature Representation Module (SFR), and a Feature Importance Calculation Module (FIC). The FLI unsupervisedly extracts high-level semantic features to provide feature-level interpretability. To accurately represent features, the SFR transforms the extraction of input units into a dimensionality scaling solution (DSS) that addresses the problem of determining the target dimension for dimensionality reduction during Non-negative Matrix Factorization (NMF), thus adaptively determining the number of features. The FIC module introduces the Shapley value method to calculate feature importance more fairly and accurately. Specifically, the unsupervised interpretation method proposed in this invention includes:

[0056] Step 1: Prepare image datasets, including the Mini-ImageNet 2012 dataset and the PASCAL VOC2012 dataset; also prepare the pre-trained VGG-16 model and the ResNet-50 model to be interpreted;

[0057] Step 2: Preprocess the two datasets separately. For the training set, randomly select 1000 image samples from the validation sets of both datasets. For the test set, the Mini-ImageNet2012 dataset has 1000 categories and is a subset of the ImageNet dataset. Randomly select one image from each category in its validation set to form the test set, for a total of 1000 images. The PASCAL VOC 2012 dataset contains 20 categories and 17125 images; randomly select 1000 images to form the test set.

[0058] Step 3: Extract a sample image X from the test set and input it into the feature-level interpretability module and the adaptive feature representation module to obtain the feature-level interpretability and feature mask image, specifically including:

[0059] Step 31: Process a sample image X from the test set into a three-channel RGB image with a resolution of (224, 224), and standardize it according to the preset mean and standard deviation;

[0060] Specifically, the mean parameter is (0.485, 0.456, 0.406) and the standard deviation parameter is (0.229, 0.224, 0.255).

[0061] Step 32: Input the sample image X and its class label into the interpreted DNN network model (VGG-16 or ResNet-50), and then extract the feature map V in the last convolutional layer.

[0062] For convolutional layers, the VGG-16 model chooses the last feature layer, and the ResNet-50 model chooses the fourth feature layer. At this point, the dimension of the feature map V is c×h×w. Next, nonnegative matrix factorization (NMF) is used to reduce the dimensionality of the feature map V, i.e., V≈DU, where D represents the basis matrix, U represents the coefficient matrix, D has a dimension of c×k, and U has a dimension of k×h×w. The column vectors of D represent k basis vectors, each of which can be considered a feature direction. The coefficient matrix U is the projection length of the feature map V onto the column vectors of the basis matrix D, where k is the target dimension, typically less than c or h×w. The coefficient matrix U represents the feature matrix of k features.

[0063] Step 321: Input the hyperparameter information entropy a′ into the Dimension Scaling Solution (DSS) module. The input of the Non-negative Matrix Factorization (NMF) module is the feature map V and the target dimension k, and the output is the coefficient matrix U. An information entropy H(k) can be calculated using the coefficient matrix U. Therefore, it is necessary to determine whether the information entropy H(k) of the coefficient matrix U is close to or equal to a′ to determine whether the information content of the coefficient matrix U meets the requirements, that is, to determine whether H(k)-a′<δ (δ=0.01) holds true. The specific approach is: repeatedly perform non-negative matrix factorization to iteratively search for the mapping of H(k) with respect to k that makes H(k)-a′<δ true. It is known that H(k) has a monotonically decreasing trend with respect to the target dimension k. Therefore, in the interval k∈[6,30], binary search is used to accelerate the iteration. After passing through the Dimension Scaling Solution (DSS) module, the optimal solution for the target dimension k and the feature matrix U are obtained.

[0064] Step 33: Treat the coefficient matrix U as a feature matrix with a target dimension of k, and it has spatial correlation with the sample image X. The heat value region in the matrix indicates that the feature has strong expression at this spatial location.

[0065] Then, the coefficient matrix U is upsampled, normalized, and binarized to obtain k binary masks U of the same resolution as the sample image X, representing the target dimension k. mask .

[0066] Specifically, upsampling uses bilinear interpolation and normalizes to [0,1] using the maximum and minimum values. During binarization, the feature region is preserved by a threshold θ (θ = 0.3), that is, the elements in the matrix that exceed the maximum value multiplied by 0.3 are set to 1, and the elements in other positions are set to 0.

[0067] Step 34: Apply the binary mask U maskRegions with a median value of 1 are used to retain pixels from the sample image X, while regions with a median value of 0 are used to blur the sample image X. Simultaneously, the boundaries between 0 and 1 are visually enhanced. The final image represents the display features in the sample image X, i.e., feature-level interpretability, such as... Figure 2 As shown, this visualization demonstrates that an entity (image) is decomposed into semantically meaningful parts (features) of a target dimension k.

[0068] Step 35: Apply the binary mask U mask The region with a median value of 1 is used to retain the pixels of the sample image X, and the region with a median value of 0 is used to mask the pixels of the sample image X. The generated feature mask image is input into the feature importance calculation module FIC.

[0069] Step 4: Using the feature mask image obtained in Step 35 as the feature to be used to calculate importance, the Shapley value algorithm is introduced next, specifically including:

[0070] Step 41: Treat the k feature mask images as k features or input units, and calculate the importance of the k features. The specific steps are as follows:

[0071] Step 411: Let N represent the set of k features, and j represent a feature in N, j = {1, 2, ..., k}. Taking feature j as an example, calculate the importance of feature j;

[0072] Step 412: Perform a multilinear sampling in N\{j}, using L as the sampling feature. i Let i = {1, 2, ..., QM} represent the multilinear sampling result L = {L1, L2, ..., L...} QM}, where Q and M are positive integers L i It is a Bernoulli sequence with values ​​of 0 or 1.

[0073] Step 413: Use combined features L i ∪{j} represents the output value obtained by adding feature j to the sampled features and feeding it back into the DNN network model (VGG-16 or ResNet), denoted by Yc(L i ∪{j}) is used to represent this. Therefore, Yc(L) is used. i ∪{j}-Yc(L i The result of )) represents the sampling feature L i The marginal contribution, where c represents the category of interest. When there is a label, c represents the labeled category; when there is no label, c represents the predicted category of the sample image X. Additionally, the combined feature L... i The input ∪{j} still needs to be standardized before being fed into the DNN network.

[0074] Step 414: According to the multilinear sampling requirements, QM sampling is required, Q=16, M=4, that is, repeat steps 412 and 413 QM times and sum the marginal contributions of each sampling to obtain the Shapley value of feature j, that is, the importance of feature j;

[0075] Step 42: Repeat step 41 to calculate the importance of all k features and obtain the importance vector;

[0076] Step 5: Assign the importance vector as weights to the corresponding features. That is, linearly combine the importance vector and the coefficient matrix U to obtain the class activation map. Then, fuse the heatmap and the class activation map and upsample it to the input size (m, n) of the sample image X. Then, normalize the maximum and minimum values ​​to [0,1] to obtain the saliency map.

[0077] Step 6: Repeat steps 3 to 5 to generate a significance map for each sample in the test set;

[0078] Step 7: The element values ​​of the saliency plot of sample image X represent the importance of corresponding pixels in sample image X. Retain the top 20% of pixels from sample image X according to their importance to generate a new image. Then, re-input this image into the DNN network model and observe the magnitude of the decrease or increase in the output value before Softmax. Statistically analyze the results of all samples to obtain two indices: average decrease (AD) and average increase (AI). Alternatively, delete 3.6% of pixels from sample image X according to their importance each time, or insert 3.6% of pixels according to their importance each time into a blank image. Generate 28 images and re-input them into the network model to observe the change curve of the output value. The area of ​​the graph below the curve and the x-axis represents the deletion or insertion indices.

[0079] The calculation of the hyperparameter information entropy a′ specifically includes:

[0080] The training set described in step 2 is used to measure the information entropy a′, which is used to adaptively determine the target dimension (number of features) during data dimensionality reduction. Specifically:

[0081] Let S represent the training set. Based on experience, the initial range of the target dimension k is set to [6, 30]. Figure 1 The algorithm calculates the feature map extracted from the last convolutional layer for each sample in the training set within the interval k∈[6, 30]. The coefficient matrix U, after non-negative matrix factorization (NMF), is then used to calculate the average information entropy *a* of each channel. The mean of the average information entropy *a* of all sample images in the training set is calculated, and the curve H is plotted. s (k), and continue generating a saliency map according to the algorithm flow. For example... Figure 4As shown, the average interpretability performance P of all samples is statistically analyzed, and curve Ps is plotted. A perpendicular line k = k' is drawn from curve Ps at the point where it reaches a local maximum, intersecting curve H. s (k) intersect, and the coordinates of the intersection point are (k′, a′), where a′ is the desired information entropy;

[0082] The determination of information entropy a′ can be used as an empirical value for repeated use and fine-tuning on different datasets. Essentially, it is a constraint on the amount of feature information. k′ can also be treated in the same way.

[0083] To verify the effectiveness of the method of this invention, it is compared with other existing methods. For a fair comparison, the officially released code of other methods is used, and their core algorithms and experimental settings are followed. Furthermore, all methods are implemented in the same computational environment and have the same data processing methods, including uniform input image resolution and normalization. The eight methods are as follows: Method 1, Grad-CAM, is a classic saliency visual interpretation method that uses backpropagation gradients to quickly and efficiently calculate the activation map weights of each channel and generate a saliency map. Method 2, Grad-CAM++, proposes a weighted gradient strategy to improve the model's ability to distinguish multiple objects of the same category during interpretation. Method 3, RISE, constructs a general black-box interpretation framework. It uses a Monte Carlo method to calculate the importance of each input unit using a large number of randomly sampled input masks and emphasizes interpreting any black-box model without changing the network structure. Method 4, Score-CAM, uses a gradient-free score-weighted strategy to calculate the activation map weights of each channel, aiming to avoid noise in the saliency map interpretation caused by gradient weighting. Method 5, Group-CAM, merges activation maps of similar classes by grouping them, reducing the number of times the model is revisited and speeding up computation. Method 6, Layer-CAM, weighted and fused activation maps from multiple layers to generate a saliency map with precise localization information and clear localization boundaries. Method 7, FD-CAM, combines the advantages of gradient and score weighting methods, using a grouping weighting strategy to selectively close activation channels of similar classes, improving interpretation accuracy. Method 8, Recipro-CAM, uses spatial feature masks to weight the importance of feature maps based on spatial correlation to quickly generate a saliency map.

[0084] Tables 1 and 2 present quantitative comparisons of the average decrease in AD, average increase in AI, deletion (Del) metric, and insertion (Ins) metric for nine explanation methods (including the method of this invention) on two different datasets and two explained models (VGG-16 and ResNet-50), respectively. Additionally, "Over All" refers to the difference between insertion and deletion.

[0085] Table 1. Quantitative comparison results of different methods and models on the ILSVRC 2012 validation set.

[0086]

[0087] Table 2. Quantitative comparison results of different methods and models on the PASCAL VOC 2012 dataset.

[0088]

[0089]

[0090] The Average Decline (AD) metric measures the decrease in the number of retained pixels compared to the original output, typically ranging from 0 to 100, with a value closer to 0 being better. Conversely, the Average Increase (AI) metric counts the number of samples where the number of retained pixels increases compared to the original output, also typically ranging from 0 to 100, with a value closer to 100 being better. Pixels are progressively deleted or inserted according to importance, and the model is revisited. The resulting output is then plotted as a curve. The area of ​​the curvilinear shape formed by the curve and the x-axis represents the Del and Ins metrics, both ranging from 0 to 100. The data in the table represents the mean of all samples. The data in the table uses a percentage count method, expanding the measurement metrics by 100 times. A comparison of the results in Tables 1 and 2 shows that the method of this invention yields better results than all other methods, indicating that its significance interpretation results have better accuracy and persuasiveness, and are closer to the true decision-making logic of deep neural network models. The performance of this invention on two different models, VGG-16 and ResNet-50, shows that it outperforms other methods, indicating that it can be used to interpret different black-box models and has good generality. The performance on different datasets also shows that this invention outperforms other methods, indicating that it is applicable to different data environments and has good versatility and robustness.

[0091] To more intuitively illustrate the effectiveness of the method of this invention, we take the white shark of category n01484850 in the ILSVRC 2012 validation set as an example, and compare the effect graphs of the existing methods and the method of this invention on the interpretation of the saliency map of VGG-16. The experimental results are as follows: Figure 5 shown. Specifically, Figure 5 (a) represents the original sample image. Figure 5 (b) shows the interpretation results of Grad-CAM using method 1. Figure 5 (c) represents the interpretation result of Grad-CAM++ for method 2. Figure 5 (d) represents the interpretation results of RISE using method 3. Figure 5(e) represents the interpretation results of Score-CAM using method 4. Figure 5 (f) represents the interpretation results of Group-CAM in Method 5. Figure 5 (g) represents the interpretation results of Layer-CAM using method 6. Figure 5 (h) represents the interpretation results of FD-CAM using method 7. Figure 5 (i) represents the interpretation result of Recipro-CAM in method 8. Figure 5 (j) indicates the method of the present invention. Comparing the figures, it can be seen that the salient region of the method of the present invention is more concentrated and accurate, and there is no noise interpretation.

[0092] It should be noted that the specific embodiments described above are exemplary, and those skilled in the art can devise various solutions inspired by the disclosure of this invention. These solutions all fall within the scope of this invention and its protection. Those skilled in the art should understand that this specification and its accompanying drawings are illustrative and not intended to limit the scope of the claims. The scope of protection of this invention is defined by the claims and their equivalents.

Claims

1. An unsupervised explanation method based on adaptive semantic features, characterized in that, The application discloses a visual explanation method for a classification task of a deep neural network, and provides feature-level explainability and significant visual explanation. The feature-level explainability module unsupervisedly extracts high-level semantic features to provide feature-level explainability, the adaptive feature expression module converts how many input units are extracted into a dimension scaling solution to solve the target dimension determination problem of dimension reduction in a non-negative matrix decomposition process, and the feature importance calculation module introduces a Shapley value method to calculate the importance of features, specifically including the following steps. Step 1: preparing image datasets, including a Mini-ImageNet 2012 dataset and a PASCAL VOC 2012 dataset; Step 2: preprocessing the two datasets, for the training set, 1000 image samples are randomly extracted from the validation sets of the two datasets, respectively, for the test set, one image is randomly extracted from 1000 classes of the Mini-ImageNet 2012 dataset to form a test set, and 1000 images are randomly extracted from the PASCAL VOC 2012 dataset to form a test set; Step 3: extracting a sample image X in the test set and inputting the sample image X into the feature-level explainability module and the adaptive feature expression module to obtain feature-level explainability and a feature mask image, specifically including the following steps. Step 31: processing the sample image X into a three-channel RGB image with a resolution of 224x224, and normalizing the sample image X according to preset mean and standard deviation; Step 32: inputting the sample image X and a class label into a DNN network to be explained, and then cutting out a feature map V in the last convolutional layer; Step 321: inputting the information entropy a' of the hyperparameter into a dimension scaling solution module DSS, inputting the feature map V and a target dimension k into a non-negative matrix decomposition module, and outputting a coefficient matrix U; Step 33: take the coefficient matrix U as a characteristic matrix of the target dimension k, and with the spatial correlation of the sample image X, up-sample, normalize and binarize the coefficient matrix U to obtain target dimension k binary masks U with the same resolution as the sample image X mask ; Step 34: binary mask U is obtained by the following formula mask The region with a median value of 1 is used to retain the pixels of the sample image X, the region with a value of 0 is used to fog the sample image X, and the boundary of 0 and 1 is visually enhanced, and finally the obtained image is the expression of the feature in the sample image X, that is, the feature interpretability. Step 35: a binary mask U is generated mask The region with a median value of 1 is used to retain the pixels of the sample image X, and the region with a value of 0 is used to mask the pixels of the sample image X, and the generated feature mask image is input into a feature importance calculation module. Step 4: taking the feature mask image obtained in step 35 as a feature needed to calculate importance, and introducing a Shapley value method, specifically including the following steps. Step 41: regarding the k feature mask images as k features, and calculating the importance of the k features, specifically including the following steps. Step 411: using N to represent a set of the k features, using j to represent one feature in N, j={1, 2,..., k}, and taking a feature j as an example to calculate the importance of the feature j; Step 412: performing a multi-linear sampling in N(j), the sampling feature is L i , i = {1, 2,..., QM}, to obtain a multi-linear sampling result L = {L1, L2,..., L QM}, wherein Q, M are positive integers, L i is a Bernoulli sequence whose value is 0 or 1; Step 413: use the combined feature L i ∪{j} represents the multi-linear sampling result plus the feature j, which is re-fed into the DNN network to obtain the output value Yc(L i ∪{j}), and similarly Yc(L i ) is obtained and Yc(L i ∪{j})-Yc(L i ) is calculated, which is the marginal contribution of the combined feature of this sampling result. Step 414: according to a multi-linear sampling requirement, QM times of sampling of steps 412 and 413 are required, and a marginal contribution of each time of sampling is accumulated and summed to obtain a Shapley value of the feature j, that is, the importance of the feature j; Step 42: repeating step 41 to calculate the importance of all the k features and obtain an importance vector. Step 5: The importance vector is assigned as a weight to the corresponding feature, that is, the importance vector and the coefficient matrix U are linearly combined to obtain the category activation map, then the category activation map is up-sampled to the input size (m, n) of the sample image X, and is normalized to [0, 1] by maximum and minimum value, and is fused with the sample image X to obtain the saliency map; Step 6: Repeat steps 3 to 5 to generate a saliency map for each sample in the test set; Step 7: According to the element value size of the saliency map of the sample image X, the importance of the corresponding pixel of the sample image X is represented.

Images