A method for interpreting a deep learning model for image classification based on visual semantic information

Abstract

The present invention relates to a model interpretation method in the field of computer vision, and particularly relates to obtaining model interpretability statements for an image classification deep learning model to improve the model interpretation effect, including the following steps: calculating the confidence score of the neurons of the classification model; using backpropagation to derive the corresponding neuron weights and multiplying them by the confidence as the final score to find important neurons; using a class activation map to visualize the attention regions of the important neurons on the image, extracting visual features, and constructing a neuron visual feature dataset for the corresponding class in the same way; annotating the corresponding semantic information for the visual feature dataset and training it using a classification network; using the trained classification network to extract the semantic information corresponding to the visual features of the important neurons; combining the neuron importance score, visual features, and semantic information to form an interpretive statement describing the classification process of this model.

Description

Technical Field

[0001] The present invention relates to a method for interpreting models in the field of computer vision, and particularly to obtaining model interpretability statements for image classification deep learning models. Background Art

[0002] In recent years, the development of deep learning represented by Convolutional Neural Network (CNN) has significantly improved the execution ability of computer vision related tasks. However, due to the inherent black box characteristics of convolutional neural networks, there are still huge challenges in interpreting deep CNNs. Even if the model produces correct outputs, reasonable reasoning is required to prevent people from distrusting the results. Therefore, in addition to performance, the interpretability of neural networks is crucial.

[0003] In order to understand the features learned by the network, more and more research uses visualization to explore neural networks. By extracting the weights between the feature map and the fully connected layer of the target category, directly weighted summation and normalization are used to obtain the Class Activation Mapping (CAM). However, this method only explores the region of interest of the features in the last layer, and does not give a clear explanation of how the network classifies according to this region. Summary of the Invention

[0004] To solve the above problems, the purpose of the present invention is to solve the problem that the existing class activation map method only focuses on the visualization of the features in the last layer and the classification process of the deep learning model is vaguely explained, and a method for interpreting an image classification deep learning model based on visual semantic information is proposed.

[0005] A method for interpreting an image classification deep learning model based on visual semantic information of the present invention includes the following steps:

[0006] 1. Calculate the difference in classification error before and after masking the neurons of the classification model and use it as the confidence score;

[0007] 2. Use backpropagation to deduce the corresponding neuron weights and multiply them by the confidence score to find the important neurons as the final score;

[0008] 3. Use the class activation map to visualize the attention regions of the important neurons on the image, extract visual features, and construct a neuron visual feature dataset for the corresponding class in the same way;

[0009] 4. Label the visual feature dataset with the corresponding semantic information and use a classification network to train it;

[0010] 5. Use the trained classification network to extract the semantic information corresponding to the visual features of the important neurons;

[0011] 6. Combine the neuron importance score, visual features, and semantic information to form an explanatory statement describing the classification process of this model.

[0012] In some embodiments, the classification model uses the VGG16 model. Forward propagation is performed by masking a certain neuron in the 18th convolutional layer to calculate the confidence score. The calculation process is as follows:

[0013]

[0014] Where X is the input image, X0 is the baseline image, and f() is the neural network for input class scores. is the feature map of the kth layer, and l is the lth convolutional layer of the neural network. ⊙ is the Hadamard product, that is, the corresponding elements of the matrices are multiplied. The obtained is the confidence score of the neuron.

[0015] The process of obtaining the weights through backpropagation is as follows:

[0016]

[0017] Where z 18 is the input value of the 18th convolutional layer, and δ 18 is the backpropagation error of the 18th pooling layer. The obtained is the importance of the 18th layer neuron obtained through backpropagation.

[0018]

[0019] CGNIS is the final important neuron score.

[0020] In some embodiments, the class activation map uses HCScore-CAM to visualize the attention area of important neurons.

[0021] In some embodiments, the classification network uses IRENet to train the visual feature dataset. IRENet improves VGG16 by replacing the intermediate layer convolution with the SIRM (Simplified inception residual module) module and adding the ECA (Efficient Channel Attention) attention mechanism.

[0022] On the other hand, the present invention also provides a deep learning model interpretation device that can implement the above embodiments. The device includes:

[0023] An important neuron acquisition module, which is used to acquire the neurons that play an important role in the model training results;

[0024] An important neuron visual feature extraction module, which is used to visualize the attention area of important neurons, intercept and label it as a visual feature data set;

[0025] An important neuron semantic information extraction module, which is used for the deep learning model to train the visual feature data set and extract the semantic information of the visual features of important neurons, and finally combines information such as visual features and the importance degree of important neurons to form an explanatory statement.

[0026] The beneficial effects of the present invention are at least: in the image classification deep learning model interpretation method and device based on visual semantic information, important neurons of the deep learning model are obtained, the visual features of important neurons are extracted and a visual feature data set is made by using the class activation map, and the semantic information of the visual features is extracted by using the classification network, which can more accurately give an explanatory statement of the learning process of the deep learning model.

[0027] The additional advantages, objects, and features of the present invention will be partially described below, and will become partially apparent to those of ordinary skill in the art after studying the following text, or can be learned from the practice of the present invention. The objects and other advantages of the present invention can be achieved and obtained by the structures specifically pointed out in the written description, its claims, and the drawings.

[0028] Those skilled in the art will understand that the objects and advantages that can be achieved by the present invention are not limited to the above specifically described, and the above and other objects that the present invention can achieve will be more clearly understood according to the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] By reading the following detailed description of the preferred embodiments, various other advantages and benefits will become clear to those of ordinary skill in the art. The drawings are only for the purpose of showing the preferred embodiments and are not considered to be a limitation of the present invention. In the drawings:

[0030] Figure 1 is a flowchart of an image classification deep learning model interpretation method based on visual semantic information of the present invention;

[0031] Figure 2 is an example diagram of the model structure of an image classification deep learning model interpretation method based on visual semantic information of the present invention;

[0032] Figure 3 is a derivation diagram of the neuron importance degree scoring algorithm;

[0033] Figure 4 is a visual feature extraction process diagram;

[0034] Figure 5It is the structural diagram of the IRENet classification network;

[0035] Figure 6 It is the structural diagram of the SIRM module. Specific implementation manner

[0036] The exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided so that the present invention can be more thoroughly understood and the scope of the present invention can be fully conveyed to those skilled in the art.

[0037] Figure 1 It is the flowchart of a method for interpreting a deep learning model for image classification based on visual semantic information. The basic process of the method for interpreting a deep learning model for image classification based on visual semantic information is described, and specifically includes the following steps:

[0038] Step 1: Calculate the confidence score of the neurons in the classification model;

[0039] Step 2: Use backpropagation to derive the corresponding neuron weights and multiply them by the confidence score as the final score to find important neurons;

[0040] Step 3: Use the class activation map to visualize the attention regions of the important neurons on the image, extract visual features, and construct a visual feature dataset of the neurons corresponding to the same class in the same way;

[0041] Step 4: Label the corresponding semantic information for the visual feature dataset and use the classification network to train it;

[0042] Step 5: Use the trained classification network to extract the semantic information corresponding to the visual features of the important neurons;

[0043] Step 6: Combine the neuron importance score, visual features, and semantic information to form an explanatory statement describing the classification process of this model.

[0044] Figure 2 It is the instance model structural diagram of a method for interpreting a deep learning model for image classification based on visual semantic information of the present invention. The present invention is described more specifically in combination with an instance, and includes the following steps:

[0045] This instance describes the specific process of the present invention with VGG16 as the prototype. The method for extracting important neurons includes the above-mentioned Step 1 and Step 2, named CGNIS, as Figure 3 shown, and the specific implementation steps are as follows:

[0046] After the convolution kernel pooling operation of the 18th layer of VGG16, a 7×7×512 feature map is obtained, which is converted into a 25088-dimensional vector as the input of the fully connected layer. The output of this process is a 18 .

[0047]

[0048] is the feature map after 18 layers of pooling, and F() is the fully connected operation. The calculation formula for the three fully connected layers is shown in Formula 2. Where w, a, and b represent the weight, input value, and bias, respectively.

[0049] z l+1 =w l+1 a l +b l+1 (2)

[0050] Finally, the output layer uses the softmax activation function, as shown in Formula 3.

[0051]

[0052] After obtaining the classification results, back propagation is used to deduce the importance of each neuron in the high-level convolutional layer.

[0053] First, the network adopts the cross entropy loss function, as shown in Formula 4, a L is the activation vector of the output layer, y is the sample label vector, and i is the number of neurons.

[0054]

[0055] Represents the output layer error, and the reverse error of the entire output layer can be obtained using Formula 5. L Represents the input value of the current layer.

[0056]

[0057] Then, according to the chain rule, the back propagation error of the jth node in the first hidden layer is calculated, and the back propagation error formula of the lth fully connected layer is obtained as shown in Figure 6.

[0058] δ l =(w l+1 ) T δ l+1 ⊙ReLU(z t )′ (6)

[0059] When the back propagation reaches the 19th layer, there will be a back propagation error from the fully connected layer to the pooling layer, as shown in Formula 7, where F -1Represents the inverse operation of the fully connected, δ 18 is the pooling layer of the 18th layer.

[0060] δ 18 = F -1 ((w 19 ) T )δ 19 (7)

[0061] When deriving the backpropagation error of the 18th convolutional layer from δ 18 for max pooling, upsampling is required to place the elements of each channel in δ 18 at the positions of the previous forward propagation maxima, and the other element values are set to 0. The backpropagation formula is as shown in Equation 8.

[0062]

[0063] The obtained from Equation 8 is a tensor of 14×14×512. By taking the average of the first two dimensions of this tensor to reduce the dimension, becomes a vector of 1×512, which is the importance degree of the neurons in the 18th layer obtained through backpropagation.

[0064] Finally, multiply the obtained confidence score bitwise with the obtained by backpropagation to obtain a vector of 1×512, where each value represents the influence degree of the neurons at the corresponding positions of the last convolutional layer on the current image. The calculation method is as shown in Equation 9.

[0065]

[0066] Furthermore, use HCScore-CAM to visualize the visual features of important neurons.

[0067] Specifically, as Figure 4 shown, first upsample these neurons to the same size as the original image, and then multiply the neurons with the original image to obtain the HCScore-CAM map of the neurons, generating the neuron class activation mapping region. To extract more discriminative local visual features, first set the hyperparameter threshold θ1, and then calculate the required mask for the neurons, that is, as shown in Equation 10.

[0068]

[0069] where A represents the original neuron feature map after upsampling, (i, j) represents the position of the pixel point, and M represents the processed feature map.

[0070] Let x min and xmax respectively represent the minimum and maximum values of the abscissa with a value of 1 in M, y min and y max respectively represent the minimum and maximum values of the ordinate with a value of 1 in M. According to these four coordinates, the area of interest of the current neuron can be located. If X is the original input image with height and width h and w respectively, and p is the padding size, then the calculation formula for the coordinates of the boundary box of the area to be cropped is as shown in 11.

[0071]

[0072] After determining the cropping area, multiply the processed convolutional feature map by the original image and crop the corresponding area: C = X[x1:x2, y1:y2] (12)

[0073] Finally, upsample the cropped image to the size of the original image.

[0074] Furthermore, use the above method for obtaining visual features to extract features and perform manual annotation to construct a neuron visual feature dataset for the corresponding class.

[0075] Furthermore, use IRENet to train the constructed visual feature dataset.

[0076] Specifically, the network structure of IRENet is as Figure 5 shown. Replace the 3*3 convolution in the middle layer of VGG16 with the SIRM module to improve the network classification accuracy. In addition, the effective channel attention mechanism ECA is also used. ECA realizes an ultra-lightweight attention module and can be flexibly integrated into the existing CNN architecture. The SIRM module uses 1*1 convolution and 3*3 convolution for multi-scale combination to replace the original 3*3 convolution, and reduces the multi-scale branches, improving the inference speed. In addition, this module adopts the residual idea of ResNet, as Figure 6 shown. There are two types of residual structures in this module. One adds a residual structure at the 1*1 convolution and 3*3 convolution, and the other adds a residual structure at the fusion of each branch, enabling the deeper layers of the network to obtain more robust feature representations.

[0077] Furthermore, use the trained IRENet to identify the important neuron visual features extracted from the input picture and extract the corresponding semantic information.

[0078] Furthermore, combine the above-obtained neuron importance scores, visual features, and semantic information to describe the interpretive statements of the VGG16 classification process for the input image.

[0079] Based on the same inventive concept, the present invention also provides a deep learning model interpretation device that can implement the above steps. The device includes:

[0080] An important neuron acquisition module, which is used to acquire neurons that play an important role in the results during model training; an important neuron visual feature extraction module, which is used to visualize the attention area of important neurons, intercept and label it as a visual feature data set; an important neuron semantic information extraction module, which is used for a deep learning model to train the visual feature data set and extract the semantic information of the visual features of important neurons, and finally combine information such as visual features and the importance of important neurons to form an explanatory statement.

[0081] The device of the above embodiment is used to implement the corresponding method in the foregoing embodiment, and has the beneficial effects of the corresponding method embodiment, which will not be elaborated here.

[0082] Those of ordinary skill in the art should understand that the discussion of any of the above embodiments is only exemplary, and is not intended to imply that the scope of the present disclosure (including the claims) is limited to these examples; under the concept of the present invention, the technical features in the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations in different aspects of the present invention as described above, which are not provided in detail for the sake of brevity.

[0083] Embodiments of the present invention are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omission, modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for interpreting a deep learning model for image classification based on visual semantic information, characterized in that Including: For the trained classification model, mask the neurons in the last convolutional layer, and use forward propagation to calculate the classification errors before and after masking as the confidence scores; Use backpropagation to derive from the last layer of the classification model to the last convolutional layer, obtain the weights of each neuron in the last convolutional layer, multiply the weights by the confidence scores as the final scores to find the important neurons; Upsample the important neurons to the same size as the original image, then multiply the neurons by the original image to obtain the class activation map of the neurons, generate the neuron class activation mapping region, extract visual features, and construct the neuron visual feature dataset of the corresponding class in the same way; Annotate the corresponding semantic information for the visual feature dataset and use the classification network to train it; Use the trained classification network to extract the semantic information corresponding to the important neuron visual features; Combine the neuron importance scores, visual features, and semantic information to form explanatory statements describing the classification process of this model.

2. The method for interpreting an image classification deep learning model according to claim 1, wherein the neuron is characterized in that: The neurons are masked by setting their attribute weight to 0.

3. The method for explaining an image classification deep learning model according to claim 1, wherein: The extraction of visual features first sets a hyperparameter threshold , and then calculates the required mask for the neuron to make the original image only show more specific visual features of this neuron, crops this feature, and upsamples it to the size of the original image as the final visual feature.

4. A deep learning model interpretation device that can implement the image classification deep learning model interpretation method described in claim 1, including: An important neuron acquisition module, which is used to acquire the neurons that play an important role in the model training; an important neuron visual feature extraction module, which is used to visualize the attention area of the important neurons, intercept and label it as a visual feature dataset; an important neuron semantic information extraction module, which is used to train the visual feature dataset by the deep learning model and is used to extract the semantic information of the important neuron visual features, and finally combine the important neuron weights, visual features, and semantic features to generate explanatory statements.

Images