Image classification method for improving attention of target region of image

Abstract

The present application relates to the field of explainable artificial intelligence, and particularly relates to an image classification method for improving the attention of image target regions, comprising obtaining an image set to be classified and a label set thereof, and preprocessing the image set to be classified to obtain a training set; using the training set and the label set to perform first-stage training on a convolutional neural network, and calculating classification loss through a cross-entropy loss function; obtaining an initial model by completing the first-stage training, and performing class feature visualization on the initial model using the training set to obtain channel semantic vectors under different label types; copying the weights of the initial model as initial weights of a second model, and performing second-stage training on the second model based on the channel semantic vectors and the training set; obtaining an explainable convolutional neural network model by completing the second-stage training, and applying the model to image classification; the method can effectively reduce false associations of the model, and improve the stability of the model deployed in different environments.

Description

Technical Field

[0001] This invention relates to the field of explainable artificial intelligence, and more specifically to an image classification method that improves the focus on target regions in images. Background Technology

[0002] Currently, deep neural network models have far surpassed human performance in many computer vision tasks, achieving very high performance. However, deep neural network models are essentially still data-driven artificial intelligence systems. Deep neural network models establish a correlation between input data and expected results. If the training and test data satisfy the independent and identically distributed (IOD) assumption—that is, the training and test data are independently sampled from the same distribution—the model can outperform human performance. However, in real-world scenarios, due to limitations and biases in the data collection process, the IOD assumption no longer holds. Therefore, this data-driven correlation learning inevitably learns spurious contextual relationships. If the model's deployment environment is constantly changing, its performance will significantly deteriorate when the distribution shifts. Furthermore, the lack of rigorous theoretical guidance in deep neural network architecture design and the end-to-end data-driven training process inherently make the models uninterpretable.

[0003] In recent years, researchers have attempted to open the "black box" of deep neural networks through interpretable methods, proposing numerous approaches to explain the models. Among these, Class Activation Mapping (CAM) is a widely used image classification attribution method. This method highlights the regions in the input image most relevant to the prediction result, facilitating human attribution of prediction outcomes. We used the widely adopted GradCAM method to generate saliency maps and found that the model's decision-making mechanism is not entirely consistent with human behavior. Our analysis revealed that the model's decision-making process utilizes image background information; that is, given limitations and biases in the training data, identifying a dog might utilize information such as grass. Furthermore, the saliency maps revealed that the model exhibits attentional distraction, meaning it tends to focus its attention on less important features. Summary of the Invention

[0004] To address the above problems, this invention provides an image classification method for improving the attention given to target regions in an image, comprising the following steps:

[0005] S1. Obtain the set of images to be classified and their label set, and preprocess the set of images to be classified to obtain the training set;

[0006] S2. The first stage of training of the convolutional neural network is performed using the training set and the label set, and the classification loss is calculated using the cross-entropy loss function.

[0007] S3. Complete the first stage of training to obtain the initial model, and use the training set to visualize the class features of the initial model to obtain the channel semantic vectors under different label types;

[0008] S4. Copy the weights of the initial model as the initial weights of the second model. Based on the channel semantic vectors and the training set, perform the second stage of training on the second model, including:

[0009] S41. Input the sample images from the training set into the sample image category of the second model, and use the cross-entropy loss function to calculate the classification loss of the second model;

[0010] S42. Use the AnnoCAM method to obtain the first saliency map of the sample image under the initial model;

[0011] S43. Use the HiResCAM method to obtain the corresponding second saliency map of the second category image of the sample image;

[0012] S44. Binarize the first saliency map to obtain a binary saliency map, and calculate the interpretation consistency loss based on the binary saliency map and the second saliency map;

[0013] S45. The training loss is obtained by weighted summation of classification loss and interpretation consistency loss, and the training loss is backpropagated to optimize the parameters of the second model.

[0014] S46. Repeat steps S41-S45 until the parameters of the second model converge;

[0015] S5. Complete the second stage of training to obtain an interpretable convolutional neural network model, and apply it to image classification.

[0016] The beneficial effects of this invention are:

[0017] This invention visualizes channel semantics based on the semantic detection function of convolutional kernels. Based on the visualization results, convolutional kernels are filtered and labeled into two categories: kernels representing foreground regions and kernels representing background regions. When generating saliency maps using HiResCAM, only kernels representing foreground regions are used for calculation, thus effectively generating saliency maps that only represent the foreground region. This method is called AnnoCAM (Annotation-based Class Activation Map).

[0018] This invention proposes a two-stage training method. The first stage uses cross-entropy loss to train the model. After training, the AnnoCAM method is used to generate a saliency map that focuses only on the foreground. Then, the saliency map is further processed... Figure 2Value-based masks that satisfy expected properties are used to constrain model interpretation. This paper uses the Energy-based Pointing Game (EPG) and Content Heatmap (CH) metrics to evaluate the proposed method. EPG primarily assesses the degree to which the model's attention is focused on the foreground region, while CH primarily assesses the degree of concentration of the model's attention. Extensive experiments effectively validate that the proposed method can effectively reduce spurious associations in the model and improve the model's stability under different environments. Attached Figure Description

[0019] Figure 1 This is a flowchart of the method of the present invention;

[0020] Figure 2 This is a flowchart of the feature visualization method of the present invention;

[0021] Figure 3 This is a schematic diagram illustrating a visualization example of the features of this invention.

[0022] Figure 4 This is a schematic diagram of the second stage of training in this invention;

[0023] Figure 5 This is a visualization result of GradCAM in an embodiment of the present invention. Detailed Implementation

[0024] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] For image classification tasks, deep neural network models have far outperformed human performance. However, based on ex post facto interpretation methods, such as GradCAM, we have found that the model's decision-making regarding the region of interest is not entirely consistent with human decision-making. The inconsistency mainly manifests in the following ways:

[0026] (1) The model is particularly distracted when making decisions. By processing the model output image through post-interpretation methods to obtain a saliency map, it can be seen that the saliency map is still highlighted at unimportant features.

[0027] (2) When making decisions, models may consider background information, such as using the features of grass to identify a dog. However, humans mostly rely on the most discriminative object features to identify categories when recognizing objects.

[0028] Although some scholars have proposed methods to incorporate prior knowledge into the model training process, the model may still use background information when making decisions. Furthermore, while adding bounding boxes or pixel-level annotations can constrain the model to focus on foreground regions, the manual annotation process is costly. To better address these issues and focus the model's attention on the most discriminative foreground regions, this invention proposes an image classification method that improves the focus on target regions in images.

[0029] Specifically, such as Figure 1 As shown, the specific scheme of this method includes:

[0030] S1. Obtain the set of images to be classified and their label set, and preprocess the set of images to be classified to obtain the training set.

[0031] Specifically, after obtaining the set of images to be classified, each image is manually labeled with a category label, and all labels form a label set; each image to be classified in the set of images to be classified is enhanced by random cropping and random flipping transformations to obtain an enhanced training set.

[0032] S2. The convolutional neural network is trained in the first stage using the training set and the label set, and the classification loss is calculated using the cross-entropy loss function.

[0033] Specifically, the sample images in the training set are used as the input of the convolutional neural network, and the output is the sample image category; in the first stage of training in step S2, 90 epochs are trained, and the learning rate is reduced by 10 times every 30 epochs, with an initial learning rate of 0.1.

[0034] Specifically, the cross-entropy loss function used in step S2 is:

[0035]

[0036] Where N represents the number of sample images in the training set, and C represents the number of categories in the training set. This indicates whether the i-th sample image belongs to the c-th category; if so, then... If not, then f c Represents a convolutional neural network, x i Let θ represent the i-th sample image, and θ1 represent the trainable parameters of the convolutional neural network (initial model).

[0037] In real-world scenarios, training data can be limited and biased, causing the initial model to learn not only causal features within the target region of the sample image but also background features that are not causally related to the target features. This results in the initial model failing to perform well in different environments and exhibiting instability. Therefore, a second stage of optimization training is needed based on the initial model. The main purpose of the second stage of optimization training is to incorporate supervised information to guide the model in learning more target features, focusing the model's attention more on the most discriminative foreground region while reducing the learning of background features.

[0038] S3. After completing the first stage of training to obtain the initial model, use the training set to visualize the class features of the initial model to obtain the channel semantic vectors under different label categories.

[0039] Specifically, through network dissection experiments, it was found that the selectivity of convolutional kernels (channels) in different layers of the network for local content of images varies from shallow to deep. Each convolutional kernel can be used as a semantic detector for analysis. Furthermore, as the number of network layers increases, the semantics corresponding to the convolutional kernels become more specific. Deeper convolutional kernels can detect specific object concepts, which are more conducive to human understanding. Therefore, this invention mainly focuses on semantic analysis of the last layer convolutional kernel in the convolutional neural network. Maximum activation response visualization is the simplest method for visualizing the semantics of convolutional kernels. This method inputs all validation set images into the model, and then the convolutional kernel generates different activation maps for different input images. The batch of input images are sorted according to their activation intensity from largest to smallest. The activation map with the largest activation intensity and its corresponding input image are selected. The region features obtained by mapping the activation region on the activation map to the input image can represent the semantics of the convolutional kernel.

[0040] For complex datasets, such as ImageNet, this invention proposes a Class-wise Feature Visualization (CFV) method to enhance the semantic consistency of convolutional kernels. This method visualizes all convolutional kernels in the last layer of the convolutional neural network using training data from each class. However, if maximum activation response visualization is used to semantically visualize the kernels, features from other classes may be treated as features of the current class. To avoid this, this invention uses gradient-based class activation mapping as the semantic visualization map of the convolutional kernels.

[0041] Specifically, the initial model is visualized using the training set to obtain channel semantic vectors under different label categories, including:

[0042] S31. Let the number of label categories be C, and the number of channels in the last convolutional layer of the initial model be M; divide the training set into C sub-class training sets according to the labels.

[0043] S32. Using the training set of the subcategories corresponding to the c = 1, 2, ..., C label categories, visualize the class features of the initial model to obtain the channel semantic vector of the initial model under the c label category, such as... Figure 2 As shown, it includes:

[0044] S321. Input the i=1,2,…,n sample images from the training set corresponding to the c-th label category into the initial model, and extract the feature image output by the last convolutional layer of the convolutional neural network; perform gradient-based class activation mapping on the feature image to obtain the semantic view of each channel of the feature image in the last convolutional layer of the initial model; n represents the number of sample images in the sub-category training set;

[0045] S322. Repeat step S321 until all sample images in the training set corresponding to the c-th label category have completed gradient-based class activation mapping, so that each channel in the last convolutional layer of the initial model corresponds to n semantic visual images;

[0046] Specifically, gradient-based class activation mapping is performed on the feature image to obtain the semantic representation of each channel of the feature image in the last convolutional layer of the initial model, as follows:

[0047]

[0048] in, A represents the semantic view of the feature image belonging to the c-th label category in the m-th channel. m This represents the activation map of the m-th channel in the last convolutional layer of the initial model. This represents the gradient calculation of the activation map of the m-th channel under the c-th label category, where ReLU represents the activation function.

[0049] S323. For each channel in the last convolutional layer of the initial model, select the semantic view with the largest activation value from all its semantic views as its own channel semantic heatmap;

[0050] S324. In the last convolutional layer of the initial model, each channel is semantically annotated based on its own channel semantic heatmap. All channel semantic annotation results are arranged and combined according to their corresponding channel order to obtain the channel semantic vector.

[0051] Specifically, in step S324, each channel performs channel semantic annotation based on its own channel semantic heatmap, including: extracting the highlighted area in the channel semantic heatmap; if the highlighted area is located in the target area, the channel semantic annotation of that channel is 1; if the highlighted area is located in the background area, the channel semantic annotation of that channel is 0. This invention uses manual annotation for channel semantic annotation. To reduce the workload of manual annotation, all channels are sorted in descending order according to the activation value of the channel semantic heatmap, and only the first 10 or 20 channels are selected for manual annotation. The remaining channels with low intensity are assigned a default channel semantic annotation of 1.

[0052] Specifically, such as Figure 3 As shown, this paper visualizes the class features of the ResNet50 pre-trained model. Figure (a) visualizes the channel semantic heatmap corresponding to the sandbag class. It can be clearly observed that the highlighted areas corresponding to the channel semantic heatmaps in rows 1, 2, 3, and 4 are located in the target region (foreground region), that is, the convolution kernels represent the foreground features of the sandbag. However, the convolution kernels in rows 5 and 6 correspond to the semantics of the boxing gloves and no longer belong to the foreground region. In Figure (b), the highlighted areas corresponding to the channel semantic heatmaps in rows 1, 2, and 6 are located in the target region, and the highlighted areas corresponding to the channel semantic heatmaps in rows 3, 4, and 5 are located in the background region. Therefore, based on the class feature visualization analysis, it can be found that the model not only learns foreground features from the training data but also learns background information. At the same time, it is possible to manually and simply label whether the semantics of each convolution kernel belongs to the foreground or background features. By forming channel semantic vectors through channel semantic annotation, the model can achieve the desired results. Figure 3 In the diagram, the channel semantic vector corresponding to (a) is [1,1,1,1,0,0], and the channel semantic vector corresponding to (b) is [1,1,0,0,0,1].

[0053] S4. Based on the channel semantic vectors and the training set, perform a second stage of training on the initial model, such as... Figure 4 As shown, it includes:

[0054] S41. Copy the initial model weights as the initial weights of the second model, input the sample images from the training set into the initial model and the second model respectively, and calculate the classification loss of the second model using the cross-entropy loss function;

[0055] S42. The HiResCAM method is used to obtain the saliency map of the sample image under the second model, which is called the second saliency map and is represented as:

[0056]

[0057] The HiResCAM method considers activation maps for all channels, so it might use convolutional kernels representing background information to compute the saliency map. Of course, to faithfully reflect the model's decision-making mechanism, all convolutional kernels should be considered. However, to improve the saliency map localization capability, convolutional kernels representing background information can hinder this capability. Therefore, convolutional kernels can be manually selected, retaining those that only represent semantic information of the foreground region and discarding those representing background semantic information, thereby improving the saliency map localization capability.

[0058] S43. The AnnoCAM method is used to obtain the salient image of the sample image under the initial model, which is called the first salient image;

[0059] Specifically, step S43 uses the AnnoCAM method to obtain the first saliency map corresponding to the sample image under the initial model, including:

[0060] S431. Input a sample image from the training set into the initial model to obtain the corresponding sample image category;

[0061] S432. The AnnoCAM method is used to interpret and visualize the sample image and its category, resulting in the first saliency map, represented as follows:

[0062]

[0063] in, F represents the first saliency map of the sample images belonging to the c-th label category, and A represents... m This represents the activation map of the m-th channel in the last convolutional layer of the initial model. This represents the gradient calculation of the activation map of the m-th channel under the c-th label category, where ReLU represents the activation function, and ⊙ represents the element-wise multiplication of two matrices. This represents the channel semantic annotation of the m-th channel in the last convolutional layer of the initial model under the c-th label category.

[0064] S44. Binarize the first saliency map to obtain a binary saliency map, and calculate the interpretation consistency loss based on the binary saliency map and the second saliency map;

[0065] Specifically, the AnnoCAM method proposed in this invention reduces the focus on the background region. However, not all features in the foreground region are discriminative features. Therefore, a thresholding operation can be used to discard unimportant features by thresholding the first saliency map, thereby improving the localization capability of the first saliency map, as expressed as:

[0066]

[0067] in, This represents the enhanced saliency image obtained by thresholding the first saliency map belonging to the c-th label category, where τ represents the threshold and T() represents the thresholding operation, specifically:

[0068]

[0069] Indicates the first saliency map The pixel value of the j-th pixel.

[0070] The enhanced saliency map generated by thresholding the first saliency map significantly improves localization capabilities, but it does not faithfully reflect the model's decision-making process. The purpose of this invention is to ensure that the model's decision region is as consistent as possible with the saliency map region, thereby allowing the model's true decision to satisfy the expected property that "the highlighted region is located in the most discriminative target region in the image." Energy loss produces the best EPG score, and energy-based loss suppresses background features, focusing more on foreground features. Therefore, using energy loss to guide the model interpretation yields the best results. Energy loss requires a binarized Ground Truth mask. Therefore, this paper uses a binarized saliency map operation to obtain the Ground Truth mask. Only a threshold needs to be set to binarize the first saliency map generated by AnnoCAM, and its formal representation is as follows:

[0071]

[0072] in, This represents the binary saliency map obtained by binarizing the first saliency map belonging to the c-th label category, where τ represents the threshold and I() represents the binarization operation, specifically:

[0073]

[0074] Indicates the first saliency map The pixel value of the j-th pixel.

[0075] Both thresholding and binarizing the first saliency map require setting a threshold τ, with two main differences. First, the thresholded first saliency map takes a value between 0 and 1, while the binarized first saliency map takes a value of either 0 or 1. Second, the thresholded first saliency map is used to improve saliency map localization capabilities, primarily to verify that more focused attention helps reduce the degree of spurious associations in the model, while the binarized first saliency map is mainly used to generate a mask, which is used as the Ground Truth for the second-stage training of the model. The reason for not using the thresholded first saliency map as the Ground Truth for model training is that the purpose of model optimization in this invention is only to require that the interpretation of the second-stage model is within a certain target region; it is not required that the saliency map pattern within the second-stage model region be consistent with the saliency map region pattern generated by AnnoCAM in the first stage. This allows the model to adaptively learn within the region, improving the model's accuracy.

[0076] S45. The training loss is obtained by weighted summation of classification loss and interpretation consistency loss, and the training loss is backpropagated to optimize the parameters of the second model.

[0077] S46. Repeat steps S41-S45 until the parameters of the second model converge.

[0078] Specifically, the formula for calculating consistency loss is as follows:

[0079]

[0080] in, This represents the second saliency map obtained using the HiResCAM method. This represents the binary saliency map after the first saliency map obtained by the binarization AnnoCAM method, x i This represents the i-th sample image. Indicates the initial model. Let N represent the second model, and N represent the number of training set sample images.

[0081] The training loss is obtained by weighted summation of the classification loss and the interpretation consistency loss, and is expressed as:

[0082] L = L CE +λL AEC

[0083] Among them, L CE L represents the classification loss. ECC Let λ represent the explanation consistency loss, and let λ represent the hyperparameter.

[0084] S5. Complete the second stage of training to obtain an interpretable convolutional neural network model, and apply it to image classification.

[0085] In one embodiment, various experiments were conducted to verify the effectiveness of the proposed method. First, the localization capabilities of the saliency maps generated by AnnoCAM and HiResCAM under different thresholds were verified. Then, the effectiveness of the proposed method was verified on ImageNet. Next, ablation experiments were conducted to verify the selection of the hyperparameter λ between the cross-entropy loss and the energy-based class activation mapping consistency loss. Finally, the effectiveness of the proposed method was qualitatively evaluated using the post-interpretation method GradCAM.

[0086] This invention utilizes the PyTorch framework for all experiments. For the second stage of model training, this invention optimizes the model based on the official PyTorch pre-trained model, primarily using ResNet18 and ResNet50. This invention employs the SGD stochastic gradient descent algorithm for model optimization, initially setting the learning rate to 0.1, and then decreasing the learning rate by a factor of 0.1 every 30 epochs. The model's operating environment is shown in Table 1.

[0087] Table 1 Experimental Environment Configuration

[0088]

[0089] Preferably, in order to verify whether the AnnoCAM method can generate a saliency map that meets the expected properties, the present invention uses the EPG index to verify whether the saliency map generated by the method focuses on the foreground region. The larger the EPG value, the more the saliency map generated by the method focuses on foreground information.

[0090] This invention evaluates the AnnoCAM method on ResNet18 and ResNet50 pre-trained models under different thresholds. Table 2 shows the effectiveness of the AnnoCAM method; as the threshold increases, the saliency map becomes more focused, and the localization ability becomes stronger. Simultaneously, this invention also evaluates the HiResCAM method on ResNet18 and ResNet50 pre-trained models under different thresholds. Table 2 shows that as the threshold increases, the localization ability of HiResCAM improves significantly, but not as much as the improvement of the AnnoCAM method. This also indicates that semantic annotation of the convolutional kernel can effectively remove background information.

[0091] Table 2 evaluates the localization capability of the interpretation method under different thresholds τ.

[0092]

[0093] Preferably, the two-stage training method proposed in this invention is called AEC (Annotation-based Explanation Consistency). To verify the effectiveness of the AEC method, this invention directly performed secondary training on the ResNet18 and ResNet50 pre-trained models. As shown in Table 3, the model trained by this invention performs best on the EPG metric with a slight loss of accuracy. This experiment demonstrates that adding a small amount of supervision information and focusing the model on more discriminative regions effectively improves the model's causal decision-making ability. Table 3 also shows that although the method of this invention is slightly lower than the CGC method on the CH metric, it is slightly higher than the CGC method on the IAUC metric, indicating that the explanation generated by the HiResCAM method using the model trained by this invention (AEC) is more faithful to the model.

[0094] Table 3 shows the evaluation results on ImageNet.

[0095]

[0096] Table 4. Evaluation results of HiResCAM interpretation based on ResNet50 using the IAUC metric.

[0097]

[0098] Preferably, the model training process of the method of the present invention is divided into two stages. In the first stage, the model training is optimized only based on cross-entropy loss. Due to the limitations and biases of the data, the model will inevitably learn spurious association features. Therefore, an explanatory constraint term is added in the second stage to guide the model to focus only on foreground region features, while limiting the model's explanation distribution to the target feature region, making the model's attention more focused. The larger the weight of the explanatory constraint term, the more the model's attention is focused on the mask region obtained by the saliency map generated by the binarized AnnoCAM method, that is, more attention is paid to the most discriminative foreground region, but it may lead to a decrease in accuracy, because some images in the ImageNet dataset do not have obvious foreground features, and it is difficult to determine their category without relying on background features. The smaller the explanatory constraint term trade-off parameter λ, the more features in the image can be used to achieve better accuracy. According to the results in Table 5, this paper selects hyperparameter values ​​that have good accuracy and EPG value: λ = 0.5, τ = 0.6.

[0099] Table 6 Ablation Experiment

[0100]

[0101]

[0102] Preferably, to verify whether the AEC method proposed in this invention focuses on the most discriminative foreground region, this section uses the GradCAM interpretation method to visualize the decision-making basis of models trained by the baseline ERM, CGC, and AEC methods. For example... Figure 5 As shown, the second row corresponds to the saliency map of the baseline method ERM, revealing that the model's attention is scattered, and it uses background information for decision-making. For the CGC method, corresponding to the third row, the model's saliency map is more focused, reducing its focus on background information; however, it still uses background information in decision-making, for example, for can openers, the model relies more on can features to make decisions. The AEC method proposed in this invention effectively reduces its focus on background information while maintaining a more focused attention; for example, for seat belts, bicycles, and sandbags, it no longer focuses on background information like the CGC method.

[0103] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "setting," "connection," "fixing," "rotation," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal connection of two components or the interaction between two components. Unless otherwise explicitly limited, those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0104] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An image classification method for improving the attention of target regions in an image, characterized in that, Includes the following steps: S1. Obtain the set of images to be classified and their label set, and preprocess the set of images to be classified to obtain the training set; S2. The first stage of training of the convolutional neural network is performed using the training set and the label set, and the classification loss is calculated using the cross-entropy loss function. S3. Complete the first stage of training to obtain the initial model, and use the training set to visualize the class features of the initial model to obtain the channel semantic vectors under different label categories; S4. Copy the weights of the initial model as the initial weights of the second model. Based on the channel semantic vectors and the training set, perform the second stage of training on the second model, including: S41. Input the sample images from the training set into the second model to obtain the sample image categories, and use the cross-entropy loss function to calculate the classification loss of the second model; S42. Calculate the first saliency map of the sample image under the initial model using the AnnoCAM method, including: S421. Input the sample image into the initial model to obtain the corresponding sample image category; S422. The AnnoCAM method is used to interpret and visualize the sample image and its category, resulting in the first saliency map, represented as follows: in, A represents the first saliency map of the sample images belonging to the c-th label category. m This represents the activation map of the m-th channel in the last convolutional layer of the initial model. This represents the gradient calculation of the activation map of the m-th channel under the c-th label category, where ReLU represents the activation function. This represents the multiplication of corresponding elements of two matrices. This represents the channel semantic annotation of the m-th channel in the last convolutional layer of the initial model under the c-th label category. The number of channels in the last convolutional layer of the initial model is M. S43. The second saliency map of the sample image under the second model is calculated using the HiResCAM method, and is represented as follows: S44. Binarize the first saliency map to obtain a binary saliency map, and calculate the interpretation consistency loss based on the binary saliency map and the second saliency map; The formula for calculating consistency loss is as follows: in, This represents the second saliency map obtained using the HiResCAM method. Represents a binary saliency map, x i This represents the i-th sample image. Indicates the initial model. This represents the second model, where N represents the number of training set sample images. This represents the multiplication of corresponding elements of two matrices; The training loss is obtained by weighted summation of the classification loss and the interpretation consistency loss, and is expressed as: Among them, L CE L represents the classification loss. AEC The loss represents the consistency of interpretation, and λ represents the hyperparameter. S45. The training loss is obtained by weighted summation of classification loss and interpretation consistency loss, and the training loss is backpropagated to optimize the parameters of the second model. S46. Repeat steps S41-S45 until the parameters of the second model converge; S5. Complete the second stage of training to obtain an interpretable convolutional neural network model, and apply it to image classification.

2. The image classification method for improving the attention of target regions in an image according to claim 1, characterized in that, The sample images in the training set are used as input to the convolutional neural network, and the output is the sample image category. In the first stage of training in step S2, the training is carried out for 90 epochs, and the learning rate is reduced by 10 times every 30 epochs, with an initial learning rate of 0.

1.

3. The image classification method for improving the attention of target regions in an image according to claim 1, characterized in that, The cross-entropy loss function used in step S2 is: Where N represents the number of training set sample images, and C represents the number of label categories. This indicates whether the i-th sample image belongs to the c-th category; if so, then... If not, then ;f c This represents the score output by the convolutional neural network for the c-th category, x. i This represents the i-th sample image. This represents the trainable parameters of the initial model convolutional neural network.

4. The image classification method for improving the attention of target regions in an image according to claim 1, characterized in that, The initial model is visualized using the training set to obtain channel semantic vectors for different label types, including: S31. Let the number of label categories be C, and the number of channels in the last convolutional layer of the initial model be M; divide the training set into C sub-class training sets according to the labels; S32. Visualize the class features of the initial model using the training set of the sub-categories corresponding to the c-th label category to obtain the channel semantic vector of the initial model under the c-th label category, where c=1,2,…,C, including: S321. Input the i-th sample image in the training set corresponding to the c-th label category into the initial model, and extract the feature image output by the last convolutional layer of the convolutional neural network; perform gradient-based class activation mapping on the feature image to obtain the semantic view of each channel of the feature image in the last convolutional layer of the initial model; where i=1,2,…,n, and n represents the number of sample images in the sub-category training set; S322. Repeat step S321 until all sample images in the training set corresponding to the c-th label category have completed gradient-based class activation mapping, so that each channel in the last convolutional layer of the initial model corresponds to n semantic visual images; S323. For each channel in the last convolutional layer of the initial model, select the semantic view with the largest activation value from all its semantic views as its own channel semantic heatmap; S324. In the last convolutional layer of the initial model, each channel is semantically annotated based on its own channel semantic heatmap. All channel semantic annotation results are arranged and combined according to their corresponding channel order to obtain the channel semantic vector.

5. The image classification method for improving the attention of target regions in an image according to claim 4, characterized in that, Gradient-based class activation mapping is applied to the feature image to obtain the semantic representation of each channel of the feature image in the last convolutional layer of the initial model, as follows: in, A represents the semantic view of the feature image belonging to the c-th label category in the m-th channel. m This represents the activation map of the m-th channel in the last convolutional layer of the initial model. This represents the gradient calculation of the activation map of the m-th channel under the c-th label category, where ReLU represents the activation function.

6. The image classification method for improving the attention of target regions in an image according to claim 4, characterized in that, In step S324, any channel performs channel semantic annotation based on its own channel semantic heatmap, including: extracting the highlighted area in the channel semantic heatmap; if the highlighted area is located in the target area, the channel semantic annotation of the channel is 1; if the highlighted area is located in the background area, the channel semantic annotation of the channel is 0.

Images