Class activation map generation method, interpretable method, device, equipment and storage medium
The BI-CAM method addresses the issue of entangled semantics in CAM by integrating positive and negative feature correlations with binary masks and point mutual information, resulting in a more interpretable class activation map.
Patent Information
- Application Number
- CN202111453470.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-01
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2041-12-01
AI Technical Summary
Existing Class Activation Mapping (CAM) methods in neural networks suffer from entangled semantics between target and non-target regions, leading to reduced interpretability due to similar features and the use of continuous masks that distort target area characteristics.
A method that integrates positive and negative feature correlation information using bipolar information (BI-CAM) to quantify the relationship between target and non-target areas, employing binary masks and point mutual information to generate a more interpretable class activation map.
The proposed method effectively reduces the influence of entangled semantics, enhancing the interpretability of the class activation map by accurately distinguishing feature and non-feature areas, and simplifying the processing steps.
Smart Images

Figure CN114359587B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of neural network visualization technology, and in particular to a class activation map generation method, an interpretable method, a device, a equipment and a storage medium. Background Art
[0002] Convolutional Neural Networks (CNN) are now widely used in the field of computer vision, and the related technology Graph Convolutional Network (GCN) has also achieved outstanding performance in various non-Euclidean data processing. However, the complex internal mechanisms of neural networks require researchers to use some interpretable methods to understand the information output by neural networks and improve them. Class Activation Mapping (CAM) is a relatively effective technical method for interpreting neural networks, which can explain the information and features extracted by neural networks from different perspectives.
[0003] In the process of class activation map generation in the existing CAM method, entangled semantics may appear between different regions in the final hidden layer before the prediction layer, which may cause the non-target region to produce similar features with the target region, causing the nodes in the non-target region in the class activation map to be activated, affecting the interpretability of the class activation map. Summary of the invention
[0004] The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, the present invention proposes a class activation map generation method, an interpretable method, an apparatus, a device and a storage medium, which can combine the positive feature correlation information between the feature area and the input image and the negative feature correlation information between the non-feature area and the input image to accurately quantify the relationship between the target area and the non-target area of the image, which is conducive to improving the interpretability of the class activation map.
[0005] The first aspect of the present invention provides a method for generating a class activation map, comprising: inputting an original image into a trained convolutional neural network to obtain a first activation map; inputting the first activation map into a positive feature correlation module to obtain positive feature correlation information; inputting the first activation map into a negative feature correlation module to obtain negative feature correlation information; and obtaining a class activation map according to the first activation map, the positive feature correlation information and the negative feature correlation information.
[0006] According to the class activation map generation method of the embodiment of the first aspect of the present invention, there are at least the following beneficial effects: the class activation map generation method of the embodiment of the present invention first inputs the original image into the trained convolutional neural network for processing, upsamples and normalizes the processing result to obtain the first activation map, inputs the first activation map into the positive feature correlation module, obtains the positive feature correlation information containing the relationship between the feature area and the original image, inputs the first activation map into the negative feature correlation module, obtains the negative feature correlation information containing the relationship between the non-feature area and the original image, combines the positive feature correlation information with the negative feature correlation information, and then processes the first activation map to obtain the class activation map. Compared with other class activation map generation methods, the relationship between the feature area, the non-feature area and the original image is integrated at the same time, so it can better reduce the influence of entangled semantics, so that the generated class activation map has better interpretability.
[0007] In some embodiments of the present invention, the inputting the first activation map into a positive feature correlation module to obtain positive feature correlation information includes: calculating a first mask based on the first activation map; calculating a first mask image based on the first mask and the original image; inputting the first mask image into the trained convolutional neural network to obtain a first class probability; and obtaining the positive feature correlation information based on the first class probability and the full image class probability of the original image, wherein the full image class probability is obtained by inputting the original image into the trained convolutional neural network.
[0008] In some embodiments of the present invention, the step of inputting the first activation map into a negative feature correlation module to obtain negative feature correlation information includes: calculating a first negative activation map based on the first activation map; calculating a second mask based on the first negative activation map; calculating a second mask image based on the second mask and the original image; inputting the second mask image into the trained convolutional neural network to obtain a second class probability; and obtaining the negative feature correlation information based on the second class probability and the full image class probability of the original image.
[0009] In some embodiments of the present invention, calculating the first mask according to the first activation map includes: calculating the average value of the first activation map to obtain a first binarization threshold; and binarizing the first activation map according to the first binarization threshold to obtain the first mask.
[0010] In some embodiments of the present invention, calculating the second mask according to the first negative activation map includes: calculating the average value of the first negative activation map to obtain a second binarization threshold; and binarizing the first negative activation map according to the second binarization threshold to obtain the second mask.
[0011] In some embodiments of the present invention, the positive feature correlation information is positive point mutual information, and the positive feature correlation information is obtained according to the first category probability and the full-image class probability of the original image, including: performing point mutual information calculation on the first category probability and the full-image class probability to obtain the positive point mutual information.
[0012] In some embodiments of the present invention, the negative feature correlation information is negative point mutual information, and the negative feature correlation information is obtained based on the second class probability and the full image class probability of the original image, including: performing point mutual information calculation on the second class probability and the full image class probability to obtain the negative point mutual information.
[0013] In some embodiments of the present invention, obtaining a class activation map based on the first activation map, the positive feature correlation information, and the negative feature correlation information includes: calculating the positive point mutual information and the negative point mutual information in combination with a corresponding preset scaling factor to obtain a feature weight; and obtaining the class activation map based on the feature weight and the first activation map.
[0014] The second aspect of the present invention provides a class activation map interpretable method, comprising: obtaining a class activation map, wherein the class activation map is obtained according to the class activation map generation method described in the first aspect of the present invention; adjusting the proportional factors corresponding to the positive feature correlation information and the negative feature correlation information to update the class activation map; and obtaining the importance of the positive feature correlation information and the negative feature correlation information corresponding to the original image respectively according to the updated class activation map.
[0015] The class activation map interpretable method according to the second aspect of the embodiment of the present invention has at least the following beneficial effects: by obtaining the class activation map, and then adjusting the proportional factors corresponding to the positive feature correlation information and the negative feature correlation information, the weights of the positive feature correlation information and the negative feature correlation information on the generation of the class activation map are changed, thereby generating a new activation map that can intuitively display the importance of the feature area relative to the original image, or generating a new activation map that can intuitively display the importance of the non-feature area relative to the original image, which is beneficial to further improve the interpretability of the generated class activation map, and also simplifies the processing steps of the class activation map.
[0016] The third aspect of the present invention provides a class activation map processing device, including: a first input unit, used to input the original image into a trained convolutional neural network to obtain a first activation map; a second input unit, used to input the first activation map into a positive feature correlation module to obtain positive feature correlation information; a third input unit, used to input the first activation map into a negative feature correlation module to obtain negative feature correlation information; a processing unit, used to obtain a class activation map according to the first activation map, the positive feature correlation information and the negative feature correlation information.
[0017] The fourth aspect of the present invention provides a class activation map processing device, comprising: at least one memory; at least one processor; at least one program; the program is stored in the memory, and the processor executes at least one program to implement: the class activation map generation method as described in the first aspect of the present invention, or the class activation map interpretable method as described in the second aspect of the present invention.
[0018] The fifth aspect of the present invention provides a computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer executable signal, and the computer executable signal is used to execute: the class activation map generation method as described in the embodiment of the first aspect of the present invention, or the class activation map interpretable method as described in the embodiment of the second aspect of the present invention.
[0019] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] The accompanying drawings are used to provide a further understanding of the technical solution of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the technical solution of the present invention and do not constitute a limitation on the technical solution of the present invention.
[0021] Figure 1 A schematic diagram of a process for generating a class activation map according to an embodiment of the present invention;
[0022] Figure 2 A schematic diagram of a process for obtaining positive feature correlation information according to an embodiment of the present invention;
[0023] Figure 3 A schematic diagram of a process for obtaining negative feature correlation information according to an embodiment of the present invention;
[0024] Figure 4 Schematic diagram of a process for obtaining a first mask according to an embodiment of the present invention;
[0025] Figure 5 Schematic diagram of the process of obtaining the second mask according to an embodiment of the present invention;
[0026] Figure 6 A schematic diagram of a process for obtaining punctual mutual information according to an embodiment of the present invention;
[0027] Figure 7 A schematic diagram of a process for obtaining negative point mutual information according to an embodiment of the present invention;
[0028] Figure 8 A schematic diagram of a process of obtaining a class activation map by using feature weights according to an embodiment of the present invention;
[0029] Fig. 9 A schematic diagram of a flow chart of a class activation graph interpretable method according to an embodiment of the present invention;
[0030] Fig.10 A schematic diagram of the overall process of a method for generating a class activation map according to an embodiment of the present invention;
[0031] Fig.11 The present invention is a schematic diagram of the structure of a class activation graph processing device according to an embodiment of the present invention. DETAILED DESCRIPTION
[0032] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.
[0033] In the description of the embodiments of the present invention, if "several" is involved, it means one or more; if "multiple" is involved, it means two or more; if "greater than", "less than", or "exceeds" is involved, it should be understood as not including the number itself; if "above", "below", or "within" is involved, it should be understood as including the number itself.
[0034] It should be noted that the logical order is shown in the flowchart, but in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims or the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.
[0035] Deep Neural Networks (DNN) based on Convolutional Neural Networks (CNN) have achieved great success in the field of computer vision. Graph Convolutional Networks (GCN) have also achieved outstanding performance in various tasks on non-Euclidean data. However, the complex internal mechanisms of neural networks allow researchers to understand and optimize neural networks only from a macro perspective, such as convex optimization and receptive field. As the most effective explanation technology, various visualization methods explain the information and features extracted by neural networks from different perspectives. The prediction results of deep learning models can be objectively evaluated through visualization, and researchers can use visualization to build trust in reasonable prediction results and fix unreasonable erroneous predictions.
[0036] In the related art, the Class Activation Mapping (CAM)-based method can provide a convincing explanation of the features extracted by the neural network from a visual and quantitative perspective. For CNN, the hidden nodes are calculated by the global pooling of its last activation map, which means that the last activation map contains the entangled semantic information at the hidden layer position level, while the existing CAM method rarely considers that the hidden layer before the prediction layer may contain entangled semantics, that is, similar features of non-target areas and target areas. In addition, the soft mask with continuous values not only contains more non-target areas of the gradient-free CAM, but also causes the characteristics and distribution of the target area to be disturbed.
[0037] In order to improve or solve the above problems, the embodiments of the present invention provide a class activation map generation method, an interpretable method, an apparatus, a device and a storage medium, which can consider the contribution of the target area and the non-target area to the image in parallel, and thereby generate a class activation map that is easier to understand than the class activation map generated by other generation methods. The class activation map generation method is based on a deep convolutional neural network with bipolar information, so the visual interpretation framework can be called BI-CAM (Bipolar Information CAM, class activation map based on bipolar information), which can combine the positive feature correlation information between the feature area and the input image and the negative feature correlation information between the non-feature area and the input image to accurately quantify the relationship between the target area and the non-target area of the image, thereby generating a higher quality class activation map.
[0038] Reference Figure 1 and Fig.10 As shown, an embodiment of the present invention provides a method for generating a class activation map, including but not limited to the following steps:
[0039] Step S100, inputting the original image into the trained convolutional neural network to obtain a first activation map;
[0040] It should be noted that the trained convolutional neural network is used to extract features from the input original image to obtain feature maps and perform full-image class probability calculations. The trained convolutional neural network extracts features from the original image through convolution calculations, selects the feature map of the last layer of feature extraction, and performs upsampling calculations on it using linear interpolation or transposed pooling, and then performs normalization calculations to obtain the first activation map of the original image size. The calculation formula can be expressed as follows:
[0041]
[0042] It can be understood that k is the kth activation map in the last layer L of feature extraction, and the feature of the last layer corresponding to the feature node v After upsampling and normalization, the first activation map can be obtained Up(·) is an upsampling function that increases the size of the first activation map to the original image size through interpolation. Norm(·) represents the Min-Max scaling normalization function. s represents the feature block including the feature node v or cluster on the activation map. Then The feature of the last layer is extracted, and the corresponding first activation map is
[0043] Step S200: input the first activation map into a positive feature correlation module to obtain positive feature correlation information;
[0044] It should be noted that the positive feature correlation module is used to calculate the positive feature correlation information, specifically including: processing the original image according to the feature area in the first activation map, inputting the processing result into the trained convolutional neural network to obtain the corresponding first-class probability, and using the first-class probability and the whole-image class probability to perform correlation calculation to obtain the positive feature correlation information. The correlation calculation can be relative entropy calculation, norm calculation or point mutual information calculation.
[0045] Step S300, inputting the first activation map into a negative feature correlation module to obtain negative feature correlation information;
[0046] It should be noted that the negative feature correlation module is used to calculate negative feature correlation information, specifically including: processing the original image containing the non-feature area after reverse mapping the first activation map, and then inputting the processing result into the trained convolutional neural network to obtain the corresponding second-class probability, and using the second-class probability and the whole-image class probability to perform correlation calculation to obtain the negative feature correlation information. The correlation calculation can be relative entropy calculation, norm calculation or point mutual information calculation.
[0047] In step S200 and step S300, the classes mentioned in the first class probability, the second class probability and the whole image class probability are the same class, and the first class probability, the second class probability and the whole image class probability are the probabilities that the corresponding input image is identified as a certain class by the trained convolutional neural network.
[0048] Step S400: obtaining a class activation map according to the first activation map, the positive feature correlation information and the negative feature correlation information.
[0049] It should be noted that the first activation map is a plurality of images, and the corresponding positive feature correlation information and negative feature correlation information are calculated for each image, and the corresponding positive feature correlation information and negative feature correlation information are calculated according to a preset ratio to obtain the corresponding weight. The corresponding weights of all images in the first activation map are normalized to obtain the feature weights of the class activation map, and the first activation map is subjected to the corresponding activation function operation based on the feature weights to obtain the class activation map.
[0050] Compared with the related art, the embodiment of the present invention simultaneously integrates the relationship between the feature area, the non-feature area and the original image, thereby effectively reducing the influence of entangled semantics and making the generated class activation map more interpretable.
[0051] Reference Figure 2 As shown, it can be understood that in step S200, the first activation map is input into the positive feature correlation module to obtain the positive feature correlation information, including but not limited to the following steps:
[0052] Step S210, obtaining a first mask by calculation according to the first activation map;
[0053] Step S220, obtaining a first mask image by calculation according to the first mask and the original image;
[0054] It can be understood that the first mask image is an image obtained by performing Hadamard product calculation on the original image and the first mask, which contains the feature area of the first activation map, while other areas are replaced by masks, thereby highlighting the role of the feature area in the convolutional neural network for the original image.
[0055] Step S230, inputting the first mask image into the trained convolutional neural network to obtain a first class probability;
[0056] Step S240, obtaining positive feature correlation information according to the first category probability and the full-image category probability of the original image, wherein the full-image category probability is obtained by inputting the original image through a trained convolutional neural network.
[0057] The first-class probability and the whole-image class probability are used to perform correlation calculation to obtain positive feature correlation information. The correlation calculation can be relative entropy calculation, norm calculation or point mutual information calculation. The positive feature correlation information is the result obtained from the correlation calculation.
[0058] Reference Figure 3 As shown, it can be understood that in step S300, the first activation map is input into the negative feature correlation module to obtain negative feature correlation information, including but not limited to the following steps:
[0059] Step S310, calculating a first negative activation map according to the first activation map;
[0060] It should be noted that, since the first activation map is normalized, the first negative activation map is obtained by subtracting the first activation map by the all-1 matrix, and the relative feature area in the first negative activation map is the non-feature area in the first activation map. The calculation formula is:
[0061]
[0062] in, is the first negative activation map, is a matrix of all 1s, is the corresponding first activation map.
[0063] Step S320, calculating and obtaining a second mask according to the first negative activation map;
[0064] Step S330, obtaining a second mask image by calculation according to the second mask and the original image;
[0065] It can be understood that the second mask image is an image obtained by calculating the Hadamard product of the original image and the second mask, which contains the relative feature area of the first negative activation map, while other areas are replaced by masks, thereby highlighting the role of the relative feature area in the convolutional neural network for the original image.
[0066] Step S340, inputting the second mask image into the trained convolutional neural network to obtain a second class probability;
[0067] Step S350: Obtain negative feature correlation information according to the second class probability and the full image class probability of the original image.
[0068] The second class probability and the whole image class probability are used to perform correlation calculation to obtain negative feature correlation information. The correlation calculation can be relative entropy calculation, norm calculation or point mutual information calculation, and the negative feature correlation information is the result obtained from the correlation calculation.
[0069] Reference Figure 4As shown, it can be understood that in step S210, obtaining the first mask by calculating according to the first activation map includes but is not limited to the following steps:
[0070] Step S211, performing an average operation on the first activation map to obtain a first binarization threshold;
[0071] Step S212: binarize the first activation map according to a first binarization threshold to obtain a first mask.
[0072] Reference Figure 5 As shown, it can be understood that in step S320, obtaining the second mask according to the first negative activation map includes but is not limited to the following steps:
[0073] Step S321, performing an average operation on the first negative activation map to obtain a second binarization threshold;
[0074] Step S322: binarize the first activation map according to a second binarization threshold to obtain a second mask.
[0075] The calculation formulas for the first mask and the second mask are as follows:
[0076]
[0077]
[0078] Among them, |V| represents the number of nodes in the graph, v represents the characteristic node, is the first mask, is the second mask, Sign() is the label function, is the first negative activation map, is the corresponding first activation map.
[0079] In the related art, many works on CNN interpretability have discussed what kind of operation is reasonable and interpretable when using the way of disturbing input. Many existing works default to using continuous values between 0 and 1 to generate masks for the original input. One reason is that the study of visual interpretability originated from the field of computer vision. Using a smooth mask with a value range of [0,1] on the original image will not have any effect on the recognition of pixels under human eye perception. Therefore, it is usually natural to think that neural networks can also recognize these features. However, studies have shown that CNN is more inclined to extract detailed features such as edges with clear boundaries. Therefore, using a smooth mask superimposed on the original image may cause unimportant features to affect the results. From the perspective of GCN, the use of a smooth mask makes the original data closer to the coordinate origin or bias point, depending on the weight of the pixel in the mask. Taking the application of human skeleton behavior recognition as an example, when using a smooth mask to represent nodes or joints, it is easy to produce an offset of the feature area relative to the coordinate origin. For example, the mask composed of continuous values used by Score-CAM not only contains non-target information, but also moves the node features to a position close to the origin, thereby destroying the local features. Therefore, under these premises, the embodiment of the present invention uses a binary mask to filter the input features, so as to better retain the corresponding feature area.
[0080] Specifically, some embodiments of the present invention use the normalized first activation map to calculate its average value, and use the average value as a threshold to binarize the first activation map to generate a first mask and a second mask.
[0081] Reference Figure 6 As shown, it can be understood that the positive feature correlation information is the positive point mutual information. In step S240, the positive feature correlation information is obtained according to the first class probability and the full image class probability of the original image, including but not limited to the following steps:
[0082] Step S241, perform point mutual information calculation on the first class probability and the whole image class probability to obtain positive point mutual information.
[0083] The punctual mutual information can be expressed as:
[0084]
[0085]
[0086] Among them, X represents the original image, M s Indicates the first mask, X s represents the first mask image, and X s =X⊙M s , and each node v of the feature block s can be a continuous or discrete value between 0 and 1, ⊙ represents the same or operation, c represents the class, p(yc |x) is the class probability of the entire image, p(y c |X,M s ) is the first type probability.
[0087] Reference Figure 7 As shown, it can be understood that the negative feature correlation information is the negative point mutual information. In step S350, the negative feature correlation information is obtained according to the second class probability and the full image class probability of the original image, including but not limited to the following steps:
[0088] Step S351: perform point mutual information calculation on the second class probability and the whole image class probability to obtain negative point mutual information.
[0089] Negative point mutual information can be expressed as:
[0090]
[0091] Among them, X represents the original image, M \s Indicates the second mask, X \s represents the first mask image, and X \s =X⊙M \s , and each node v of the feature block s can be a continuous or discrete value between 0 and 1, ⊙ represents the same or operation, c represents the class, p(y c |X) is the class probability of the entire image, p(y c |X,M \s ) is the second type of probability.
[0092] It should be noted that p(y c |X s )+p(y c |X \s )≠p(y c |X).
[0093] Reference Figure 8 As shown, it can be understood that in step S400, obtaining a class activation map according to the first activation map, the positive feature correlation information and the negative feature correlation information includes but is not limited to the following steps:
[0094] Step S410, calculating the positive point mutual information and the negative point mutual information in combination with the corresponding preset scaling factor to obtain a feature weight;
[0095] Step S420: Obtain a class activation map based on the feature weights and the first activation map.
[0096] In the related art, when the most important nodes or factions in the graph are removed or disturbed, the distribution of neural network decisions will change dramatically. In contrast, if these operations are applied to unimportant nodes or factions, the distribution of classifications will hardly change. In this regard, some embodiments of the present invention use point-wise mutual information (PMI) to measure the difference between the predicted distributions of the original data and the changed data. Compared with the results of other correlation calculation methods, point mutual information is used to measure the correlation between the original image and the feature area of the first activation map and the relative feature area of the first negative activation map, which is more concise and effective, and it is also easier to quantitatively calculate the correlation.
[0097] Specifically, the point mutual information calculated by the first class probability and the whole image class probability is called positive point mutual information (PPMI), and the point mutual information calculated by the second class probability and the whole image class probability is called negative point mutual information (NPMI). The more important the content of the first mask image is to the original image, the closer the corresponding positive point mutual information is to 0, and the closer the corresponding negative point mutual information of the second mask image is to negative infinity. The importance of the features and nodes corresponding to the first mask image to the original image in the convolutional neural network can be evaluated by positive point mutual information and negative point mutual information. Obviously, considering the relevant metrics of the activation map, for the activation map containing important features, it is necessary to maximize the positive point mutual information and minimize the negative point mutual information, and introduce a preset scaling factor to control the ratio of positive point mutual information and negative point mutual information, so as to obtain a class activation map that can reflect the influence of important features or unimportant features on the judgment of the convolutional neural network.
[0098] Reference Fig.10 , the corresponding feature weight calculation method is: each image of the first activation map corresponds to positive point mutual information and negative point mutual information, the product of the positive point mutual information and the preset first scale factor is added to the product of the negative negative point mutual information and the preset second scale factor, the result weight is calculated, and the result weights of other images are combined to perform normalization calculation through the Softmax logistic regression function to obtain the feature weight, and then each image of the first activation map and the corresponding feature weight are operated by the ReLU (Rectified Linear Unit) activation function to obtain the class activation map. It can be understood that the sum of the first scale factor and the second scale factor is 1.
[0099] The result weight calculation process can be expressed as:
[0100]
[0101] in, is the result weight, α is the preset first proportional factor, (1-α) is the preset second proportional factor, is the punctual mutual information, is the negative point mutual information.
[0102] It should be noted that since the point mutual information is not necessarily a positive number, and usually the probability of the first and second categories is less than the probability of the entire image, the relative value of the activation map weight is considered, and the result weight is normalized as follows:
[0103]
[0104] in, is the feature weight.
[0105] The class activation map calculation formula can be expressed as:
[0106]
[0107] in, is the class activation graph, Up(F Lk ) is the result of upsampling the feature image corresponding to the feature weight.
[0108] Reference Fig. 9 As shown, one embodiment of the present invention further provides a class activation map interpretable method, including but not limited to the following steps:
[0109] Step S500, obtaining a class activation map;
[0110] It can be understood that the class activation map is obtained according to the class activation map generating method of the embodiment of the present invention.
[0111] Step S600: adjusting the proportional factors corresponding to the positive feature correlation information and the negative feature correlation information to update the class activation map;
[0112] Step S700: According to the updated class activation map, the importance of the positive feature correlation information and the negative feature correlation information corresponding to the original image are obtained.
[0113] It is understandable that when the first scale factor and the second scale factor change, the corresponding class activation map will also change. When the first scale factor is 1, the feature weights of the class activation map are obtained from the positive feature correlation information, that is, the importance of positive features to the original image in the judgment process of the trained convolutional neural network can be intuitively displayed. If the corresponding first mask is generated by binarization, the interpretability of the class activation map can be significantly improved. When the first scale factor is 0, the feature weights of the class activation map are obtained from the negative feature correlation information, that is, the influence of negative features on the original image in the judgment process of the trained convolutional neural network can be intuitively displayed, and can be used as a reference for evaluating the defect location of the trained convolutional neural network. If the corresponding second mask is generated by binarization, the interpretability of the class activation map can be significantly improved. When the first scale factor is adjusted between 0 and 1, the degree of influence of different features on the original image in the judgment of the trained convolutional neural network can be displayed.
[0114] It can be understood that the first scaling factor of the embodiment of the present invention corresponds to positive feature correlation information, and the second scaling factor corresponds to negative feature correlation information.
[0115] Reference Fig.11 An embodiment of the present invention further provides a class activation graph processing device, comprising:
[0116] A first input unit 100, used to input the original image into the trained convolutional neural network to obtain a first activation map;
[0117] A second input unit 200 is used to input the first activation map into a positive feature correlation module to obtain positive feature correlation information;
[0118] A third input unit 300 is used to input the first activation map into a negative feature correlation module to obtain negative feature correlation information;
[0119] The processing unit 400 is used to obtain a class activation map according to the first activation map, the positive feature correlation information and the negative feature correlation information.
[0120] It should be noted that the contents of the method embodiments of the present invention are all applicable to the system embodiments. The functions specifically implemented by the system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method, which will not be repeated here.
[0121] In addition, an embodiment of the present invention further provides a class activation map processing device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor.
[0122] The processor and the memory may be connected via a bus or other means.
[0123] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0124] It should be noted that the class activation map processing device in this embodiment can be applied as a class activation map generation method or a class activation map interpretable method as in the above-mentioned embodiment. The class activation map processing device in this embodiment and the class activation map generation method or the class activation map interpretable method as in the above-mentioned embodiment have the same inventive concept, so these embodiments have the same implementation principles and technical effects, which will not be described in detail here.
[0125] The non-transient software program and instructions required to implement the class activation map generation method and / or the class activation map interpretable method of the above-mentioned embodiment are stored in the memory. When executed by the processor, the class activation map generation method and / or the class activation map interpretable method of the above-mentioned embodiment are executed, for example, the above-described Figure 1 The method steps S100 to S400, Figure 2 Steps S210 to S240 of the method, Figure 3 Steps S310 to S350 of the method, Figure 4 Steps S211 to S212 of the method, Figure 5 Steps S321 to S322 of the method, Figure 6 Step S241 of the method, Figure 7 Step S351 of the method, Figure 8 Steps S410 to S420 of the method, Fig. 9 Method steps S500 to S700 in.
[0126] The device embodiments and equipment embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e., may be located in one place, or may be distributed over multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.
[0127] In addition, one embodiment of the present invention further provides a computer-readable storage medium, which stores computer-executable instructions. The computer-executable instructions are executed by a processor or a controller, for example, by a processor in the above-mentioned class activation map processing device embodiment, so that the above-mentioned processor can execute the class activation map generation method and / or the class activation map interpretable method in the above-mentioned embodiment, for example, to execute the above-mentioned Figure 1 The method steps S100 to S400, Figure 2 Steps S210 to S240 of the method, Figure 3 Steps S310 to S350 of the method, Figure 4 Steps S211 to S212 of the method, Figure 5 Steps S321 to S322 of the method, Figure 6 Step S241 of the method, Figure 7 Step S351 of the method, Figure 8 Steps S410 to S420 of the method, Fig. 9 Method steps S500 to S700 in.
[0128] Through the description of the above embodiments, it will be appreciated by those skilled in the art that all or some steps and systems in the disclosed methods above can be implemented as software, firmware, hardware and appropriate combinations thereof. Some physical components or all physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a readable medium, and the readable medium can include a computer storage medium (or a non-transitory medium) and a communication medium (or a temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as a computer-readable signal, a data structure, an instruction module or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassette, magnetic tape, disk storage or other magnetic storage device, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media generally include computer readable signals, data structures, instruction modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.
[0129] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the above-mentioned implementation mode. In the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other. Those skilled in the art can also make various equivalent deformations or substitutions without violating the spirit of the present invention. These equivalent deformations or substitutions are all included in the scope defined by the claims of the present invention.
Claims
1. A method for generating class activation maps, It is characterized in that include: Inputting the original image into the trained convolutional neural network to obtain a first activation map; Inputting the first activation map into a positive feature correlation module to obtain positive feature correlation information; Inputting the first activation map into a negative feature correlation module to obtain negative feature correlation information; Obtaining a class activation map according to the first activation map, the positive feature correlation information, and the negative feature correlation information; The step of inputting the first activation map into a positive feature correlation module to obtain positive feature correlation information includes: Calculating a first mask according to the first activation map; Calculating a first mask image according to the first mask and the original image; Inputting the first mask image into the trained convolutional neural network to obtain a first class probability; Obtaining the positive feature correlation information according to the first class probability and the full-image class probability of the original image, wherein the full-image class probability is obtained by inputting the original image into the trained convolutional neural network; The step of inputting the first activation map into a negative feature correlation module to obtain negative feature correlation information includes: Calculate a first negative activation map according to the first activation map; Calculating a second mask according to the first negative activation map; Calculating a second mask image according to the second mask and the original image; Inputting the second mask image into the trained convolutional neural network to obtain a second class probability; The negative feature correlation information is obtained according to the second class probability and the full-image class probability of the original image.
2. The class activation map generation method according to claim 1, It is characterized in that The step of calculating a first mask according to the first activation map includes: Calculating the average value of the first activation map to obtain a first binarization threshold; Binarization is performed on the first activation map according to the first binarization threshold to obtain the first mask.
3. The class activation map generation method according to claim 1, It is characterized in that The step of calculating a second mask according to the first negative activation map comprises: Calculating the average value of the first negative activation map to obtain a second binarization threshold; Binarization is performed on the first negative activation map according to the second binarization threshold to obtain the second mask.
4. The class activation map generation method according to claim 1, It is characterized in that The positive feature correlation information is positive point mutual information, and the positive feature correlation information is obtained according to the first class probability and the full image class probability of the original image, including: Point mutual information is calculated for the first class probability and the full-image class probability to obtain the positive point mutual information.
5. The class activation map generation method according to claim 4, It is characterized in that The negative feature correlation information is negative point mutual information, and the negative feature correlation information is obtained according to the second class probability and the full image class probability of the original image, including: Point mutual information is calculated for the second class probability and the full-image class probability to obtain the negative point mutual information.
6. The class activation map generation method according to claim 5, It is characterized in that The obtaining of a class activation map according to the first activation map, the positive feature correlation information and the negative feature correlation information includes: The positive point mutual information and the negative point mutual information are combined with corresponding proportional factors to calculate a feature weight; The class activation map is obtained by calculation according to the feature weight and the first activation map.
7. A class activation map interpretable method, It is characterized in that include: Obtain a class activation map, wherein the class activation map is obtained according to the class activation map generation method according to any one of claims 1 to 6; Adjusting the scaling factors corresponding to the positive feature correlation information and the negative feature correlation information to update the class activation map; According to the updated class activation map, the positive feature correlation information and the negative feature correlation information are obtained, and the importance of the original image corresponds to each other.
8. A class activation graph processing device, It is characterized in that include: A first input unit, used to input the original image into the trained convolutional neural network to obtain a first activation map; A second input unit, configured to input the first activation map into a positive feature correlation module to obtain positive feature correlation information; A third input unit, used to input the first activation map into a negative feature correlation module to obtain negative feature correlation information; a processing unit, configured to obtain a class activation map according to the first activation map, the positive feature correlation information, and the negative feature correlation information; Wherein, the second input unit is further used for: Calculating a first mask according to the first activation map; Calculating a first mask image according to the first mask and the original image; Inputting the first mask image into the trained convolutional neural network to obtain a first class probability; Obtaining the positive feature correlation information according to the first class probability and the full-image class probability of the original image, wherein the full-image class probability is obtained by inputting the original image into the trained convolutional neural network; Wherein, the third input unit is further used for: Calculate a first negative activation map according to the first activation map; Calculating a second mask according to the first negative activation map; Calculating a second mask image according to the second mask and the original image; Inputting the second mask image into the trained convolutional neural network to obtain a second class probability; The negative feature correlation information is obtained according to the second class probability and the full-image class probability of the original image.
9. A class activation graph processing device, It is characterized in that include: at least one memory; at least one processor; at least one program; The program is stored in the memory, and the processor executes at least one of the programs to implement: The method for generating a class activation map according to any one of claims 1 to 6, or the method for interpreting a class activation map according to claim 7.
10. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a computer-executable signal, wherein the computer-executable signal is used to execute: The method for generating a class activation map according to any one of claims 1 to 6, or the method for interpreting a class activation map according to claim 7.
