A small target detection model visual explainability method, device and medium

Abstract

The application discloses a kind of small target detection model visual explainability method, comprising: S1. deep feature map is obtained by CNN forward transmission, and feature map corresponding gradient is obtained by specific class target classification score back propagation;S2. after being processed based on sequencing grouping, mask containing high fine-grained local sensitivity information is obtained by feature image pixel level gradient weighting;S3. fuzzy mask occlusion image is input into CNN, and global decision contribution is obtained by containing N times integral sampling forward transmission, and contribution importance score is calculated and mean is taken as group importance score;S4. class activation map is obtained by linear weighting to grouping mask, and importance score is used to weight mask, to obtain visual explanation chart.The application is based on global decision contribution and parallel computing grouping mask importance score, fuzzy boundary and integral method are carried out to processing to mask, improve the sensitivity and algorithm speed of global decision contribution to different inputs.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision data processing, and particularly relates to a small target visual interpretation method based on model weight correlation and integral score, a device and a storage medium. BACKGROUND

[0002] The visual interpretation method is a technology for analyzing the decision results of a deep neural network model represented by a convolutional neural network through a visual means, and the goal is to help users understand the internal mechanism of the model and trust the model decision.

[0003] With the rapid development and improvement of the visual interpretation method, the existing technology generally adopts a class activation mapping (CAM) based visual interpretation method for use. The CAM visual interpretation method forms a class activation map by weighted combination of the feature maps of the last convolutional layer, so that the image pixel value size represents the correlation degree of the input and the output result at the corresponding position.

[0004] Although the CAM visual interpretation method has become a hot direction in this field due to its intuitive and good explanation advantages, the existing CAM method has the problem of poor visual interpretation effect and timeliness due to the insufficient and ineffective combination of local sensitivity and global decision contribution.

[0005] Therefore, the present application proposes a small target detection model visual interpretability method to solve the above technical problems. SUMMARY

[0006] The main purpose of the present application is to provide a small target detection model visual interpretability method to solve the technical problems proposed in the background art.

[0007] The present application solves the above technical problems by adopting the following technical solutions:

[0008] A small target detection model visual interpretability method, comprising the following steps:

[0009] S1. Obtain a deep feature map through CNN forward transmission, and obtain a feature map corresponding gradient through specific class target classification score back propagation;

[0010] S2. Obtain a group of masks containing high-fine-grained local sensitivity information after feature map pixel-level gradient weighting and based on sequencing grouping processing;

[0011] S3. Input the blurred mask occlusion image into the CNN, obtain global decision contribution through N times integral sampling forward transmission, calculate the contribution importance score and take the mean value as the group importance score;

[0012] S4. Linearly weighting the group masks to obtain class activation maps, the class activation maps are weighted by the importance scores to obtain visual interpretation maps.

[0013] Preferably, the specific process of feature image pixel-level gradient weighting in the S2 step includes:

[0014] S21. Let f be an image classification CNN model, θ be the model parameters, and for a given input image I0, the feature map weight is obtained by global average pooling After network forward inference, the classification score formula y c is:

[0015]

[0016] wherein represents the kth channel feature map, i and j represent the row index and column index of the feature map respectively;

[0017] S22. Explicitly encode each feature map weight in the classification score formula y c , and the calculation formula of the feature map weight is:

[0018]

[0019] wherein is the weighting coefficient of the pixel gradient corresponding to the class c and the feature map, and the calculation formula is:

[0020]

[0021] wherein, is the pixel gradient value of the kth channel of the feature map A at the (i, j) position;

[0022] S23. Weight each pixel gradient of the feature map A at the (i, j) position to obtain K groups of weighted feature maps

[0023] Preferably, the generation process of the mask obtained by the sequencing and grouping processing in the S2 step includes:

[0024] K groups of weighted feature maps are equally divided into G groups according to the fixed order of the generated weighted feature maps and G<K, and each group of weighted feature activation maps is superimposed to obtain G masks M l for generating class activation maps, and the calculation formula is:

[0025]

[0026] ​The obtained low-resolution mask is normalized and up-sampled to the original image size by bilinear interpolation to obtain a smoother mask, and the calculation formula is:

[0027]

[0028] M’ l is the processed smooth mask.

[0029] Preferably, the specific calculation process of obtaining the global decision contribution in the S3 step includes:

[0030] S31. Use Gaussian blur processing on the occlusion area, and obtain a blurred mask occlusion image I’ l , and the calculation expression is:

[0031]

[0032] wherein, is the input image after Gaussian blur, and is Hadamard product;

[0033] S32. Gradually accumulate the input image features with linear interpolation as the integration path, calculate the average of N times of forward transmission classification scores on the target class, and take it as the group mask importance score, to obtain a more accurate global decision contribution corresponding to the input.

[0034] Preferably, the specific operation process of calculating the contribution importance score in the S3 step includes:

[0035] Let the output f c of the baseline image M’0 be zero, take a pure black picture or a noise picture as the baseline image, and perform the no gradient method, and there is:

[0036]

[0037] wherein, is the importance score of the lth group of masks, and N is the integration sampling number;

[0038] The forward transmission of N masks is calculated in parallel.

[0039] Preferably, the S4 step uses the importance score to weight G group masks to obtain a more accurate visual interpretation map, and the calculation expression is:

[0040]

[0041] wherein is the group mask of the ith baseline image.

[0042] In another aspect, the present invention also discloses a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the method described above.

[0043] In another aspect, the present invention also discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the method described above.

[0044] As can be seen from the above technical solution, the present invention provides a method for the visual interpretability of a small target detection model. Compared with the prior art, the present invention has the following advantages:

[0045] 1. This invention mines high-fine-grained local sensitivity information in feature maps by pixel-level gradient weighting and generates weighted feature maps that are highly correlated with decision results. By merging weighted feature maps using an ordered grouping method, a mask containing local sensitivity information with low redundancy and strong decision correlation can be obtained, thereby improving the visual accuracy of target detection results and making it easy to use.

[0046] 2. This invention is based on parallel calculation of the importance score of the group mask according to the global decision contribution. The mask is processed by fuzzy boundary and integral methods, which improves the sensitivity of different inputs to the global decision contribution and the algorithm speed.

[0047] 3. This invention combines the idea of ​​non-gradient global contribution, resulting in less random noise in the model operation results, producing a smoother class activation map, more concentrated, accurate and complete target region localization, better visual continuity, and easier reflection of the model's classification decision.

[0048] 4. This invention can detect and locate multiple similar targets in complex backgrounds with minimal background interference; at the same time, it is more effective in activating small targets, and the location area is more accurate, concentrated and complete, resulting in better visual effects.

[0049] It should be understood that the descriptions in this section are not intended to identify key or essential features of embodiments of the invention, nor are they intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Of course, implementing any product of the invention does not necessarily require achieving all of the advantages described above simultaneously. Attached Figure Description

[0050] The accompanying drawings, which form part of this application, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings:

[0051] Figure 1 This is a schematic diagram of the LG-CAM method of the present invention;

[0052] Figure 2 A schematic diagram illustrating the problem identification of existing CAM methods;

[0053] Figure 3 This is a schematic diagram comparing the channel-level weight distribution of the feature activation map in this invention;

[0054] Figure 4 This is a comparative visual explanation diagram of different methods of the present invention;

[0055] Figure 5 This is a schematic diagram illustrating the different categories of distinguishable identification according to the present invention;

[0056] Figure 6 This is a schematic diagram comparing the insertion and deletion metric curves of the present invention;

[0057] Figure 7 This is a schematic diagram illustrating the target positioning effect of the present invention;

[0058] Figure 8 This is a schematic diagram comparing the positioning accuracy under different thresholds according to the present invention. Detailed Implementation

[0059] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0060] For details in the embodiments, please refer to Figures 1 to 8 .

[0061] In existing technologies, gradient-free CAM methods, represented by ScoreCAM, use multiple sets of feature maps as masks and multiply them with the original image to occlude the original image. This occlusion serves as a perturbation image, which is then input into the neural network. The difference between the forward propagation classification confidence score on the target class and the baseline is used to measure the importance of each layer's feature map, reflecting the overall decision-making of the CNN, as shown in the following equation:

[0062]

[0063] Where I0 is the given input image, ⊙ is the Hadamard product, and I b For baseline image, For the input feature map, is the feature map upsampled to the size of the input image, and f(·) is the confidence score calculated.

[0064] While a global decision contribution approach can yield a relatively accurate class activation map, its importance score acquisition is limited by the selection of the baseline image and the input interference image. It lacks sensitivity to global decision contributions for different input features and neglects local sensitivity information during forward propagation, resulting in performance limitations. Furthermore, the high computational cost and large memory footprint of forward inference lead to poor real-time performance and can easily result in undesirable outcomes. Figure 2 The problems shown include positioning deviation, artifacts, and noise.

[0065] Therefore, in order to solve such Figure 2 The existing technical problems shown are referred to Figure 1 This invention proposes a visual interpretability method (LG-CAM) for small target detection models, comprising the following steps:

[0066] S1. Obtain deep feature maps through forward propagation of CNN, and obtain the gradients corresponding to the feature maps through backpropagation of the classification scores of specific target classes.

[0067] S2. A set of masks containing high-fine-grained local sensitivity information is obtained by weighting the pixel-level gradient of the feature map and using an ordered grouping method.

[0068] The specific process of pixel-level gradient weighting of feature maps includes:

[0069] S21. Let f be an image classification CNN model, and θ be the model parameters. For a given input image I0, the feature map weights are obtained through global average pooling. After forward inference through the network, the classification score formula y is obtained. c for:

[0070]

[0071] in This is represented as the feature map of the k-th channel, where i and j represent the feature maps respectively. Row indexes and column indexes in the data;

[0072] S22. Formula for categorical fractions y c Weights of each feature map within For explicit encoding, the formula for calculating the feature map weights is:

[0073]

[0074] in The weighting coefficients for the pixel gradients of class c and the corresponding feature map are calculated using the following formula:

[0075]

[0076] in, is the pixel gradient value of the K-th channel at the (i, j) position of feature map A;

[0077] High-fine-grained local sensitivity in the feature map is mined through pixel-level gradient weighting, enabling better activation of multiple similar targets as a whole;

[0078] At this time, referring to Figure 3 As shown, for the mean distribution of the feature map and the channel-level weight distributions corresponding to the two algorithms, it is found by comparing with the prior art that another traditional visual interpretation method, Grad-CAM, contains a large number of negative-direction activations. The non-negative weight activation values obtained by the method of this application have a higher correlation with the feature map, reflecting the ability of the gradient to correspondingly affect the key regions of the decision result, and can better reflect the importance of the feature map.

[0079] S23. Therefore, at this time, each pixel gradient at the (i, j) position of feature map A is weighted to obtain k groups of high-fine-grained weighted feature maps of different decision-related regions There is a calculation formula:

[0080]

[0081] where k ∈ {1, 2,..., K}. <s

[0082] In addition, in a specific embodiment, it should be further noted that the generation process of obtaining the mask by ordinal grouping processing includes:

[0083] S24. Divide the K groups of weighted feature maps Equidistantly into G groups in a fixed order according to the generated weighted feature maps, where G < K, and stack the weighted feature activation maps of each group to obtain G masks M for generating class activation maps l , and the calculation formula is:

[0084]

[0085] where the hyperparameter G is determined by experiments to obtain the optimal value.

[0086] At this time, the grouping method adopts ordinal grouping by merging in a fixed order of the feature map, and the algorithm can obtain better performance and consistency. In the model of this application, K = 512, G = 64, and through grouping and merging, the subsequent calculation amount can be reduced by 8 times.

[0087] S25. To obtain a smoother mask, the obtained low-resolution mask is normalized and upsampled to the original image size by bilinear interpolation to obtain a smoother mask, and the calculation formula is:

[0088]

[0089] M'l This is the smoothed mask after processing.

[0090] S3. Input the blurred masked image into the CNN, obtain the global decision contribution through a forward pass containing N integral samplings, calculate the contribution importance score and take the average as the group importance score.

[0091] The specific calculation process for obtaining the global decision contribution includes:

[0092] S31. Apply Gaussian blur to the occluded area to obtain a blurred masked occlusion image I'. l The calculation expression is:

[0093]

[0094] in, The input image is Gaussian blurred, and ⊙ represents the Hadamard product;

[0095] S32. The attribution between model prediction and input features should conform to two basic axioms: sensitivity (if different inputs have feature bias in a certain dimension and the predicted output is different, then the attribution in that dimension is not zero) and realization invariance (functionally equivalent models have the same input attribution). Therefore, the input image features are gradually accumulated by using linear interpolation as the integration path, the forward propagation classification scores on the target class are calculated N times and averaged as the group mask importance score, so as to obtain a more accurate global decision contribution corresponding to the input.

[0096] Meanwhile, many gradient-free methods use pure black images or noisy images as baseline images. Therefore, the output f of the baseline image M'0 is also set at this time. c When (M”0) is zero, a pure black image or a noisy image is used as the baseline image, and a gradient-free method is applied, resulting in:

[0097]

[0098]

[0099] in, Let N be the importance score of the l-th mask, and N be the number of integral samples.

[0100] It can further avoid interference from unreasonable baseline images and improve the accuracy of representing the global decision contribution of different input features;

[0101] Meanwhile, to improve the running speed, the accumulation operation is expanded and the forward propagation of N masks is calculated in parallel, which effectively improves the algorithm speed.

[0102] S4. Linearly weight the grouped masks to obtain the class activation map. The class activation map is then weighted with importance scores to obtain the visual interpretation map. The calculation expression is as follows:

[0103]

[0104] in is the grouping mask for the i-th group of baseline images.

[0105] In summary, by mining high-fine-grained local sensitivity information in feature maps through pixel-level gradient weighting and generating weighted feature maps that are highly correlated with the decision results, and merging the weighted feature maps using an ordered grouping method, a mask containing local sensitivity information with low redundancy and strong decision relevance can be obtained, thereby improving the visual accuracy of target detection results. At the same time, the importance score of the grouped mask is calculated in parallel based on the global decision contribution, and the mask is processed using fuzzy boundary and integral methods, which improves the sensitivity of different inputs to the global decision contribution and the algorithm speed, making it easy to use.

[0106] Furthermore, this method is illustrated using the following system programming code:

[0107] LG-CAM algorithm

[0108] Input: Image I0, model f, class c, number of groups G, Gaussian blur parameters.

[0109] Output: Class activation graph

[0110] 1. Initialize the class activation graph Reference input image

[0111] 2. Obtain the feature map A of the target layer k layers. c The pixel-level weighted gradient W is calculated using the formula for calculating feature map weights. c The weighted feature map F is calculated using the formula for calculating the weighted feature map with high fine-grained weighting. c ;

[0112] 3. Using the formula for calculating the mask of the generated activation map and the formula for calculating bilinear interpolation upsampling, the weighted feature maps are merged into G groups, and an upsampled group mask M' is generated. l ;

[0113] 4 while l=1 and l≤G

[0114] 5 while i = 0 and i <N

[0115] 6. Calculate the forward pass fraction f of this integral. c (M' l );

[0116] 7 i = i + 1

[0117] 8 end

[0118] 9. The mask importance score is calculated using the formula for calculating the importance score contributed by the gradient-free method. The formula is then superimposed onto the visual interpretation map.

[0119] 10 l=l+1

[0120] 11 end

[0121] 12 Return

[0122] Experimental Example

[0123] Based on the above embodiments, in a specific experimental demonstration process, the visual interpretability method of the small object detection model proposed in the above embodiments was also experimentally verified based on the ImageNet classification dataset.

[0124] (1) Establishing datasets and evaluation metrics

[0125] The ImageNet validation set contains 50,000 images across 1,000 object types, and provides the ground truth bounding boxes for the highest-predicted class. 2,000 images are extracted from the ImageNet validation set, and a visual explanation map is generated for the highest-confidence class in each image. The following preprocessing is typically used with the ImageNet image dataset: image size is resized to (224×224×3), pixel values ​​are converted to the range [0,1], and normalization is performed using the mean vector [0.485, 0.456, 0.406] and standard deviation vector [0.229, 0.224, 0.225]. To ensure fairness, the weights of the VGG19 pre-trained model from the PyTorch library are used uniformly for validation.

[0126] The evaluation of visual interpretability methods is divided into qualitative and quantitative evaluation. Qualitative evaluation assesses visual interpretability methods by analyzing the visual continuity, class discrimination, and visualization angles of multi-object and small-object salience maps obtained from different methods. Quantitative evaluation can be conducted from two aspects: confidence level assessment and location assessment.

[0127] Confidence evaluation employs an image occlusion experiment, forcing the model to change its decisions and verifying the importance of salient regions in the class activation map for prediction confidence. Metrics include deletion (Del) and insertion (Ins) measures. Considering both measures, the result can be calculated using the following formula:

[0128] ID = AUC(Ins) - AUC(Del)

[0129] Among them, the smaller the deletion metric, the better, while the larger the insertion metric and ID metric, the better.

[0130] The localization accuracy of the method was evaluated using a weakly supervised target localization experiment. Localization accuracy was measured by loc1 and loc5 metrics. The Intersection over Union (IoU) threshold between the predicted bounding box and the ground truth bounding box was set at 0.1 intervals within the range of (0,1) and tested sequentially to obtain the target localization accuracy of loc1 and loc5 at different thresholds, denoted as mloc1 and mloc5. Higher target localization accuracy is better.

[0131] (2) Based on the above dataset and evaluation indicators, an improved ablation experiment was conducted.

[0132] The parameters in the improved method were tested using ablation experiments on the ImageNet dataset to determine the optimal model structure and achieve adaptability to different input images. Since the number of groups and the number of integral samples are key parameters that significantly impact computational cost and accuracy, this section first determined the number of groups that could contain the most efficient information through multiple sets of experiments to ensure basic algorithm accuracy and efficiency. Secondly, accuracy tests were conducted on a model with a fixed number of groups and multiple sets of integral samples to determine a reasonable number of integral samples. Subsequently, the grouping method, Gaussian blur parameters, and negative gradient operation, which have independent effects on the results, were tested and validated. Finally, the improved method was validated to demonstrate its effectiveness and speed advantage compared to the baseline model.

[0133] (a) Group Quantity Test

[0134] The LG-CAM algorithm was tested with different values ​​for the number of groups G. In the experiment, the integral sampling number was set to 4, the grouping method was set to ordered grouping and negative gradient operation was used. The test results are shown in the table below. As the number of groups increases, the computational cost increases, the confidence comprehensive evaluation index ID continuously rises, and the localization accuracy index first rises and then decreases, reaching the maximum mloc1 when the number of groups is 32 and 64. This indicates that there is some redundancy in the CNN feature map, and an excessively large number of groups only brings a small improvement in accuracy at a greater computational cost. Considering that the ID index at G=64 is significantly improved compared to G=32, and the two have similar time consumption, the number of groups was chosen to be 64.

[0135]

[0136]

[0137] (b) Integral sampling quantity test

[0138] The integral sampling number represents the number of intervals from 0 to all image features in the input image. The impact of different integral sampling numbers on the performance of the LG-CAM algorithm was verified. In the experiment, the number of groups was set to 64, the grouping method was ordered grouping, and negative gradient operation was used. The test results are shown in the table below. As the integral sampling number increases, the localization accuracy and confidence index first rise and then fall. The molecular localization accuracy reaches its highest point at N=4 and N=10, respectively. The ID index reaches its highest point at N=2, and is second best at N=4. This indicates that by using integral sampling to gradually accumulate input image features, this method can combine the contribution of input features to the interpretation results in different dimensions, supplementing global decision-making information, and thus improving the algorithm's performance.

[0139] Meanwhile, the time consumption increases significantly with the increase in the number of samples. Therefore, the number of integral samples is chosen to be 4.

[0140]

[0141] (c) Grouping method test

[0142] The impact of different grouping methods on algorithm performance was tested. The number of groups was set to 64, and the integral sampling quantity was set to 4. The test results are shown in the table below. K-means clustering, hierarchical clustering, and spectral clustering methods unfold the feature maps into one-dimensional feature vectors for clustering. The Structural Similarity (SSIM) grouping method calculates the structural similarity matrix between feature activation maps before clustering. It should be noted that the resulting class activation maps are inconsistent due to the influence of the initial values ​​of the clustering algorithms. The clustering method results in the table were tested 5 times and the average was taken. From the table below, spectral clustering achieved the best localization accuracy in mloc1 and mloc5, while the ordinal grouping method was second best. The ordinal grouping method was the best in terms of ID metrics and had the lowest processing time. The ordinal grouping method does not perform manual clustering, thus avoiding inconsistent results and suboptimal clustering. This method performs equidistant grouping according to the fixed order of the weighted feature maps generated by feature extraction, fully utilizing the information structure of the CNN features themselves, and obtaining a set of feature masks with low information redundancy and strong decision relevance more concisely and efficiently. Therefore, the ordered grouping method is chosen.

[0143]

[0144] (d) Gaussian blur parameter test

[0145] In the calculation of the global decision contribution of the grouped mask, Gaussian blurring is applied to the occluded region to determine a reasonable Gaussian kernel size, namely:

[0146]

[0147]

[0148] in, The input image is Gaussian blurred, ⊙ represents the Hadamard product, and I' l To mask the image using a blurred mask.

[0149] The results are shown in the table below. As the Gaussian kernel size increases, the ID index improves, reaching its highest value when the Gaussian kernel size is 51. The Gaussian kernel size has no significant impact on positioning accuracy and latency. Therefore, considering all factors, a Gaussian kernel size of 51 (with a kernel standard deviation of 50) was chosen.

[0150]

[0151] (e) Negative gradient elimination test

[0152] In the pixel gradient weighting formula, the LG-CAM algorithm uses ReLU to filter negative gradients, illustrating the impact of negative gradients on the model results. Experimental results are shown in the table below; w / oReLU indicates that the LG-CAM algorithm does not use ReLU in the formula calculation. It can be seen that removing negative gradients can improve both localization accuracy and confidence scores by 2%.

[0153]

[0154] (f) Validation of the effectiveness of the improved method

[0155] To verify the impact of the improved method on the algorithm's interpretability, the proposed improved method was gradually introduced based on the GroupCAM algorithm to verify its effectiveness. The test results are shown in the table below. Comparing the GroupCAM+pp algorithm with the GroupCAM algorithm, it can be seen that the improved method, using high-fine-grained local sensitivity weighting of feature maps, improves the localization accuracy of mloc1 and mloc5 by 0.76% and 1.01%, respectively. The improved method, using global decision contribution calculation with group masks, improves the ID confidence index by 0.121%. Simultaneously, parallel processing of group masks in the code improves the algorithm speed. Overall, the algorithm in this application improves the mloc1, mloc5, and ID indices by 1.01%, 1.28%, and 0.023%, respectively, compared to the benchmark algorithm, demonstrating the effectiveness of the proposed improved method.

[0156] (3) Comparison experiment with other methods

[0157] The proposed LG-CAM method is qualitatively and quantitatively compared with seven interpretable methods. The comparison algorithms include gradient-based guided backpropagation (Guided-BP), perturbation-based RISE, class activation mapping-based GradCAM, GradCAM++, ScoreCAM, ISCAM, and GroupCAM. RISE generates 4000 masks, GroupCAM uses 64 groups, and ISCAM uses 10 integral samples.

[0158] (a) Qualitative analysis

[0159] For four typical example images, different visual interpretation methods generate corresponding visual interpretation images, as follows: Figure 4 .

[0160] From a visual perspective, the Guided-BP method reflects all image features, such as textures and edges, learned by the convolutional layers of the CNN model. However, this type of method modifies gradient information during computation and cannot truly reflect the model's decisions. The perturbation-based RISE method can reflect the key parts of the image that influence the CNN's decisions, exhibiting good visual continuity, but its scope tends to diverge and is difficult to focus on the target region. Conversely, the GradCAM method struggles to activate all relevant regions of the target, resulting in poor visual continuity. In comparison, GradCAM++, ScoreCAM, and GroupCAM offer some improvement in the completeness and visual continuity of the target localization region, but for features like... Figure 4 Example (4) Targets with complex backgrounds may still have a lot of background noise, making it difficult to reflect the decision-making features of CNN. Compared with the comparison algorithm, the LG-CAM method combines the idea of ​​non-gradient global contribution, resulting in less random noise, producing a smoother class activation map, more concentrated, accurate and complete target region localization, better visual continuity, and reflecting the main basis for the model to make classification decisions.

[0161] From the perspective of visualization effects for multi-objectives and small-objectives, Figure 4 In Examples (2) and (3), RISE and GradCAM tend to capture only a single target in the image and are difficult to activate small target regions; GradCAM++ can locate multiple targets, while ScoreCAM and GroupCAM methods have more complete and focused target activation and can locate and discover small targets, but artifacts and noise interference are greater when the background is complex. The LG-CAM proposed in this application can discover and locate multiple similar targets in complex backgrounds with less background interference; at the same time, it is more effective in activating small targets, and the localized area is more accurate, concentrated and complete, with better visual effects.

[0162] From the perspective of category differentiation ability Figure 5This demonstrates that the LG-CAM method can effectively distinguish between different target categories. For the two target categories of butterflies and flowers in the image, the VGG19 model achieves a confidence score of 94.8% for classifying the input image as a butterfly and 1% for classifying it as a flower. Although the latter confidence score is significantly lower than the former, by backpropagating different category confidence scores to obtain different activation feature map gradient weights, the method in this application can still provide class activation maps corresponding to different target regions.

[0163] (b) Quantitative analysis

[0164] L1. Image Occlusion Experiment

[0165] Figure 6 They correspond to Figure 4 The images in the middle are insertion and deletion metric curves and AUC values ​​obtained through image occlusion experiments (left is the insertion metric curve, right is the deletion metric curve). It can be seen that LG-CAM exhibits a faster increase and larger AUC in the insertion metric curve, and a faster decrease and smaller AUC in the deletion metric curve. Figure 4 The ID indexes in Examples (1), (2), and (4) reached the highest levels of 90.11%, 36.28%, and 63.41%, respectively. Figure 4 The ID index of the middle example (3) reached the second highest at 79.17%, indicating that LG-CAM has better interpretability and maintains good consistency with the visual qualitative evaluation results.

[0166] The table below shows the average metrics of LG-CAM and the comparison methods in the image occlusion experiment on the validation set. It can be seen that LG-CAM achieves the best performance on the Del metric, is only 0.07 lower than ISCAM on the Ins metric, and is only 0.07 lower than ISCAM on the ID comprehensive metric, exceeding GradCAM++ by 4.17%. The image occlusion experiment demonstrates that the visual interpretation map provided by the LG-CAM method corresponds to better classification confidence and has stronger interpretive power.

[0167]

[0168] L2. Experiment on the accuracy of weakly supervised target localization

[0169] Experimental tests were conducted on the weakly supervised target localization accuracy of the LG-CAM method and six visual interpretation methods. Figure 7 Visual effects of different target localization methods are shown, with red boxes representing the ground truth bounding boxes and green boxes representing the bounding boxes obtained by the algorithms. From a visual perspective, the target localization box obtained by the algorithm in this application fits the ground truth bounding box better, resulting in more accurate localization.

[0170] To more comprehensively and objectively evaluate positioning accuracy, positioning accuracy under different thresholds was tested on the validation set, resulting in target positioning accuracy curves for loc1 and loc5. (Refer to...) Figure 8 .

[0171] The average positioning accuracy at each threshold was used to obtain the mloc1 and mloc5 positioning accuracies. The algorithm's running speed was also statistically analyzed. In the weakly supervised target localization experiment, compared to the comparative algorithms, the algorithm in this application achieved higher positioning accuracy at each threshold, with mloc1 and mloc5 accuracies reaching the highest, exceeding the second-place GradCAM++ by 0.12% and 0.1%, respectively. This indicates that the algorithm has superior positioning capabilities. Simultaneously, the algorithm in this application balances speed and performance, with a single-frame image processing time of 70ms, which is 76 times that of ISCAM. This demonstrates that the algorithm in this application exhibits superior overall performance.

[0172] In terms of computational results, the proposed method combines the idea of ​​non-gradient global contribution, resulting in less random noise in the model operation results, producing a smoother class activation map, more concentrated, accurate and complete target region localization, better visual continuity, and easier reflection of the model's classification decision.

[0173] Therefore, this method can also detect and locate multiple similar targets in complex backgrounds with little background interference; at the same time, it is more effective in activating small targets, and the localization area is more accurate, concentrated and complete, resulting in better visual effects.

[0174] In another aspect, the present invention also discloses a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the method described above.

[0175] In another aspect, the present invention also discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the method described above.

[0176] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the small target detection model visual interpretability methods in the above embodiments.

[0177] It is understood that the system provided in the embodiments of the present invention corresponds to the method provided in the embodiments of the present invention, and the explanation, examples and beneficial effects of the relevant content can be referred to the corresponding parts of the above methods.

[0178] This application also provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, communication interface, and memory communicate with each other via the communication bus.

[0179] Memory, used to store computer programs;

[0180] The processor, when executing the program stored in memory, implements the visual interpretability method of the small target detection model described above.

[0181] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc.

[0182] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0183] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0184] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0185] It should also be noted that electronic devices include terminal devices, which can also be called terminals, user equipment, mobile stations, mobile terminals, etc. Terminal devices can be mobile phones, smart TVs, wearable devices, tablets, computers with wireless transceiver capabilities, virtual reality terminal devices, augmented reality terminal devices, wireless terminals in industrial control, wireless terminals in autonomous driving, wireless terminals in remote surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, and so on. The embodiments of this application do not limit the specific technologies or device forms used in the terminal devices.

[0186] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium, or a semiconductor medium (e.g., solid-state drive SSD), etc.

[0187] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0188] Furthermore, it should be noted that if any directional indication (such as up, down, left, right, front, back, etc.) is involved in the embodiments of the present invention, the directional indication is only used to explain the relative positional relationship and movement of each component in a specific posture. If the specific posture changes, the directional indication will also change accordingly.

[0189] Furthermore, if the embodiments of this invention involve descriptions such as "first" or "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, the meaning of "and / or" throughout the text includes three parallel solutions; for example, "A and / or B" includes solution A, solution B, or a solution where both A and B are satisfied simultaneously. Furthermore, in the embodiments of this invention, "multiple" refers to two or more. Moreover, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by this invention.

Claims

1. A method for visual interpretability of a small target detection model, characterized in that, Includes the following steps: S1. Obtain deep feature maps through forward propagation of CNN, and obtain the gradients of the feature maps through backpropagation using the classification scores of specific target classes; S2. After pixel-level gradient weighting of the feature map and order-based grouping, a set of masks containing high-fine-grained local sensitivity information is obtained; S3. Input the blurred masked image into the CNN, obtain the global decision contribution through a forward pass containing N integral sampling, calculate the contribution importance score and take the average as the group importance score; S4. Linearly weight the grouped mask to obtain the class activation map. The class activation map is then weighted with importance scores to obtain the visual interpretation map. The specific process of pixel-level gradient weighting of the feature map in step S2 includes: S21. Let... For image classification CNN models, These are the model parameters for a given input image. Feature map weights are obtained through global average pooling. After forward inference through the network, the classification score formula is obtained. for: in Represented as the first Each channel feature map and They represent feature maps respectively. Row indexes and column indexes in the data; S22. Formula for Classification Scores Weights of each feature map within For explicit encoding, the formula for calculating the feature map weights is: in For class The weighting coefficients of the pixel gradients corresponding to the feature map are calculated using the following formula: in, Let be the pixel gradient value of the Kth channel at position (i, j) in feature map A; S23 is a feature map. exist The gradient of each pixel at a given location is weighted to obtain K sets of weighted feature maps. ; The process of generating the mask by ordering grouping in step S2 includes: Will Group weighted feature map The weighted feature maps are divided into equal intervals according to a fixed order. group and By overlaying the weighted feature activation maps of each group, we obtain A mask used to generate a class activation map. The calculation formula is: The obtained low-resolution mask is normalized and then upsampled to the original image size using bilinear interpolation to obtain a smoother mask. The calculation formula is as follows: This is the processed smooth mask; The specific calculation process for obtaining the global decision contribution in step S3 includes: S31. Apply Gaussian blur to the occluded area to obtain a blurred masked occlusion image. The calculation expression is: in, The input image has been Gaussian blurred. For Hadamaji; S32. Accumulate the input image features step by step using linear interpolation as the integration path, calculate the forward propagation classification score on the target class N times and take the average as the group mask importance score to obtain the global decision contribution; The specific operational procedure for calculating the contribution importance score in step S3 includes: Let the baseline image Output If the baseline image is zero, and a pure black image or a noisy image is used as the baseline image, then a gradient-free method is applied, resulting in: in, For the first Importance score of group mask This represents the number of integral samples. parallel computing Forward propagation of the mask.

2. The visual interpretability method for small target detection model as described in claim 1, characterized in that, In step S4, the importance scores are used to weight the G-group masks to obtain a more refined and accurate visual interpretation map. The calculation expression is as follows: in For the first Grouping mask.

3. A computer-readable storage medium, characterized in that, The system stores a computer program that, when executed by a processor, causes the processor to perform the steps of the method as described in any one of claims 1 to 2.

4. A computer device, characterized in that, It includes a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of the method as described in any one of claims 1 to 2.

Images