Thermodynamic diagram evaluation method and system based on class activation mapping and dynamic weight

Abstract

The invention relates to a thermodynamic diagram evaluation method and system based on class activation mapping and dynamic weight, and the method comprises the steps: obtaining a crop image, inputting the crop image into a pre-trained deep learning model, and outputting a crop detection result; processing the crop detection result into a thermodynamic diagram based on class activation mapping, and performing de-noising processing on the thermodynamic diagram to obtain a thermodynamic diagram without background interference; color space conversion is carried out on the thermodynamic diagram without background interference, the color difference between each pixel and a standard color is calculated, and the standard color comprises a standard reference color preset according to the thermodynamic diagram and a corresponding weight; and distributing a weight value for each pixel based on the color difference between each pixel and the reference color, calculating an average temperature weight value, and completing quantitative evaluation of the thermodynamic diagram based on the average temperature weight value. According to the method, a novel interpretive dimension and a novel quantitative standard are provided for application of the deep learning model in agriculture.

Description

Technical Field

[0001] The present invention relates to the technical fields of deep learning and image recognition result evaluation, and particularly to a heatmap evaluation method and system based on class activation mapping and dynamic weights. Background Art

[0002] With the digital and intelligent transformation of agricultural production, the application of object detection technology in agricultural scenarios has become increasingly widespread. The convolutional neural network in the field of deep learning has become the mainstream method in crop detection and recognition due to its powerful feature extraction and learning capabilities. Among them, the YOLO algorithm is widely used in fields such as crop health detection, fruit recognition, and pest control due to its real-time performance and high efficiency. Its advantage lies in the ability to quickly detect multiple targets and adapt to complex agricultural environments (such as different lighting conditions, occlusions, etc.). However, despite the excellent performance of YOLO in these tasks, the "end-to-end" nature of its deep learning model makes the internal decision-making process of the model still an invisible "black box" to users.

[0003] The non-interpretability of the model has always been a bottleneck in the application of deep learning, especially in fields such as agriculture that require high precision and credibility. Users not only hope that the model can provide high-accuracy detection results, but also hope to understand how the model makes these decisions. Therefore, the interpretability of the model has become an important direction in deep learning research. To address this challenge, in recent years, techniques based on class activation mapping and its improved gradient-weighted class activation mapping have been widely applied in the agricultural field to visually present the attention areas of the neural network to different input features. However, although visualization techniques such as Grad-CAM can generate heatmaps of the model to show the degree of attention of the network to specific regions in the image, most of its evaluation metrics rely on subjective visual observations. Current quantitative metrics, such as the coverage rate of high-temperature areas, still lack a unified evaluation standard and do not fully quantify the importance of different regions in the heatmap to the detection results. Therefore, in view of the deficiencies still existing in the application of the above-mentioned existing visualization methods in fine-grained interpretation and multi-object detection, as well as technical problems such as a single application background, the present invention proposes a heatmap evaluation method and system based on class activation mapping and dynamic weights. Summary of the Invention

[0004] The purpose of the present invention is to provide a heatmap evaluation method and system based on class activation mapping and dynamic weights, to make up for the deficiencies of existing heatmap quantization methods, and also to provide a new interpretability dimension and quantization standard for the application of deep learning models in agriculture, and to provide a more interpretable and reproducible reference basis for the intelligent detection of crops.

[0005] To achieve the above object, on the one hand, the heatmap evaluation method based on class activation mapping and dynamic weights of the present invention includes:

[0006] Obtain a crop image, input the crop image into a pre-trained deep learning model, and output a crop detection result;

[0007] Based on class activation mapping, process the crop detection result into a heat map, and perform denoising processing on the heat map to obtain a heat map without background interference;

[0008] Perform color space conversion on the heat map without background interference, and calculate the color difference between each pixel and a reference color, where the reference color includes a reference color preset according to the heat map and corresponding weights;

[0009] Assign a weight value to each pixel based on the color difference between each pixel and the reference color, and calculate an average temperature weight value, and complete the quantitative evaluation of the heat map based on the average temperature weight value.

[0010] Optionally, performing denoising processing on the heat map includes: setting the pixel values of the background part in the heat map to zero.

[0011] Optionally, performing color space conversion on the heat map without background interference and calculating the color difference between each pixel and the reference color includes:

[0012] Convert the heat map without background interference from the RGB color space to the LAB color space, and calculate the color difference between each pixel on the heat map without background interference and the reference color in the LAB color space. The calculation method for the color difference between each pixel and the reference color is:

[0013]

[0014] where, ΔE 00 is the color difference value between the compared pixel and the reference color, ΔL, ΔC, and ΔH respectively represent the differences in brightness, chromaticity, and hue, S L 、S C 、S H are normalization factors, k L 、k C 、k H are weight coefficients, and R T is a hue correction factor.

[0015] Optionally, assigning a weight value to each pixel based on the color difference between each pixel and the reference color includes:

[0016] Based on the color difference between each pixel and the reference color, use a dynamic interpolation mechanism to assign a weight value to each pixel in the heat map without background interference, and set the pixel area as a positive weight value and the background area as a negative weight value.

[0017] Optionally, the dynamic interpolation mechanism is as follows:

[0018] W = interp(ΔE 00 , [ΔE min , ΔE max , [w min , w max );

[0019] Wherein, W represents the weight value of the pixel, and ΔE 00 is the color difference value between the pixel to be compared and the reference color, and ΔE min and ΔE max are respectively the maximum and minimum values of the color difference value, and w min and w max are respectively the maximum and minimum values of the weight value.

[0020] Optionally, the calculation of the average temperature weight value is as follows:

[0021]

[0022] Wherein, ATW is the average temperature weight value, W i represents the weight value of the pixel point i within the target detection frame, W j represents the weight value of the pixel point j outside the target detection frame, and N is the normalization factor.

[0023] On the other hand, the present invention also provides a heat map evaluation system based on class activation mapping and dynamic weights, including:

[0024] A detection result acquisition module, configured to acquire a crop image, input the crop image into a pre-trained deep learning model, and output a crop detection result;

[0025] A heat map processing module, configured to process the crop detection result into a heat map based on class activation mapping, and perform denoising processing on the heat map to obtain a heat map without background interference;

[0026] A color difference calculation module, configured to perform color space conversion on the heat map without background interference, and calculate the color difference between each pixel and the reference color, wherein the reference color includes a reference reference color preset according to the heat map and corresponding weights;

[0027] A weight assignment module, configured to assign a weight value to each pixel based on the size of the color difference between each pixel and the reference color;

[0028] A quantitative evaluation module, configured to calculate the average temperature weight value based on the weight value of each pixel, and complete the quantitative evaluation of the heat map based on the average temperature weight value.

[0029] On the other hand, the present invention also provides a processor for running a program, wherein when the program runs, it executes the steps of the heatmap evaluation method based on class activation mapping and dynamic weights.

[0030] On the other hand, the present invention also provides an electronic device including one or more memories and a processor, wherein the memory is used to store one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement the steps of the heatmap evaluation method based on class activation mapping and dynamic weights.

[0031] The beneficial effects of the present invention are as follows:

[0032] Based on the dynamic weight interpolation calculation of color difference, by quantifying the weights of different color regions in the heatmap, the present invention provides an objective quantification index called Average Temperature Weight (ATW); by dynamically evaluating the color distribution of the crop target area, the interference of the background and low-temperature areas is eliminated, thereby obtaining a more reference-worthy heatmap quantification result. The ATW evaluation system not only makes up for the deficiencies of existing heatmap quantification methods, but also provides a new interpretive dimension and quantification standard for the application of deep learning models such as YOLO in agriculture, provides a more interpretable and reproducible reference basis for the intelligent detection of crops, and is expected to promote the further application of deep learning technology in the field of agricultural detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention, and those of ordinary skill in the art can also obtain other drawings based on these drawings without creative efforts.

[0034] Figure 1 It is a flowchart of the heatmap evaluation method based on class activation mapping and dynamic weights according to an embodiment of the present invention;

[0035] Figure 2 It is a schematic diagram before and after heat treatment according to an embodiment of the present invention;

[0036] Figure 3 It is a schematic diagram before and after denoising treatment according to an embodiment of the present invention;

[0037] Figure 4 It is a schematic diagram of positive and negative weight value annotation according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0038] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0039] To make the above objects, features, and advantages of the present invention more obvious and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0040] This embodiment provides a heatmap evaluation method based on class activation mapping and dynamic weights, as Figure 1 shown, including:

[0041] Obtain a crop image, input the crop image into a pre-trained deep learning model, and output a crop detection result;

[0042] Based on class activation mapping, process the crop detection result into a heatmap, and perform denoising processing on the heatmap to obtain a heatmap without background interference;

[0043] Perform color space conversion on the heatmap without background interference, and calculate the color difference between each pixel and a reference color, where the reference color includes a reference color preset according to the heatmap and the corresponding weight;

[0044] Assign a weight value to each pixel based on the color difference between each pixel and the reference color, and calculate an average temperature weight value, and complete the quantitative evaluation of the heatmap based on the average temperature weight value.

[0045] Specifically, this embodiment first completes the detection of crop targets through a deep learning model, and then generates a heatmap to intuitively display the attention degree of the model to different regions. To further quantify this visualization process, a dynamic weight interpolation method based on color difference is designed to measure the relationship between the color distribution in the heatmap and the model confidence.

[0046] This embodiment adopts a weight distribution method based on color difference, which can dynamically adapt to the color distribution of different images and avoid the limitations of fixed weight methods in complex scenes. By eliminating the interference of background and absolute low temperature areas, focusing on target areas and high temperature areas, the quantification results are more accurate and have a higher reference value. The introduction of the mechanism of positive and negative weights can not only evaluate the effectiveness of target detection, but also quantify the degree of misjudgment of the model in non-target areas, providing a clear direction for further optimization of the model. The improved average temperature weight (ATW) calculation method is based on a scientific color difference model and dynamic interpolation method, combined with positive and negative weights and effective area analysis, so that the quantification results are more interpretable, and more in line with the human eye's perception of thermal maps.

[0047] Furthermore, the crop image is input into the pre-trained deep learning model, and the crop detection results are output, including the location (bounding box), category label and confidence score of the target crop, which specifically include the following:

[0048] Specifically, the deep learning model in this embodiment takes the YOLO (You Only Look Once) model as an example. YOLO is a target detection algorithm based on deep learning. Through a single neural network model, it simultaneously completes the location and classification of the target in a forward propagation process. The biggest feature of YOLO is that it can detect targets in the entire image at a fast speed, which is suitable for real-time detection tasks.

[0049] The goal of the YOLO model is to divide the input image into grids and then predict the target in each grid. The specific process is as follows:

[0050] (1) Image segmentation: The input image is first divided into an S×S grid, where each grid is responsible for detecting an object, especially when the center point of the grid falls on the target object.

[0051] (2) Prediction: The convolutional layer of the neural network (CNN) is used to extract image features, and the fully connected layer is used to make the final prediction. The output is usually a vector of S×S×(B×5+C), where B is the number of bounding boxes predicted for each grid and C is the number of target categories.

[0052] (3) Target detection results: The prediction results of all grids are summarized, and the duplicate prediction results are filtered out through non-maximum suppression technology, and finally the target category label, location coordinates and confidence are output.

[0053] The YOLO model divides the image into multiple grids and performs target detection in each grid. However, the process is neither visible nor explainable, so explainable visualization technology is needed to assist in the optimization of model parameters.

[0054] Further, based on class activation mapping, the crop detection results are processed into a heat map, and the heat map is denoised to obtain a heat map without background interference, which specifically includes the following:

[0055] During the generation of the heat map, this embodiment adopts a two-step processing method: first, a heat map with the background of the original image is generated, and then a pure heat map without the background, that is, a heat map without background interference, is generated.

[0056] (1) Generate a heat map with the background of the original image:

[0057] This embodiment uses Grad-CAM (Gradient-weighted Class Activation Mapping), which is an improvement on the model visualization technology CAM (Class Activation Mapping, CAM) based on class activation mapping, and solves the problem that CAM can only be applied to the last convolutional layer. Grad-CAM makes the generated heat map more accurate by using gradient information to weight the convolutional feature map, and can handle complex neural network structures. The specific improvements are as follows:

[0058] 1) Utilization of gradient information: Different from CAM, Grad-CAM not only depends on the product of the convolutional layer feature map and the weight, but also introduces gradient information. Grad-CAM calculates the gradient of the category with respect to the convolutional layer feature map, and these gradients reflect the regions of interest of the model for specific categories.

[0059] 2) Weighted feature map: Grad-CAM combines gradient information with the convolutional feature map. Specifically, Grad-CAM calculates the influence of each feature map on the category prediction, and uses this influence as the weight to weight the feature map. The final heat map is the sum of the weighted feature maps:

[0060]

[0061] 3) Generate heat map: The heat map generated by Grad-CAM is mapped back to the original image to visualize the regions of interest of the model. Compared with CAM, the advantage of Grad-CAM is that it can effectively process more layers of convolutional feature maps and can give more detailed regions of interest, as Figure 2 shown.

[0062] (2) Generate a pure heat map without background:

[0063] By denoising the heat map, specifically, the pixel values of the background part in the heat map are set to zero to generate a pure heat map without background, as Figure 3 shown.

[0064] The above two-step processing method can meet the requirements of intuitive display and quantitative analysis at the same time. The heatmap with background enables researchers to clearly see the detection effect of the target object and the attention of the model to the target area; while the pure heatmap without background eliminates the pixels irrelevant to the target, providing a cleaner input for subsequent quantitative calculations.

[0065] Further, perform a color space conversion on the heatmap without background interference, and calculate the color difference between each pixel and the reference color, including: converting the heatmap without background interference from the RGB color space to the LAB color space, and calculating the color difference between each pixel on the heatmap without background interference and the reference color in the LAB color space, where the reference color includes a preset reference color and corresponding weights according to the heatmap.

[0066] Assign a weight value to each pixel based on the color difference between each pixel and the reference color, including: based on the color difference between each pixel and the reference color, use a dynamic interpolation mechanism to assign a weight value to each pixel in the heatmap without background interference, and set the pixel area as a positive weight value and the background area as a negative weight value.

[0067] Specifically, in this embodiment, a reference color and corresponding weights are first set based on the LAB color space:

[0068] In the generated class activation heatmap, different color regions represent the attention of the model to different regions of the input image. Therefore, selecting a suitable reference color and assigning reasonable weights to it are key steps in the heatmap quantization process. Through the analysis of the heatmaps of a large number of excellent agricultural detection models and the evaluation of the model detection results, this embodiment selects red, yellow, green, and blue as the reference colors. These colors not only represent the hierarchical changes in the model's attention to the target object, but also reflect the confidence distribution in different regions of the heatmap.

[0069] 1) Red is given the highest weight because after testing and verification, in the heatmap generated by the model, the red region often corresponds to the target detection result with the highest confidence of the model. When the target object is detected by the model with high confidence, a large area of red region often appears in the heatmap. By comparing the actual detection results and the coverage range of the red region, it is found that the red region highly coincides with the position of the detected target object. Therefore, red is given the highest weight in the heatmap.

[0070] 2) Yellow, as the color with the second highest weight, represents the region with the second highest confidence of the model. By comparing the test results, it is found that the detection accuracy rate of the yellow region is slightly lower than that of the red region, but still has high accuracy. Usually, the yellow region appears in the detection results of the model for the edges or some features of the target object, so it is given the second highest weight.

[0071] 3) Green represents the detection areas with relatively low model confidence. During the detection process, the green areas often cover the blurred parts or occluded areas around the target object. In a large number of tests, the detection results in the green areas are less accurate than those in the red and yellow areas, so the corresponding weights for green are lower. However, considering that these areas may contain useful information in some cases, a relatively low positive weight is assigned to the green areas.

[0072] 4) Blue and its near - blue areas are considered as areas that the model deems irrelevant or with extremely low confidence. In actual detection, these areas usually correspond to the background or irrelevant areas in the image. To ensure that the evaluation of the attention to the target object during the heatmap quantization process is not interfered by the background areas, in this embodiment, the weights of the blue and near - blue areas are fixed at 0. Through this processing, the heatmap only contains the parts that the model deems important, excluding the influence caused by the low - temperature areas.

[0073] Then, the difference between each pixel and its corresponding reference color is accurately calculated through the CIEDE2000 color difference formula. CIEDE2000 color difference is one of the mainstream methods for measuring the visual difference between two colors at present, which can more accurately reflect the position of colors in the LAB color space. The calculation formula is as follows:

[0074]

[0075] Among them, ΔE 00 is the color difference value between the compared pixel and the reference color, ΔL, ΔC, and ΔH respectively represent the differences in brightness, chroma, and hue, S L , S C , S H are normalization factors, k L , k C , k H are weight coefficients, and R T is the hue correction factor

[0076] Then, for each pixel, according to its color difference from the reference color, a dynamic weight interpolation method is used to assign a weight value to it. As the color difference increases, the weight value gradually decreases. The process of assigning weight values to different color areas is based on the mechanism of color difference and dynamic interpolation. The calculation formula is:

[0077] W = interp(ΔE 00 , [ΔE min , ΔE max , [w min , w max ) ;

[0078] Among them, W represents the weight value of the pixel, and ΔE 00is the color difference value between the pixel to be compared and the reference color, ΔE min and ΔE max are respectively the maximum and minimum values of the color difference value, w min and w max are respectively the maximum and minimum values of the weight value.

[0079] Through this dynamic weight assignment, this embodiment effectively converts the color information in the heat map into a quantified weight value, making the subsequent calculation of the average temperature weight ATW more interpretable and consistent.

[0080] This embodiment also includes dividing the pixels in the heat map into positive weights (target area) and negative weights (non-target area) according to the position of the target detection frame. Among them, the pixels in the target area are regarded as positive values, and the pixels in the non-target area are regarded as negative values. As Figure 4 shown.

[0081] Furthermore, calculate the average temperature weight value, and complete the quantitative evaluation of the heat map based on the average temperature weight value. The calculation of the average temperature weight value is as follows:

[0082]

[0083] Among them, ATW is the average temperature weight value, W i represents the weight value of the pixel point i within the target detection frame, W j represents the weight value of the pixel point j outside the target detection frame, and N is the normalization factor.

[0084] Specifically, in order to objectively quantify the heat map, this embodiment proposes a new quantization index, the average temperature weight ATW. ATW quantifies the attention of the model to different color regions by calculating the weight distribution of each region in the image. First, the image is converted to the LAB color space, and then the color difference between each pixel in the image and the reference colors (red, yellow, green, blue) is calculated through the CIEDE2000 color difference. Through the dynamic interpolation algorithm, weights are assigned to each pixel according to different color difference values. During the quantization process, this embodiment obtains the overall quantization result of the heat map by weighted averaging the pixels in all non-zero weight regions of the heat map. This quantization process excludes the interference of the background and low-temperature regions, ensuring that the ATW index is more referenceable and consistent, and providing an effective measurement standard for the evaluation of subsequent experimental results.

[0085] Next, analyze the superiority of this method compared with the existing technology:

[0086] Compare this method with the calculation method of the "average temperature weight value" mentioned in the existing technical solution 1: "Real-time Detection Method and Experimental Research of Field Chinese Cabbage Seedlings Based on YOLOv8-CGB",

[0087] 1) Method for calculating weights of reference colors:

[0088] Solution 1: Directly assign fixed weight values to reference colors (e.g., 1.0 for red, 0.6 - 0.8 for yellow, 0.4 - 0.6 for green, and 0 for blue). Although this method is simple, it lacks dynamic adaptability to the weight distribution in actual scenarios.

[0089] This method: By calculating the CIEDE2000 color difference between the pixel color and the reference color, the fixed weight is changed to dynamic allocation based on the color difference, making the weight distribution better reflect the actual contribution of different pixels to the model's attention.

[0090] Effect: The weight assignment method based on the CIEDE2000 color difference can dynamically adapt to the color distribution of different images, avoiding the limitations of the fixed weight method in complex scenarios.

[0091] 2) Range of weight calculation:

[0092] Solution 1: The weight assignment range covers the entire image, including the background area.

[0093] This method: Exclude the absolute low-temperature area (pixels with a weight of 0), and the background and irrelevant areas no longer participate in the calculation, thus significantly improving the accuracy and consistency of the calculation results.

[0094] Effect: By eliminating the interference of the background and the absolute low-temperature area, the improved method focuses on the target area and the high-temperature area, and the quantization result is more accurate and has higher reference value.

[0095] 3) Introduction of positive and negative weights:

[0096] Solution 1: Do not distinguish between the positive and negative nature of pixel weights, and all weight pixels are regarded as positive contributions.

[0097] This method: Distinguish the weight nature of pixels inside and outside the target detection box: The weights inside the detection box are regarded as positive values, indicating the effective attention of the model to the target area. The high-temperature weights outside the detection box are regarded as negative values, reflecting the misjudgment of the model in the non-target area.

[0098] Effect: The introduction of the mechanism of positive and negative weights enables the method to not only evaluate the effectiveness of target detection but also quantify the misjudgment degree of the model in the non-target area, providing a clear direction for further optimizing the model.

[0099] 4) Improvement of the ATW calculation formula:

[0100] Solution 1: Accumulate the weights of all pixels and divide by the total number of pixels in the entire image to obtain the average temperature weight.

[0101] This method: comprehensively evaluates the positive and negative contributions of all pixels in the image to obtain the average temperature weight.

[0102] Effect: The improved ATW calculation method is based on a scientific color difference model and a dynamic interpolation method, combined with positive and negative weights and effective region analysis, making the quantization result more interpretable and more in line with the human eye's perception of the heat map.

[0103] The core advantage of this method is that it can not only quantify the attention degree of the target area, but also clarify the attention difference between the target area and the non-target area in the heat map by distinguishing the positive and negative weights in the heat map. Although the heat map of Grad-CAM can show the attention degree of the target area, for some misjudged areas or background areas, the high temperature values in the heat map may not represent the correct attention (such as background misjudgment). This method can clearly show which areas are actually effectively detected by the model and which areas are just background or misjudgment interference through the distinction of positive and negative weights. This is particularly important, especially when dealing with tasks of complex backgrounds or multi-object detection. Directly extracting the weights in the heat map may not clearly reflect the impact of misjudged areas.

[0104] This method not only provides the overall quantization result of the heat map, but also can conduct a fine-grained evaluation of the target area and the non-target area. This is particularly important for the interpretability of the model, especially in refined application fields such as agriculture, where it is possible to more deeply understand the performance of the model in specific areas. In practical applications, especially in agricultural monitoring, it is very important to understand why the model shows a high degree of attention in a certain area. If the ATW result shows that some high-temperature areas actually belong to misjudgments, this can provide a clear direction for the optimization of the model. For example, through ATW, it is possible to confirm which areas are actually not correctly identified as targets but are background noise or non-target areas, which is of great significance for the improvement of training data and the optimization of the model.

[0105] In multi-object detection, the uneven distribution and size of objects may cause some small or marginal objects to be ignored or misjudged. The heat map generated by Grad-CAM shows different intensities of colors within the target area. However, relying solely on the color intensity in the heat map may not be sufficient to evaluate the differences between multiple objects. This method can provide a clearer quantitative evaluation for the multi-object detection task through the classification of positive and negative weights and the precise weighting of the effective regions in the heat map. It can provide better discrimination between the target area and the non-target area, avoiding some misjudged areas from occupying too much weight in the overall evaluation.

[0106] This method uses ATW as a quantitative evaluation metric. Its results are not only used to evaluate the interpretability of the model but also serve as a feedback mechanism to drive model improvement. In practical applications, it may be necessary to optimize the model according to the ATW results, especially when adjusting the attention area of the network or eliminating certain background areas. By continuously monitoring the ATW value, especially during the training process, if it is found that the ATW value of some target areas is much lower than that of other areas, it may mean that the model pays insufficient attention to these target areas, thus providing a basis for subsequent training optimization. ATW can be used as an optimization goal to promote the model to focus more precisely on the target area, thereby improving the detection accuracy.

[0107] Although Grad-CAM itself provides the generation of heatmaps and color intensity mapping, the calculation of ATW is still relatively simple and can effectively reduce the interference of the background area on the heatmap. ATW is not only a supplement to the existing heatmaps, but its flexibility enables it to be adapted to a variety of practical scenarios. The calculation of ATW is just a weighted average of the pixel weights in the heatmap, and its formula and implementation method are relatively simple, and meaningful quantitative metrics can be generated quickly. Especially during multiple experiments and scenario switches, ATW can quickly evaluate and compare the performance of the model in different datasets or different tasks.

[0108] This method not only strengthens the quantitative evaluation of the target area and non-target area, but also improves the interpretability by introducing positive and negative weight differentiation, and provides clear feedback for subsequent model optimization. Especially when dealing with tasks such as complex backgrounds and multi-object detection, this method can provide more refined analysis and avoid the possible misguidance simply relying on the heatmap itself.

[0109] To further optimize the technical solution, this embodiment also provides a heatmap evaluation system based on class activation mapping and dynamic weights, including:

[0110] A detection result acquisition module, configured to acquire a crop image, input the crop image into a pre-trained deep learning model, and output a crop detection result;

[0111] A heatmap processing module, configured to process the crop detection result into a heatmap based on class activation mapping, and perform denoising processing on the heatmap to obtain a heatmap without background interference;

[0112] A color difference calculation module, configured to perform color space conversion on the heatmap without background interference and calculate the color difference between each pixel and a reference color, where the reference color includes a reference reference color preset according to the heatmap and the corresponding weight;

[0113] A weight assignment module, configured to assign a weight value to each pixel based on the size of the color difference between each pixel and the reference color;

[0114] A quantitative evaluation module, configured to calculate an average temperature weight value based on the weight value of each pixel, and complete the quantitative evaluation of the heat map based on the average temperature weight value.

[0115] To further optimize the technical solution, this embodiment further provides a processor, which is used to run a program. When the program runs, it executes the steps of the heat map evaluation method based on class activation mapping and dynamic weights.

[0116] To further optimize the technical solution, this embodiment further provides an electronic device, including one or more memories and processors. The memory is used to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the steps of the heat map evaluation method based on class activation mapping and dynamic weights.

[0117] The above-described embodiments are only descriptions of the preferred embodiments of the present invention, and do not limit the scope of the present invention. Without departing from the design spirit of the present invention, various deformations and improvements made by those of ordinary skill in the art to the technical solution of the present invention shall fall within the protection scope determined by the claims of the present invention.

Claims

1. A heat map evaluation method based on class activation mapping and dynamic weights, characterized in that: include: Acquire crop images, input the crop images into a pre-trained deep learning model, and output crop detection results; Processing the crop detection result into a heat map based on class activation mapping, and performing denoising on the heat map to obtain a heat map without background interference; Performing color space conversion on the heat map without background interference, and calculating the color difference between each pixel and a reference color, wherein the reference color includes a reference reference color preset according to the heat map and a corresponding weight; A weight value is assigned to each pixel based on the color difference between each pixel and the reference color, and an average temperature weight value is calculated, and a quantitative evaluation of the thermal map is completed based on the average temperature weight value.

2. The heat map evaluation method based on class activation mapping and dynamic weight according to claim 1, characterized in that: The denoising process for the heat map includes: setting the pixel values ​​of the background part of the heat map to zero.

3. The heat map evaluation method based on class activation mapping and dynamic weight according to claim 1, characterized in that: Performing color space conversion on the heat map without background interference and calculating the color difference between each pixel and the reference color includes: The heat map without background interference is converted from the RGB color space to the LAB color space, and the color difference between each pixel and the reference color on the heat map without background interference is calculated in the LAB color space. The color difference calculation method between each pixel and the reference color is: Where, ΔE 00 is the color difference between the compared pixel and the reference color, ΔL, ΔC, and ΔH represent the differences in brightness, chromaticity, and hue, respectively, and S L , S C , S H is the normalization factor, k L , k C , k H is the weight coefficient, R T is the hue correction factor.

4. The heat map evaluation method based on class activation mapping and dynamic weight according to claim 1, characterized in that: Assigning a weight value to each pixel based on the color difference between each pixel and the reference color includes: Based on the color difference between each pixel and the reference color, a dynamic interpolation mechanism is used to assign a weight value to each pixel in the thermal map without background interference, and the pixel area is set to a positive weight value and the background area is set to a negative weight value.

5. The heat map evaluation method based on class activation mapping and dynamic weight according to claim 4, characterized in that: The dynamic interpolation mechanism is: W=interp(ΔE 00 ,[ΔE min ,ΔE max ],[w min ,w max ]); Where W represents the weight value of the pixel, ΔE 00 is the color difference between the compared pixel and the reference color, ΔE min and ΔE max are the maximum and minimum values ​​of the color difference, w min and w max are the maximum and minimum weight values ​​respectively.

6. The heat map evaluation method based on class activation mapping and dynamic weight according to claim 1, characterized in that: The average temperature weight value is calculated as: Among them, ATW is the average temperature weight value, W i Represents the weight value of pixel i in the target detection box, W j represents the weight value of pixel j outside the target detection box, and N is the normalization factor.

7. A heat map evaluation system based on class activation mapping and dynamic weights, characterized in that: include: A detection result acquisition module, used to acquire crop images, input the crop images into a pre-trained deep learning model, and output crop detection results; A heat map processing module, used for processing the crop detection result into a heat map based on class activation mapping, and performing denoising on the heat map to obtain a heat map without background interference; A color difference calculation module, used to perform color space conversion on the heat map without background interference, and calculate the color difference between each pixel and a reference color, wherein the reference color includes a reference reference color preset according to the heat map and a corresponding weight; A weight allocation module, used for allocating a weight value to each pixel based on the color difference between each pixel and a reference color; The quantitative evaluation module is used to calculate an average temperature weight value based on the weight value of each pixel, and complete the quantitative evaluation of the thermal map based on the average temperature weight value.

8. A processor, characterized in that: The processor is used to run a program, wherein the program, when running, executes the steps of the heat map evaluation method based on class activation mapping and dynamic weights according to any one of claims 1 to 6.

9. An electronic device, characterized in that: It includes one or more memories and processors, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the steps of the heat map evaluation method based on class activation mapping and dynamic weights as described in any one of claims 1 to 6.

Images