Heatmap evaluation method and system based on class activation mapping and dynamic weights

Abstract

This invention relates to a heatmap evaluation method and system based on class activation mapping and dynamic weights, comprising: acquiring crop images; inputting the crop images into a pre-trained deep learning model to output crop detection results; processing the crop detection results into a heatmap based on class activation mapping, and performing noise reduction processing on the heatmap to obtain a heatmap without background interference; performing color space conversion on the heatmap without background interference, and calculating the color difference between each pixel and a reference color, wherein the reference color includes a preset reference color and corresponding weight based on the heatmap; assigning a weight value to each pixel based on the color difference between each pixel and the reference color, and calculating an average temperature weight value; and completing a quantitative evaluation of the heatmap based on the average temperature weight value. This invention provides a new interpretive dimension and quantitative standard for the application of deep learning models in agriculture.

Description

Technical Field

[0001] This invention relates to the field of deep learning and image recognition result evaluation technology, and in particular to a heatmap evaluation method and system based on class activation mapping and dynamic weights. Background Technology

[0002] With the digital and intelligent transformation of agricultural production, target detection technology is being applied more and more widely in agricultural scenarios. Convolutional neural networks in the field of deep learning, with their powerful feature extraction and learning capabilities, have become the mainstream method for crop detection and recognition. Among them, the YOLO algorithm, with its real-time performance and efficiency, is widely used in crop health monitoring, fruit identification, and pest and disease control. Its advantages lie in its ability to quickly detect multiple targets and adapt to complex agricultural environments (such as different lighting conditions and occlusion). However, despite YOLO's excellent performance in these tasks, the "end-to-end" nature of its deep learning model means that the model's internal decision-making process remains an invisible "black box" to the user.

[0003] The lack of interpretability of models has always been a bottleneck in deep learning applications, especially in fields like agriculture where accuracy and reliability are crucial. Users not only want models to provide high-accuracy detection results, but also want to understand how the models make these decisions. Therefore, model interpretability has become an important direction in deep learning research. To address this challenge, in recent years, techniques based on class activation mapping and its improved gradient-weighted class activation mapping have been widely applied in agriculture to visually represent the regions of interest of neural networks for different input features. However, although visualization techniques such as Grad-CAM can generate heatmaps of models, showing the degree of attention the network pays to specific regions in an image, their evaluation metrics mostly rely on subjective visual observation. Currently, some quantitative metrics, such as high-temperature zone coverage, still lack unified evaluation standards and fail to fully quantify the importance of different regions in the heatmap to the detection results. Therefore, addressing the shortcomings of existing visualization methods in fine-grained interpretation and multi-target detection applications, as well as their limited application context, this invention proposes a heatmap evaluation method and system based on class activation mapping and dynamic weights. Summary of the Invention

[0004] The purpose of this invention is to provide a heatmap evaluation method and system based on class activation mapping and dynamic weights, which makes up for the shortcomings of existing heatmap quantification methods, and also provides a new interpretive dimension and quantification standard for the application of deep learning models in agriculture, and provides a more interpretable and reproducible reference for intelligent detection of crops.

[0005] To achieve the above objectives, on the one hand, the present invention provides a heatmap evaluation method based on class activation mapping and dynamic weights, comprising:

[0006] Acquire crop images, input the crop images into a pre-trained deep learning model, and output crop detection results;

[0007] The crop detection results are processed into a heat map based on class activation mapping, and the heat map is then denoised to obtain a heat map without background interference.

[0008] The heatmap without background interference is converted to a color space, and the color difference between each pixel and the reference color is calculated. The reference color includes a preset reference color based on the heatmap and its corresponding weight.

[0009] A weight value is assigned to each pixel based on the color difference between each pixel and the reference color, and an average temperature weight value is calculated. A quantitative evaluation of the heat map is then completed based on the average temperature weight value.

[0010] Optionally, denoising the heatmap includes setting the pixel values ​​of the background portion of the heatmap to zero.

[0011] Optionally, performing color space conversion on the heatmap without background interference and calculating the color difference between each pixel and the reference color includes:

[0012] The heatmap without background interference is converted from the RGB color space to the LAB color space. The color difference between each pixel in the heatmap without background interference and the reference color is calculated in the LAB color space. The method for calculating the color difference between each pixel and the reference color is as follows:

[0013]

[0014] Where, ΔE 00 The color difference between the compared pixel and the reference color is represented by ΔL, ΔC, and ΔH, which represent the differences in brightness, chromaticity, and hue, respectively. L S C S H It is the standardization factor, k L k C k H It is the weighting coefficient, R T It is a hue correction factor.

[0015] Optionally, assigning a weight value to each pixel based on the color difference between each pixel and the reference color includes:

[0016] Based on the color difference between each pixel and the reference color, a dynamic interpolation mechanism is used to assign a weight value to each pixel in the heatmap without background interference, and the pixel area is set as a positive weight value and the background area as a negative weight value.

[0017] Optionally, the dynamic interpolation mechanism is as follows:

[0018] W = interp(ΔE) 00 , [ΔE min ΔE max ],[w min w max ]);

[0019] Where W represents the pixel weight value, ΔE 00 ΔE represents the color difference between the compared pixel and the reference color. min and ΔE max These are the maximum and minimum values ​​of the color difference, w min and w max These are the maximum and minimum values ​​of the weight, respectively.

[0020] Optionally, the average temperature weighting value is calculated as follows:

[0021]

[0022] Where ATW is the average temperature weighting value, W i W represents the weight value of pixel i within the target detection bounding box. j This represents the weight value of pixel j outside the target detection box, where N is the normalization factor.

[0023] On the other hand, the present invention also provides a heatmap evaluation system based on class activation mapping and dynamic weights, including:

[0024] The detection result acquisition module is used to acquire crop images, input the crop images into a pre-trained deep learning model, and output crop detection results.

[0025] The heatmap processing module is used to process the crop detection results into a heatmap based on class activation mapping, and to perform noise reduction processing on the heatmap to obtain a heatmap without background interference.

[0026] The color difference calculation module is used to perform color space conversion on the heatmap without background interference and calculate the color difference between each pixel and the reference color, wherein the reference color includes a preset reference color based on the heatmap and the corresponding weight.

[0027] The weight allocation module is used to assign a weight value to each pixel based on the color difference between each pixel and the reference color.

[0028] The quantitative evaluation module is used to calculate the average temperature weight value based on the weight value of each pixel, and to complete the quantitative evaluation of the heat map based on the average temperature weight value.

[0029] On the other hand, the present invention also provides a processor for running a program, wherein the program executes the steps of the heatmap evaluation method based on class activation mapping and dynamic weights during runtime.

[0030] On the other hand, the present invention also provides an electronic device including one or more memories and a processor, the memories being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the steps of the heatmap evaluation method based on class activation mapping and dynamic weights.

[0031] The beneficial effects of this invention are as follows:

[0032] This invention, based on dynamic weighted interpolation calculation of color difference, provides an objective quantitative index called Average Temperature Weight (ATW) by quantifying the weights of different color regions in a heatmap. By dynamically evaluating the color distribution of the target crop region and eliminating interference from background and low-temperature areas, it yields a more meaningful heatmap quantification result. The ATW evaluation system not only compensates for the shortcomings of existing heatmap quantification methods but also provides a new interpretive dimension and quantitative standard for the application of deep learning models such as YOLO in agriculture. It offers a more interpretable and reproducible reference for intelligent crop detection and is expected to promote the further application of deep learning technology in agricultural detection. Attached Figure Description

[0033] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0034] Figure 1 This is a flowchart of the heatmap evaluation method based on class activation mapping and dynamic weights according to an embodiment of the present invention;

[0035] Figure 2 This is a schematic diagram showing the heat treatment process before and after an embodiment of the present invention;

[0036] Figure 3 This is a schematic diagram showing the noise reduction process before and after in an embodiment of the present invention;

[0037] Figure 4 This is a schematic diagram illustrating the labeling of positive and negative weight values ​​in an embodiment of the present invention. Detailed Implementation

[0038] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0039] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0040] This embodiment provides a heatmap evaluation method based on class activation mapping and dynamic weights, such as Figure 1 As shown, it includes:

[0041] Acquire crop images, input the crop images into a pre-trained deep learning model, and output crop detection results;

[0042] The crop detection results are processed into a heat map based on class activation mapping, and the heat map is then denoised to obtain a heat map without background interference.

[0043] The heatmap without background interference is converted to a color space, and the color difference between each pixel and the reference color is calculated. The reference color includes a preset reference color based on the heatmap and its corresponding weight.

[0044] A weight value is assigned to each pixel based on the color difference between each pixel and the reference color, and an average temperature weight value is calculated. A quantitative evaluation of the heat map is then completed based on the average temperature weight value.

[0045] Specifically, this embodiment first uses a deep learning model to detect crop targets, and then generates a heatmap to visually demonstrate the model's attention to different areas. To further quantify this visualization process, a dynamic weighted interpolation method based on color difference is designed to measure the relationship between the color distribution in the heatmap and the model's confidence level.

[0046] This embodiment employs a color difference-based weighting method, which dynamically adapts to the color distribution of different images, avoiding the limitations of fixed-weight methods in complex scenes. By eliminating interference from the background and extremely low-temperature regions, it focuses on the target and high-temperature areas, resulting in more accurate quantification results with higher reference value. The introduction of a positive and negative weighting mechanism not only evaluates the effectiveness of target detection but also quantifies the degree of misjudgment by the model in non-target areas, providing a clear direction for further model optimization. The improved Average Temperature Weight (ATW) calculation method is based on a scientific color difference model and dynamic interpolation method, combined with positive and negative weighting and effective area analysis, making the quantification results more interpretable and more consistent with human perception of heatmaps.

[0047] Furthermore, the crop images are input into a pre-trained deep learning model, which outputs crop detection results, including the location (boundary box) of the target crop, the category label, and the confidence score, specifically including the following:

[0048] Specifically, in this embodiment, the deep learning model used is YOLO (You Only Look Once). YOLO is a deep learning-based object detection algorithm that uses a single neural network model to simultaneously locate and classify objects during a forward propagation process. YOLO's greatest advantage is its ability to perform object detection on the entire image at a relatively high speed, making it suitable for real-time detection tasks.

[0049] The goal of the YOLO model is to segment the input image into a grid and then predict the target within each grid. The specific process is as follows:

[0050] (1) Image segmentation: The input image is first divided into an S×S grid, and each grid is responsible for detecting a target, especially when the center point of the grid falls on the target object.

[0051] (2) Prediction: Image features are extracted through the convolutional layer (CNN) of the neural network, and the final prediction is performed through the fully connected layer. The output is usually an S×S×(B×5+C) vector, where B is the number of bounding boxes predicted for each grid and C is the number of target categories.

[0052] (3) Target detection results: The prediction results of all grids are summarized and duplicate prediction results are filtered out by non-maximum suppression technology. Finally, the target's category label, location coordinates and confidence score are output.

[0053] The YOLO model divides an image into multiple grids and performs object detection within each grid. However, the process is not visible or interpretable, so interpretable visualization techniques are needed to assist in the parameter tuning and optimization of the model.

[0054] Furthermore, the crop detection results are processed into a heatmap based on class activation mapping, and the heatmap is then denoised to obtain a heatmap free of background interference. Specifically, this includes the following:

[0055] In the process of generating the heatmap, this embodiment adopts a two-step processing method: first, a heatmap with the original image background is generated, and then a pure heatmap with the background removed is generated, that is, a heatmap without background interference.

[0056] (1) Generate a heatmap with the original image background:

[0057] This embodiment employs Grad-CAM (Gradient-weighted Class Activation Mapping), an improvement upon CAM (Class Activation Mapping) model visualization technology, which addresses the limitation of CAM only being applicable to the last convolutional layer. Grad-CAM utilizes gradient information to weight convolutional feature maps, resulting in more accurate heatmaps and the ability to handle complex neural network structures. Specific improvements are as follows:

[0058] 1) Utilization of gradient information: Unlike CAM, Grad-CAM not only relies on the product of convolutional layer feature maps and weights, but also introduces gradient information. Grad-CAM calculates the gradient of the class with respect to the convolutional layer feature maps, and these gradients reflect the model's focus on the region of interest for a specific class.

[0059] 2) Weighted Feature Maps: Grad-CAM combines gradient information with convolutional feature maps. Specifically, Grad-CAM calculates the impact of each feature map on class prediction and uses this impact as a weight to weight the feature maps. The final heatmap is the sum of the weighted feature maps:

[0060]

[0061] 3) Generating Heatmaps: The heatmaps generated by Grad-CAM are mapped back to the original image to visualize the regions of interest for the model. Compared to CAM, Grad-CAM's advantage lies in its ability to effectively process convolutional feature maps from more layers and to provide more detailed regions of interest, such as... Figure 2 As shown.

[0062] (2) Generate a pure heatmap with background removed:

[0063] By denoising the heatmap, specifically by setting the pixel values ​​of the background portion of the heatmap to zero, a pure heatmap without the background is generated. Figure 3 As shown.

[0064] The two-step processing method described above can simultaneously meet the needs of intuitive display and quantitative analysis. The heatmap with background allows researchers to clearly see the detection effect of the target object and the model's focus on the target area; while the pure heatmap without background removes pixels irrelevant to the target, providing a cleaner input for subsequent quantitative calculations.

[0065] Furthermore, performing color space conversion on the heatmap without background interference and calculating the color difference between each pixel and the reference color includes: converting the heatmap without background interference from the RGB color space to the LAB color space, and calculating the color difference between each pixel on the heatmap without background interference and the reference color in the LAB color space, wherein the reference color includes a preset reference color and corresponding weight based on the heatmap.

[0066] Assigning a weight value to each pixel based on the color difference between each pixel and the reference color includes: assigning a weight value to each pixel in the heatmap without background interference using a dynamic interpolation mechanism based on the color difference between each pixel and the reference color, and setting the pixel area as a positive weight value and the background area as a negative weight value.

[0067] Specifically, this embodiment first sets a reference color and its corresponding weight based on the LAB color space:

[0068] In the generated class activation heatmap, different colored regions represent the model's attention to different areas of the input image. Therefore, selecting appropriate reference colors and assigning them reasonable weights is a crucial step in the heatmap quantization process. Through heatmap analysis of numerous excellent agricultural detection models and evaluation of model detection results, this embodiment selects red, yellow, green, and blue as baseline reference colors. These colors not only represent the hierarchical changes in the model's attention to the target object but also reflect the confidence distribution of different regions in the heatmap.

[0069] 1) Red is given the highest weight because, after testing and validation, red areas in the heatmap generated by the model often correspond to the target detection results with the highest model confidence. When a target is detected by the model with high confidence, it often appears as a large red area in the heatmap. By comparing the actual detection results with the coverage of the red areas, it was found that the red areas highly match the locations of the detected target objects. Therefore, red is given the highest weight in the heatmap.

[0070] 2) Yellow, as the second-highest weighted color, represents the second-highest confidence region of the model. Comparative testing revealed that the detection accuracy in the yellow region is slightly lower than that in the red region, but still maintains high accuracy. Typically, the yellow region appears in the model's detection results for object edges or partial features, and therefore is assigned the second-highest weight.

[0071] 3) Green represents detection areas with low model confidence. During detection, green areas often cover blurred or occluded areas around the target object. In numerous tests, the detection results for green areas are less accurate than those for red and yellow, hence the lower weight for green. However, considering that these areas may contain useful information in certain situations, a lower positive weight is assigned to green areas.

[0072] 4) Blue and near-blue regions are considered irrelevant or have extremely low confidence levels by the model. In actual detection, these regions typically correspond to background or irrelevant areas in the image. To ensure that the assessment of the target's importance during heatmap quantization is not interfered with by background regions, this embodiment fixes the weight of blue and near-blue regions to 0. Through this processing, the heatmap only includes the parts that the model considers important, eliminating the influence of low-temperature regions.

[0073] Then, the difference between each pixel and its corresponding reference color is precisely calculated using the CIEDE2000 color difference formula. CIEDE2000 color difference is one of the mainstream methods for measuring the visual difference between two colors, and it can more accurately reflect the position of a color in the LAB color space. The calculation formula is as follows:

[0074]

[0075] Where, ΔE 00 The color difference between the compared pixel and the reference color is represented by ΔL, ΔC, and ΔH, which represent the differences in brightness, chromaticity, and hue, respectively. L S C S H It is the standardization factor, k L k C k H It is the weighting coefficient, R T It is a hue correction factor

[0076] Then, for each pixel, a weight value is assigned using dynamic weighted interpolation based on its color difference from the reference color. As the color difference increases, the weight value gradually decreases. The process of assigning weight values ​​to different color regions is based on a mechanism of color difference and dynamic interpolation, and the calculation formula is as follows:

[0077] W = interp(ΔE) 00 , [ΔE min ΔE max ],[w min w max ]);

[0078] Where W represents the pixel weight value, ΔE 00ΔE represents the color difference between the compared pixel and the reference color. min and ΔE max These are the maximum and minimum values ​​of the color difference, w min and w max These are the maximum and minimum values ​​of the weight, respectively.

[0079] Through this dynamic weight allocation, this embodiment effectively converts the color information in the heatmap into quantified weight values, making the subsequent average temperature weight (ATW) calculation more interpretable and consistent.

[0080] This embodiment also includes dividing pixels in the heatmap into positive weights (target region) and negative weights (non-target region) based on the position of the target detection box. Pixels within the target region are considered positive values, and pixels outside the target region are considered negative values. Figure 4 As shown.

[0081] Furthermore, the average temperature weight value is calculated, and a quantitative evaluation of the heat map is completed based on the average temperature weight value. The average temperature weight value is calculated as follows:

[0082]

[0083] Where ATW is the average temperature weighting value, W i W represents the weight value of pixel i within the target detection bounding box. j This represents the weight value of pixel j outside the target detection box, where N is the normalization factor.

[0084] Specifically, to objectively quantify the heatmap, this embodiment proposes a new quantification metric: Average Temperature Weight (ATW). ATW quantifies the model's attention to different color regions by calculating the weight distribution of each region in the image. First, the image is converted to the LAB color space, and then the color difference between each pixel in the image and the reference colors (red, yellow, green, and blue) is calculated using CIEDE2000 color difference. A dynamic interpolation algorithm is used to assign weights to each pixel based on different color difference values. During the quantization process, this embodiment obtains the overall quantization result of the heatmap by weighted averaging of pixels in all non-zero weight regions. This quantization process eliminates interference from the background and low-temperature regions, ensuring that the ATW metric is more referential and consistent, providing an effective standard for evaluating subsequent experimental results.

[0085] The advantages of this method compared to existing technologies are analyzed below:

[0086] This method is compared with the calculation method of "average temperature weighting value" mentioned in the existing technical solution 1: "Real-time detection method and experimental study of Chinese cabbage seedlings in the field based on YOLOv8-CGB".

[0087] 1) Calculation method for reference color weight:

[0088] Option 1: Directly assign fixed weight values ​​to the reference colors (e.g., red is 1.0, yellow is 0.6-0.8, green is 0.4-0.6, and blue is 0). While this method is simple, it lacks dynamic adaptability to the weight distribution in real-world scenarios.

[0089] This method calculates the CIEDE2000 color difference between a pixel color and a reference color, and replaces the fixed weights with a dynamic allocation based on the color difference, making the weight distribution more reflective of the actual contribution of different pixels to the model's attention.

[0090] Results: The weighting method based on CIEDE2000 color difference can dynamically adapt to the color distribution of different images, avoiding the limitations of fixed weight methods in complex scenes.

[0091] 2) Scope of weight calculation:

[0092] Option 1: The weight allocation range covers the entire image, including the background area.

[0093] This method eliminates the absolutely low temperature region (pixels with a weight of 0), and the background and irrelevant regions are no longer included in the calculation, thus significantly improving the accuracy and consistency of the calculation results.

[0094] Results: By eliminating interference from the background and the absolute low temperature region, the improved method focuses on the target region and the high temperature region, resulting in more accurate quantification results with higher reference value.

[0095] 3) Introduction of positive and negative weights:

[0096] Option 1: It does not distinguish between positive and negative pixel weights, and all weighted pixels are considered to make positive contributions.

[0097] This method distinguishes the weight properties of pixels inside and outside the target detection box: weights inside the detection box are considered positive values, indicating that the model effectively focuses on the target region. Weights outside the detection box are considered negative values, reflecting the model's misjudgment of non-target regions.

[0098] Results: By introducing a mechanism of positive and negative weights, the method can not only evaluate the effectiveness of object detection, but also quantify the degree of misjudgment by the model in non-target regions, providing a clear direction for further model optimization.

[0099] 4) Improvement of the ATW calculation formula:

[0100] Option 1: Accumulate the weights of all pixels and divide by the total number of pixels in the image to obtain the average temperature weight.

[0101] This method comprehensively evaluates the positive and negative contributions of all pixels in the image to derive the average temperature weight.

[0102] Results: The improved ATW calculation method is based on a scientific color difference model and dynamic interpolation method, combined with positive and negative weights and effective area analysis, making the quantification results more interpretable and more in line with human perception of heat maps.

[0103] The core advantage of this method lies in its ability to not only quantify the attention given to target regions but also to clearly define the difference in attention the model gives to target and non-target regions by distinguishing between positive and negative weights in the heatmap. While Grad-CAM's heatmap can show the attention given to target regions, high temperatures in the heatmap may not represent correct attention for some misclassified or background areas (e.g., background misclassification). This method, by distinguishing between positive and negative weights, can clearly show which regions are indeed effectively detected by the model and which regions are merely background or misclassified interference. This is particularly important, especially when dealing with complex backgrounds or multi-object detection tasks, where directly extracting weights from the heatmap may not clearly reflect the impact of misclassified regions.

[0104] This method not only provides overall quantitative results for heatmaps but also enables fine-grained evaluation of target and non-target regions. This is particularly important for model interpretability, especially in precision applications such as agriculture, allowing for a deeper understanding of the model's performance in specific areas. In practical applications, especially in agricultural monitoring, understanding why the model generates high attention in a particular area is crucial. If ATW results show that some high-temperature areas are actually misclassified, this provides clear direction for model tuning. For example, ATW can identify which areas are not correctly identified as targets but are actually background noise or non-target regions, which is significant for improving training data and optimizing the model.

[0105] In multi-object detection, uneven distribution and size of targets can lead to the neglect or misclassification of small or edge targets. Grad-CAM generates heatmaps that display varying color intensities within target regions; however, relying solely on color intensity in the heatmap may be insufficient to assess the differences between multiple targets. This method, through the classification of positive and negative weights and precise weighting of effective regions in the heatmap, provides a clearer quantitative evaluation for multi-object detection tasks. It offers better differentiation between target and non-target regions, preventing misclassified regions from carrying excessive weight in the overall evaluation.

[0106] This method uses ATW as a quantitative evaluation metric. The results are not only used to assess model interpretability but also serve as a feedback mechanism to drive model improvement. In practical applications, it may be necessary to fine-tune the model based on ATW results, especially when adjusting the network's regions of interest or eliminating certain background regions. By continuously monitoring ATW values, particularly during training, if the ATW values ​​of certain target regions are found to be significantly lower than those of other regions, it may indicate that the model is not paying sufficient attention to these target regions, thus providing a basis for subsequent training and optimization. ATW can serve as an optimization target, promoting more precise model attention to target regions and thereby improving detection accuracy.

[0107] While Grad-CAM itself provides heatmap generation and color intensity mapping, ATW's computation remains relatively simple and effectively reduces background interference. ATW not only complements existing heatmaps but also offers flexibility for adaptation to various real-world scenarios. ATW's computation involves a weighted average of pixel weights in the heatmap; its formula and implementation are relatively simple, allowing for the rapid generation of meaningful quantitative metrics. Especially during multiple experiments and scene changes, ATW can quickly evaluate and compare model performance across different datasets or tasks.

[0108] This method not only enhances the quantitative evaluation of target and non-target regions but also improves interpretability by introducing positive and negative weights, providing clear feedback for subsequent model optimization. Especially when dealing with complex backgrounds and multi-target detection tasks, this method offers more refined analysis, avoiding the potential misleading results of simply relying on heatmaps.

[0109] To further optimize the technical solution, this embodiment also provides a heatmap evaluation system based on class activation mapping and dynamic weights, including:

[0110] The detection result acquisition module is used to acquire crop images, input the crop images into a pre-trained deep learning model, and output crop detection results.

[0111] The heatmap processing module is used to process the crop detection results into a heatmap based on class activation mapping, and to perform noise reduction processing on the heatmap to obtain a heatmap without background interference.

[0112] The color difference calculation module is used to perform color space conversion on the heatmap without background interference and calculate the color difference between each pixel and the reference color, wherein the reference color includes a preset reference color based on the heatmap and the corresponding weight.

[0113] The weight allocation module is used to assign a weight value to each pixel based on the color difference between each pixel and the reference color.

[0114] The quantitative evaluation module is used to calculate the average temperature weight value based on the weight value of each pixel, and to complete the quantitative evaluation of the heat map based on the average temperature weight value.

[0115] To further optimize the technical solution, this embodiment also provides a processor for running a program, wherein the program executes the steps of the heatmap evaluation method based on class activation mapping and dynamic weights during runtime.

[0116] To further optimize the technical solution, this embodiment also provides an electronic device, including one or more memories and a processor. The memories are used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the steps of the heatmap evaluation method based on class activation mapping and dynamic weights.

[0117] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made by those skilled in the art to the technical solutions of the present invention without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A heatmap evaluation method based on class activation mapping and dynamic weights, characterized in that, include: Acquire crop images, input the crop images into a pre-trained deep learning model, and output crop detection results; The crop detection results are processed into a heat map based on class activation mapping, and the heat map is then denoised to obtain a heat map without background interference. The heatmap without background interference is subjected to color space conversion, and the color difference between each pixel and the reference color is calculated, wherein the reference color includes a reference color preset according to the heatmap and the corresponding weight. A weight value is assigned to each pixel based on the color difference between each pixel and the reference color, and an average temperature weight value is calculated. A quantitative evaluation of the heat map is then completed based on the average temperature weight value.

2. The heatmap evaluation method based on class activation mapping and dynamic weights according to claim 1, characterized in that, The denoising process for the heatmap includes setting the pixel values ​​of the background portion of the heatmap to zero.

3. The heatmap evaluation method based on class activation mapping and dynamic weights according to claim 1, characterized in that, The process of performing color space conversion on the heatmap without background interference and calculating the color difference between each pixel and the reference color includes: The heatmap without background interference is converted from the RGB color space to the LAB color space. The color difference between each pixel in the heatmap without background interference and the reference color is calculated in the LAB color space. The method for calculating the color difference between each pixel and the reference color is as follows: Where, ΔE 00 The color difference between the compared pixel and the reference color is represented by ΔL, ΔC, and ΔH, which represent the differences in brightness, chromaticity, and hue, respectively. L S C S H It is the standardization factor, k L k C k H It is the weighting coefficient, R T It is a hue correction factor.

4. The heatmap evaluation method based on class activation mapping and dynamic weights according to claim 1, characterized in that, Each pixel is assigned a weight value based on the color difference between it and the reference color, including: Based on the color difference between each pixel and the reference color, a dynamic interpolation mechanism is used to assign a weight value to each pixel in the heatmap without background interference, and the pixel area is set as a positive weight value and the background area as a negative weight value.

5. The heatmap evaluation method based on class activation mapping and dynamic weights according to claim 4, characterized in that, The dynamic interpolation mechanism is as follows: W=interp(ΔE 00 ,[ΔE min ,ΔE max ],[w min ,w max ]); Where W represents the pixel weight value, ΔE 00 ΔE represents the color difference between the compared pixel and the reference color. min and ΔE max These are the maximum and minimum values ​​of the color difference, w min and w max These are the maximum and minimum values ​​of the weight, respectively.

6. The heatmap evaluation method based on class activation mapping and dynamic weights according to claim 1, characterized in that, The average temperature weighting value is calculated as follows: Where ATW is the average temperature weighting value, W i W represents the weight value of pixel i within the target detection bounding box. j This represents the weight value of pixel j outside the target detection box, where N is the normalization factor.

7. A heatmap evaluation system based on class activation mapping and dynamic weights, characterized in that, include: The detection result acquisition module is used to acquire crop images, input the crop images into a pre-trained deep learning model, and output crop detection results. The heatmap processing module is used to process the crop detection results into a heatmap based on class activation mapping, and to perform noise reduction processing on the heatmap to obtain a heatmap without background interference. The color difference calculation module is used to perform color space conversion on the heatmap without background interference and calculate the color difference between each pixel and the reference color, wherein the reference color includes a preset reference color based on the heatmap and the corresponding weight. The weight allocation module is used to assign a weight value to each pixel based on the color difference between each pixel and the reference color. The quantitative evaluation module is used to calculate the average temperature weight value based on the weight value of each pixel, and to complete the quantitative evaluation of the heat map based on the average temperature weight value.

8. A processor, characterized in that, The processor is used to run a program, wherein the program executes the steps of the heatmap evaluation method based on class activation mapping and dynamic weights as described in any one of claims 1 to 6.

9. An electronic device, characterized in that, The device includes one or more memories and a processor, the memories being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the steps of the heatmap evaluation method based on class activation mapping and dynamic weights as described in any one of claims 1 to 6.

Images