Reconstruction task interpretability analysis method based on encoder-decoder architecture
By introducing the EDRAM method into the encoder-decoder architecture, the problem of insufficient interpretability analysis in the reconstruction task is solved, multi-level and multi-scale interpretability analysis is realized, and the interpretability of the model in the image reconstruction task is improved.
Patent Information
- Application Number
- CN202411662569.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-20
- Publication Date
- 2025-02-25
AI Technical Summary
The interpretability analysis method for reconstruction tasks in the prior art is insufficient, making it difficult to provide comprehensive, multi-scale analysis, especially in the case of multi-category output.
A method of interpretability analysis of reconstruction tasks based on encoder-decoder architecture is proposed, called Encoder-Decoder Reconstruction-oriented Feature Activation Mapping (EDRAM). This method captures feature maps and gradient information and calculates EDRAM values by registering forward and reverse hook functions at the target layer of the model, thereby realizing multi-level and multi-scale interpretability analysis.
It improves the interpretability of deep learning models in image reconstruction tasks, enables researchers to better understand and analyze the decision-making process of the model, and is suitable for multi-category reconstruction tasks, providing a comprehensive and intuitive explanation, providing an important basis for model optimization and performance improvement.
Smart Images
Figure CN119514601A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence and machine learning, and in particular to an interpretability analysis method for deep learning models. Specifically, the present invention proposes an interpretability analysis method for reconstruction tasks based on an encoder-decoder architecture, called Encoder-Decoder Reconstruction-oriented Feature Activation Mapping (EDRAM). Background Art
[0002] With the rapid development of deep learning technology, models based on encoder-decoder architecture have achieved remarkable results in tasks such as image reconstruction, image segmentation, and semantic segmentation. Such models usually consist of an encoder network and a decoder network. The encoder is responsible for extracting the features of the input data, while the decoder is responsible for reconstructing or converting these features into target outputs. Among them, models such as UNet have performed well in fields such as medical image segmentation, demonstrating the powerful potential of the encoder-decoder architecture.
[0003] However, despite the excellent performance of these models, their working principles are often regarded as "black boxes" and lack interpretability. This opacity limits our understanding of the model's decision-making process and makes it difficult to debug, optimize, and assess the credibility of the model. This limitation is particularly prominent in some application scenarios that require high interpretability, such as medical diagnosis and autonomous driving.
[0004] In recent years, interpretability methods for classification tasks, such as Class Activation Mapping (CAM) and its variants, have made some progress. These methods can generate heat maps that highlight the image regions that the model focuses on when making classification decisions. However, these methods are mainly designed for classification tasks and are difficult to directly apply to reconstruction tasks.
[0005] For reconstruction tasks, existing interpretability methods are relatively scarce. The particularity of reconstruction tasks is that the output of the model is usually data with the same or similar dimensions as the input, rather than a simple category label. This complex mapping relationship between output and input makes it difficult for traditional interpretability methods to apply. Especially in the case of multi-category output, such as reconstructing multiple organelle structures at the same time, traditional methods find it difficult to distinguish the model's focus on different categories.
[0006] In addition, existing interpretability methods often only focus on one or a few layers of the network, and it is difficult to provide comprehensive, multi-scale analysis. For the encoder-decoder architecture, features at different levels have different semantic information and abstraction levels. It is difficult to fully understand the decision-making process of the model by analyzing only a single layer. Summary of the invention
[0007] The purpose of this invention is to provide a reconstruction task interpretability analysis method based on an encoder-decoder architecture, called Encoder-Decoder Reconstruction-oriented Feature Activation Mapping (EDRAM). This method aims to solve the problem of insufficient reconstruction task interpretability analysis methods in the prior art and improve the interpretability of deep learning models in image reconstruction tasks.
[0008] The technical solution of the present invention is as follows: A reconstruction task interpretability analysis method based on an encoder-decoder architecture includes the following steps: Step 1: Load the pre-trained encoder-decoder network model for reconstruction tasks: Load the trained encoder-decoder network model for image reconstruction tasks, such as UNet and other encoder-decoder network models.
[0009] Step 2. Build the EDRAM class: Create an EDRAM class that inherits from nn.Module. This class contains the target model and the target layer list, and implements the interpretable analysis method.
[0010] Step 3: Preprocess and standardize the input image: Preprocess the input image, including converting the image into an appropriate data type and format, and standardizing it.
[0011] Step 4: Input the preprocessed image into the encoder-decoder network model for forward propagation: Use the preprocessed image as input, perform forward propagation through the encoder-decoder model, and obtain the output result of the model.
[0012] Step 5: Calculate the gradient corresponding to the reconstructed output of the model and calculate EDRAM for each layer of the network: backpropagate the output of the model, calculate the gradient, and use these gradients and feature maps to calculate the EDRAM of each layer.
[0013] Step 6: Fuse the EDRAM of each layer to obtain the overall EDRAM: perform weighted averaging or other fusion methods on the calculated EDRAM results of each layer to obtain the overall EDRAM representation.
[0014] Step 7: Visualize the EDRAM of each layer and the overall EDRAM, the original image, and the reconstruction result: Use a visualization method to display the EDRAM of each layer, the overall EDRAM, the original input image, and the reconstruction result for intuitive analysis and comparison.
[0015] Furthermore, the feature map and gradient of the target layer are extracted as follows: By registering the forward hook function save_activation at the target layer of the model, the feature map during the forward propagation process is captured and stored. , feature map The dimension is [B, C, H, W], where B is the batch size, C is the number of channels, and H and W are the height and width of the feature map respectively; By registering the reverse hook function save_gradient in the target layer of the model, the gradient during the back-propagation process is captured and stored. .gradient Dimensions and feature maps of Same, [B, C, H, W];
[0016] Calculate the global average pooling of the gradient to get the weight, and Perform global average pooling in the spatial dimension to obtain the weight ; Among them, the weight The dimension of is [B, C, 1, 1]; Weight With feature map Perform element-by-element multiplication in the channel dimension and sum it in the channel dimension to obtain the EDRAM result; apply the ReLU activation function to the EDRAM result, retain the positive values and set the negative values to zero, and then perform bilinear interpolation and normalization to ensure that the value range is between [0, 1].
[0017] Weight The calculation formula is: in, is the weight of the kth feature map; Z is the spatial dimension of the feature map, usually width × height; i, j are the spatial coordinate indexes of the feature map; Output Y for the kth feature map The partial derivative of is the gradient value at position (i, j).
[0018] Furthermore, the calculation formula of EDRAM result is: in, is the EDRAM result of the kth layer; ReLU is the activation function used to retain positive values and set negative values to zero; c is the channel index of the feature map; is the weight of the cth channel of the kth layer; is the feature map of the cth channel of the kth layer; The calculation formula for bilinear interpolation is: in, is the bilinear interpolation function, The target size is specified; The normalized calculation formula is: Among them, min( )for The minimum value in )for The maximum value in ; is a constant.
[0019] Furthermore, the fusion formula in step six is: in, is the overall class activation mapping result after fusion; N is the number of EDRAM layers involved in the fusion; is the EDRAM result of the i-th layer.
[0020] Furthermore, the superposition formula in step seven is: in, is the superimposed image; I is the original image or the reconstruction result; is the normalized EDRAM; α is the superposition weight, the larger α is, the higher the transparency of the original image is.
[0021] Another object of the present invention is to provide a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the above method.
[0022] Another object of the present invention is to provide a computing device, comprising a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the above method is implemented.
[0023] Compared with the prior art, the present invention has the following beneficial effects: The method of the present invention can improve the interpretability of deep learning models in image reconstruction tasks, enable researchers and developers to better understand and analyze the decision-making process of the model, reveal the focus of the model in the reconstruction process, explain how the model uses input information to generate output, and adapt to the feature analysis requirements of different levels and scales. At the same time, the method of the present invention is also applicable to multi-category reconstruction tasks, can handle multi-category output situations, and provide independent interpretability analysis for each category and network level. Through the fusion and visualization of multi-level EDRAM results, the EDRAM method realizes a comprehensive and intuitive explanation of the decision-making process of the encoder-decoder model, providing an important basis for model optimization and performance improvement. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] Figure 1 The structure diagram of the Encoder-Decoder Reconstruction-oriented FeatureActivation Mapping (EDRAM) of the present invention; Figure 2 It is a schematic diagram of EDRAM in the codec model; Figure 3 Reconstructing the image to be separated for testing; Figure 4 Interpretable analyzed images for mitochondria.
[0025] Figure 5 The images after analysis can be interpreted as microtubules. DETAILED DESCRIPTION
[0026] The present invention will be further described below in conjunction with the accompanying drawings: like Figure 1 As shown in the figure, the overall structure of EDRAM includes input layer, encoder network, decoder network, output layer and interpretability analysis module (UNetReconstructionEDRAM, Visualization Process). The input layer receives the original image, the encoder network is responsible for extracting image features, the decoder network reconstructs the extracted features into the target output, and the interpretability analysis module is responsible for generating a heat map to explain the decision process of the model.
[0027] pass Figure 2, we can more clearly understand the workflow of EDRAM in the encoder-decoder model. First, the original image is preprocessed and input into the encoder-decoder network. During the forward propagation process, EDRAM captures and stores the feature maps of each layer of the network through pre-registered hook functions. After the model completes the forward propagation, the output results are back-propagated to calculate the gradient corresponding to each layer of the feature map. EDRAM uses this gradient information to generate preliminary class activation maps through weighted sum operations. Subsequently, these maps are ReLU activated, bilinear interpolated, and normalized to obtain the final EDRAM heat map (red box).
[0028] In this embodiment, the interpretability of the model in the image reconstruction task is analyzed by taking the organelle structure (including mitochondria and microtubules) as an example, so as to further illustrate the method of the present invention. Taking the biological science field as an example, the trained multi-structure encoder-decoder network model is integrated into the microscope system. Through a single staining process, a mixed superposition image of multiple structures is obtained, which is then input into the deployed deep learning model for reasoning. The EDRAM method will save the feature map and gradient information, and visualize each step of the deep learning model while generating the classification reconstruction results. In this way, researchers can obtain interpretable analysis diagrams of mitochondria and microtubules, thereby enhancing confidence and trust in the deep learning model. In addition, the present invention can also be widely used in other scenarios that require classification reconstruction. It only needs to provide model weights based on encoders and decoders to achieve interpretable analysis of the results.
[0029] A reconstruction task interpretability analysis method based on an encoder-decoder architecture includes the following steps: Step 1: Load the pre-trained encoder-decoder network model for reconstruction tasks: Load the trained encoder-decoder network model for image reconstruction tasks, such as UNet and other encoder-decoder network models.
[0030] Step 2. Build the EDRAM class: Create an EDRAM class that inherits from nn.Module. This class contains the target model and the target layer list, and implements the interpretable analysis method.
[0031] Step 3: Preprocess and standardize the input image: Preprocess the input image, including converting the image into an appropriate data type and format, and standardizing it.
[0032] Step 4: Input the preprocessed image into the encoder-decoder network model for forward propagation: Use the preprocessed image as input, perform forward propagation through the encoder-decoder model, and obtain the output result of the model.
[0033] Step 5: Calculate the gradient corresponding to the reconstructed output of the model and calculate the EDRAM for each layer of the network: back-propagate the output of the model, calculate the gradient, and use these gradients and feature maps to calculate the EDRAM of each layer; By registering the forward hook function save_activation at the target layer of the model, the feature map during the forward propagation process is captured and stored. , feature map The dimension is [B, C, H, W], where B is the batch size, C is the number of channels, and H and W are the height and width of the feature map respectively; By registering the reverse hook function save_gradient in the target layer of the model, the gradient during the back-propagation process is captured and stored. .gradient Dimensions and feature maps of Same, [B, C, H, W];
[0034] Calculate the global average pooling of the gradient to get the weight, and Perform global average pooling in the spatial dimension to obtain the weight ; Among them, the weight The dimension of is [B, C, 1, 1]; Weight With feature map Perform element-by-element multiplication in the channel dimension and sum it in the channel dimension to obtain the EDRAM result; apply the ReLU activation function to the EDRAM result, retain the positive values and set the negative values to zero, and then perform bilinear interpolation and normalization to ensure that the value range is between [0, 1].
[0035] Weight The calculation formula is: in, is the weight of the kth feature map; Z is the spatial dimension of the feature map, usually width × height; i, j are the spatial coordinate indexes of the feature map; Output Y for the kth feature map The partial derivative of is the gradient value at position (i, j).
[0036] The calculation formula for EDRAM result is: in, is the EDRAM result of the kth layer; ReLU is the activation function used to retain positive values and set negative values to zero; c is the channel index of the feature map; is the weight of the cth channel of the kth layer; is the feature map of the cth channel of the kth layer; The calculation formula for bilinear interpolation is: in, is the bilinear interpolation function, The target size is specified; The normalized calculation formula is: Among them, min( )for The minimum value in )for The maximum value in ; is a constant; in this embodiment, The value of is 1e-8.
[0037] Step 6: Fusing the EDRAM of each layer to obtain the overall EDRAM: performing weighted averaging or other fusion methods on the calculated EDRAM results of each layer to obtain the overall EDRAM representation; The fusion formula is: in, is the overall class activation mapping result after fusion; N is the number of EDRAM layers involved in the fusion; is the EDRAM result of the i-th layer.
[0038] Step 7: Visualize the EDRAM of each layer and the overall EDRAM, the original image, and the reconstruction result: Use a visualization method to display the EDRAM of each layer, the overall EDRAM, the original input image, and the reconstruction result for intuitive analysis and comparison; The superposition formula is: in, is the superimposed image; I is the original image or the reconstruction result; is the normalized EDRAM; α is the superposition weight, the larger α is, the higher the transparency of the original image is. In this embodiment, the value of α is 0.6.
[0039] To verify the effectiveness of the EDRAM method, we selected Figure 3The image shown in Figure 1 is used for testing. This image contains multiple organelle structures that need to be reconstructed simultaneously, such as mitochondria and microtubules. This complex multi-category reconstruction task provides an ideal test scenario for the EDRAM method because it requires the model to accurately identify and reconstruct different cell structures, and we need to understand the basis for the model's decision-making in this process.
[0040] Figure 4 , Figure 5 The results of EDRAM interpretability analysis are shown. In these two figures, we can clearly see the image regions that the model focuses on when reconstructing different organelle structures. The highlighted parts in the heat map represent the areas that the model pays close attention to when making reconstruction decisions. By comparing the heat maps of different categories (such as mitochondria and microtubules), we can observe the differences in the model's focus when dealing with different structures. These visualizations not only verify the correctness of the model, but also provide an opportunity to gain a deeper understanding of the model's decision-making process.
[0041] The implementation process of the EDRAM method can be summarized in the following key steps: First, we load the pre-trained encoder-decoder model and register forward and backward hook functions for the specified target layer. These hook functions are responsible for capturing the necessary feature map and gradient information during the model operation. Next, we normalize the input image to ensure that it is suitable for the model input. Then, we perform the forward propagation of the model to obtain the reconstructed output. After obtaining the output, we perform backpropagation to calculate the gradient of each target layer. Using these gradients and the previously saved feature maps, we calculate the EDRAM value of each layer. This process includes global average pooling of the gradients to obtain weights, multiplying the weights with the feature maps, applying the ReLU activation function, performing bilinear interpolation to adjust the size, and finally normalizing.
[0042] Specifically, the EDRAM class contains a list of target models and target layers, and uses PyTorch's hook mechanism to register the save operations of feature maps and gradients. The class implements the save_activation and save_gradient methods, which are used to save the feature maps of forward propagation and the gradients of backpropagation, respectively. The EDRAM class contains the normalize_input method to normalize the input image. The forward method performs the forward propagation of the model and returns the model output result of the normalized input. The get_reconstruction_edram method is used to calculate the class activation map of the reconstruction task and supports multi-category output processing. The __call__ method integrates the forward propagation and class activation map calculation steps, and returns the activation maps of multiple categories and their reconstruction results.
[0043] For the case of multi-category output, the EDRAM method calculates the EDRAM value for each output category separately. This allows us to analyze the differences in the model's behavior when dealing with different categories. By comparing the EDRAM results of different categories, we can gain insight into how the model distinguishes and reconstructs different organelle structures. In addition, the EDRAM method also supports the fusion of multiple layers of results. Through weighted averaging or other fusion methods, we can obtain an overall EDRAM representation that fully reflects the model's decision-making process.
[0044] Specifically, a refined EDRAM calculation method is used for the multi-category output in the reconstruction task. First, the model output results are carefully separated by category to obtain independent reconstruction results. Then, for each category of output, the back propagation process is performed independently to accurately calculate the corresponding gradient. When calculating the EDRAM value, the get_reconstruction_edram function is used. This part accurately calculates the EDRAM value for each target layer and each category based on the previously stored feature map and gradient information. The output of this process is a dictionary containing multiple categories of EDRAM results, which lists the EDRAM value of each target layer and the corresponding reconstruction output tensor for each category. In the visualization stage, the method displays the EDRAM results of different categories and different network levels, as well as their corresponding reconstructed images. The calculation of EDRAM is closely integrated with the reconstruction output, and the method achieves end-to-end interpretable analysis from input to output. By visualizing the EDRAM results of different categories and different levels, the method achieves multi-scale and multi-target interpretability.
[0045] The method of the present invention supports EDRAM analysis of multiple levels of the encoder-decoder network. The target_layers parameter is used to specify the network layer to be analyzed, and EDRAM is calculated separately for each specified target layer. The compute_overall_edram method is implemented to fuse the EDRAM results of multiple levels. The method displays the EDRAM results of each level and the overall EDRAM results after fusion. By analyzing the EDRAM of different levels, the interpretability information of the network at different abstraction levels is provided. In order to display the EDRAM results more intuitively, the method implements the function of superimposing EDRAM on the original image or the reconstruction result.
[0046] An important feature of the EDRAM method is its visualization capability. Figure 4As shown in the figure, we can overlay the EDRAM heat map onto the original image or the reconstruction result to intuitively show the model's focus area. This visualization not only helps researchers understand and verify the behavior of the model, but also provides an important basis for model debugging and optimization. By analyzing these visualization results, researchers can find possible problems with the model, such as over-focusing on certain specific areas or ignoring important image features.
[0047] In the drawings of the embodiments disclosed in the present invention, only the structures involved in the embodiments disclosed in the present invention are involved, but the above is only a preferred embodiment of the present invention, which can be fully applied to various fields suitable for the present invention. Therefore, without departing from the general concept defined by the claims and the equivalent scope, the present invention is not limited to the specific details and the illustrations shown and described here.
Claims
1. A reconstruction task interpretability analysis method based on an encoder-decoder architecture, characterized in that: The method comprises the following steps: Step 1: Load a pre-trained encoder-decoder network model for the reconstruction task; the encoder-decoder network model includes an input layer, an encoder network, a decoder network, an output layer, and an interpretability analysis module; Step 2: Build the Encoder-Decoder Reconstruction-oriented Feature Activation Mapping (EDRAM) class containing the target model and the target layer list to implement feature activation mapping; Step 3: Preprocess and normalize the input image in the EDRAM class; Step 4: Input the preprocessed and standardized image into the encoder-decoder network model, perform forward propagation, and obtain the output result of the network model; Step 5: Back-propagate the output results of the network model, calculate the gradient, and use these gradients and feature maps to calculate the EDRAM results of each layer of the network model; Step 6: Fuse the EDRAM results of each layer to obtain the overall EDRAM result; Step 7: Overlay the EDRAM results of each layer and the overall EDRAM results onto the original image and the reconstructed results for visualization.
2. The reconstruction task interpretability analysis method based on the encoder-decoder architecture according to claim 1 is characterized in that: The EDRAM class contains the target model and target layer list, and uses PyTorch's hook mechanism to register the save operations of feature maps and gradients, saving the feature maps of forward propagation and the gradients of back propagation; the EDRAM class contains the normalize_input method to normalize the input image; the forward method is used to execute the forward propagation of the model and return the model output result of the standardized input; the get_reconstruction_cam method is used to calculate the class activation map of the reconstruction task and support multi-category output processing; the __call__ method is used to integrate the forward propagation and class activation map calculation steps, and return the activation maps of multiple categories and their reconstruction results.
3. The reconstruction task interpretability analysis method based on the encoder-decoder architecture according to claim 2 is characterized in that: The feature maps and gradients of the target layer are extracted as follows: By registering the forward hook function save_activation at the target layer of the model, the feature map during the forward propagation process is captured and stored. , feature map The dimension is [B, C, H, W], where B is the batch size, C is the number of channels, and H and W are the height and width of the feature map respectively; By registering the reverse hook function save_gradient in the target layer of the model, the gradient during the back-propagation process is captured and stored. ;gradient Dimensions and feature maps Same, [B, C, H, W]; Calculate the global average pooling of the gradient to get the weight, and Perform global average pooling in the spatial dimension to obtain the weight ; Among them, the weight The dimension of is [B, C, 1, 1]; Weight With feature map Perform element-by-element multiplication in the channel dimension and sum it in the channel dimension to obtain the EDRAM result; apply the ReLU activation function to the EDRAM result, retain the positive values and set the negative values to zero, and then perform bilinear interpolation and normalization to ensure that the value range is between [0, 1].
4. The reconstruction task interpretability analysis method based on the encoder-decoder architecture according to claim 3 is characterized in that: Weight The calculation formula is: in, is the weight of the kth feature map; Z is the spatial dimension of the feature map, usually width × height; i, j are the spatial coordinate indexes of the feature map; Output Y for the kth feature map The partial derivative of is the gradient value at position (i, j).
5. The reconstruction task interpretability analysis method based on the encoder-decoder architecture according to claim 3 is characterized in that: The calculation formula for EDRAM result is: in, is the EDRAM result of the kth layer; ReLU is the activation function used to retain positive values and set negative values to zero; c is the channel index of the feature map; is the weight of the cth channel of the kth layer; is the feature map of the cth channel of the kth layer; The calculation formula for bilinear interpolation is: in, is the bilinear interpolation function, The target size is specified; The normalized calculation formula is: Among them, min( )for The minimum value in )for The maximum value in ; is a constant.
6. The reconstruction task interpretability analysis method based on the encoder-decoder architecture according to claim 1, characterized in that: The fusion formula in step six is: in, is the overall class activation mapping result after fusion; N is the number of EDRAM layers involved in the fusion; is the EDRAM result of the i-th layer.
7. The reconstruction task interpretability analysis method based on the encoder-decoder architecture according to claim 1, characterized in that: The superposition formula in step seven is: in, is the superimposed image; I is the original image or the reconstruction result; is the normalized EDRAM; α is the superposition weight.
8. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the method according to any one of claims 1 to 7.
9. A computing device, comprising a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the method according to any one of claims 1 to 7 is implemented.