CNN Activation Masking for Pixel-Level Input-Output Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The difficulty in interpreting parameters and understanding the operation of deep learning systems, particularly convolutional neural networks, limits their optimization and improvement.
Innovation Solution
A method is introduced to convert convolutional neural networks into linear systems by recording and fixing the activation state of the ReLU function, allowing for analysis of impulse response to visualize the effect of input pixels on output pixels and vice versa, using an activation recorder layer and masking layer to modify the network.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If standard fully supervised convolutional models are used, then image recognition performance is improved, but interpretability of network parameters and operations deteriorates
Solution Approach 1:
The patent introduces an activation recorder layer as an intermediary component between the convolutional layers and the output. This recorder captures and stores the activation states of neurons during network operation, enabling external analysis tools to visualize and interpret which input pixels influence which output pixels without modifying the core recognition functionality. The intermediary layer thus bridges the gap between high-performance black-box models and interpretable analysis.
2Difficulty of detecting and measuring
If convolutional neural networks are modified to enable analysis, then interpretability is improved, but network complexity increases
Solution Approach 1:
The patent segments the convolutional neural network by inserting a distinct activation recorder layer that separates the forward propagation computation from the analysis functionality. This segmentation allows the core network to remain focused on recognition tasks while the recorder layer handles the additional complexity of tracking and storing activation states for interpretability analysis.
Data Source
Figure 1~2
Figure 3
Figure 4~6
AI summary
Provided are a convolutional neural network and a processing method, apparatus and system therefor, and a medium. The method comprises: using an activation recorder layer as an activation function layer in a convolutional neural network, wherein in response to inputting a probe image with contents to the convolutional neural network, the activation recorder layer performs the same activation operation as the activation function layer and records an activation result of the activation operation; modifying the convolutional neural network, with the modification step comprising replacing the activation recorder layer with a masking layer, wherein the masking layer uses the recorded activation result; and inputting, to the modified convolutional neural network, an analysis image as an input image to output an output image of the modified convolutional neural network, so as to analyze a positive influence or a negative influence between the input image and the output image, wherein the analysis image is a pixel-level binary image.