Explanatable method for decision basis of medical image classification model
By introducing channel and spatial attention mechanisms, SVM discriminative learning, and CARAFE upsampling method, the accuracy and reliability issues of existing medical image classification model prototype interpretation methods are solved. This enables effective evaluation of prototype concept vectors and detailed interpretation of lesion structures, thereby improving the model's interpretation accuracy and credibility.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SHENYANG AEROSPACE UNIVERSITY
- Filing Date
- 2026-01-13
- Publication Date
- 2026-05-01
AI Technical Summary
Existing prototype interpretation methods for medical image classification models suffer from insufficient accuracy in prototypal semantic focus, limited spatial visualization accuracy due to traditional upsampling methods, lack of quantitative assessment of prototype contributions, difficulty in identifying redundant or erroneous associations, and weak overall expressive ability for complex medical structures.
Channel attention and spatial attention mechanisms are introduced to generate attention-enhanced feature maps. Prototype concept vectors are obtained through SVM discriminative learning. A content-aware CARAFE upsampling method is adopted, and a counterfactual intervention mechanism is designed to evaluate the contribution of the prototype concept vectors. Combined with the U-Net network, lesion and structural information are preserved to improve the accuracy and reliability of interpretation.
It significantly improves the semantic discriminativeness and stability of prototype concepts, enhances the interpretation accuracy and credibility of medical image classification tasks, strengthens the reliability and spatial visibility of model interpretation results, and can objectively evaluate the contribution of prototype concept vectors to model prediction.
Smart Images
Figure CN121962733A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to interpretability methods for classification models, specifically to an interpretability method for the decision-making basis of a medical image classification model. Background Technology
[0002] In the field of medical image analysis, interpretability is particularly crucial. It not only helps researchers and clinicians validate diagnostic criteria and improve the credibility and clinical acceptance of medical image classification models, but also provides a clear basis for optimizing these models. The interpretability of neural networks refers to the ability to understand the model's prediction or decision-making mechanisms. Its core lies in analyzing the key features relied upon by the model in generating prediction results and the influence of these features on the final output. This is essential for analyzing the model's internal mechanisms, identifying potential biases, and optimizing performance.
[0003] Existing methods for interpreting neural networks primarily rely on visualization. These methods map internal model features or gradient information back into the input space, generating heatmaps with color coding to indicate the importance of pixels or regions, thus showcasing the areas of interest for the model. However, such methods only provide locational information and cannot reveal the specific semantic basis of the model's decisions. In fine-grained or complex medical imaging scenarios, they suffer from a lack of discriminative power and stability in interpretation. To overcome the shortcomings of visualization methods, researchers have proposed concept-based interpretation methods. By identifying activation patterns of human-understandable concepts (such as texture and shape) in high-level feature spaces, these methods quantify the contribution of concepts to prediction scores, achieving global and sample-level semantic interpretation and improving the intuitiveness and credibility of the interpretation. However, this method depends on concept definitions; incomplete or low-quality concept coverage can easily lead to misleading interpretations, and the accuracy of local interpretations is limited, with unstable correspondences between concepts and input features.
[0004] To address the aforementioned issues, a small number of researchers have begun exploring a more stable interpretation method to make the model's decision-making process more understandable and intuitive while maintaining classification performance. This method is known as prototype interpretation. The prototype concept is a semantic concept corresponding to the typical features of each category learned by the neural network from training data, which can correspond to local regions of the input image. Prototype interpretation methods derived from this concept achieve classification decisions by calculating the similarity between the features of the input sample and each prototype, providing a similarity-based explanation for the prediction results. XProtoNet is a widely used prototype interpretation method. It uses training constraints to ensure that prototypes reflect typical local features of the category and maintains visual consistency between the prototype concept and the local regions of the input image. During the inference phase, it completes classification and provides an explanation using similarity calculations.
[0005] However, existing prototype interpretation methods also have significant drawbacks: first, the semantic focus of the prototype is not accurate enough and is easily affected by background interference; second, traditional upsampling methods result in limited spatial visualization accuracy and blurred edges and structures; and third, there is a lack of quantitative evaluation of the prototype contribution, making it difficult to identify redundant or erroneous associations and resulting in a weak overall ability to express complex medical structures. Summary of the Invention
[0006] To address the problems existing in the prior art, this invention provides an interpretability method for the decision-making basis of medical image classification models. It aims to improve the semantic focus capability of prototype concepts, preserve the spatial details of lesion and structural information, and quantify the contribution of prototypes to medical image classification through a prototype effectiveness evaluation mechanism, thereby improving the interpretability and reliability of medical image classification tasks.
[0007] The technical solution of this invention is:
[0008] An interpretability method for the decision-making basis of a medical image classification model, comprising the following steps:
[0009] Step 1: Obtain the required number of raw CT images and select two-dimensional image slices containing lesion areas from each raw CT image. Preprocess each image slice to obtain a standardized dataset of two-dimensional image slices containing lesion areas. Divide this dataset into two parts according to a certain ratio. One part is used as the test set, and the other part is combined with two-dimensional image slices of uniform size that do not contain lesion areas in an equal ratio to form the training set.
[0010] Step 2: Input the images from the training and test sets into a ResNet-50 convolutional neural network (with ResNet-50 as the backbone) to obtain the low-level texture features and high-level semantic features of the input images, and derive the feature map output by the last convolutional layer of the model. ,in The number of feature map channels. The width of the feature map. The feature map height;
[0011] Step 3: [Regarding...] Applying channel attention and spatial attention mechanisms to generate attention-enhanced feature maps ;
[0012] Step 4: [Regarding...] Feature learning is performed on high-response regions in the dataset to obtain several prototype feature vectors corresponding to images in the test set (test images) and several prototype feature vectors corresponding to images in the training set (training images). Each prototype feature vector is... Semantic representations are used to characterize semantic patterns with discriminative significance in medical images;
[0013] Step 5: Flatten the prototype feature vectors of each training image, and input the flattened prototype feature vectors into SVM for discriminative learning to obtain the prototype concept vector corresponding to each category.
[0014] Step 6: Calculate the similarity between the prototype feature vector of the test image and the prototype concept vector corresponding to each category, determine the category corresponding to the maximum similarity as the predicted category of the test image, and regard the prototype concept vector corresponding to the category with the maximum similarity as the target prototype concept vector.
[0015] Step 7: Filter out the similarities calculated in Step 6 that are higher than the preset threshold, and use the target prototype concept vectors corresponding to these similarities as prototype concept vectors to be mapped.
[0016] Step 8: Map the prototype concept vector to be mapped to the test image;
[0017] Step 9: Design a counterfactual intervention mechanism. By suppressing the components of the corresponding class prototype concept vectors in the high-level semantic vectors of the test images, analyze their impact on the classification results, and thus evaluate the contribution of each prototype concept vector to the decision of the ResNet-50 model.
[0018] Furthermore, according to the interpretability method, the preprocessing of each image slice in step 1 includes window width and window level processing, size unification, pixel intensity normalization, and noise suppression processing.
[0019] Furthermore, according to the explained interpretability method, step 3 includes: firstly, for Channel attention enhancement is performed, global information is aggregated in the spatial dimension, and the importance weight of each channel is calculated according to equation (1) to obtain the feature map. Then The attention-enhanced feature map is generated by weighting each spatial location according to equation (2). ;
[0020] (1)
[0021] Wherein, GAP represents global average pooling; and The weight matrix is a learnable weight matrix; It is the ReLU activation function; Use the Sigmoid activation function; Indicates channel weighting operation;
[0022] (2)
[0023] in Indicates the kernel size as Convolution operations; This indicates a channel splicing operation; and These represent spatial average pooling and max pooling, respectively.
[0024] Furthermore, according to the explained interpretability method, step 4 regarding... The process of feature learning for high-response regions in the model is as follows: Each spatial location is considered as a local semantic candidate unit. Then, regions with activation values higher than a preset threshold are selected from each local semantic candidate unit as high-response regions. The corresponding local feature vectors are then extracted from the high-response regions and mapped to fixed-dimensional feature representations, i.e., prototype feature vectors, through feature aggregation and normalization.
[0025] Further, according to the interpretability method described above, the specific process of inputting the flattened prototype feature vector into the SVM for discriminative learning in step 5 is as follows: For each category, train a corresponding binary classification SVM to distinguish the prototype feature vectors of this category from those of other categories; construct a CAV corresponding to each prototype feature vector through the classification hyperplane normal vector of each category SVM to represent the semantic direction of this category in the conceptual space; calculate the activation value of each prototype feature vector on its category CAV according to equation (3); select the category with the highest activation value according to the magnitude of the activation value. Each prototype feature vector serves as the prototype concept vector for that category. ;
[0026] (3)
[0027] in This indicates the prototype feature vector within its class. The activation value on the CAV; Indicate category The classification hyperplane normal vector of the SVM; Indicate category The prototype feature vector.
[0028] Furthermore, according to the interpretability method described above, the method for mapping the prototype concept vector to be mapped to the test image in step 8 is as follows: [The text abruptly ends here, likely due to an incomplete sentence or a formatting error.] The similarity between the feature vectors of each spatial location and the prototype concept vector to be mapped is measured to obtain the concept activation feature map. ;exist Regions with a similarity to the prototype concept vector to be mapped exceeding a set threshold will be marked as regions to be mapped; Input the kernel prediction unit based on the content-aware CARAFE upsampling method, and through Convolution performs channel compression to obtain low-dimensional feature representations. ;Will Edge structure map extracted from test images using the U-Net network By fusing the features, we obtain a structurally enhanced feature representation. ;based on predict Adaptive recombining convolution kernels for each spatial location And on The weights are spatially normalized to highlight salient regions and suppress noise responses; based on right Weighted recombination operations are performed on the local neighborhood features at each location to generate a high-resolution concept activation feature map. ;Establish The mapping relationship between the image and the test image will The middle corresponds to The location of the region to be mapped is projected onto the coordinates of the test image according to the scale mapping rules. Determine the spatial region in the test image that corresponds to the target prototype concept vector to be mapped.
[0029] Furthermore, according to the explained interpretability method, step 9 includes: firstly, processing the attention enhancement feature map corresponding to the test image... High-level semantic vectors are obtained through global average pooling. Then calculate the target prototype concept vector corresponding to the category of the test image. In the high-level semantic vector of the test image Response strength Then, for each target prototype concept vector... A concept suppression operation is performed by weakening the corresponding semantic components in the high-level semantic vector along the direction of the target prototype concept vector, resulting in an intervened high-level semantic vector. ,in The suppression coefficient, The time represents the maximum magnitude suppression in the direction corresponding to the target prototype concept vector. This suppression operation does not assume that different prototype concept vectors are orthogonal to each other. The suppression operation is used to directionally weaken the contribution of the target prototype concept vector to the high-level semantic representation, achieving suppression of a single target prototype concept vector. The impact of the target prototype concept vector on the classification result is evaluated by applying the suppression of the target prototype concept vector; then... With the target prototype concept vector set for each category Perform matching, where This indicates the corresponding category, specifically: calculation. The cosine similarity with all target prototype concept vectors in the category is used, and the average value is taken as the matching score for the category. Finally, the classification result is selected based on the matching score, and the contribution of a single target prototype concept vector to the decision of the ResNet-50 model is quantitatively analyzed based on the change in classification accuracy before and after the intervention.
[0030] Compared with the prior art, the present invention has the following beneficial effects:
[0031] (1) The present invention introduces channel attention and spatial attention mechanisms in the prototype concept vector extraction stage, which can effectively guide the model to focus on key semantic regions related to the disease, reduce background interference, significantly improve the semantic discriminativeness and stability of the prototype concept vector, thereby enhancing the reliability of the interpretation results of the ResNet-50 model.
[0032] (2) The present invention adopts the content-aware CARAFE upsampling method to replace the traditional upsampling method. In the process of prototypal concept vector localization and deconvolution visualization, it effectively alleviates the problems of edge blurring and structural distortion, better preserves the important anatomical structure and lesion contour information in medical images, and significantly improves the spatial visibility and structural consistency of the interpreted heatmap.
[0033] (3) The present invention further proposes a prototype concept vector validity evaluation mechanism. Combined with counterfactual intervention strategy, it performs quantitative analysis on the semantic inhibition of prototype concept vectors in high-level semantic space. It can objectively evaluate the actual contribution of each prototype concept vector to the model prediction, reveal potential erroneous associations or redundant prototype concept vectors, thereby improving the precision and credibility of the ResNet-50 model decision explanation. Attached Figure Description
[0034] Figure 1 This is a flowchart illustrating the interpretability method for the decision-making basis of the medical image classification model in this embodiment.
[0035] Figure 2 The following are examples of misclassification of malignant nodule images in the test set of this embodiment. The visualization of the prototype concept vector in the training or test images is referred to as the prototype concept. Figure (a) is the test image, Figure (b) is the prototype concept in the training set images similar to the prototype concept in the test image, Figure (c) is the position of the prototype concept in the training set images, and Figure (d) is the activation heatmap of the prototype concept in the training set images.
[0036] Figure 3Figure 1 shows an example of the correct classification of benign nodule images in the test set of this embodiment. Figure 2 shows the test image, Figure 3 shows the prototype concept in the training set images that are similar to the prototype concept in the test image, Figure 4 shows the position of the prototype concept in the training set images, and Figure 5 shows the activation heatmap of the prototype concept in the training set images.
[0037] Figure 4 This is a schematic diagram of the prototype concept suppression experiment for this implementation method. Detailed Implementation
[0038] To facilitate understanding of this application, a more comprehensive description of this application will be provided below with reference to the accompanying drawings.
[0039] Figure 1 This is a flowchart illustrating the interpretability method of the medical image classification model decision-making basis in this embodiment. For example... Figure 1 As shown, the interpretability method for the decision-making basis of the medical image classification model includes the following steps:
[0040] Step 1: Obtain the required number of raw CT images, then select two-dimensional image slices containing lesion areas from each raw CT image according to the corresponding annotation file, and preprocess each image slice. Divide the standardized two-dimensional image slice dataset containing lesion areas into two parts according to a certain ratio. One part is used as the test set, and the other part is combined with two-dimensional image slices of uniform size that do not contain lesion areas in an equal ratio to form the training set.
[0041] This implementation first obtains the LIDC-IDRI public dataset from the internet, then extracts the required number of raw CT images from the LIDC-IDRI public dataset, and then selects two-dimensional image slices containing lesion regions from each raw CT image according to the corresponding annotation file. Then, each two-dimensional image slice containing lesion regions is preprocessed, including window width and window level processing, size uniformity, pixel intensity normalization, and noise suppression processing. Then, the preprocessed two-dimensional image slice dataset is divided into two parts according to a certain ratio. One part is used as the test set, and the other part is combined with the two-dimensional image slices with uniform size that do not contain lesion regions at a 1:1 ratio to form the training set.
[0042] In a preferred embodiment, when preprocessing each two-dimensional image slice containing the lesion area, the window width and window level are first adjusted according to the imaging characteristics of lung tissue, and the window level is set to... Window width set to Then, the size of the two-dimensional image slices was uniformly adjusted to... Pixels, and linearly map pixel intensity to The interval is then used; finally, Gaussian filtering is applied for noise suppression, where the standard deviation of the Gaussian filter is set to... This is to preserve lesion edge information while suppressing noise.
[0043] Step 2: Input the images from the training and test sets into a ResNet-50 convolutional neural network (convolutional neural network) with ResNet-50 as the backbone. The model extracts features from the input images through a multi-layer residual structure, acquiring low-level texture features and high-level semantic features layer by layer, and finally deriving the feature map output by the last convolutional layer. ,in The number of feature map channels. The width of the feature map. The feature map height;
[0044] This implementation introduces a deep residual feature extraction network with ResNet-50 as the backbone to effectively model multi-level semantic information of medical images, generating stable and highly discriminative high-dimensional feature map representations, thereby providing reliable feature support for prototype concept modeling and causal interpretability analysis.
[0045] Step 3: [Regarding...] By applying channel attention and spatial attention mechanisms, the contextual modeling ability and spatial local representation ability between feature channels are enhanced, generating attention-enhanced feature maps. ;
[0046] In this embodiment, firstly... Channel attention enhancement is performed, global information is aggregated in the spatial dimension, and the importance weight of each channel is calculated to obtain the feature map. This operation is used to enhance semantically relevant channel features of lesions and suppress irrelevant channel responses. Then... The above process generates a spatial weight map, which is the final attention-enhanced feature map, by weighting each spatial location. This operation is used to highlight the lesion area and suppress background interference, thereby enhancing the ResNet-50 network model's ability to focus on key lesion areas by combining channel and spatial attention.
[0047] In this step, the importance weight of each channel is calculated using equation (1):
[0048] (1)
[0049] GAP represents global average pooling. and For learnable weight matrix, It is the ReLU activation function. It is the Sigmoid activation function. This indicates a channel weighting operation.
[0050] Equation (2) is used to perform a weighted calculation for each spatial location:
[0051] (2)
[0052] in Indicates the kernel size as Convolution operation, This indicates a channel splicing operation. and These represent spatial average pooling and max pooling, respectively.
[0053] Step 4: [Regarding...] Feature learning is performed on the high-response regions in the image to obtain several prototype feature vectors corresponding to the test images in the test set and several prototype feature vectors corresponding to the training images in the training set. Each prototype feature vector is used to characterize a semantic pattern with discriminative significance in the medical image.
[0054] Will Each spatial location is considered a local semantic candidate unit. Regions with activation values higher than a preset threshold are then selected as high-response regions. Local feature vectors are extracted from these high-response regions and mapped to fixed-dimensional feature representations, i.e., prototype feature vectors, through feature aggregation and normalization. Each prototype feature vector is... Semantic representations are used to characterize semantic patterns with discriminative significance in medical images.
[0055] Step 5: Flatten the prototype feature vectors of each training image, and input the flattened prototype feature vectors into SVM (Support Vector Machine) for discriminative learning to obtain the prototype concept vector corresponding to each category;
[0056] In this embodiment, the specific process of inputting the flattened prototype feature vector into the SVM for discriminative learning is as follows: For each category, a corresponding binary classification SVM is trained to distinguish the prototype feature vectors of this category from those of other categories; a CAV (Concept Activation Vector) corresponding to each prototype feature vector is constructed through the classification hyperplane normal vector of each category SVM to represent the semantic direction of this category in the concept space; the activation value of each prototype feature vector is calculated on its respective category CAV according to equation (3); based on the magnitude of the activation value, the highest activation value corresponding to each category is selected. Each prototype feature vector serves as the prototype concept vector for that category. .
[0057] (3)
[0058] in This indicates the prototype feature vector within its class. The activation value on the CAV; Indicate category The classification hyperplane normal vector of the SVM; Indicate category The prototype feature vector;
[0059] Step 6: Calculate the cosine similarity between the prototype feature vector of the test image and the prototype concept vector corresponding to each category to quantify the degree of matching between the prototype feature vector of the test image and the prototype of each category, and then complete the classification prediction based on the similarity score.
[0060] In this embodiment, the cosine similarity between the prototype feature vector of the test image and the prototype concept vector corresponding to each category is calculated according to equation (4), and the category corresponding to the maximum similarity is determined as the predicted category of the test image, thereby realizing classification decision based on concept similarity. Wherein, the prototype concept vector corresponding to the category with the maximum similarity is regarded as the target prototype concept vector;
[0061] (4)
[0062] in Represents the prototype feature vector of the test image; This represents the target prototype concept vector corresponding to each category.
[0063] Step 7: Based on the prototype feature vector of the test image calculated in Step 6 Based on the similarity with prototype concept vectors of each category, target prototype concept vectors with similarity scores exceeding a preset threshold are selected. , as the prototype concept vector to be mapped.
[0064] Step 8: Map the prototype concept vector to be mapped to the test image;
[0065] The method for mapping the prototype concept vector to be mapped to the test image in this embodiment is as follows: [The method involves mapping the original feature map corresponding to the test image.] The similarity between the feature vectors of each spatial location and the prototype concept vector to be mapped is measured to obtain the concept activation feature map. .exist Regions with a similarity exceeding a set threshold to the prototype concept vector to be mapped are marked as regions to be mapped, emphasizing key semantic patterns while preserving the original feature values of other locations. The feature map corresponding to the test image is then selected. This allows for the preservation of complete spatial information for each feature, enabling more accurate reconstruction of local neighborhood features during CARAFE upsampling.
[0066] Activate feature maps of concepts Input the kernel prediction unit based on the content-aware CARAFE upsampling method, and then... Convolution performs channel compression to obtain low-dimensional feature representations. This reduces the computational complexity of the kernel prediction stage. It represents low-dimensional features. Edge structure map extracted from test images using the U-Net network By fusing the features, we obtain a structurally enhanced feature representation. The edge structure map is used to describe the boundary and shape information of the lesion region in the test image, providing constraints on the display structure for the convolution kernel prediction process. Based on structure-enhanced feature representation... ,predict Adaptive recombining convolution kernels for each spatial location and to The adaptive recombined convolutional kernel weights corresponding to each spatial location are spatially normalized to highlight salient regions and suppress noise responses.
[0067] Based on the predicted adaptive recombined convolutional kernel, the concept activation feature map is processed. Weighted recombination operations are performed on the local neighborhood features at each location to generate a high-resolution concept activation feature map. :
[0068] (5)
[0069] in, Concept activation feature map after upsampling The target location in Represents the concept activation feature map before upsampling. Center and target position The corresponding source location, Activate feature maps for concepts China and Israel Centered The corresponding local neighborhood, Represents the local neighborhood The specific spatial location index within, These are the adaptive recombined convolutional kernel weights at the corresponding positions.
[0070] Establish high-resolution concept activation feature maps The mapping relationship between the high-resolution concept activation feature map and the test image will enable the high-resolution concept activation feature map. The middle corresponds to The location of the region to be mapped is projected onto the coordinates of the test image according to the scale mapping rule. Determine the target prototype concept vector in the test image that corresponds to the target prototype concept vector to be mapped. Corresponding spatial regions:
[0071] (6)
[0072] in, This is a mapping function between feature locations and pixel locations corresponding to the target prototype concept vector. This function is used to describe the high-resolution concept activation feature map. The spatial correspondence between the feature locations and the pixel coordinates of the test image is determined by the downsampling rate of the ResNet-50 network structure and the upsampling rate of CARAFE.
[0073] This step enables a precise mapping of the target prototype concept from the feature space to the pixel space, giving ResNet-50 the spatial interpretation capability "from feature vector to image".
[0074] Step 9: Design a counterfactual intervention mechanism. By suppressing the components of the corresponding class prototype concept vectors in the high-level semantic vectors of the test images, analyze their impact on the classification results, and thus evaluate the contribution of each prototype concept vector to the decision of the ResNet-50 model.
[0075] First, the attention-enhanced feature map corresponding to the test image. High-level semantic vectors are obtained through global average pooling. :
[0076] (7)
[0077] in, This indicates global average pooling.
[0078] Calculate the target prototype concept vector corresponding to the category of the test image. In the high-level semantic vector of the test image Response strength Then, for each target prototype concept vector... A concept suppression operation is performed by weakening the corresponding semantic components in the high-level semantic vector along the direction of the target prototype concept vector. ,in The suppression coefficient, The time interval represents the maximum magnitude suppression in the direction corresponding to the target prototype concept vector. This suppression operation does not assume that different prototype concept vectors are orthogonal to each other. The suppression operation is used to directionally weaken the contribution of the target prototype concept vector to the high-level semantic representation, achieving suppression of a single target prototype concept vector. The impact of the target prototype concept vector on the classification result is evaluated by applying the suppression. The semantic vector after intervention is then... With the target prototype concept vector set for each category Perform matching, where Indicates the corresponding category. The specific content is: Calculation. The cosine similarity with all target prototype concept vectors in this category is used, and the average is taken as the matching score for this category:
[0079] (8)
[0080] Select the category result based on the matching score: ,in To predict the category, the contribution of individual target prototype concept vectors to the ResNet-50 model's decision is quantitatively analyzed based on the changes in classification accuracy before and after the intervention. If the suppression operation significantly reduces the prediction probability, it indicates that the prototype concept vector has a strong positive effect on classification. This evaluation method can not only explain the interpretability of prototype concept vectors within the model, but also quantitatively measure the magnitude of the effect of each prototype concept vector on the final prediction result from the perspective of causal intervention, thus providing a more reliable basis for interpretable diagnosis of medical images.
[0081] Figure 2 Figure 1 shows an example of misclassification of malignant nodule images in the test set of this embodiment. Figure 2(a) is the test image, Figure 3(b) is the prototype concept in the training set images similar to the prototype concept in the test image, Figure 4(c) shows the location of the prototype concept in the training set images, and Figure 5(d) is the activation heatmap of the prototype concept in the training set images. This invention compares the texture and boundary features of nodules (malignant nodules) in the test image with the prototype concept of benign nodules in the training set images, thus incorrectly predicting them as benign.
[0082] Figure 3Figure 1 shows an example of the correct classification of benign nodule images in the test set of this embodiment. Figure 2 shows the test image, Figure 3 shows the prototype concept in the training set images similar to the prototype concept in the test image, Figure 4 shows the position of the prototype concept in the training set images, and Figure 5 shows the activation heatmap of the prototype concept in the training set images. This invention can learn the boundary smoothness and texture features of benign nodules and establish associations between previously unseen nodule images and these learned prototype concepts. As shown in the figure, when the network model encounters a nodule image it has never seen before, it compares the image with the prototype concept of benign nodules and classifies it based on the similarity between the new image and the learned prototype concept.
[0083] Figure 4 This is a schematic diagram of the prototype concept suppression experiment in this embodiment. The horizontal axis represents 10 prototype concepts, the vertical axis represents the change in accuracy, negative values indicate a decrease in accuracy, and the curves represent different suppression coefficients. Impact on accuracy, curve type distinction Size, The larger the value, the stronger the suppression of the prototype. Black dots represent... The maximum inhibitory effect at that time. It can be observed that, as The increase in the number of classes significantly increases the rate of decrease in classification accuracy, indicating that the ResNet-50 model is highly dependent on some key prototype concepts.
[0084] It should be understood that, inspired by the technical concept of this invention, those skilled in the art can make various improvements or modifications based on the above content without departing from the scope of this invention, and these modifications still fall within the protection scope of this invention.
Claims
1. A method for interpretability of decision-making criteria in a medical image classification model, characterized in that, The method includes the following steps: Step 1: Obtain the required number of raw CT images and select two-dimensional image slices containing lesion areas from each raw CT image. Preprocess each image slice to obtain a standardized dataset of two-dimensional image slices containing lesion areas. Divide this dataset into two parts according to a certain ratio. One part is used as the test set, and the other part is combined with two-dimensional image slices of uniform size that do not contain lesion areas in an equal ratio to form the training set. Step 2: Input the images from the training and test sets into a ResNet-50 convolutional neural network (with ResNet-50 as the backbone) to obtain the low-level texture features and high-level semantic features of the input images, and derive the feature map output by the last convolutional layer of the model. ,in The number of feature map channels. The width of the feature map. The height of the feature map; Step 3: [Regarding...] Applying channel attention and spatial attention mechanisms to generate attention-enhanced feature maps ; Step 4: [Regarding...] Feature learning is performed on high-response regions in the dataset to obtain several prototype feature vectors of images in the test set (test images) and several prototype feature vectors of images in the training set (training images), where each prototype feature vector is... Semantic representations are used to characterize semantic patterns with discriminative significance in medical images; Step 5: Flatten the prototype feature vectors of each training image, and input the flattened prototype feature vectors into SVM for discriminative learning to obtain the prototype concept vector corresponding to each category. Step 6: Calculate the similarity between the prototype feature vector of the test image and the prototype concept vector corresponding to each category, determine the category corresponding to the maximum similarity as the predicted category of the test image, and regard the prototype concept vector corresponding to the category with the maximum similarity as the target prototype concept vector. Step 7: Filter out the similarities calculated in Step 6 that are higher than the preset threshold, and use the target prototype concept vectors corresponding to these similarities as prototype concept vectors to be mapped. Step 8: Map the prototype concept vector to be mapped to the test image; Step 9: Design a counterfactual intervention mechanism. By suppressing the components of the corresponding class prototype concept vectors in the high-level semantic vectors of the test images, analyze their impact on the classification results, and thus evaluate the contribution of each prototype concept vector to the decision of the ResNet-50 model.
2. The interpretability method according to claim 1, characterized in that, The preprocessing of each image slice described in step 1 includes window width and level processing, size unification, pixel intensity normalization, and noise suppression processing.
3. The interpretability method according to claim 1, characterized in that, Step 3 includes: firstly... Channel attention enhancement is performed, global information is aggregated in the spatial dimension, and the importance weight of each channel is calculated according to equation (1) to obtain the feature map. Then The attention-enhanced feature map is generated by weighting each spatial location according to equation (2). ; (1) Wherein, GAP represents global average pooling; and The weight matrix is a learnable weight matrix; It is the ReLU activation function; Use the Sigmoid activation function; Indicates channel weighting operation; (2) in Indicates the kernel size as Convolution operations; This indicates a channel splicing operation; and These represent spatial average pooling and max pooling, respectively.
4. The interpretability method according to claim 3, characterized in that, Step 4 describes the... The process of feature learning for high-response regions in the model is as follows: Each spatial location is considered as a local semantic candidate unit. Then, regions with activation values higher than a preset threshold are selected from each local semantic candidate unit as high-response regions. The corresponding local feature vectors are then extracted from the high-response regions and mapped to fixed-dimensional feature representations, i.e., prototype feature vectors, through feature aggregation and normalization.
5. The interpretability method according to claim 4, characterized in that, The specific process of inputting the flattened prototype feature vector into the SVM for discriminative learning in step 5 is as follows: For each category, train a corresponding binary classification SVM to distinguish the prototype feature vectors of this category from those of other categories; construct a CAV corresponding to each prototype feature vector through the classification hyperplane normal vector of each category SVM to represent the semantic direction of this category in the concept space; calculate the activation value of each prototype feature vector on its category CAV according to equation (3); select the category with the highest activation value according to the size of the activation value. Each prototype feature vector serves as the prototype concept vector for that category. ; （3） in This indicates the prototype feature vector within its class. The activation value on the CAV; Indicate category The classification hyperplane normal vector of the SVM; Indicate category The prototype feature vector.
6. The interpretability method according to claim 5, characterized in that, The method for mapping the prototype concept vector to be mapped to the test image in step 8 is as follows: Map the original feature map corresponding to the test image... The similarity between the feature vectors of each spatial location and the prototype concept vector to be mapped is measured to obtain the concept activation feature map. ;exist Regions with a similarity to the prototype concept vector to be mapped exceeding a set threshold are marked as regions to be mapped; Input the kernel prediction unit based on the content-aware CARAFE upsampling method, and then... Convolution performs channel compression to obtain low-dimensional feature representations. ;Will Edge structure map extracted from test images using U-Net network By fusing the features, we obtain a structurally enhanced feature representation. ;based on predict Adaptive recombining convolution kernels for each spatial location And on The weights are spatially normalized to highlight salient regions and suppress noise responses; based on right Weighted recombination operations are performed on the local neighborhood features at each location to generate a high-resolution concept activation feature map. ;Establish The mapping relationship between the image and the test image will The middle corresponds to The location of the region to be mapped is projected onto the coordinates of the test image according to the scale mapping rules. Determine the spatial region in the test image that corresponds to the target prototype concept vector to be mapped.
7. The interpretability method according to claim 6, characterized in that, Step 9 includes: First, the attention-enhanced feature map corresponding to the test image... High-level semantic vectors are obtained through global average pooling. Then calculate the target prototype concept vector corresponding to the category of the test image. In the high-level semantic vector of the test image Response strength Then, for each target prototype concept vector... A concept suppression operation is performed by weakening the corresponding semantic components in the high-level semantic vector along the direction of the target prototype concept vector, resulting in an intervened high-level semantic vector. ,in The suppression coefficient, The time represents the maximum magnitude suppression in the direction corresponding to the target prototype concept vector. This suppression operation does not assume that different prototype concept vectors are orthogonal to each other. The suppression operation is used to directionally weaken the contribution of the target prototype concept vector to the high-level semantic representation, achieving suppression of a single target prototype concept vector. The impact of the target prototype concept vector on the classification result is evaluated by applying the suppression of the target prototype concept vector; then... With the target prototype concept vector set for each category Perform matching, where This indicates the corresponding category, specifically: calculation. The cosine similarity with all target prototype concept vectors in the category is used, and the average value is taken as the matching score for the category. Finally, the classification result is selected based on the matching score, and the contribution of a single target prototype concept vector to the decision of the ResNet-50 model is quantitatively analyzed based on the change in classification accuracy before and after the intervention.