A SPECT-MPI image classification method and system
Through the combination of the dual-channel three-dimensional residual network and the differential feature attention module, the model overfitting and noise feature problems in the SPECT-MPI dataset are solved, and high-accuracy image classification is achieved, especially in small-scale imbalanced data. Excellent performance under set.
Patent Information
- Application Number
- CN202411054265.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-02
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-08-02
AI Technical Summary
The prior art is problem of overfitting models under small-scale unbalanced SPECT-MPI data sets and noise characteristics lead to classification errors.
The dual-channel three-dimensional residual network model is used to combine the differential feature attention module to feature extraction and fusion of the load and resting features of the SPECT-MPI image, enhance noise robustness through the attention mechanism, and use nearest neighbor decisions for classification.
Achieving 96.88% and 92.7% accuracy on small-scale unbalanced datasets significantly improves noise robustness and classification accuracy and reduces noise impact.
Smart Images

Figure CN118587511B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a fine-grained image classification method and system with enhanced noise robustness, and in particular to a noise robust differential feature attention processing SPECT-MPI myocardial ischemia and infarction image classification, belonging to the field of artificial intelligence image processing technology. Background Art
[0002] Coronary Artery Disease (CAD) is one of the major causes of morbidity and mortality. One of the effective tools for detecting CAD is single-photon emission computed tomography-myocardial perfusion imaging (SPECT-MPI). As an important advanced technology, SPECT-MPI uses the ability of normal or functional myocardial cells to selectively absorb certain imaging agents. The amount of myocardial uptake is proportional to the local myocardial blood flow. Nuclear medicine imaging equipment is used to scan the short axis (SA), horizontal long axis (HLA) and vertical long axis (VLA) of the patient's heart at rest and under load. Under load, the blood flow of the myocardium with normal blood supply increases, and the uptake of imaging agents also increases accordingly; while the myocardium in the coronary artery stenosis area cannot increase blood perfusion accordingly with the load, which increases the difference in the distribution of imaging agents between the lesion area and the normal area, thereby showing ischemic lesions.
[0003] As far as computer-aided diagnosis of medical images is concerned, traditional machine learning algorithms require pre-defined features, and the pre-defined features are applied in various combinations for effective diagnosis, but these combinations are usually not comprehensive, resulting in information loss and thus affecting accuracy. The prior art has publicly used the superior characteristics of deep learning (DL) methods. DL methods can avoid tedious manual feature extraction by automatically calculating features, and can also extract features from the data layer by layer to obtain deep discrimination information of SPECT-MPI images. For SPECT-MPI, it can generate three cross-sectional images of the heart, and simultaneously present the effect of myocardial absorption of tracers at different times, so it can present three-dimensional anatomical structure information. However, previous studies used two-dimensional convolutional neural network (CNN) modeling on SPECT-MPI images, ignoring the temporal and spatial correlation between slices, resulting in classification errors.
[0004] SPECT-MPI image classification belongs to fine-grained classification. Compared with ordinary image classification problems, the image data it faces has more similar appearance features. Since SPECT-MPI images are arranged according to the tracer perfusion time, there will be common perfusion abnormality features at the end of the sequence; individual differences in subjects, such as patient gender, age, and heart size; attenuation artifacts affected by body tissues during the acquisition process, and noise caused by extracardiac activities such as the gastrointestinal tract. These factors cause the data to show small inter-class differences and large intra-class differences. At the same time, the public SPECT-MPI data set has a small sample size and an unbalanced category distribution, which makes CNN easy to overfit. To solve the above problems, the present invention proposes to use three-dimensional convolution to process SPECT-MPI myocardial slices that have both temporal and spatial correlations based on the twin network structure; at the same time, the myocardial information at rest and load states is fused through the attention mechanism to enhance the processing of noise. Summary of the invention
[0005] The purpose of the present invention is to overcome the deficiencies in the prior art, provide a fine-grained image classification method and system with enhanced noise robustness, and solve the technical problems of model overfitting and classification errors caused by noise characteristics in small-scale unbalanced SPECT-MPI datasets.
[0006] In a first aspect, the present invention provides a SPECT-MPI image classification method, comprising:
[0007] Preprocessing the SPECT-MPI image to be classified, and pairing the preprocessed SPECT-MPI image with a standard image in a standard image set to obtain a plurality of sample pairs; wherein the standard image is a preprocessed image randomly selected from a classification model training set;
[0008] The classification model is used to process multiple sample pairs to obtain the classification results of SPECT-MPI images;
[0009] Among them, the process of the classification model processing multiple sample pairs includes:
[0010] A dual-channel three-dimensional residual network model is used to extract features from sample pairs respectively, and the load features and rest features of the samples in each sample pair are obtained;
[0011] The difference feature attention module, connection module and fully connected layer are used in sequence to fuse the load feature and rest feature of the same sample to obtain the feature vector of the sample in each sample pair;
[0012] According to the feature vector, calculate the similarity of samples in each sample pair;
[0013] According to the similarity, the nearest neighbor decision classification is adopted to obtain the classification results of SPECT-MPI images.
[0014] In combination with the first aspect, further, the image is preprocessed, including:
[0015] Perform redundant denoising on the image based on the color threshold denoising algorithm;
[0016] Cut and reconstruct the redundant denoised images to obtain the load group images and the rest group images;
[0017] By performing image registration on the load group images and the rest group images, the registered load group images and the rest group images are obtained;
[0018] Random rotation, translation and flipping operations are performed on the registered load group images and rest group images.
[0019] Furthermore, the preprocessed SPECT-MPI images are paired with standard images in the standard image set to obtain multiple sample pairs, including:
[0020] The load group images and rest group images of the same image are taken as samples, and the samples corresponding to the SPECT-MPI images are paired with the samples corresponding to all standard images to obtain multiple sample pairs.
[0021] Furthermore, the dual-channel 3D residual network model includes two weight-shared 3D-ResNets, and the two 3D-ResNets respectively extract features from the load group image and the rest group image of the same sample;
[0022] The 3D-ResNet consists of an initial block and four residual blocks; the initial block includes a convolutional layer with a maximum pooling layer, and each of the residual blocks consists of a bottleneck structure and a residual connection, and the bottleneck structure includes three convolutional layers.
[0023] Furthermore, the load characteristics include VLA axis load characteristics, HLA axis load characteristics and SA load characteristics, and the rest characteristics include VLA axis rest characteristics, HLA axis rest characteristics and SA rest characteristics;
[0024] The difference feature attention module includes the deep spatial attention module, the planar spatial attention module and the channel attention module;
[0025] The load feature and rest feature of the same sample are fused by using the difference feature attention module, connection module and full connection layer in turn to obtain the feature vector of the sample in each sample pair, including:
[0026] The deep spatial attention module, the planar spatial attention module and the channel attention module are used in sequence to process the load features and rest features of the same axis;
[0027] Use the connection module to splice the processed features of all axes;
[0028] The fully connected layer is used to process the concatenated features to obtain the feature vector of the sample.
[0029] Furthermore, the deep spatial attention module is used to process the load characteristics and rest characteristics of the same axis to obtain ,
[0030] ,
[0031] in, represents the first-level spatial features processed by the deep spatial attention module, represents element-wise multiplication, Denotes the deep spatial attention module according to The calculated deep spatial attention map, Indicates load characteristics and resting characteristics The absolute value difference of
[0032] The calculation is as follows:
[0033] ,
[0034] in, represents the average pooling of feature channels, represents the average pooling of the HW plane, represents a convolution operation with a kernel size of 1×1×3 and a stride of 1. ReLU is the activation function, express Sigmoid Activation function.
[0035] Furthermore, the plane space attention module is used to process the first-level spatial features to obtain ;
[0036] ,
[0037] in, represents the secondary spatial features after being processed by the plane spatial attention module, Represents the plane spatial attention module according to The calculated features have spatial attention maps in the HW plane;
[0038] The calculation formula is as follows:
[0039] ,
[0040] in, represents the maximum pooling calculation based on the channel dimension, represents the average pooling calculation based on the channel dimension, Represents a splicing operation, Represents a convolution operation with a kernel size of 3×3×1 and a stride of 1.
[0041] Furthermore, the channel attention module is used to process the secondary spatial features to obtain ;
[0042] ,
[0043] in, represents the calibration characteristic, Represents the channel attention module according to The calculated channel attention map;
[0044] The calculation formula is as follows:
[0045] ,
[0046] in, and represents the weights of the two fully connected layers, MLP represents a multilayer perceptron, is the number of channels, is the ratio used to set the number of hidden layer nodes, The activation function is added in back, , , and Represent 3D global average and 3D maximum pooling respectively.
[0047] Furthermore, the fully connected layer includes two linear connection structures, and a ReLU Activation function and dropout Layer; the calculation formula is as follows:
[0048] ,
[0049] in, represents the feature vector after the fully connected layer, represents the concatenated feature vector, FCL It means that after the fully connected layer, is the neuron node discard operation, and represents the weight of the linear transformation, and Indicates bias.
[0050] In a second aspect, the present invention provides a SPECT-MPI image classification system, comprising:
[0051] Sample pairing module: used to preprocess the SPECT-MPI image to be classified, and pair the preprocessed SPECT-MPI image with the standard image in the standard image set to obtain multiple sample pairs; wherein the standard image is a preprocessed image randomly selected from the classification model training set;
[0052] Feature extraction module: used to extract features from sample pairs using a dual-channel three-dimensional residual network model to obtain the load features and resting features of the samples in each sample pair; the load features and resting features of the same sample are fused using the difference feature attention module, connection module and fully connected layer in turn to obtain the feature vectors of the samples in each sample pair;
[0053] Classification module: used to calculate the similarity of samples in each sample pair based on the feature vector; based on the similarity, the nearest neighbor decision classification is adopted to obtain the classification result of the SPECT-MPI image.
[0054] Compared with the prior art, the present invention has the following beneficial effects:
[0055] The present invention proposes a SPECT-MPI image classification method for classification and identification of CAD, and adopts a dual-channel three-dimensional residual network model to extract features of sample pairs respectively, so as to obtain the load features and rest features of the samples in each sample pair; and adopts a difference feature attention module, a connection module and a fully connected layer in turn to fuse the load features and rest features of the same sample, so as to obtain the feature vector of the samples in each sample pair.
[0056] The dual-channel 3D residual network model consists of a dual-channel 3D-ResNet, which is designed to extract features from the stress group images and the rest group images of SPECT-MPI images. Unlike the two-dimensional CNN, the dual-channel 3D residual network model uses a three-dimensional convolution kernel, which can make full use of the temporal information between the stress group images and the rest group images, and can obtain sufficient discriminant information from the SPECT-MPI images, more easily capture the abnormal perfusion features in the SPECT-MPI images, and obtain sufficient discriminant information from small-scale and unbalanced data.
[0057] The difference feature attention module can use the difference features to improve the significance of perfusion abnormalities of the load features. By adding spatial and channel weights to each load feature to fuse the above deep features, the noise of SPECT-MPI images can be reduced from multiple angles, thereby improving the noise robustness of the classification method.
[0058] The accuracy of the proposed method on the random sampling set and the K-fold cross validation set is 96.88% and 92.7% respectively, showing excellent performance in dealing with unbalanced, noisy and small-scale data. BRIEF DESCRIPTION OF THE DRAWINGS
[0059] Figure 1 is a flow chart of a SPECT-MPI image classification method provided by an embodiment of the present invention;
[0060] Figure 2 The figures are effect diagrams before and after the dynamic color threshold algorithm processing provided by the embodiment of the present invention, wherein (a) is the original image of the ROI, and (b) is the processed image;
[0061] Figure 3 is an image registration effect diagram provided by an embodiment of the present invention, wherein (a) is a stress myocardial image, (b) is a resting myocardial image, and (c) is a resting myocardial image after registration;
[0062] Figure 4 is a flow chart of feature extraction performed by a feature encoder provided by an embodiment of the present invention;
[0063] Figure 5 It is a schematic diagram of the difference feature attention module provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0064] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.
[0065] The purpose of the present invention is to solve the problem of poor classification effect in SPECT-MPI small data sets, unbalanced data sets and noisy data sets in the prior art, and propose a SPECT-MPI image classification method and system, which uses a classification model to classify SPECT-MPI images to determine myocardial ischemia or infarction. The classification model is specifically a 3D Siamese network with a stress-rest feature fusion module (3DSiam-SRFNet).
[0066] The present invention comprises a preprocessing stage, a training stage and a classification stage, wherein the training stage comprises sample pairing, feature extraction, similarity calculation and parameter optimization steps; the classification stage comprises preprocessing, sample pairing, feature extraction and classification steps.
[0067] In the preprocessing stage, the present invention uses a dynamic color threshold algorithm to extract myocardial regions of interest (ROI) to reduce redundant information irrelevant to classification. Due to the different time points of image acquisition, the spatial positions of the heart in the load state and the rest state are inconsistent, and the spatial positions of the heart in the two states on the image cannot be directly corresponding. The present invention performs image registration on rest imaging and load imaging to facilitate the implementation of the difference feature attention module in the feature extraction stage.
[0068] In the sample pairing steps of the training stage and the classification stage, the present application constructs sample pairs through the training set and the classification set, and initializes the category labels of the paired samples.
[0069] In the feature extraction step of the training phase and the classification phase, the feature encoder of 3DSiam-SRFNet uses a dual-channel 3D-ResNet (D3D-ResNet) model and a Feature Difference-based Attention Module (FDAM) to encode the data. D3D-ResNet extracts deep features of the load group and the rest group of SPECT-MPI. Then, a FDAM with a three-level attention mechanism is designed to fuse the myocardial load features and the rest features from the feature space dimension and the channel dimension. Finally, 3DSiam-SRFNet embeds the features into a feature space through a fully connected layer.
[0070] During the training phase, the network is optimized by back-propagation through the designed contrastive loss function.
[0071] In the classification stage, the feature similarity between the sample to be classified and all samples in the support set is calculated by Euclidean distance, and finally the category label of the sample to be classified is obtained by the nearest neighbor decision.
[0072] The advantage of the 3D twin network is that the similarity between the supporting data and the training data / data to be classified is utilized in the classification process. First, the 3D twin network uses sample pairs as the data to be processed, which has more diversity than a single sample. Then, the features of each sample are extracted using D3D-ResNet and FDAM with shared weights, the similarity between the sample to be classified and the supporting sample is calculated, and the similarity level of the sample to be classified is judged. Since the 3D twin network uses two identical feature extraction modules with shared weights, the supporting data and the training / data to be classified can be learned together. Compared with other CNN methods, the 3D twin network provided by the present invention can more easily capture the perfusion abnormality features in SPECT-MPI images, so that sufficient discriminant information can be obtained from small-scale and unbalanced data.
[0073] like Figure 1 As shown, it is a flow chart of a SPECT-MPI image classification method provided by an embodiment of the present invention. The two-dimensional SPECT-MPI image is reconstructed into a three-dimensional sequence image through a series of preprocessing, and then the sample pairing is performed, and the paired sample pairs are input into 3DSiam-SRFNet. For the samples of each branch, D3D-ResNet is first used to extract the load feature and the rest feature, and then the difference feature attention module is used to realize the information fusion of the two feature layers, and then the above features are sent to the fully connected layer to obtain the final embedded feature vector, and in the classification stage, the Euclidean distance is used to calculate the feature vector similarity between the samples in the classification set and the samples in the support set, and the nearest neighbor decision is used to obtain the final classification result.
[0074] Embodiment 1:
[0075] This example uses the SPECT-MPI images from the Kaggle website as the experimental database. The experimental database contains 192 patients, including 150 CAD patients (78.1%) and 42 normal people (22.9%). Table 1 shows the detailed information of the experimental database. All selected SPECT-MPI images were independently interpreted and classified by expert readers using two different category labels (1: Normal and 2: CAD). All CAD samples are recorded as ,in Representative CAD samples, Represents the total number of CAD samples; all Normal samples are recorded as ,in Representative Normal samples, Represents the total number of Normal samples.
[0076] Table 1: Detailed information of the experimental database
[0077]
[0078] Next, and Preprocessing is performed, including redundant denoising, image reconstruction, image registration, data enhancement, and data set division. The specific steps are as follows:
[0079] Step 1: Redundant denoising: Use the color threshold denoising algorithm to remove the digital label information of the SPECT-MPI image and the myocardial background noise.
[0080] This application uses an RGB-based color threshold algorithm to segment SPECT-MPI images according to the formula Extract ROI. Given a SPECT-MPI image with a shape of The matrix ,in is the height, is the width, is the number of channels, and assume that the first channel is the red channel R and the second channel is the green channel G.
[0081] This application is for each position Perform the following operation: If the value of the red channel is greater than the value of the green channel, keep the values of all channels at that position, otherwise set the values of all channels at that position to zero.
[0082] First, use the indicator function Construct a new matrix :
[0083] ,
[0084] in, Representation Matrix In Location and Channel The value of . is an indicator function. Define the indicator function as follows:
[0085] ,
[0086] when When, keep When When The value of is set to zero.
[0087] In matrix form, this application uses element-wise product and extended Boolean mask matrix representation:
[0088] ,
[0089] in, represents element-wise product, is a and is expanded to a Boolean matrix with the same first two dimensions (height and width) as Three-dimensional matrices of the same shape. Each channel uses the same Boolean matrix to perform element-by-element multiplication. The operation is performed at all positions and the results are stored in a new matrix middle.
[0090] like Figure 2 As shown in FIG. 1 , the effect diagram before and after the color threshold algorithm processing provided by the embodiment of the present invention is shown, where (a) is
[0091] matrix The corresponding ROI original image, (b) is the new matrix The corresponding processed image; ROI The original image contains many white annotations and wireframes for the convenience of the doctor's naked eye observation. In addition, the flow of tracer in the heart is concentrated in the orange, pink and purple areas called ROI, while the green area is almost useless. It can be seen from (b) that the white wireframe and digital labels as well as the low tracer area of the myocardium are eliminated.
[0092] Step 2: Image reconstruction: the SPECT-MPI images after redundant denoising were cut and reorganized, and stacked into load group images and rest group images according to the short axis, vertical axis and horizontal axis.
[0093] This application cuts the redundant denoised SPECT-MPI image into 80 sub-images (myocardial slices) according to the position of the original wireframe, and the size of each sub-image is 89*89. Then, these sub-images are divided into load group images and rest group images according to the patient's status during data collection, where the sequence data in the load group image and the rest group image are respectively represented as and ,in , and The sub-images represent data obtained from the short axis, horizontal axis, and vertical axis of the heart, respectively.
[0094] Furthermore, since the classification of SPECT-MPI images is based on the brightness differences between classes, the present application converts the RGB values of the sequence data into grayscale values.
[0095] Step 3: Image registration; the present application uses an optimizer to translate the images grouped in step 2 in the Y-axis direction so that the rest group images and the load group images are aligned under the mean square error metric.
[0096] for Each sub-graph in , all in The subgraph corresponding to the same index in By initializing the translation transformation, setting the mean square error as the metric, and using the regular step gradient descent method to optimize the Y-axis translation, the sub-graph Resampling is performed to achieve The specific steps are as follows:
[0097] Sub-step 3.1: Initial transformation
[0098] The initial translation transform is set to zero, i.e. there is no initial displacement:
[0099] .
[0100] Sub-step 3.2: Mean Squared Error Metric
[0101] Use Mean Squared Error (MSE) as the registration metric function:
[0102] ,
[0103] in, are the pixel coordinates in the fixed image, It is transforming The corresponding pixel coordinates of the moving image.
[0104] Sub-step 3.3: Optimizer settings
[0105] The MSE is optimized using a regular step gradient descent method, allowing the optimized parameter to be only the Y-axis translation. The goal of the optimizer is to find the optimal Y-axis translation , so that the MSE is minimized.
[0106] Sub-step 3.4: Translation Transformation
[0107] The translation transformation is only performed in the Y-axis direction, so the Y-axis translation amount is optimized , the final translation transformation matrix It is expressed as:
[0108] .
[0109] Sub-step 3.5: Gradient Descent Optimization
[0110] In each iteration, the translation is updated according to the gradient , the formula is as follows:
[0111] ,
[0112] in, is the iteration step index, is the learning rate, MSE About The gradient of Represents the updated translation amount, Indicates the current translation amount.
[0113] Sub-step 3.6: Resampling
[0114] Use the final translation transformation matrix Resample the resting images to align with the stress images:
[0115] ,
[0116] Finally, the resampled registration image is obtained ,renew Atomic diagram in The registration effect is shown in the figure Figure 3 As shown, (a) is stress myocardial imaging, (b) is resting myocardial imaging to be registered, and (c) is resting myocardial imaging after registration. In (a), (b), and (c), the first row represents the short-axis slices of the heart at rest and under stress, the second row represents the horizontal long-axis slices, and the third row represents the vertical long-axis slices.
[0117] Step 4: Data enhancement: randomly rotate, translate, and flip the registered load group images and rest group images to expand data diversity.
[0118] This application adopts a random data enhancement method to increase the diversity of sequence data. Specifically, MONAI (MedicalOpen Network for AI) is used to process the load group images and the rest group images through random flipping, rotation, translation and scaling transformations to maintain the consistency of the load group images and the rest group images to obtain the preprocessed data set.
[0119] The preprocessed data set is divided. Specifically, this application uses the data set on Kaggle. First, two experimental data sets are set, where set-A is a randomly selected data set and set-B is a k-fold cross-validation data set.
[0120] a) Set-A: In order to compare with other research results, set-A was constructed based on random partitioning. This application obtained preprocessed SPECT-MPI images for partitioning, randomly selected 1 / 6 of them as the classification set, and the rest as the classification set.
[0121] b) Set-B: In order to avoid the randomness of an experiment, set-B was constructed based on K-fold cross validation. The preprocessed SPECT-MPI images were divided into 6 non-overlapping subsets. Each time, one subset was selected as the classification set, and the other subsets were used as training sets.
[0122] in , , the data set of the training set is recorded as , the data set of the classification set is recorded as , for the next step of processing.
[0123] Embodiment 2:
[0124] This embodiment performs 3DSiam-SRFNet training, and the training phase includes sample pairing, feature extraction, similarity calculation, and parameter optimization steps.
[0125] Step 1: Sample pairing.
[0126] This application combines the preprocessed SPECT-MPI images into sample pairs. In the training phase, sample pairing randomly selects samples from the training set to form sample pairs in the training phase;
[0127] Specifically, this application uses the preprocessed training set data , construct the sample pair set in the training phase according to the following formula .
[0128] ,
[0129] in, and Respectively indicate from The sample selected in represents a sample pair, represents the total number of sample pairs, Represents the index of the sample pair in the dataset.
[0130] In addition, the sample pairs in the training phase Tags It is expressed as follows:
[0131] ,
[0132] The label of each pair of samples belonging to the same class is 1, and the label of each pair of samples belonging to different classes is 0.
[0133] Step 2: Feature extraction.
[0134] The 3DSiam-SRFNet of this application uses two weight-shared feature encoders to extract features for sample pairs. Figure 4 As shown in FIG. 1 , a feature encoder according to an embodiment of the present invention performs feature extraction, wherein the feature encoder includes a dual-channel three-dimensional residual network module D3D-ResNet and a difference feature attention module FDAM. In this step, the sample pairs in the training phase are Mapping to vector space to generate feature vector representation in the training phase , , the specific steps are as follows:
[0135] Step 2.1: D3D-ResNet.
[0136] ResNet is a common convolutional neural network model with skip connections, through which the outputs of the first few layers can bypass the outputs of subsequent layers to propagate residual information to prevent gradient vanishing or exploding. Since the general ResNet model processes single-channel two-dimensional data, it cannot fully process the three-dimensional data of the load group and the rest group. To solve this problem, the present application first converts the convolutional layer and pooling layer of ResNet from a two-dimensional mode to a three-dimensional mode.
[0137] like Figure 4 As shown in Figure 1, D3D-ResNets consists of two weight-sharing 3D-ResNet branches, each of which consists of an initial block and four residual blocks. The initial block includes a convolutional layer with a maximum pooling layer, while each residual block consists of a bottleneck structure and a residual connection. The bottleneck structure includes three convolutional layers. Each convolutional layer is followed by batch normalization and a rectified linear unit ( ReLU activation function). Finally, the output of the last module is the deep features of the SPECT-MPI image. For each 3D-ResNet, the number of residual blocks, the size of the convolutional layer, and the size of the output are recorded in Table 2. To deal with overfitting, 3D-ResNet loads pre-trained models trained by multiple medical datasets.
[0138] Table 2: Architecture parameters of 3D-ResNet:
[0139]
[0140] This application uses the above method to process load group images and rest group images Then, the load characteristics are obtained. and resting characteristics ,in and Respectively and The sub-images and sub-images.
[0141] Step 2.2: FDAM.
[0142] The load features and rest features in SPECT-MPI images are fused to effectively identify perfusion abnormalities. This application uses FDAM, which consists of a three-level attention module. and The difference information is The three-level attention of FDAM includes deep spatial attention module, planar spatial attention module and channel attention module.
[0143] like Figure 5 As shown, this is the principle diagram of the difference feature attention module. After D3D-ResNet processing, this application obtains the load feature and resting characteristics , are all three-dimensional features, among which , , and Represent the height, width, depth and number of channels of each 3D feature. , and The coordinate axes of the three-dimensional features are set to the x, z, and y axes respectively.
[0144] First, calculate and The absolute value difference Next, a three-level attention module is used to calculate the weight of the difference feature in the load feature.
[0145] The three-level attention module used has the following specific processing steps:
[0146] 1) Deep Spatial Attention Module
[0147] Specifically, this application uses a one-dimensional differential spatial attention module for processing The output of this module is recorded as , we can get .
[0148] (4)
[0149] in, represents the first-level spatial features processed by the deep spatial attention module, , Denotes the deep spatial attention module according to The calculated deep spatial attention map, Represents element-wise multiplication.
[0150] It can be calculated according to formula (5):
[0151] (5)
[0152] in, represents the average pooling of feature channels, represents the average pooling of the HW plane, represents a convolution operation with a kernel size of 1×1×3 and a stride of 1. ReLU is the activation function, express Sigmoid Activation function.
[0153] 2) Planar Spatial Attention Module
[0154] right Perform two-dimensional plane spatial attention calculation, and the output of the plane spatial attention module is recorded as , obtained by formula (6) .
[0155] (6)
[0156] in, represents the secondary spatial features after being processed by the plane spatial attention module, , Represents the plane spatial attention module according to The calculated features have spatial attention maps in the HW plane.
[0157] It can be calculated by formula (7).
[0158] (7)
[0159] in, represents the maximum pooling calculation based on the channel dimension, represents the average pooling calculation based on the channel dimension, Represents a splicing operation, Represents a convolution operation with a kernel size of 3×3×1 and a stride of 1.
[0160] 3) Channel Attention Module
[0161] right Perform channel attention calculation, and the output of the channel attention module is recorded as , obtained by formula (8) .
[0162] (8)
[0163] in, represents the calibration characteristic, , Represents the channel attention module according to The calculated channel attention map.
[0164] It can be calculated by formula (9).
[0165] (9)
[0166] in, and represents the weights of the two fully connected layers, is the number of channels, is the ratio used to set the number of hidden layer nodes, MLP represents a multilayer perceptron, The activation function is added in back. , ,in and Represent 3D global average and 3D maximum pooling respectively.
[0167] Use the above feature extraction method to process , and After that, this application received three , then expand them into vectors and concatenate them into feature vectors Finally, a fully connected layer (FCL) is used to obtain the final embedded feature vector .
[0168] FCL Contains two linear connection structures. In order to improve the nonlinear mapping and generalization capabilities of the module, a ReLU Activation function and dropout The calculation formula of the fully connected layer is expressed by formula (1):
[0169] (1)
[0170] in, is the embedded feature vector after the fully connected layer, is the neuron node discard operation, represents the activation function of the fully connected layer, and represents the weight of the linear transformation, and Indicates bias.
[0171] Step 3: Similarity calculation.
[0172] This step is used to calculate the feature vector corresponding to the training phase sample pair calculated in step 2 and The Euclidean distance between them is calculated according to formula (10): and The Euclidean distance between as a similarity score.
[0173] (10)
[0174] in, and Respectively and The feature vector point of .
[0175] Step 4: Parameter optimization.
[0176] This application calculates the contrast loss based on the similarity score obtained in step 3 and then performs error back propagation. By minimizing the contrast loss To back-propagate and optimize the model weight parameters of 3DSiam-SRFNet until 3DSiam-SRFNet converges. 3DSiam-SRFNet generates embedding vectors that are closer for the same type of input and farther for different types of input. When the distance between pairs of samples of the same type exceeds a certain threshold or the distance between pairs of samples of different types is lower than another threshold, Penalize 3DSiam-SRFNet. Contrastive loss The definition of is shown in formula (11).
[0177] (11)
[0178] in, d represents the threshold value, is the label of each sample pair, represents the number of sample pairs in the training process, Indicates the maximum value between the two. and Belong to the same category, that is hour, The first item will minimize the Euclidean distance between pairs of samples. and belong to different classes, namely hour, The second item The Euclidean distance between sample pairs will be at least the threshold d. In summary, in the embedded feature space, samples of the same class will be pulled closer, and samples of different classes will be pushed further away, so that the distance between Normal samples and CAD samples will continue to increase, making them easier to distinguish.
[0179] Repeat steps 2 to 4 to obtain the converged model weight parameters of 3DSiam-SRFNet.
[0180] Embodiment three:
[0181] In the classification stage, some samples are randomly selected from the training set to form a support set, which is the standard image set. Then, the samples in the classification set are paired with the standard images in the support set to obtain multiple sample pairs in the classification stage.
[0182] Step 1: Sample pairing
[0183] For Example 1 The samples to be classified in First, from Example 1, Randomly select some data to build a support set , then the samples to be classified With support set Samples in Pairing, constructing a set of sample pairs for the classification stage .
[0184] Step 2: Feature Extraction
[0185] After 3DSiam-SRFNet loads the 3DSiam-SRFNet model weight parameters obtained in Example 2, 3DSiam-SRFNet classifies the sample pairs in the classification stage. Perform feature encoding to obtain the feature vector of the image to be classified and the standard image feature vector .
[0186] Step 3: Similarity calculation
[0187] This step is used to calculate the feature vector of the image to be classified obtained by step 3 and the standard image feature vector The Euclidean distance between them is calculated according to formula (10): and The Euclidean distance between as a similarity score.
[0188] (10)
[0189] in, and Respectively and No. feature vector points.
[0190] Step 4: Nearest Neighbor Decision
[0191] In the classification stage, according to the support set Data and classification sets in The similarity scores between the data in the support set are compared to predict the category of each image to be classified. The nearest neighbor decision method is used for classification, as shown in formula (12), that is, the sample most similar to the image to be classified is found in the support set. , and Classification Belong to the category, get the sample The final category.
[0192] (12)
[0193] in, To find the minimum value, express and The feature similarity.
[0194] The present invention proposes a SPECT-MPI image classification method for classification and identification of CAD, wherein the 3DSiam-SRFNet adopts two weight-shared feature encoders, each of which integrates D3D-ResNet and FDAM. D3D-ResNet is composed of a dual-channel 3D-ResNet, which is intended to extract deep features from images of the load group and the rest group of SPECT-MPI images. Unlike two-dimensional CNN, D3D-ResNet can make full use of the temporal information between the load group and the rest group sub-images, and can obtain sufficient discriminant information from the SPECT-MPI images. Therefore, D3D-ResNet has excellent performance in extracting features from SPECT-MPI images.
[0195] FDAM can use the difference features to improve the significance of perfusion abnormalities in the stress features, and fuse the above deep features by adding spatial and channel weights to each stress feature. The deep features of the stress group images and the rest group images are weighted and fused, thereby improving the noise robustness of the classification method.
[0196] Specifically, since the noise in SPECT-MPI images is generated by external aspects of the myocardium (such as human bones and breasts) and extracardiac activities such as the gastrointestinal tract, the noise between the load group and the rest group of the SPECT-MPI images is usually consistent. Therefore, the first-level attention mechanism enhances the difference information between the load group and the rest group of the SPECT-MPI images to reduce similar components, including noise. Then the second-level and third-level attention mechanisms are used to adaptively adjust the weights of the spatial features and channel features of the load group images, respectively, to further filter out useless information including noise. A SPECT-MPI image classification method described in the present invention can reduce the noise of SPECT-MPI images from multiple angles and has excellent noise robustness.
[0197] Finally, the present invention adopts a similarity classifier to estimate the category of the sample to be classified according to the similarity score between the supporting sample and the sample to be classified through the nearest neighbor decision method.
[0198] After obtaining the sample class labels of all predicted classification sets from step 4, the confusion matrix of true positive (TP), false positive (FP), true negative (TN) and false negative (FN) values is used as the main evaluation indicator. Then, this application uses several secondary indicators, including overall accuracy (OA), sensitivity (SEN), precision (PRE), specificity (SPE) and F1 score, as evaluation indicators.
[0199] Specifically, , , , , .
[0200] The specific results are analyzed as follows:
[0201] Set-A classification experiment results:
[0202] Table 3 shows the evaluation results of set-A based on random sampling. It can be seen that 96.88% of SPECT-MPIs were correctly classified. In terms of identifying CAD, only one SPECT-MPI image was misclassified, and the positive predictive value (PPV) and SPE reached 100%. When detecting normal SPECT-MPI images, all normal SPECT-MPI images were correctly identified, with SPE and F1 scores of 96% and 97.96%, respectively. The average values of the above indicators exceeded 93%, and the method of the present invention can achieve excellent results on set-A.
[0203] Table 3: Set-A evaluation results
[0204]
[0205] Set-B classification experimental results:
[0206] Table 4 shows the results obtained by the method of the present invention on set-B based on six-fold cross validation. The average scores of PPV, SPE and F1 reached 87.95%, 92.77% and 92.7%, respectively. In terms of identifying CAD, the method of the present invention achieved 92.67% SEN, 97.9% PPV, and 92.86% SPE; therefore, based on six-fold cross validation, the method of the present invention can effectively distinguish CAD images from Normal SPECT-MPI images.
[0207] Table 4: Evaluation results of set-B
[0208]
[0209] Comparison of random sampling data results:
[0210] This application uses the CAD index for comparison. Table 5 shows the performance comparison between the method proposed in this application and related studies based on random sampling data. Compared with the prior art, the method of this application achieves the highest OA, PRE and SPE in distinguishing abnormal SPECT images, and performs better in detecting CAD in SPECT-MPI images based on random sampling data.
[0211] Table 5: Related research on CAD detection in SPECT-MPI images based on random sampling data
[0212]
[0213] Comparison of results based on K-fold cross-validation: This application compares the performance of 3DSiam-SRFNet with that of four common CNN models, namely VGGNet-16, VGGNet-19, RestNet-34, and ResNet-50. Table 6 shows the classification results of different methods.
[0214] Table 6: Classification results of different methods on SPECT-MPI images on set-A and set-B
[0215]
[0216] As can be seen from Table 6, the proposed method has the best performance among all the methods. In the experiments of set-A, although the SEN of VGGNet-16 is 100%, its OA and SPE are much lower than those of the proposed method. In addition, the OA and SEN produced by the proposed method are significantly higher than those of VGGNet-19, ResNet-34, and ResNet-50. For the experiments of set-B, the SEN of the proposed method is more than 25% higher than that of the other methods. In terms of OA and SPE, the method of the present application also achieves the highest values among all the methods. Therefore, the above comparison shows that the method of the present application performs better than other methods when processing K-fold cross-validation data.
[0217] The accuracy of the proposed method on the random sampling set and the K-fold cross validation set is 96.88% and 92.7% respectively, showing excellent performance in dealing with unbalanced, noisy and small-scale data.
[0218] The present invention also provides a SPECT-MPI image classification system, comprising:
[0219] Sample pairing module: used to preprocess the SPECT-MPI image to be classified, and pair the preprocessed SPECT-MPI image with the standard image in the standard image set to obtain multiple sample pairs; wherein the standard image is a preprocessed image randomly selected from the classification model training set;
[0220] Feature extraction module: used to extract features from sample pairs using a dual-channel three-dimensional residual network model to obtain the load features and resting features of the samples in each sample pair; the load features and resting features of the same sample are fused using the difference feature attention module, connection module and fully connected layer in turn to obtain the feature vectors of the samples in each sample pair;
[0221] Classification module: used to calculate the similarity of samples in each sample pair based on the feature vector; based on the similarity, the nearest neighbor decision classification is adopted to obtain the classification result of the SPECT-MPI image.
[0222] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.
Claims
1. A SPECT-MPI image classification method, characterized in that: include: Preprocessing the SPECT-MPI image to be classified, and pairing the preprocessed SPECT-MPI image with a standard image in a standard image set to obtain a plurality of sample pairs; wherein the standard image is a preprocessed image randomly selected from a classification model training set; The classification model is used to process multiple sample pairs to obtain the classification results of SPECT-MPI images; Among them, the process of the classification model processing multiple sample pairs includes: A dual-channel three-dimensional residual network model was used to extract features from sample pairs, and the load characteristics and rest characteristics of the samples in each sample pair were obtained. The load characteristics included VLA axis load characteristics, HLA axis load characteristics, and SA load characteristics, and the rest characteristics included VLA axis rest characteristics, HLA axis rest characteristics, and SA rest characteristics. The load feature and rest feature of the same sample are fused by using the difference feature attention module, connection module and fully connected layer in sequence to obtain the feature vector of the sample in each sample pair. The difference feature attention module includes the deep spatial attention module, the plane spatial attention module and the channel attention module. The deep spatial attention module, the plane spatial attention module and the channel attention module are used in sequence to process the load feature and rest feature of the same axis. According to the feature vector, calculate the similarity of samples in each sample pair; According to the similarity, the nearest neighbor decision classification is adopted to obtain the classification results of SPECT-MPI images; Use the deep spatial attention module to process the load features and rest features of the same axis to obtain , , in, represents the first-level spatial features processed by the deep spatial attention module, represents element-wise multiplication, Denotes the deep spatial attention module according to The calculated deep spatial attention map, Indicates load characteristics and resting characteristics The absolute value difference of The calculation is as follows: , in, represents the average pooling of feature channels, represents the average pooling of the HW plane, represents a convolution operation with a kernel size of 1×1×3 and a stride of 1. is the activation function, express Sigmoid Activation function; Use the planar spatial attention module to process the first-level spatial features and obtain ; , in, represents the secondary spatial features after being processed by the plane spatial attention module, Represents the plane spatial attention module according to The calculated feature spatial attention map in the HW plane; The calculation formula is as follows: , in, Indicates the maximum pooling calculation based on the channel dimension, Represents a splicing operation, Represents a convolution operation with a kernel size of 3×3×1 and a stride of 1; Use the channel attention module to process the secondary spatial features and obtain ; , in, represents the calibration characteristic, Represents the channel attention module according to The calculated channel attention map; The calculation formula is as follows: , in, and represents the weights of the two fully connected layers, MLP represents a multilayer perceptron, is the number of channels, is the ratio used to set the number of hidden layer nodes, The activation function is added in back, , , and Represent 3D global average and 3D maximum pooling respectively.
2. A SPECT-MPI image classification method according to claim 1, characterized in that: Preprocess the image, including: Perform redundant denoising on the image based on the color threshold denoising algorithm; Cut and reconstruct the redundant denoised images to obtain the load group images and the rest group images; By performing image registration on the load group images and the rest group images, the registered load group images and the rest group images are obtained; Random rotation, translation and flipping operations are performed on the registered load group images and rest group images.
3. A SPECT-MPI image classification method according to claim 2, characterized in that: The preprocessed SPECT-MPI images are paired with standard images in the standard image set to obtain multiple sample pairs, including: The load group images and rest group images of the same image are taken as samples, and the samples corresponding to the SPECT-MPI images are paired with the samples corresponding to all standard images to obtain multiple sample pairs.
4. A SPECT-MPI image classification method according to claim 2, characterized in that: The dual-channel three-dimensional residual network model includes two weight-shared 3D-ResNets, which respectively extract features from the load group images and the rest group images of the same sample; The 3D-ResNet consists of an initial block and four residual blocks; the initial block includes a convolutional layer with a maximum pooling layer, and each of the residual blocks consists of a bottleneck structure and a residual connection, and the bottleneck structure includes three convolutional layers.
5. A SPECT-MPI image classification method according to claim 1, characterized in that: The load feature and rest feature of the same sample are fused by using the difference feature attention module, connection module and full connection layer in turn to obtain the feature vector of the sample in each sample pair, including: The deep spatial attention module, the planar spatial attention module and the channel attention module are used in sequence to process the load features and rest features of the same axis; Use the connection module to splice the processed features of all axes; The fully connected layer is used to process the concatenated features to obtain the feature vector of the sample.
6. A SPECT-MPI image classification method according to claim 1, characterized in that: The fully connected layer includes two linear connection structures, between which a ReLU activation function and a dropout layer are arranged; the calculation formula is as follows: , in, represents the feature vector after the fully connected layer, represents the concatenated feature vector, FCL represents the fully connected layer, is the neuron node discard operation, and represents the weight of the linear transformation, and Indicates bias.
7. A SPECT-MPI image classification system, characterized in that: include: Sample pairing module: used to preprocess the SPECT-MPI image to be classified, and pair the preprocessed SPECT-MPI image with the standard image in the standard image set to obtain multiple sample pairs; wherein the standard image is a preprocessed image randomly selected from the classification model training set; Feature extraction module: used to extract features from sample pairs using a dual-channel three-dimensional residual network model, and obtain the load features and rest features of the samples in each sample pair. The load features include VLA axis load features, HLA axis load features, and SA load features, and the rest features include VLA axis rest features, HLA axis rest features, and SA rest features; the difference feature attention module, connection module, and fully connected layer are used in sequence to fuse the load features and rest features of the same sample to obtain the feature vectors of the samples in each sample pair, wherein the difference feature attention module includes a deep spatial attention module, a planar spatial attention module, and a channel attention module, and the deep spatial attention module, the planar spatial attention module, and the channel attention module are used in sequence to process the load features and rest features of the same axis; Use the deep spatial attention module to process the load features and rest features of the same axis to obtain , , in, represents the first-level spatial features processed by the deep spatial attention module, represents element-wise multiplication, Denotes the deep spatial attention module according to The calculated deep spatial attention map, Indicates load characteristics and resting characteristics The absolute value difference of The calculation is as follows: , in, represents the average pooling of feature channels, represents the average pooling of the HW plane, represents a convolution operation with a kernel size of 1×1×3 and a stride of 1. is the activation function, express Sigmoid Activation function; Use the planar spatial attention module to process the first-level spatial features and obtain ; , in, represents the secondary spatial features after being processed by the plane spatial attention module, Represents the plane spatial attention module according to The calculated features have spatial attention maps in the HW plane; The calculation formula is as follows: , in, represents the maximum pooling calculation based on the channel dimension, Represents a splicing operation, Represents a convolution operation with a kernel size of 3×3×1 and a stride of 1; Use the channel attention module to process the secondary spatial features and obtain ; , in, represents the calibration characteristic, Represents the channel attention module according to The calculated channel attention map; The calculation formula is as follows: , in, and represents the weights of the two fully connected layers, MLP represents a multilayer perceptron, is the number of channels, is the ratio used to set the number of hidden layer nodes, The activation function is added in back, , , and Represent 3D global average and 3D maximum pooling respectively; Classification module: used to calculate the similarity of samples in each sample pair based on the feature vector; based on the similarity, the nearest neighbor decision classification is adopted to obtain the classification result of the SPECT-MPI image.
