Medical image classification method, use, device and storage medium of low-dose CT

Abstract

This invention discloses a medical image classification method for low-dose CT, which involves collecting ICA, CCTA, and NCCT images of the same body part; using ICA as a reference image to annotate CCTA and NCCT images; constructing a pre-trained model using data augmentation and self-supervised learning methods; building a dual-stream network architecture of teacher-student network, and initializing the teacher network and student network separately using the parameters of the pre-trained model; inputting different images from the same body into the teacher network and student network respectively, while simultaneously training the teacher-student network, and transferring knowledge from the teacher network to the student network through distillation loss; after the teacher-student network training is completed, removing the teacher network and using only the student network to classify the image data. This invention reduces the pain and cost experienced by patients during ICA or CCTA examinations and solves the problem of difficulty in capturing lesion features due to the low resolution of NCCT.

Description

Technical Field

[0001] This invention belongs to the fields of deep learning and medical image processing, and relates to medical image data enhancement technology, self-supervised learning technology, knowledge distillation technology and medical image classification technology. In particular, it relates to a method, application, equipment and storage medium for medical image classification of low-dose CT. Background Technology

[0002] Currently, with the continuous development of the social economy and significant changes in people's lifestyles, the incidence of cardiovascular and cerebrovascular diseases such as coronary heart disease is increasing year by year, leading to a continuous increase in related diseases and economic burdens. Those undergoing physical examinations typically do not exhibit clinical symptoms such as prolonged pain, and relying solely on these symptoms to determine the severity of the disease often results in missed diagnoses of high-risk patients. Therefore, accurate screening of individuals undergoing physical examinations suspected of having cardiovascular and cerebrovascular diseases has significant clinical and social benefits.

[0003] Accurate diagnosis of coronary artery disease (CA) relies on invasive coronary angiography (ICA) and coronary computed tomographic angiography (CCTA). ICA provides high-resolution images and can accurately measure the stenosis and blockage of the coronary arteries, making it the gold standard for diagnosing CA. However, ICA is an invasive procedure with high risks, making it unsuitable for screening suspected CA patients during routine checkups. While CCTA is a non-invasive medical imaging technique that provides three-dimensional images to help doctors observe the stenosis of the coronary arteries, it requires the injection of contrast agents to enhance the visibility of blood vessels, which may raise other health risks, such as contrast agent-related allergies and kidney damage. Compared to ICA and CCTA, noncontrast chest CT (NCCT) has the advantages of requiring no contrast agents, low radiation dose, speed, and convenience, and includes complete imaging information of the entire heart and surrounding tissues. Therefore, fully utilizing NCCT imaging for early screening of suspected CA patients during routine checkups has significant clinical value.

[0004] With the rapid development of computer technology, deep learning has made tremendous progress in the field of natural image processing, with large-scale datasets such as ImageNet playing a crucial role. These datasets provide rich training samples for deep learning, promoting the continuous optimization of pre-trained models and enabling them to better adapt to various downstream tasks. However, medical images differ significantly from natural images. Compared to natural images, medical images are more complex, containing detailed anatomical structures, lesion signs, and clinical features. These factors require deep learning models to possess specialized medical knowledge, making pre-trained models based on natural images unsuitable for processing medical images. Furthermore, the scarcity of medical data is a major challenge, as there are no publicly available large-scale medical image datasets available for model pre-training. Therefore, how to efficiently utilize medical image data to build high-quality pre-trained models has always been a hot research topic. Due to the low radiation dose and lack of contrast agents in NCCT, its image resolution is relatively low, making the anatomical features of blood vessels and surrounding tissues less apparent, especially subtle coronary artery lesions, which are difficult for the human eye to accurately identify. This problem presents another major challenge for deep learning models, as models struggle to directly capture the detailed anatomical features from NCCT. Summary of the Invention

[0005] This invention provides a method, application, device, and storage medium for medical image classification using low-dose CT to address the technical problems existing in the prior art.

[0006] The technical solution adopted by this invention to solve the technical problems existing in the prior art is as follows:

[0007] A medical image classification method for low-dose CT, characterized by comprising the following steps:

[0008] Step 1: Collect ICA, CCTA, and NCCT images of the same part of the human body; use the ICA images as annotation reference images to annotate the CCTA and NCCT images;

[0009] Step 2: Construct a pre-trained model using data augmentation and self-supervised learning methods;

[0010] Step 3: Construct a two-stream network architecture for the teacher-student network. The teacher network is used to perform the classification task of CCTA images, and the student network is used to perform the classification task of NCCT images. Initialize the teacher network and the student network respectively using the parameters of the pre-trained model.

[0011] Step 4: Input CCTA images from the same human body into the teacher network and NCCT images from the same human body into the student network to train the teacher-student network, and transfer knowledge from the teacher network to the student network through distillation loss.

[0012] Step 5: After the teacher-student network training is completed, remove the teacher network and use only the student network to classify the image data; input the NCCT images into the student network to obtain the classification of the NCCT images.

[0013] Furthermore, in step 2, the original CCTA image is first augmented with data. Then, based on the original CCTA image and the data obtained after data augmentation, a pre-trained model is constructed by performing three proxy tasks: occlusion volume restoration, image rotation angle prediction, and contrast encoding.

[0014] Furthermore, the method for performing data augmentation on the original CCTA imagery includes the following steps:

[0015] Step A1: Construct a data augmentation operation space; this space includes the following data augmentation operation types: brightness, contrast, tonal distribution, sharpness, Gaussian blur, Gaussian noise, rotation, horizontal flip, vertical flip, scaling, X-axis translation, Y-axis translation, X-axis clipping, and Y-axis clipping.

[0016] Step A2: Randomly sample M data augmentation operations from the constructed data augmentation operation space, where M takes the value {2,3};

[0017] Step A3: Shuffle the order of the selected M data augmentation operations;

[0018] Step A4: Randomly generate operation magnitudes and probabilities for M data augmentation operations;

[0019] Step A5: Apply the final data augmentation operations sequentially to the original CCTA image;

[0020] Step A6: Perform the above steps 5 times to obtain a data-enhanced image that is 5 times larger than the original medical image.

[0021] Furthermore, a method for performing self-supervised learning on three proxy tasks—occlusion volume restoration, image rotation angle prediction, and contrastive encoding—includes the following steps:

[0022] Step B1: Based on the region of interest in anatomy, crop the input CCTA image so that the cropped sub-volume contains the object to be analyzed and its surrounding related tissues.

[0023] Step B2: Randomly occlude and rotate the obtained sub-volume. The rotation angles include 0°, 90°, 180°, and 270°.

[0024] Step B3: The obtained sub-volume is divided into a sequence of image patches, and the dimensionality transformation of the data is as follows:

[0025]

[0026] Where N = H × W × D / P 3 ;

[0027] In the formula:

[0028] X is a three-dimensional subvolume;

[0029] X P It is a sequence of image patches;

[0030] H, W, and D correspond to the resolution of the three dimensions of the sub-volume;

[0031] P,P,P corresponds to the resolution of each image patch in three dimensions;

[0032] C represents the number of channels;

[0033] N is the number of image patches;

[0034] Step B4: Add positional encoding to the image patch sequence, and then input it into the feature extractor;

[0035] Step B5 involves mining anatomical features by simultaneously performing occlusion volume restoration, image rotation angle prediction, and contrastive encoding tasks.

[0036] Furthermore, in step B5, when performing the occlusion volume repair task, the occlusion volume is randomly occluded, and then a transposed convolutional layer is appended after the feature extractor as a reconstruction head, and the output of the reconstruction head is defined as... Reconstruction loss L inpaint The calculation formula is as follows:

[0037]

[0038] When performing an image rotation angle prediction task, the sub-volume is randomly rotated by a certain angle category, and then a fully connected layer is added after the feature extractor to predict the rotation category; the classification loss function adopts cross-entropy loss, and the classification loss L rotation The calculation formula is as follows:

[0039]

[0040] In the formula:

[0041] The predicted category is the r-th class;

[0042] y r For the r-th real category;

[0043] r is the index of the category;

[0044] In performing the contrastive encoding task, a linear mapping layer is appended after the feature extractor to obtain the representation of the sub-volume in the feature space v. Contrastive encoding learns better feature representations by bringing augmented samples from the same sub-volume closer in the feature space v, while simultaneously pushing augmented samples from different sub-volumes further apart. Augmented samples are obtained by rotating the original samples by different angles. Augmented samples from the same class are called positive samples, and augmented samples from different classes are called negative samples. An equal number of positive and negative samples are selected to construct positive-negative sample pairs. Cosine similarity is used as the distance measurement for the encoded representation, and the contrastive encoding loss L... contrast The calculation formula is as follows:

[0045]

[0046] In the formula:

[0047] v i Let i be the feature space of the i-th sub-volume;

[0048] v j Let j be the feature space of the j-th sub-volume;

[0049] v m Let m be the feature space of the m-th sub-volume;

[0050] i, j, and m are the sub-volume indices;

[0051] t is a measure of the normalized temperature scale;

[0052] u is an indicator function whose value is 1 when m is not equal to i;

[0053] sim is the cosine similarity;

[0054] Q is the number of positive and negative sample pairs;

[0055] When performing the above three tasks simultaneously to pre-train the feature extractor, the total loss is calculated as follows:

[0056] L total =λ1L inpaint +λ2L rotation +λ3L contrast ;

[0057] In the formula:

[0058] λ1, λ2, and λ3 correspond to the weights of the loss function when performing the occlusion volume restoration task, the image rotation angle prediction task, and the contrastive coding task, respectively.

[0059] L total The total loss is calculated when simultaneously performing the occlusion volume restoration task, the image rotation angle prediction task, and the contrastive encoding task.

[0060] Furthermore, in step 4, when training the teacher network, firstly, CCTA images are selected and segmented into image patch sequences; then, these sequences are input into a feature extractor to extract features; finally, the extracted features are subjected to binary classification through three fully connected layers; the teacher network training loss L... tcls The calculation formula is as follows:

[0061]

[0062] In the formula:

[0063] g e The label for Class e CCTA images;

[0064] The prediction results of the teacher network for the e-th type CCTA image;

[0065] e represents the category index of the CCTA image;

[0066] When training the student network, firstly, NCCT images are selected and segmented into image patch sequences; then, these sequences are input into a feature extractor to extract features; finally, the extracted features are subjected to binary classification through three fully connected layers; the student network training loss L... scls The calculation formula is as follows:

[0067]

[0068] In the formula:

[0069] h f The label for NCCT images of type f;

[0070] The prediction results of the student network for the f-th type NCCT image;

[0071] f is the category index of NCCT images.

[0072] Furthermore, in step 4, the method of transferring knowledge from the teacher network to the student network through distillation loss includes the following steps:

[0073] Step C1: Take the absolute values ​​of the feature maps output by the feature extractors at each stage in the teacher network and student network, and sum them up according to the channel dimension.

[0074]

[0075]

[0076] In the formula:

[0077] This is the feature map output by the k-th stage of the feature extractor in the teacher network.

[0078] The feature map output by the k-th stage of the feature extractor in the student network;

[0079] The attention map is obtained by summing the absolute values ​​of the feature maps output by the k-th stage of the feature extractor in the teacher network according to the channel dimension.

[0080] The attention map is obtained by summing the absolute values ​​of the feature maps output by the k-th stage of the feature extractor in the student network according to the channel dimension.

[0081] The feature map of the p-th channel output by the k-th stage of the feature extractor of the teacher network;

[0082] The feature map of the p-th channel output by the k-th stage of the feature extractor for the student network;

[0083] p is the index of the channel number;

[0084] k is the index of the feature extractor stage number;

[0085] C represents the number of channels;

[0086] φ(·) represents taking the absolute values ​​of the input feature maps along the channel dimension and summing them;

[0087] Step C2, will and Vectorization is performed, and the distillation loss is calculated using L2 loss. Let the distillation loss function be L. d L d The calculation formula is as follows:

[0088]

[0089] In the formula: vec(·) represents vectorizing the feature map;

[0090] Step C3, let the total loss of the teacher-student network be L, and the formula for calculating L is as follows:

[0091] L = L tcls +L scls +λd L d ;

[0092] In the formula: λ d This represents the weight of distillation losses.

[0093] The present invention also provides an application of the medical image classification method of low-dose CT as described above, which is applied to the classification of stenosis and occlusion in coronary artery images.

[0094] The present invention also provides a medical image classification device based on low-dose CT, including a memory and a processor. The memory is used to store a computer program; the processor is used to execute the computer program and, when executing the computer program, to implement the steps of the low-dose CT medical image classification method as described above.

[0095] The present invention also provides a storage medium storing a computer program that, when executed by a processor, implements the steps of the low-dose CT medical image classification method described above.

[0096] The advantages and positive effects of this invention are:

[0097] This invention optimizes data augmentation techniques to better adapt to the characteristics of medical imaging and constructs a pre-trained model of medical images through self-supervised learning. Building upon this, based on the concept of knowledge distillation in transfer learning, this invention transfers anatomical knowledge learned by the teacher network from CCTA to the student network, guiding it to more accurately capture lesion features in NCCT. This screening method based on low-dose CT not only reduces the pain and financial burden on examinees but also provides doctors with effective auxiliary information. This invention addresses the problems of scarce medical data and the lack of publicly available high-quality pre-trained models through optimized data augmentation operations and a self-supervised learning approach. Furthermore, based on the concept of knowledge distillation, this invention solves the problem of difficulty in capturing lesion features due to the low resolution of NCCT. Attached Figure Description

[0098] Figure 1 This is a flowchart of a low-dose CT medical image classification method according to the present invention.

[0099] Figure 2 This is an optimized data augmentation operation workflow diagram of the present invention.

[0100] Figure 3 This is a flowchart of a self-supervised learning process according to the present invention.

[0101] Figure 4 This is a teacher-student network architecture diagram of the present invention.

[0102] In the picture:

[0103] Prediction T This represents the prediction results of the teacher network.

[0104] Prediction S This represents the predicted results of the student network. Detailed Implementation

[0105] The present invention will now be described in detail with reference to the accompanying drawings and embodiments. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0106] The Chinese translations of the English words in this application are as follows:

[0107] ICA: Coronary angiography.

[0108] CCTA: Coronary CT angiography.

[0109] NCCT: Non-enhanced chest CT.

[0110] ID: Examinee's ID number.

[0111] Label: Marking.

[0112] Swin Transformer: A feature extractor.

[0113] Masked Volume Inpainting: Repairing masked volumes.

[0114] Image Rotation Prediction: Predicting the rotation angle of an image.

[0115] Contrastive Coding: Comparative coding.

[0116] ImageNet: An image recognition database.

[0117] patch: an image block.

[0118] stage: phase or stage.

[0119] Patch Partition: Divide an image into patches.

[0120] Rotation: to rotate.

[0121] Contrastive: comparison.

[0122] Inpainting: Repairing.

[0123] Please see Figures 1 to 4 A medical image classification method for low-dose CT includes the following steps:

[0124] Step 1: Collect ICA, CCTA, and NCCT images of the same part of the human body; use the ICA images as annotation reference images to annotate the CCTA and NCCT images.

[0125] Step 2: Construct a pre-trained model using data augmentation and self-supervised learning methods.

[0126] Step 3: Construct a two-stream network architecture with a teacher-student network. The teacher network performs the classification task for CCTA images, and the student network performs the classification task for NCCT images. Initialize the teacher and student networks using the parameters of the pre-trained model.

[0127] Step 4: Input CCTA images from the same human body into the teacher network, and simultaneously input NCCT images from the same human body into the student network to train the teacher-student network. Then, transfer knowledge from the teacher network to the student network through distillation loss.

[0128] Step 5: After the teacher-student network training is complete, remove the teacher network and use only the student network to classify the image data. Input the NCCT images into the student network to obtain the classification of the NCCT images.

[0129] Preferably, in step 2, the original CCTA image is first augmented with data. Then, based on the original CCTA image and the data obtained after data augmentation, a pre-trained model can be constructed by performing three proxy tasks: occlusion volume repair, image rotation angle prediction, and contrast encoding.

[0130] Preferably, the method for performing data augmentation on raw CCTA images may include the following steps:

[0131] Step A1: Construct a data augmentation operation space; this space may include the following data augmentation operation types: brightness, contrast, tonal distribution, sharpness, Gaussian blur, Gaussian noise, rotation, horizontal flip, vertical flip, scaling, X-axis translation, Y-axis translation, X-axis clipping, and Y-axis clipping.

[0132] Step A2 involves randomly sampling M data augmentation operations from the constructed data augmentation operation space, where M can take the value {2,3}.

[0133] Step A3 can be used to shuffle the order of the selected M data enhancement operations.

[0134] Step A4 can randomly generate operation magnitudes and probabilities for M data augmentation operations.

[0135] Step A5: Apply the final data augmentation operations sequentially to the original CCTA image.

[0136] Step A6 can be performed 3-10 times, preferably 5 times, to obtain a data-enhanced image that is 5 times larger than the original medical image.

[0137] Preferably, the method for performing self-supervised learning of three proxy tasks—occlusion volume restoration, image rotation angle prediction, and contrastive encoding—may include the following steps:

[0138] Step B1 involves cropping the input CCTA image based on the region of interest in anatomy, so that the cropped sub-volume contains the object to be analyzed and its surrounding related tissues.

[0139] Step B2 involves randomly occluding and rotating the obtained sub-volume. The rotation angle can be 0°, 90°, 180°, or 270°.

[0140] Step B3: Divide the obtained sub-volume into a sequence of image patches. The dimensionality transformation of the data can be as follows:

[0141]

[0142] Where N = H × W × D / P 3 ;

[0143] In the formula:

[0144] X is a three-dimensional subvolume;

[0145] X P It is a sequence of image patches;

[0146] H, W, and D correspond to the resolution of the three dimensions of the sub-volume;

[0147] P,P,P corresponds to the resolution of each image patch in three dimensions;

[0148] C represents the number of channels;

[0149] N is the number of image patches.

[0150] Step B4 involves adding positional encoding to the image patch sequence, which is then input into the feature extractor.

[0151] Step B5 involves mining anatomical features by simultaneously performing occlusion volume restoration, image rotation angle prediction, and contrastive encoding tasks.

[0152] Preferably, in step B5, when performing the occlusion volume repair task, the occlusion volume can be randomly occluded, and then a transposed convolutional layer can be appended after the feature extractor as a reconstruction head, and the output of the reconstruction head is defined as... Reconstruction loss L inpaint The calculation formula is as follows:

[0153]

[0154] When performing an image rotation angle prediction task, a sub-volume can be randomly rotated by a certain angle category. Then, a fully connected layer can be attached after the feature extractor to predict the rotation category. The classification loss function can be cross-entropy loss, with a classification loss L... rotation The calculation formula is as follows:

[0155]

[0156] In the formula:

[0157] The predicted category is the r-th class;

[0158] y r For the r-th real category;

[0159] r is the index of the category.

[0160] When performing contrastive encoding, a linear mapping layer can be attached after the feature extractor to obtain the representation of the sub-volume in the feature space v. Contrastive encoding learns better feature representations by bringing augmented samples from the same sub-volume closer in the feature space v, while simultaneously pushing augmented samples from different sub-volumes further apart. Augmented samples can be obtained by rotating the original samples by different angles. Augmented samples from the same class are called positive samples, and augmented samples from different classes are called negative samples. An equal number of positive and negative samples can be selected to construct positive-negative sample pairs. Cosine similarity can be used as the distance measurement for the encoded representation. The contrastive encoding loss L... contrast The calculation formula is as follows:

[0161]

[0162] In the formula:

[0163] v i Let i be the feature space of the i-th sub-volume;

[0164] v j Let j be the feature space of the j-th sub-volume;

[0165] v m Let m be the feature space of the m-th sub-volume;

[0166] i, j, and m are the sub-volume indices;

[0167] t is a measure of the normalized temperature scale;

[0168] u is an indicator function whose value is 1 when m is not equal to i;

[0169] sim is the cosine similarity;

[0170] Q represents the number of positive and negative sample pairs.

[0171] When performing the above three tasks simultaneously to pre-train the feature extractor, the total loss can be calculated as follows:

[0172] L total =λ1L inpaint +λ2L rotation +λ3L contrast ;

[0173] In the formula:

[0174] λ1, λ2, and λ3 correspond to the weights of the loss function when performing the occlusion volume restoration task, the image rotation angle prediction task, and the contrastive coding task, respectively.

[0175] L total The total loss is calculated when simultaneously performing the occlusion volume restoration task, the image rotation angle prediction task, and the contrastive encoding task.

[0176] Preferably, in step 4, when training the teacher network, firstly, CCTA images are selected and can be segmented into image patch sequences; then, these are input into a feature extractor to extract features; finally, the extracted features can be binary-classified through three fully connected layers; the teacher network training loss L tcls The calculation formula is as follows:

[0177]

[0178] In the formula:

[0179] g e The label for Class e CCTA images;

[0180] The prediction results of the teacher network for the e-th type CCTA image;

[0181] e represents the category index of the CCTA image.

[0182] When training the student network, firstly, NCCT images are selected and segmented into image patch sequences; then, these sequences are input into a feature extractor to extract features; finally, the extracted features are subjected to binary classification through three fully connected layers; the student network training loss L...scls The calculation formula is as follows:

[0183]

[0184] In the formula:

[0185] h f The label for NCCT images of type f;

[0186] The prediction results of the student network for the f-th type NCCT image;

[0187] f is the category index of NCCT images.

[0188] Preferably, in step 4, the method of transferring knowledge from the teacher network to the student network through distillation loss may include the following steps:

[0189] Step C1 involves taking the absolute values ​​of the feature maps output by each stage of the feature extractor in both the teacher and student networks, along with their respective channel dimensions, and then summing them together.

[0190]

[0191]

[0192] In the formula:

[0193] This is the feature map output by the k-th stage of the feature extractor in the teacher network.

[0194] The feature map output by the k-th stage of the feature extractor in the student network;

[0195] The attention map is obtained by summing the absolute values ​​of the feature maps output by the k-th stage of the feature extractor in the teacher network according to the channel dimension.

[0196] The attention map is obtained by summing the absolute values ​​of the feature maps output by the k-th stage of the feature extractor in the student network according to the channel dimension.

[0197] The feature map of the p-th channel output by the k-th stage of the feature extractor of the teacher network;

[0198] The feature map of the p-th channel output by the k-th stage of the feature extractor for the student network;

[0199] p is the index of the channel number;

[0200] k is the index of the feature extractor stage number;

[0201] C represents the number of channels;

[0202] φ(·) represents taking the absolute values ​​of the input feature maps along the channel dimension and adding them together.

[0203] Step C2 can and Vectorization is performed, and the distillation loss is calculated using L2 loss. Let the distillation loss function be L. d L d The calculation formula is as follows:

[0204]

[0205] In the formula: vec(·) represents vectorizing the feature map.

[0206] Step C3: Let the total loss of the teacher-student network be L. The formula for calculating L is as follows:

[0207] L = L tcls +L scls +λ d L d ;

[0208] In the formula: λ d This represents the weight of distillation losses.

[0209] The present invention also provides an application of the medical image classification method of low-dose CT as described above, which is applied to the classification of stenosis and occlusion in coronary artery images.

[0210] The present invention also provides a medical image classification device based on low-dose CT, including a memory and a processor. The memory is used to store a computer program; the processor is used to execute the computer program and, when executing the computer program, to implement the steps of the low-dose CT medical image classification method as described above.

[0211] The present invention also provides a storage medium storing a computer program that, when executed by a processor, implements the steps of the low-dose CT medical image classification method described above.

[0212] Feature extractors in the teacher and student networks can employ RNNs, CNNs, or Transformers. The Swing Transformer is preferred among Transformers. RNNs, CNNs, and Transformers are commonly used feature extractors.

[0213] The working principle of the present invention will be further explained below with reference to a preferred embodiment:

[0214] This invention uses the Swing Transformer as the feature extractor for both the teacher and student networks. Anatomical features learned by the teacher network are transferred to the student network through distillation learning. After network training is complete, the teacher network is removed, and only the student network is used. By inputting the NCCT images of the examinee, the extent of coronary artery lesions in the examinee can be determined. The specific process is as follows:

[0215] 1. Construct and initialize a two-stream network architecture for the teacher-student network. Both the teacher and student networks use the Swin Transformer as the feature extractor, followed by a fully connected layer as the classification head. After completing the network construction, initialize the Swin Transformer using pre-trained weights obtained through self-supervised learning.

[0216] 2. The teacher network performs a CCTA classification task. First, CCTA images are selected based on the examinee's ID and segmented into a sequence of image patches. Then, these patches are input into a Swin Transformer to extract features. Finally, the extracted features are processed through three fully connected layers for binary classification to determine the risk of coronary heart disease. The loss function for this task is as follows:

[0217]

[0218] In the formula:

[0219] g e The label for Class e CCTA images;

[0220] The prediction results of the teacher network for the e-th type CCTA image;

[0221] e represents the category index of the CCTA image.

[0222] 3. The student network performs an NCCT classification task. First, NCCT images are selected based on the examinee's ID and segmented into a sequence of image patches. Then, these patches are input into a Swin Transformer to extract features. Finally, the extracted features are processed through three fully connected layers for binary classification to determine the risk of coronary heart disease. The loss function for this task is as follows:

[0223]

[0224] In the formula:

[0225] h f The label for NCCT images of type f;

[0226] The prediction results of the student network for the f-th type NCCT image;

[0227] f is the category index of NCCT images.

[0228] 4. The teacher network and student network are trained synchronously. Anatomical features learned from CCTA are transferred to the student network's feature extractor via distillation learning, guiding the student network to learn lesion features in NCCT. This invention uses the Swin Transformer feature extractor, which has four stages. In each stage, the network calculates the distillation loss. First, the absolute values ​​of the feature maps output from each stage are summed along the channel dimension, then vectorized. Finally, the distillation loss is calculated based on L2 loss. The specific process is as follows:

[0229] 4.1 Take the absolute values ​​of the feature maps output by the Swin Transformer feature extractor at each stage in the teacher network and student network, and sum them up according to the channel dimension. That is:

[0230]

[0231]

[0232] In the formula:

[0233] This is the feature map output by the k-th stage of the feature extractor in the teacher network.

[0234] The feature map output by the k-th stage of the feature extractor in the student network;

[0235] The attention map is obtained by summing the absolute values ​​of the feature maps output by the k-th stage of the feature extractor in the teacher network according to the channel dimension.

[0236] The attention map is obtained by summing the absolute values ​​of the feature maps output by the k-th stage of the feature extractor in the student network according to the channel dimension.

[0237] The feature map of the p-th channel output by the k-th stage of the feature extractor of the teacher network;

[0238] The feature map of the p-th channel output by the k-th stage of the feature extractor for the student network;

[0239] p is the index of the channel number;

[0240] k is the index of the feature extractor stage number;

[0241] C represents the number of channels;

[0242] φ(·) represents taking the absolute values ​​of the input feature maps along the channel dimension and adding them together.

[0243] 4.2 will and Vectorization is performed, and the distillation loss is calculated using L2 loss. Let the distillation loss function be L. d L d The calculation formula is as follows:

[0244]

[0245] In the formula: vec(·) represents vectorizing the feature map;

[0246] Let L be the total loss of the teacher-student network. The formula for calculating L is as follows:

[0247] L = L tcls +L scls +λ d L d ;

[0248] In the formula: λ d This represents the weight of distillation losses.

[0249] 5. After the network training is completed, the risk of coronary heart disease in the examinee can be obtained by using only the student network in the two-stream network structure and inputting the examinee's NCCT image.

[0250] Given the significant differences between medical images and natural images, pre-trained models based on traditional natural images are not suitable for medical image processing. Furthermore, due to the scarcity of medical data, publicly available large-scale medical image datasets are relatively limited. Therefore, this invention optimizes data augmentation techniques to better adapt to the characteristics of medical images and constructs a pre-trained model for medical images through self-supervised learning.

[0251] Traditional data augmentation operations such as inverting, equalizing, and solarizing, as well as repeated data augmentation operations, can destroy details and features in medical images, resulting in unrealistic medical images. Therefore, this invention filters data augmentation operations and ultimately constructs a data augmentation space DA = {brightness, contrast, posterize, sharpness, Gaussian blur, Gaussian noise, rotate, horizontal flip, vertical flip, scale, X-axis translation, Y-axis translation, X-axis shear, Y-axis shear}. Furthermore, to ensure image realism and maintain the diversity of image augmentation, this invention sets the maximum number of consecutive applications of augmentation operations to 3.

[0252] Because the features in medical images are extremely subtle and difficult to capture, this invention designs three proxy tasks for self-supervised learning to comprehensively mine the anatomical features, including occlusion volume restoration, image rotation angle prediction, and contrastive encoding. The occlusion volume restoration task focuses on repairing blood vessels and surrounding tissues. By attempting to restore this information, the model can more accurately capture the anatomical features of blood vessels and their surrounding tissues, features closely related to the pathogenesis of coronary heart disease. On the other hand, the contrastive encoding task aims to bring similar samples closer together in the embedding space, while separating dissimilar samples. This process helps the model more sensitively capture more discriminative details, improving the model's ability to extract and distinguish important information in images.

[0253] Because NCCT images have relatively low resolution, they do not clearly display the anatomical features of blood vessels and surrounding tissues, especially subtle coronary artery lesions. This makes it difficult to identify the lesion features contained within. However, CCTA images have relatively high resolution, making anatomical features easier to observe, and they share anatomical similarities with NCCT. This invention is based on the knowledge distillation concept in transfer learning, constructing a two-stream network architecture of teacher-student networks. The teacher network learns the anatomical features from CCTA and transfers them to the student network through knowledge distillation, thereby guiding the student network to better learn the lesion features from NCCT. This network architecture significantly reduces the difficulty for the model to capture lesion features from NCCT.

[0254] The aforementioned two-stream network architecture of the teacher-student network can adopt applicable teacher-student networks in the existing technology, or it can adopt neural networks in the existing technology and build it using conventional technical means.

[0255] In summary, this invention alleviates the problems of scarce medical data and the lack of publicly available high-quality pre-trained models through optimized data augmentation operations and a self-supervised learning approach. Building upon this foundation, this invention, based on the concept of knowledge distillation, addresses the difficulty in capturing lesion features due to the low resolution of NCCT.

[0256] The embodiments described above are only used to illustrate the technical ideas and features of the present invention. Their purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The patent scope of the present invention should not be limited by these embodiments. That is, any equivalent changes or modifications made in accordance with the spirit disclosed in the present invention still fall within the patent scope of the present invention.

Claims

1. A medical image classification method of low-dose CT, characterized by, Includes the following steps: Step 1: Collect ICA, CCTA, and NCCT images of the same part of the human body; use the ICA images as annotation reference images to annotate the CCTA and NCCT images; Step 2: Construct a pre-trained model using data augmentation and self-supervised learning methods; Step 3: Construct a two-stream network architecture for the teacher-student network. The teacher network is used to perform the classification task of CCTA images, and the student network is used to perform the classification task of NCCT images. Initialize the teacher network and the student network respectively using the parameters of the pre-trained model. Step 4: Input CCTA images from the same human body into the teacher network and NCCT images from the same human body into the student network to train the teacher-student network, and transfer knowledge from the teacher network to the student network through distillation loss. Step 5: After the teacher-student network training is completed, remove the teacher network and use only the student network to classify the image data; input the NCCT images into the student network to obtain the classification of the NCCT images; In step 2, the original CCTA image is first augmented with data. Then, based on the original CCTA image and the data obtained after augmentation, a pre-trained model is built by performing three proxy tasks: occlusion volume repair, image rotation angle prediction, and contrast encoding. A method for performing self-supervised learning on three proxy tasks—occlusion volume restoration, image rotation angle prediction, and contrastive encoding—includes the following steps: Step B1: Based on the region of interest in anatomy, crop the input CCTA image so that the cropped sub-volume contains the object to be analyzed and its surrounding related tissues. Step B2: Randomly occlude and rotate the obtained sub-volume. The rotation angles include 0°, 90°, 180°, and 270°. Step B3: The obtained sub-volume is divided into a sequence of image patches, and the dimensionality transformation of the data is as follows: ； wherein ; In the formula: X is a three-dimensional subvolume; X P is a sequence of image blocks; corresponding to the resolution of the three dimensions of the sub-volume; corresponding to the resolution of three dimensions for each image block; N is the number of channels; is the number of image blocks; Step B4: Add positional encoding to the image patch sequence, and then input it into the feature extractor; Step B5 involves mining anatomical features by simultaneously performing occlusion volume restoration, image rotation angle prediction, and contrastive encoding tasks. In step B5, when performing the occlusion volume repair task, the sub-volume is randomly occluded, then a transpose convolution layer is attached behind the feature extractor as a reconstruction head, and the reconstruction head output is defined as ; reconstruction loss The calculation formula is as follows: ； In performing the image rotation angle prediction task, the sub-volume is randomly rotated by a certain angle category, and then a fully connected layer is attached after the feature extractor to predict the rotation category; the classification loss function adopts cross-entropy loss, and the classification loss The calculation formula is as follows: ； In the formula: is the rth prediction class; For the r-th real category; r is the index of the category; When performing a contrastive encoding task, a linear mapping layer is appended after the feature extractor to obtain the subvolume in the feature space. The representation in the feature space; contrastive coding works by combining augmented samples from the same subvolume in the feature space. The distance is shortened, while the augmented samples from different sub-volumes are brought closer in the feature space. The distance is increased to learn better feature representations; augmented samples are obtained by rotating the original samples by different angles; augmented samples from the same class are called positive samples, and augmented samples from different classes are called negative samples; positive and negative sample pairs are constructed by selecting the same number of positive and negative samples; cosine similarity is used as the distance measured in the encoding representation, and the encoding loss is compared. The calculation formula is as follows: ； In the formula: characteristic space for the i-th sub-volume; characteristic space for the jth sub-volume; characteristic space for the mth sub-volume; i, j, and m are the sub-volume indices; is a measure of the normalized temperature scale; u is an indicator function that has the value 1 when is not equal to 1. is the cosine similarity; Q is the number of positive and negative sample pairs; When performing the above three tasks simultaneously to pre-train the feature extractor, the total loss is calculated as follows: ； In the formula: , and correspond to the weights of the loss functions when performing the occlusion volume inpainting task, the image rotation angle prediction task, and the contrastive coding task, respectively. Ltotal is the total loss for performing the occlusion volume inpainting task, the image rotation angle prediction task, and the contrastive coding task simultaneously. 2.The method of claim 1, wherein, The method for performing data augmentation on raw CCTA images includes the following steps: Step A1: Construct a data augmentation operation space; this space includes the following data augmentation operation types: brightness, contrast, tonal distribution, sharpness, Gaussian blur, Gaussian noise, rotation, horizontal flip, vertical flip, scaling, X-axis translation, Y-axis translation, X-axis clipping, and Y-axis clipping. Step A2: Randomly sample M data augmentation operations from the constructed data augmentation operation space, where M takes the value {2,3}; Step A3: Shuffle the order of the selected M data augmentation operations; Step A4: Randomly generate operation magnitudes and probabilities for M data augmentation operations; Step A5: Apply the final data augmentation operations sequentially to the original CCTA image; Step A6: Perform the above steps 5 times to obtain a data-enhanced image that is 5 times larger than the original medical image. 3.The medical image classification method of low-dose CT according to claim 1, characterized in that, In step 4, when training the teacher network, firstly, CCTA images are selected and segmented into image patch sequences; then, these are input into a feature extractor to extract features; finally, the extracted features are subjected to binary classification through three fully connected layers; the teacher network training loss... The calculation formula is as follows: ； In the formula: Label for Class e CCTA images; pred e is the prediction result of the teacher network for the e-th class of the CCTA image; e represents the category index of the CCTA image; When training the student network, first, the NCCT image is selected and cut into an image block sequence; then, the image block sequence is input into the feature extractor to extract features; finally, the extracted features are classified by three fully connected layers; and a calculation formula of a training loss of the student network is as follows: ​ ； In the formula: Label for class f NCCT image; Student network prediction of class f NCCT image; f is the category index of NCCT images. 4.The method of claim 3, wherein, Step 4, the method of transferring knowledge from the teacher network to the student network through distillation loss, includes the following steps: Step C1: Take the absolute values ​​of the feature maps output by the feature extractors at each stage in the teacher network and student network, and sum them up according to the channel dimension. ； ； In the formula: the feature map outputted by the feature extractor in the teacher network in the first stage the feature map outputted by the feature extractor in the teacher network in the first stage the feature extractor in the student network for the feature map output by the stage The feature map output by the feature extractor in the teacher network is taken as an input of the attention mechanism. The attention map is obtained by taking the absolute value and summing up the channel dimension of the feature map output in the stage. The feature extractor in the student network is trained by The attention map is obtained by taking the absolute value and summing up the feature maps in the channel dimension output in the stage. a feature map of the p-th channel output by the feature extractor of the k-th stage for the teacher network; a feature map of the p-th channel output by the feature extractor for the k-th stage of the student network; p is the index of the channel number; k is the index of the feature extractor stage number; N is the number of channels; represents taking the absolute value of the input feature map along the channel dimension and adding them up; Step C2, to and Vectorization, and by Loss calculation distillation loss, set distillation loss function as , The calculation formula is as follows: ； In the formula: denotes vectorization of the feature map; Step C3, let the total loss of the teacher-student network be L, and the formula for calculating L is as follows: ； In the formula: is the weight of the distillation loss.

5. Use of a medical image classification method of low-dose CT according to any one of claims 1 to 4, characterized in that, This method is applied to the classification of stenosis and occlusion in coronary artery imaging.

6. A medical image classification device based on low-dose CT, comprising a memory and a processor, characterized in that, The memory is used to store a computer program; the processor is used to execute the computer program and, when executing the computer program, to implement the steps of the low-dose CT medical image classification method as described in any one of claims 1 to 4.

7. A storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the medical image classification method for low-dose CT as described in any one of claims 1 to 4.

Images