Lung ct abnormality detection method based on multi-scale cropping and self-supervised reconstruction

Abstract

This invention relates to a lung CT anomaly detection method based on multi-scale cropping and self-supervised reconstruction. First, a self-supervised enhancement strategy, CropMixPaste, suitable for lung CT scan images, is established to generate anomalies resembling ground-glass opacities, thereby encouraging the model to detect local irregularities in lung CT scan images. Then, a Simple Mask Attention Prediction Block (SMAPB) is inserted into the convolutional network, utilizing this self-supervised reconstruction module to predict masking information to refine local features. Finally, an anomaly detector is constructed using representations learned in the self-supervised agent task. Experimental results on a real lung CT scan dataset demonstrate the effectiveness and superiority of the proposed method.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to a method for detecting lung CT abnormalities based on multi-scale cropping and self-supervised reconstruction. Background Technology

[0002] Computed tomography (CT) scans of the lungs are an effective imaging method that helps clinicians quickly identify areas of lung infection. Studies have shown that CT scans have become one of the important tools for assisting in the diagnosis and treatment of viral pneumonia. A prominent feature on chest CT images is ground-glass opacity (GGO), which in severe cases can appear as a "crazy paving" pattern. This is an abnormal CT sign that can occur in various lung diseases and is divided into two main categories: localized and diffuse. Therefore, developing computer-aided diagnostic systems to locate abnormal areas and assist in discovering abnormalities that might be missed due to physician overwork or inattention can help improve the efficiency and quality of diagnosis for patients with viral pneumonia and prevent the widespread and rapid spread of the disease.

[0003] Currently, various deep learning methods have made significant progress in CT scan image analysis. However, supervised deep learning is limited by the availability of datasets and computational resources. In most cases, the number of lung CT images is limited, and the resolution is low. Furthermore, the time-consuming and expensive manual annotation of CT scan images requires the knowledge of medical experts. Therefore, unsupervised anomaly detection methods without additional labels have been a focus of research.

[0004] Unlike typical supervised classification problems, anomaly detection in lung CT scans faces two challenges. First, the unknowability of anomalies makes it difficult to obtain a wide range of anomaly data (e.g., rare lesions). Second, the differences between normal and anomalous images in medical images are more subtle than those in natural datasets, requiring models to be sensitive enough to recognize both subtle differences and salient features.

[0005] In recent years, scholars both domestically and internationally have conducted a series of studies on anomaly detection in medical images. These methods can generally be categorized into three types: reconstruction-based methods, feature similarity-based methods, and self-supervised learning-based methods. Reconstruction-based methods train encoder networks (AEs), generative models (GANs), or normalized flows to reconstruct similar samples. Since the model only obtains the feature distribution of normal samples, the difference between the generated or reconstructed sample and the input is considered anomaly region. Feature similarity-based methods obtain anomaly scores by calculating the similarity between the feature vectors of normal training samples and test samples. Common similarity settings include Gaussian distribution, representative memory databases, and nearest neighbor algorithms. Simultaneously, some methods also adjust the features of training samples through techniques such as feature coupling, early stopping, and distillation to obtain more discriminative normal features. Recently, self-supervised learning-based methods have demonstrated excellent performance. These methods learn better feature representations by setting surrogate tasks, such as predicting geometric transformations, predicting rotations, and predicting data augmentation. Thanks to the targeted settings of surrogate tasks, these methods have been widely applied in auxiliary diagnostic techniques such as MRI of the brain, CT of the lungs, CT of the abdomen, chest X-ray, and OCT of the eye, demonstrating significant application value. Therefore, the detection of abnormalities on lung CT aims to distinguish abnormal images from lung CT scans and locate abnormal areas of ground-glass opacities (GGO), such as... Figure 1 As shown. Summary of the Invention

[0006] The purpose of this invention is to address the shortcomings of existing lung CT anomaly detection methods based on self-supervised learning. Therefore, this invention provides a lung CT anomaly detection method based on multi-scale cropping and self-supervised reconstruction to improve the anomaly detection performance of lung CT scan images.

[0007] To achieve the above objectives, the technical solution of this invention is: a lung CT anomaly detection method based on multi-scale cropping and self-supervised reconstruction. First, a self-supervised enhancement strategy, CropMixPaste, suitable for lung CT scan images is set to generate anomalies similar to ground-glass opacities, encouraging the model to detect local irregularities in lung CT scan images. Then, a simple masking attention prediction block (SMAPB) is inserted into a convolutional network for self-supervised learning to predict masking information to refine local features. Finally, the representations learned through self-supervised learning are used to construct an anomaly detector.

[0008] In one embodiment of the present invention, given a set of normal training samples x1, x2…x N ∈x train The purpose of this method is to distinguish between normal and abnormal test samples; the anomaly detection problem is defined as follows:

[0009] For test set Y testy1,y2…y N Each sample is associated with d-dimensional features. The goal is to learn a scoring function. Using a threshold t for sample y i Categorize:

[0010]

[0011] Where s i Indicates y i The label indicates the normal class and the abnormal class.

[0012] In one embodiment of the present invention, the basic process of CropMixPaste is as follows:

[0013] (1) Obtain multiple cropping blocks of different sizes from a normal image;

[0014] (2) Use Mixup to blend cropped blocks to form a new blended image, where intermediate enhancement is performed before each blending operation;

[0015] (3) Randomly blur and blend images and randomly adjust image size;

[0016] (4) Paste the image generated in step (3) back to any position of the original image.

[0017] In one embodiment of the present invention, the CropMixPaste is specifically implemented as follows:

[0018] 1) Multi-scale cropping

[0019] Given from normal data Selected normal image Cutting block x c Represented as:

[0020] x c =r(c(x))

[0021] Here, c(x) represents the clipping operation, and r(x) represents the resizing operation; then, a set containing N clipping operations is defined. The upper and lower limits of the cutting ratio are (s) low ,s up They are evenly distributed among each cropping operation. Small cropping operations capture fine details, while large cropping operations cover more pixels, thus achieving multi-scale information acquisition. Therefore, a set of N cropping blocks of different sizes is obtained in the following way:

[0022]

[0023] 2) MixUp operation

[0024] To more naturally blend clipping blocks of different sizes to obtain multi-scale information, from χ c Two clipping blocks are randomly selected from x. i x j The images are then blended using the MixUp operation, and channel permutation is performed on the images with smaller blending weights, as defined below:

[0025]

[0026] Where p(x) represents the channel arrangement, the mixing weight λ is randomly sampled from Beta(α,α), and α is set to 0.3; then, the above operation is repeated to mix the image z with the randomly selected cropped block x. k Mix; by repeating the mixing operation, χ c The cropped blocks in the image are merged into the final image x. min ;

[0027] 3) Blur and paste

[0028] To make the synthesized image x as close as possible mix Aligned with a real image, random blurring is used to simulate anomalous regions similar to ground-glass opacities (GGOs) in the lungs; furthermore, this operation blurs the edges of the blocks and improves the continuity of the pasting; subsequently, the randomly blurred image x blur The image is randomly resized and then pasted back into any position on the original image to form the final composite anomalous image. aug ;

[0029] 4) Training objectives of self-supervised learning

[0030] Using anomalous and normal samples generated by CropMixCatse, a deep neural network can be trained to learn useful feature representations, thereby improving the generalization ability to real anomalies and thus enabling better CT scan anomaly detection. Therefore, the training objective of self-supervised learning is defined as:

[0031]

[0032] Where CMP(x) is CropMixPaste, and f(x) is a deep neural network. This represents the cross-entropy loss.

[0033] In one embodiment of the present invention, the simple masked attention prediction block (SMAPB) is inserted into the convolutional network, that is, a self-supervised reconstruction block is introduced into the self-supervised depth encoder f. The SMAPB consists of a masked convolutional layer and a parameterless attention layer, with an independent loss function to minimize the reconstruction error between the masked input and output. The masked convolutional layer, also known as a masked convolutional filter, has its central region masked, and the sub-kernels at the four corners obtain activation maps through convolution operations to predict the center of the mask. The subsequent parameterless attention ensures that the learned information is not oversimplified and can refine key features.

[0034] In one embodiment of the present invention, the masking convolutional filter is implemented as follows:

[0035] First, set the camouflage area at the center of the receptive field to Where c represents the number of channels; the distance extends outwards from M in all directions. Learnable sub-kernels The kernels are set at the four corners; therefore, masking kernels of size L will be created, where L = 2k + 2d + 1.

[0036] Let the input tensor be Where c is the number of channels, and h and w are the height and width, respectively; then, for each pixel, a designed masked convolutional kernel is used to obtain information about the entire tensor X; specifically, convolution operations are only performed at the locations of the sub-kernels within the receptive field; then, the results of the four convolution operations are summed, just like the sub-kernel K. i It belongs to the same kernel K; the result is the predicted value at the same position as the masked M; perform convolution filtering on each channel to predict the masking information from all channels; to ensure that the spatial dimensions of the input and output are the same and that each pixel is masked, add k+d zero padding pixels around the input and set the stride to 1; finally, activate the output tensor through ReLU.

[0037] In one embodiment of the present invention, the parameterless attention is used to process the output of the masked convolutional filter, i.e., using an attention SimAM with full 3D weights. Considering the energy function defined for each neuron, the following minimum energy equation is defined:

[0038]

[0039] in δ represents the energy hyperparameter; the above equation shows that the lower the energy, the greater the difference between neuron t and surrounding neurons, and the higher its importance; therefore, the importance of a neuron is determined by... We obtain the features and then use the Sigmoid function to enhance and optimize them, as defined below:

[0040]

[0041] E will all Grouping across channels and spatial dimensions.

[0042] In one embodiment of the present invention, the SMAPB has a separate reconstruction loss function to calculate the error between each masking region information and its location information reconstructed by the masking filter of the receptive field; for the input tensor X, the mean squared error (MSE) used is as follows:

[0043]

[0044] Where G(X) represents the SMAPB process, Indicates the final output;

[0045] Therefore, the loss function of the method is as follows:

[0046] L total =L CMP +ω·L SMAPB

[0047] Where L CMP The training objective for self-supervised learning, It is a hyperparameter representing the importance of SMAPB loss.

[0048] In one embodiment of the present invention, the anomaly detector is constructed using a self-supervised learning representation, specifically, the representation learned from the encoder f using CropMixPaste and SMAPB is used to construct an out-of-distribution detector g for calculating anomaly scores; then, the anomaly score is obtained by testing the Mahalanobis distance between the Gaussian distributions of test samples and normal samples, as shown in the following formula:

[0049]

[0050] in This represents the average value. Represents Gaussian distribution The covariance in the equation is given by D, where D is the dimension size; therefore, the Mahalanobis distance matrix M of the anomaly map can be calculated using the above formula; high scores indicate salient regions of anomalies; finally, the highest score is used to determine the anomaly score of the entire image.

[0051] In one embodiment of the present invention, the method uses GradCAM visual interpretation technology to provide a visual anomaly heatmap, thereby guiding the localization of anomalies and defects.

[0052] Compared to existing technologies, this invention offers the following advantages: This invention proposes a lung CT anomaly detection method based on multi-scale cropping and self-supervised reconstruction. The algorithm first establishes a self-supervised enhancement strategy, CropMixPaste, suitable for lung CT scan images to generate ground-glass opacities, thereby encouraging the model to detect local irregularities in lung CT scan images. Then, a Simple Mask Attention Prediction Block (SMAPB) is inserted into the convolutional network, utilizing this self-supervised reconstruction to predict masking information and refine local features. Finally, an anomaly detector is constructed using representations learned in the self-supervised agent task. Experimental results on a real lung CT scan dataset demonstrate the effectiveness and superiority of the algorithm. Attached Figure Description

[0053] Figure 1 This refers to the abnormal lung CT images, the corresponding ground-glass opacity areas, and the localization areas that can be provided by the abnormality detection methods.

[0054] Figure 2 This is a flowchart of the method of the present invention.

[0055] Figure 3 Examples of lung CT scan images augmented using (a) CutPaste, (b) CropMix, (c) FPI, (d) NSA, and the CropMixMixPaste method proposed in this invention.

[0056] Figure 4 The Simple Masked Attention Prediction Block (SMAPB) structure was designed for this invention.

[0057] Figure 5 The visualization results of abnormal lung CT samples from the SC2CT dataset are shown from left to right as follows: original image, STFPM method result, Reverse Distillation method result, CutPaste method result, NSA method result, and the method of this invention.

[0058] Figure 6 The visualization results of abnormal lung CT samples from the COVID CTset dataset are shown from left to right as follows: original image, STFPM method result, Reverse Distillation method result, CutPaste method result, NSA method result, and the method of this invention. Detailed Implementation

[0059] The technical solution of the present invention will now be described in detail with reference to the accompanying drawings.

[0060] This invention discloses a lung CT anomaly detection method based on multi-scale cropping and self-supervised reconstruction. First, a self-supervised enhancement strategy, CropMixPaste, suitable for lung CT scan images is set to generate anomalies similar to ground-glass opacities, encouraging the model to detect local irregularities in lung CT scan images. Then, a Simple Mask Attention Prediction Block (SMAPB) is inserted into a convolutional network for self-supervised learning to predict masking information and refine local features. Finally, the self-supervised learned representations are used to construct an anomaly detector.

[0061] The technical solution of the present invention is described in detail below.

[0062] 1. Method Structure

[0063] This invention proposes a method for anomaly detection in lung CT scan images. Given a set of normal training samples... The goal is to distinguish between normal and abnormal test samples. The anomaly detection problem is defined as follows:

[0064] For test set Y test y1,y2…y N Each sample is associated with d-dimensional features. The goal is to learn a scoring function. Using a threshold t for sample y i Categorize:

[0065]

[0066] Where s i Indicates y i The labels are 0 for the normal class and 1 for the abnormal class (other classes). Therefore, this invention uses a two-stage method to construct the learned scoring function s. i Anomaly detectors. For example... Figure 2 As shown, firstly, a self-supervised data augmentation agent task, CropMixPaste, designed to generate ground-glass opacity-like enhancements, is proposed to learn feature representations from normal data while paying attention to subtle differences in lung CT scans. Then, a ResNet encoder with a Simple Masked Attention Prediction Block (SMAPB) is used instead of a pure ResNet encoder for self-supervised learning. This block is a self-supervised reconstruction setting used to learn and refine local features. Finally, an anomaly detector is constructed based on the learned representations. During the testing phase, the invention uses Mahalanobis distance to calculate anomaly scores and generates anomaly localization maps using GradCAM.

[0067] 1.1 CropMixPaste Self-Supervised Augmentation Task

[0068] To further prevent the model from overfitting to simple tasks and falling into the predicament of being unable to distinguish anomalies, this invention sets up a data augmentation task, CropMixPaste, suitable for lung CT scan images, for self-supervised feature representation learning. Its basic process is as follows:

[0069] (1) Obtain multiple cropping blocks of different sizes from a normal image;

[0070] (2) Use Mixup to blend these cropped pieces to form a new blended image, where intermediate enhancement is performed before each blending operation;

[0071] (3) Randomly blur and blend images and randomly adjust image size;

[0072] (4) Paste the image generated in step (3) back to any position of the original image.

[0073] Figure 3 Examples of lung CT scan images augmented using (a) CutPaste, (b) CropMix, (c) FPI, (d) NSA, and the CropMixMixPaste method proposed in this invention.

[0074] 1.1.1 Multi-scale cropping

[0075] A single cropped view lacks rich local information; therefore, this invention considers obtaining multiple cropped blocks from the same image. Given normal data... Selected image Cutting block x c It can be represented as:

[0076] x c =r(c(x))

[0077] Here, c(x) represents the clipping operation, and r(x) represents the resizing operation. Then, a set containing N clipping operations is defined. The upper and lower limits of the cutting ratio are (s) low ,s up These are evenly distributed among each cropping operation. For example, when the cropping ratio range is (0.05, 0.50), the cropping ratios for the three cropping operations are (0.05, 0.20), (0.20, 0.35), and (0.35, 0.50), respectively. Smaller cropping ratios capture finer details, while larger cropping ratios cover more pixels, thus enabling multi-scale information acquisition. Therefore, a set of N different-sized cropping blocks can be obtained as follows:

[0078]

[0079] 1.1.2 MixUp Mixing Operation

[0080] To more naturally blend clipping blocks of different sizes to obtain multi-scale information, from χ c Two clipping blocks are randomly selected from x. i ,x j The images are then blended using the MixUp operation, and channel permutation is performed on the images with smaller blending weights, as defined below:

[0081]

[0082] Where p(x) represents the channel arrangement, and the mixing weight λ is randomly sampled from Beta(α,α), with α set to 0.3. Then, the above operation is repeated to mix the image z with a randomly selected cropped block x. k Mix. By repeating the mixing operation, χ c The cropped blocks in the image are merged into the final image x. mix .

[0083] 1.1.3 Blur and Paste

[0084] In order to make the synthesized image x as close as possible mix Aligned with a real image, random blurring is used to simulate abnormal areas resembling ground-glass opacities (GGOs) in the lungs. Furthermore, this operation blurs the edges of the blocks and improves the continuity of the pasting. Afterwards, the randomly blurred image x... blur The image is randomly resized and then pasted back into any position on the original image to form the final composite anomalous image. aug .

[0085] 1.1.4 Training Objectives of Self-Supervised Learning

[0086] Using anomalous and normal samples generated by CropMixCatse, a deep neural network can be trained to learn useful feature representations, improving generalization to real anomalies and thus enhancing CT scan anomaly detection. Therefore, the training objective of the self-supervised task is defined as:

[0087]

[0088] Where CMP(x) is CropMixPaste enhancement, and f(x) is a deep neural network. This represents the cross-entropy loss. Similar to general self-supervised learning methods, this invention applies conventional data augmentation, such as color jitter and random horizontal flipping, before inputting x into f or CMP.

[0089] 1.2 Simple Masked Attention Prediction Block (SMAPB) for Refining Features

[0090] Considering the disadvantage of pure ResNet in dealing with local irregularities, this invention introduces a self-supervised reconstruction block (called a simple masking attention prediction block) into the self-supervised depth encoder f (e.g. Figure 4 (As shown), to refine local features. When the self-supervised reconstruction task is a low-level architectural component, it can be flexibly integrated into a pure ResNet architecture, even at any layer, regardless of whether the encoder is based on the reconstruction model.

[0091] This is a reconstruction-based approach that forces the learning of a global structure from local patterns to achieve lower reconstruction errors. It consists of a masking convolutional layer and a parameter-free attention block with an independent loss function to minimize the reconstruction error between the masked input and output. The masking convolutional layer is also called a masking convolutional filter. Its central region is masked, and the four corner sub-kernels are convolutionally processed to obtain activation maps to predict the masked center. The subsequent attention block ensures that the learned information is not oversimplified and is able to refine key features.

[0092] 1.2.1 Masked Convolution Filter

[0093] First, set the camouflage area at the center of the receptive field to Where c represents the number of channels. The distance extends outwards in all directions from M as the center. Learnable sub-kernels The kernels are positioned at the four corners. Therefore, masking kernels of size L will be created, where L = 2k + 2d + 1.

[0094] Let the input tensor be Here, c is the number of channels, and j and w are the height and width, respectively. Then, for each pixel, a designed masked convolutional kernel is used to obtain information about the entire tensor X. Specifically, convolution operations are performed only at the locations of the sub-kernels within the receptive field. Then, the results of the four convolution operations are summed, just like the sub-kernel K. i The result is the predicted value at the same position as the masked M, belonging to the entire kernel K. Performing convolutional filtering on each channel predicts masking information from all channels. To ensure the input and output have the same spatial dimensions and that each pixel is masked, k+s pixels of zero-padding are added around the input, with a stride of 1. Finally, the output tensor is activated using ReLU.

[0095] 1.2.2 Simple and Parameter-Free Attention SimAM

[0096] Next, attention is used to process the masked convolution output. Since the size of M is set to 1, the masked convolution is performed pixel-wise. However, existing spatial or channel attention can only refine features along the spatial or channel dimensions. Therefore, this invention uses an attention SimAM with full 3D weights, which is simple and lightweight, requiring no additional parameters. Using this mechanism, adaptive recalibration of the pixel-wise masked output can be performed to learn more about the selectivity of global information. Another motivation for using SimAM is to maintain lightweight performance while increasing non-linear processing capabilities.

[0097] As a novel interpretable attention mechanism, SimAM is based on the spatial inhibition effect in neuroscience, where information-rich neurons often exhibit firing patterns distinct from surrounding neurons, while simultaneously inhibiting them. Therefore, the goal is to give higher priority to neurons exhibiting spatial inhibition. Considering the energy function defined for each neuron, the following minimum energy equation is defined:

[0098]

[0099] in δ represents the energy hyperparameter. The above formula shows that the lower the energy, the greater the difference between neuron t and surrounding neurons, and the higher its importance. Therefore, the importance of a neuron can be determined by... We obtain the features and then use the Sigmoid function to enhance and optimize them, as defined below:

[0100]

[0101] E will all Grouping across channels and spatial dimensions.

[0102] 1.2.3 Loss Function

[0103] This has a separate reconstruction loss function to calculate the error between the information of each masked region and the location information reconstructed by the masking filter of the receptive field. The mean squared error (MSE) used for the input tensor X is as follows:

[0104]

[0105] Where G(X) represents the SMAPB process, This represents the final output. SMAPB is inserted before the penultimate convolutional layer of the ResNet network to construct the self-supervised encoder f.

[0106] Finally, the loss function of the proposed self-supervised method is defined as follows, based on multi-scale pruning and simple masking attention prediction using equations (1) and (2):

[0107] L total =L CMP +ω·L SMAPB (3)

[0108] in It is a hyperparameter representing the importance of SMAPB loss.

[0109] 1.3 Calculation and Visualization of Anomaly Scores

[0110] During the testing phase, the representation learned from the encoder f is reconstructed using data augmentation strategies and masking predictions, which can then be used to construct an out-of-distribution detector g for calculating anomaly scores. The anomaly score is then obtained by measuring the Mahalanobis distance between the Gaussian distributions of the test samples and the normal samples, as shown in the following formula:

[0111]

[0112] in This represents the average value. Represents Gaussian distribution The covariance in the matrix is ​​given by the formula, where D is the dimension. Therefore, the Mahalanobis distance matrix M of the anomaly map can be calculated using this formula. High scores indicate salient anomaly regions. Finally, the highest score is used to determine the anomaly score for the entire image.

[0113] Furthermore, most lung CT scan datasets do not include pixel-level anomaly annotations due to the high cost of labeling. Therefore, this invention utilizes GradCAM visual interpretation technology to provide visual anomaly heatmaps, thereby guiding the localization of abnormalities and defects.

[0114] 2 Implementation Examples

[0115] During the experiment, two publicly available lung CT scan datasets were used to evaluate the performance of the proposed method. Statistics for the datasets are shown in Table 1. SC2CT (SARS-CoV-2CT-Scan) was collected from a hospital in São Paulo, Brazil, and includes 2482 CT scan images, of which 1252 were positive for SARS-CoV-2 infection (COVID-19) and 1230 were negative for non-SARS-CoV-2 infected patients. The COVID CTset was collected from the Negin Medical Center in Sari, Iran. This dataset contains the entire original CT scans of 377 individuals. During data preprocessing, images lacking clear information were excluded to ensure dataset quality, such as some closed, normal lung slices that do not carry information. The final dataset contains 1053 normal images and 666 pneumonia images. All images were resized to 256×256 pixels. These datasets were reorganized according to a standard anomaly detection protocol and divided into an 80% training set and a 20% test set. According to the standard protocol, only negative (normal) images were used during training.

[0116] Table 1. Data statistics for the SC2CT / and COVID-CTset datasets.

[0117]

[0118] To evaluate the performance of the proposed method, the area under the receiver operating characteristic curve (ROC curve) was calculated in the experiment. This is a commonly used metric for anomaly detection, which defines the false positive rate (FPR) on the X-axis and the true positive rate (TPR) on the Y-axis. The formula is as follows:

[0119]

[0120]

[0121] In this system, TP stands for True Positive, meaning a positive sample predicted as positive by the model; FP stands for False Positive, meaning a negative sample predicted as positive by the model; TN stands for True Negative, meaning a positive sample predicted as positive by the model; and FN stands for False Negative, meaning a positive sample predicted as negative by the model. The closer the AUROC value is to 1, the better the classifier can distinguish between positive and negative samples.

[0122] The experimental environment for this invention was an Ubuntu 20.04.3LTS operating system, using an NVIDIA GEFORCE RTX3080Ti GPU with 12GB of VRAM, 128MB of RAM, and an Intel(R) Xeon(R) Silver4210 CPU @ 2.20GHz. The method follows standard protocols, using an anomaly detector with ResNet18 to learn normal representations from scratch. SMAPB is inserted before the penultimate convolutional layer of ResNet18. After the ResNet backbone, an MLP projection head and a linear layer are added to the anomaly detector. The optimizer uses SGD with momentum of 0.9 and weight decay of 0.00003. The initial learning rate is 0.03, and a cyclic cosine annealing schedule is used. The self-supervised data augmentation is implemented using the proposed CropMixPaste strategy. The number of pruning operations is randomly selected from (2, 3, 4), the pruning ratio ranges from (0.05, 0.25), the mixing ratio is 0.3, and channel permutation is used as an intermediate augmentation. The patch size range is (0.05 × image size, 0.25 × image size). For SMAPB, the subkernel size k is set to 1, and the inflation rate d is set to 1. The hyperparameter ω of equation (3) is set to 0.03. In the experiment, the model was trained for 100 epochs with a batch size of 32, and the weights were saved when the AUROC was highest. Mahalanobis distance was used for anomaly detection scoring.

[0123] 2.1 Qualitative Comparison

[0124] To qualitatively compare the detection performance of five anomaly detection methods, anomaly localization visualization was performed on lung CT images from two real datasets. The results are as follows: Figure 5-6 As shown, it is difficult to obtain pixel-by-pixel markers of local abnormalities on lung CT scan images. Therefore, GradCAM visualization is used to highlight regions that influence abnormality detection decisions.

[0125] Figure 5 This paper presents the anomaly detection results for four lung CT images from the SC2CT dataset. The five columns from left to right show the original image, the results of the STFPM method, the results of the Reverse Distillation method, the results of the CutPaste method, the results of the NSA method, and the results of the method proposed in this invention. Figure 5 As can be seen, the method of this invention tends to focus more on specific local areas and is more accurate than other methods. These localization results almost perfectly match the attention physicians pay to abnormal areas (such as frosted glass shadows) on CT scan images.

[0126] Figure 6This image presents the anomaly detection results for four lung CT images from the COVID CTset dataset. The five columns from left to right show the original image, the results from the STFPM method, the results from the Reverse Distillation method, the results from the CutPaste method, the results from the NSA method, and the results from the method of this invention. Figure 6 As can be seen, the method of this invention has a more accurate localization effect on both localized and diffuse frosted glass shadows. In contrast, the CutPaste method and NSA method, due to their simple proxy task settings, cannot accurately locate specific abnormal regions, while the STFPM method and Reverse Distillation method tend to focus on overall features rather than local anomalies.

[0127] Table 2. Quantitative comparison of the present invention with state-of-the-art algorithms on real datasets.

[0128]

[0129] 2.2 Quantitative comparison

[0130] To quantitatively compare the algorithm of this invention with twelve state-of-the-art algorithms, it was tested on two real-world lung CT scan image datasets, SC2CT and COVID-CTset, and the anomaly detection performance was quantitatively evaluated using AUROC (%). Table 2 shows the quantitative evaluation results on the two datasets, with the best results bolded, the second best underlined, and the third best underlined. As can be seen from Table 2, the algorithm of this invention achieved the best performance on both datasets, exceeding the second best results by 1.50% and 2.09%, and exceeding the third best results by 4.25% and 2.19%. Therefore, the method proposed in this invention can achieve satisfactory anomaly detection performance on different types and resolutions of lung CT scan datasets.

[0131] The above are preferred embodiments of the present invention. Any changes made to the technical solution of the present invention that do not exceed the scope of the technical solution of the present invention shall fall within the protection scope of the present invention.

Claims

1. A method for detecting lung CT abnormalities based on multi-scale cropping and self-supervised reconstruction, characterized in that, First, a self-supervised enhancement strategy, CropMixPaste, is set up for lung CT scan images to generate ground-glass opacities-like anomalies, encouraging the model to detect local irregularities in lung CT scan images. Then, a Simple Mask Attention Prediction Block (SMAPB) is inserted into the convolutional network for self-supervised learning, predicting masking information to refine local features. Finally, the self-supervised learned representations are used to construct an anomaly detector. Specifically… Given a set of normal training samples The purpose of this method is to distinguish between normal and abnormal test samples; the anomaly detection problem is defined as follows: For the test set Each sample is related to Dimensional features are related The goal is to learn a scoring function. To be based on threshold For the sample Categorize: in express The labels are 0 for normal classes and 1 for abnormal classes; The basic process of CropMixPaste is as follows: (1) Obtain multiple cropping blocks of different sizes from a normal image; (2) Use Mixup to blend cropped blocks to form a new blended image, where intermediate enhancement is performed before each blending operation; (3) Randomly blur and blend the image and randomly adjust the image size, wherein the random blurring generates an anomaly similar to a frosted glass shadow; (4) Paste the image generated in step (3) back to any position of the original image; The method involves inserting the Simple Masked Attention Prediction Block (SMAPB) into the convolutional network, which essentially introduces the self-supervised reconstruction block into the self-supervised depth encoder. The Simple Masked Attention Prediction Block (SMAPB) consists of a masked convolutional layer and a parameterless attention module, with an independent loss function to minimize the reconstruction error between the masked input and output. The masked convolutional layer, also known as a masked convolutional filter, has its central region masked, and the sub-kernels at the four corners obtain activation maps through convolution operations to predict the center of the mask. The subsequent parameterless attention module ensures that the learned information is not oversimplified and is able to refine key features. The anomaly detector is constructed using self-supervised learning representations, specifically through CropMixPaste and SMAPB from the encoder. The learned representations are used to construct an out-of-distribution detector for computing anomaly scores. Then, the anomaly score is obtained by testing the Mahalanobis distance between the Gaussian distributions of the test samples and the normal samples, as shown in the following formula: in This represents the average value. Represents Gaussian distribution The covariance in, where It refers to the dimension size; therefore, the Mahalanobis distance matrix of the anomaly map can be calculated using the above formula. ; High scores indicate abnormally significant regions; Finally, the highest score is used to determine the anomaly score for the entire image.

2. The method for detecting lung CT abnormalities based on multi-scale cropping and self-supervised reconstruction according to claim 1, characterized in that, The CropMixPaste is implemented as follows: 1) Multi-scale cropping Given from normal data Selected normal image Cutting blocks Represented as: in, This indicates a cropping operation. This indicates a resizing operation; then, define a containing A collection of cropping operations The upper and lower limits of the cutting ratio are: They are evenly distributed among each cropping operation. Small cropping operations capture fine details, while large cropping operations cover more pixels, thus achieving multi-scale information acquisition. Therefore, a set of numbers is obtained through the following method. Different sized cutting blocks: 2) MixUp operation To more naturally blend clipping blocks of different sizes to obtain multi-scale information, from Randomly select two clipping blocks , The images are then blended using the MixUp operation, and channel permutation is performed on the images with smaller blending weights, as defined below: in Indicates channel arrangement and mixed weights from Random sampling in the middle, Set it to 0.3; then repeat the above steps to blend the images. With randomly selected clipping blocks Mix; through repeated mixing operations, The cropped blocks in the image are merged into the final image. ; 3) Blur and paste To make the synthesized image as close as possible Aligned with a real image, random blurring is used to simulate abnormal regions similar to ground-glass opacities (GGOs) in the lungs; furthermore, this operation blurs the edges of the blocks and improves the continuity of the pasting; subsequently, the image is randomly blurred. The images are randomly resized and then pasted back into any position on the original image to create the final composite anomalous image. ; 4) Training objectives of self-supervised learning Using anomalous and normal samples generated by CropMixPatse, a deep neural network can be trained to learn useful feature representations, thereby improving the generalization ability to real anomalies and thus enabling better CT scan anomaly detection. Therefore, the training objective of self-supervised learning is defined as: in It's CropMixPaste. For deep neural networks, This represents the cross-entropy loss.

3. The method for detecting lung CT abnormalities based on multi-scale cropping and self-supervised reconstruction according to claim 1, characterized in that, The masked convolutional filter is implemented as follows: First, set the camouflage area at the center of the receptive field to ,in Indicates the number of channels; Expand the distance in all directions from the center Learnable sub-cores Set in the four corners; Therefore, a size of will be created The masking convolution kernel, where ; Let the input tensor be ,in It is the number of channels. and These are the height and width, respectively; then, the designed masking convolution kernel is used for each pixel to obtain the entire tensor. Specifically, the module only performs convolution operations at the location of the sub-kernel of the receptive field; then, the results of the four convolution operations are added together, and the result is the sum of the values ​​of the masked kernel and the kernel. Predicted values ​​at the same location; perform convolutional filtering on each channel to predict masking information from all channels; to ensure that the spatial dimensions of the input and output are the same, and that each pixel is masked, add [masking information] around the input. Zero-padding is applied to each pixel, and the stride is set to 1; finally, the output tensor is activated via ReLU.

4. The method for detecting lung CT abnormalities based on multi-scale cropping and self-supervised reconstruction according to claim 3, characterized in that, The parameterless attention module is used to process the output of the masked convolutional filter, i.e., using an attention module SimAM with full 3D weights. Considering the energy function defined for each neuron, the following minimum energy equation is defined: in , , Represents energy hyperparameters; The above formula shows that the lower the energy, the more neurons... The greater the difference between a neuron and its surrounding neurons, the higher its importance; therefore, the importance of a neuron is determined by… We obtain the features and then use the Sigmoid function to enhance and optimize them, as defined below: in All Grouping across channels and spatial dimensions.

5. The lung CT abnormality detection method based on multi-scale cropping and self-supervised reconstruction module according to claim 1, characterized in that, The SMAPB has a separate reconstruction loss function to calculate the error between each masking region information and its reconstructed location information through the masking filter of the receptive field; for the input tensor The mean squared error (MSE) used is shown below: in The process of representing SMAPB, Indicates the final output; Therefore, the loss function of the method is as follows: in It is a hyperparameter representing the importance of SMAPB loss.

6. The lung CT abnormality detection method based on multi-scale cropping and self-supervised reconstruction module according to claim 1, characterized in that, This method uses GradCAM visual interpretation technology to provide visual anomaly heatmaps, thereby guiding the location of anomalies and defects.

Images