A COVID-19 Lesion Segmentation Method Based on Grad-CAM

Abstract

The present invention discloses a COVID-19 lesion segmentation method based on Grad-CAM. The main content of this method is as follows: First, perform conventional image preprocessing on CT images, and then use an image binarization preprocessing method with OSTU as the core to enhance the contrast between lung lesions and healthy areas and highlight the edge information of the lesions; input the preprocessed CT images into a classification convolutional neural network for training; call Grad-CAM in the trained classification convolutional neural network to generate a feature region localization heat map for CT image classification. Finally, set a segmentation threshold in the heat map and obtain the lung lesion segmentation result according to the threshold. Through the present invention, automatic segmentation of COVID-19 lesions can be achieved in the case of only class labels and no segmentation labels, thereby saving manpower and ensuring the objectivity of the segmentation result.

Description

Technical Field

[0001] The present invention belongs to the field of COVID-19 lesion segmentation, and particularly relates to a COVID-19 lesion segmentation method based on Grad-CAM. Background Art

[0002] Due to the defects such as a relatively high false negative rate of RT-PCR, the accurate diagnosis of suspected cases will be delayed. As an important supplement to RT-PCR, CT is a radiological imaging technique for effectively screening COVID-19 because it has high spatial resolution and can detect small lesions. However, manual annotation of lung lesions is cumbersome and time-consuming, and has a high degree of subjectivity. There are usually personal biases and it is affected by clinical experience, and the characteristics of the lesion boundary are diffuse, resulting in large errors in manual annotation. Using deep learning methods to automatically detect COVID-19 and segment lesions in CT images can help quantify the severity of COVID-19, thus assisting doctors in faster diagnosis and treatment.

[0003] Since the novel coronavirus mutates relatively fast, the pathological features and pulmonary imaging changes caused by different strains are different. It is difficult to quickly locate the lesions and collect enough labeled data to train the deep model in a short time; at the same time, most current public COVID-19 image datasets are only used for case diagnosis, and very few provide segmentation labels. In addition, due to the characteristics of shadows and ground-glass in lung lesions, the boundaries are not clear, so it is difficult to label diffuse lesions; and manual annotation of a large amount of data is likely to introduce noisy labels, affecting the training effect of deep learning. However, most current deep learning-based lesion segmentation algorithms must provide a certain amount of data with segmentation labels during the training process. When the novel coronavirus mutates, it is impossible to quickly locate the lesions and achieve the effect of assisting doctors in diagnosis and treatment. Summary of the Invention

[0004] Aiming at the above problems existing in the prior art, the present invention aims to provide a COVID-19 lesion segmentation method based on Grad-CAM. By performing conventional image preprocessing such as denoising on two-dimensional CT images to remove the noise in the images and only retain the lung region, and performing image binarization preprocessing with OSTU as the core to enhance the contrast between the lung lesions and the healthy region and highlight the lesion edge information, and by constructing and training a COVID-19 lesion segmentation model based on Grad-CAM, the segmentation of COVID-19 chest CT lesions can be realized under the condition of only requiring class labels and not requiring segmentation labels.

[0005] To achieve the purpose of the present invention, the technical solutions adopted by the present invention are as follows:

[0006] A COVID-19 lesion segmentation method based on Grad-CAM, the method comprising the following steps:

[0007] Step 1: Perform general preprocessing on the CT image;

[0008] Step 1.1: Denoise the CT image using median filtering;

[0009] Step 1.2: Crop the CT image to retain the ROI;

[0010] Step 1.3: Unify the correspondence between the CT image orientation and the human body orientation;

[0011] Step 1.4: Unify the size of the CT image;

[0012] Step 2: Use the image binarization preprocessing method with OSTU as the core to enhance the contrast between the lung lesions and the healthy area and highlight the lesion edge information;

[0013] Step 2.1: Adjust the brightness of the CT image;

[0014] Step 2.2: Adjust the contrast of the CT image using adaptive histogram equalization;

[0015] Step 2.3: Use OSTU to determine the binarization threshold of the CT image and perform binarization processing;

[0016] Step 2.4: Retain the complete lung contour by performing union operation on the binary image;

[0017] Step 2.5: Invert the result of the binarization processing;

[0018] Step 2.6: Use hole filling and dilation to extract the lung contour edge information;

[0019] Step 2.7: Use the algorithm designed in the present invention to remove the invalid information in the image edge area;

[0020] Step 3: Input the preprocessed CT image into a classification convolutional neural network for training;

[0021] Step 4: Call Grad-CAM in the trained classification convolutional neural network to generate a feature region localization heat map for CT image classification;

[0022] Step 5: Set a segmentation threshold in the heat map and obtain the lung lesion segmentation result according to the threshold.

[0023] As an improvement of the present invention, in the said Step 1, the method for performing general preprocessing on the CT image is:

[0024] In step 1.1, since there is noise in some CT images, image denoising is first performed by means of median filtering. In step 1.2, for CT images obtained by scanning from different devices, the proportion and spatial distribution of the lung region in the whole image are different. At the same time, most CT images also have the problem that the spatial proportion of the lung region is relatively small. Therefore, all CT images are cropped to retain only the region of interest (ROI) as much as possible and remove the non-lung region in the image. In step 1.3, since the directions of CT images obtained by scanning from different devices are also different, the directions of all CT images are uniformly adjusted to the relationship that the upper part of the CT image corresponds to the front of the human body. Finally, in step 1.4, the sizes of all CT images are uniformly adjusted to 512×512.

[0025] As an improvement of the present invention, in step 2, an image binarization preprocessing method with OSTU as the core is used to enhance the contrast between lung lesions and healthy regions and highlight the edge information of the lesions.

[0026] In step 2.1, the present invention first adjusts the brightness of the CT image after conventional image preprocessing to highlight the lesions that may be misjudged as the background during binarization due to being too dark in the CT image. In step 2.2, the contrast of the CT image is adjusted, and the contrast between lung lesions and lung healthy regions is enhanced through adaptive histogram equalization processing to improve the edge sharpness of the lesions. In step 2.3, then the present invention determines the optimal threshold for binarization processing of each CT image by the maximum inter-class variance method, and the threshold calculation method is as follows:

[0027] In CT image I, assuming there are L (L≥1) gray levels, the number of pixels with gray level i (1≤i≤L) is s i , and the total number of pixels in image I is S, then the probability P i that all pixels with gray level i appear is

[0028]

[0029] The image pixel set is divided into the target pixel set C t and the background pixel set C b . Assuming the optimal threshold is k, the pixels with gray level i in [1,k] are classified into C t , and the pixels with gray level i in [k + 1,L] are classified into C b . Then the proportion ω t of the pixels in the target pixel set C b and the background pixel set C t in all the pixels of image I and ωb respectively

[0030]

[0031]

[0032] From the above two equations, the mean value μ of the target pixels can be obtained t and the mean value μ of the background pixels b respectively

[0033]

[0034]

[0035] The between-class variance σ t and σ b respectively

[0036]

[0037]

[0038] The total gray mean value μ of the two is

[0039]

[0040] The total between-class variance σ 2 is

[0041] σ 2 = ω t σ t 2 + ω b σ b 2

[0042] The optimal threshold Th is

[0043]

[0044] The above equation means taking k when the total between-class variance σ 2 reaches the maximum value as the optimal threshold

[0045] According to this threshold, each pixel point in the image is assigned 0 or 1, thus completing the binarization process of the CT image. However, there are still some problems in the binarization result at this time, such as overfitting in the processing of the lung contour, loss of lung edge information, and invalid information in the image edge area, etc. Therefore, it is still necessary to further optimize the binarization result at this time

[0046] In step 2.4, to solve the problem of overfitting in lung contour processing, the present invention performs binarization on the CT image after conventional image preprocessing, and performs an OR operation on the corresponding pixel points in the binarization result and the above binarization result, so as to retain the complete lung contour while highlighting the detailed information. In step 2.5, the binarization result after the OR operation is inverted, so that the main body of the lung appears as a white area, which is convenient for subsequent optimization processing.

[0047] In step 2.6, to improve the lung edge information, the present invention extracts the contour information from the CT image after conventional image preprocessing, and performs hole filling and dilation operations before contour extraction, so as to finally obtain the complete lung contour information. The extracted lung contour information is subjected to an OR operation with the inverted binarization result above to obtain a binarization result with contour information.

[0048] In step 2.7, to remove the invalid information in the image edge area, the present invention designs an algorithm that can automatically remove the white invalid information at the edge of the binary image without manual intervention, and an anti-misjudgment algorithm is added thereto. Finally, the above algorithm is called in the binarization result with improved lung edge information to generate the final CT image after conventional image preprocessing and image binarization preprocessing.

[0049] As an improvement of the present invention, in step 3, since Grad-CAM is a technology for identifying the feature region with the highest correlation with a specific category during the classification process, we can utilize the fact that the main difference between the patient's CT and the healthy person's CT is the presence or absence of lesions, and use Grad-CAM to identify and distinguish the feature region of the two, that is, the lesion, during the classification process, so as to achieve the segmentation of chest lesions without pixel-level segmentation labels. Therefore, before using Grad-CAM, the present invention uses a classification convolutional neural network to perform binary classification on the patient's CT and the healthy person's CT.

[0050] As an improvement of the present invention, in step 4, since Grad-CAM can be applied to any CNN-based architecture, it can be directly used in any classification neural network without changing the network structure to generate a heat map and perform lesion segmentation. The present invention uses the trained classification convolutional neural network to classify the CT image, uses Grad-CAM to identify the feature region on the CT image with the classification prediction result of the patient's CT, and generates a feature region localization heat map according to the probability size, that is, a lesion probability distribution heat map is obtained in the CT image with the classification prediction result of the patient's CT. The specific calculation method is as follows:

[0051] To obtain the class discriminant localization map for any class c Grad-CAM first performs forward propagation to calculate the score gradient y of class c before the softmax layer c , where y c is related to the feature map A of the convolutional layer k (k represents the k-th channel in the feature layer A); the calculated gradient is propagated backward and globally average pooled to obtain the neuron importance weights

[0052]

[0053] where Z is the product of the width and height of the feature layer, is the data at the position of ij in the k-th channel of the feature layer A. The weight represents the partial linearization of the deep network downstream of the feature map A and obtains the importance of the k-th feature map of the target class c; then Grad-CAM performs a weighted combination of the forward activation maps and obtains the class discriminative localization map of class c through ReLU

[0054]

[0055] Finally, through a heat map with the same size as the convolutional feature map is generated to achieve the localization of the lesion area.

[0056] (5) Set a segmentation threshold in the heat map and obtain the lung lesion segmentation result according to the threshold.

[0057] As an improvement of the present invention, in step 5, the present invention processes the generated heat map of the lesion probability distribution to determine the final lesion segmentation result. The present invention calls Grad-CAM on a large number of publicly available COVID-19 chest CT datasets to generate a heat map of the lesion probability distribution, sets different thresholds by the dichotomy method in the generated heat map, and evaluates the lesion segmentation effect at this threshold to continuously approach the optimal segmentation threshold. Finally, the area higher than the optimal segmentation threshold in the heat map of the lesion probability distribution is retained as the final lesion segmentation result.

[0058] Compared with the prior art, the technical solution of the present invention has the following beneficial technical effects: For the problem of COVID-19 lesion segmentation, the present invention proposes a novel deep learning-based lesion segmentation method that only depends on class labels and does not depend on segmentation labels. Compared with other deep learning algorithms that rely on segmentation labels, the present invention can not only greatly reduce the manpower and time consumed by manual annotation, but also avoid the noisy labels introduced by manual annotation, thus ensuring the objectivity and accuracy of the segmentation results. The present invention can automatically classify CT images and segment lesions in the images whose classification prediction results are for patients' CT, while most other deep learning algorithms still require manual classification before lesion segmentation can be performed. At the same time, the present invention has strong generalization ability, can quickly locate the area where the lesion is located in the absence of lesion segmentation labels, and provide the probability information that the area is a lesion in the form of a heat map, further assisting doctors in finding the cause of the disease. In addition, the technical route of this research is not complicated to implement, requires a small computational cost, and has low requirements for hardware conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 is a flowchart of a COVID-19 lesion segmentation method based on Grad-CAM of the present invention;

[0060] Figure 2 is a flowchart of the conventional image preprocessing method adopted by the present invention;

[0061] Figure 3 is a flowchart of the image binarization preprocessing method adopted by the present invention;

[0062] Figure 4 is a flowchart of the algorithm for removing invalid information in the edge area in the image binarization preprocessing method adopted by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0063] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0064] Embodiment 1: As Figure 1 shown, the present invention proposes a COVID-19 lesion segmentation method based on Grad-CAM, and the detailed steps of this method are as follows:

[0065] (1) Perform conventional image preprocessing on CT images;

[0066] The conventional image preprocessing method generally includes image denoising, image enhancement, contrast adjustment, brightness adjustment, and size adjustment, etc. The present invention performs the conventional preprocessing as Figure 2 shown according to the more common problems in CT images.

[0067] Due to the presence of noise in some CT images, image denoising is first performed by median filtering. For CT images obtained from scans of different devices, the proportion and spatial distribution of the lung region in the entire image vary. At the same time, most CT images also have the problem of a relatively small spatial proportion of the lung region. Therefore, all CT images are cropped to retain only the region of interest (ROI) as much as possible and remove the non-lung regions in the images. In addition, the orientations of CT images obtained from scans of different devices are also different, so the orientations of all CT images are uniformly adjusted to the relationship where the upper part of the CT image corresponds to the front of the human body. Finally, the sizes of all CT images are uniformly adjusted to 512×512.

[0068] (2) Use an image binarization preprocessing method with OSTU as the core to enhance the contrast between lung lesions and healthy regions and highlight the edge information of the lesions;

[0069] Although non-lung regions have been removed as much as possible from the CT images after conventional image preprocessing, only the ROI is retained, and the image quality has been improved. However, in CT images, due to the characteristics of unclear contrast and edges of COVID-19 lesions, and the fact that CT images are presented in the form of grayscale images, it is still impossible to accurately and clearly distinguish between lung lesions and lung healthy regions in the CT images after conventional image preprocessing. Therefore, it is necessary to solve the above problems through further image preprocessing.

[0070] Based on the conventional image preprocessing method, the present invention presents a CT image binarization preprocessing method based on OSTU, that is, a binarization processing method with the maximum inter-class variance method as the core and main feature is used to enhance the contrast between lung lesions and lung healthy regions, provide more effective information, which is more conducive to the training of deep learning algorithms for COVID-19 chest CT classification and lesion segmentation, and improve the classification and lesion segmentation accuracy. The method flow is shown in Figure 3 .

[0071] To improve the effect of image binarization processing and retain as many detail information as possible, the present invention first adjusts the brightness of the CT images after conventional image preprocessing. By doubling its brightness, the lesions that may have been misjudged as the background during binarization due to being too dark are highlighted in the CT images, thus ensuring the integrity of the lesions as much as possible. Then the present invention continues to adjust the contrast of the CT images. Through adaptive histogram equalization processing, the contrast between lung lesions and lung healthy regions is enhanced, and the edge clarity of the lesions is improved, so that the edge and contour information of the lesions can be retained as accurately and completely as possible during the binarization process.

[0072] After completing preparatory work such as brightness adjustment and contrast adjustment, the present invention determines the optimal threshold for binarizing each CT image by the Otsu method. The threshold calculation method is as follows:

[0073] In the CT image I, assume there are L (L≥1) gray levels, and the number of pixels with gray level i (1≤i≤L) is s i , and the total number of pixels in the image I is S. Then the probability P i that all pixels with gray level i appear is

[0074]

[0075] Divide the image pixel set into the target pixel set C t and the background pixel set C b . Assume the optimal threshold is k. Pixels with gray level i in [1,k] are assigned to C t , and pixels with gray level i in [k + 1,L] are assigned to C b . Then the proportion ω t of the pixels in the target pixel set C b and the background pixel set C t in all the pixels of the image I are respectively b as follows

[0076]

[0077]

[0078] From the above two formulas, the target pixel mean μ t and the background pixel mean μ b are respectively

[0079]

[0080]

[0081] The between-class variance σ t and σ b are respectively

[0082]

[0083]

[0084] The total gray mean μ of the two is

[0085]

[0086] The total between-class variance σ 2 is

[0087] σ2 = ω t σ t 2 + ω b σ b 2

[0088] The optimal threshold Th is

[0089]

[0090] The above formula means taking the k when the total variance between classes σ 2 reaches the maximum value as the optimal threshold.

[0091] Each pixel point in the image is assigned 0 or 1 according to this threshold, thus completing the binarization process of the CT image. However, there are still some problems in the binarization result at this time, such as overfitting in the processing of the lung contour, loss of lung edge information, and invalid information in the image edge area. Therefore, further optimization processing is still required for the binarization result at this time.

[0092] To solve the problem of overfitting in the processing of the lung contour, the present invention performs binarization processing on the CT image before brightness adjustment and contrast adjustment, and the method used is still the maximum inter-class variance method; then the corresponding pixel points in the binarization result and the binarization result generated after brightness adjustment and contrast adjustment are subjected to an OR operation, so as to retain the complete lung contour while highlighting the detail information. Then, the binarization result after the OR operation is inverted, so that the main body of the lung appears as a white area, which is convenient for subsequent optimization processing.

[0093] To improve the lung edge information, the present invention extracts contour information from the CT image before brightness adjustment and contrast adjustment through the bwperim algorithm. However, since there are many textures inside the lung, various hole textures inside the lung are also extracted while extracting the lung contour. Therefore, before contour extraction, filling holes and dilation operations must be performed, and finally the complete lung contour information is obtained. The extracted lung contour information is subjected to an OR operation with the inverted binarization result above to obtain a binarization result with contour information.

[0094] To remove the invalid information in the image edge area, the present invention designs an algorithm that can automatically remove the white invalid information at the edge of the binary image without manual intervention. The algorithm flow is shown in Figure 4 .

[0095] This algorithm first classifies each pixel point in the image to determine whether it is in the white area or the black area in the binary image. Since there are also a small number of black pixel points in the white area of the binary image, when any one of the 9 related pixel points in the nine-grid centered on a certain pixel point is white, it is determined that the pixel point is in the white area, and the corresponding position in the classification matrix WorB is set to 1; otherwise, it is determined that the pixel point is in the black area, and the corresponding position in the classification matrix WorB is set to 0.

[0096] Then it is determined whether the classified white pixel points are located in the edge area. The algorithm traverses the left half of each row in the classification matrix WorB from left to right and the right half from right to left in turn; for each column, it traverses the upper half from top to bottom and the lower half from bottom to top to determine whether all white pixel points are located in the edge area. Finally, the judgment results in the four directions are subjected to a union operation to obtain the edge judgment matrix EorN. Taking the direction from left to right as an example, the judgment method is specifically described. For a certain row in the classification matrix WorB, starting from the leftmost element and traversing, when n consecutive 0s (i.e., black pixel points) are continuously traversed, the corresponding positions of all elements to the left of the first 0 in the continuous sequence are assigned 1 in the edge judgment matrix EorN, and the corresponding positions of all elements to the right and itself are assigned 0.

[0097] However, in some CT images, there are also cases where the lung area is located at the edge. Therefore, an anti-misjudgment algorithm needs to be added to avoid misjudging the lung area as edge invalid information and thus mistakenly removing it. Therefore, on the basis of the above traversal to judge whether it is in the edge area, an anti-misjudgment algorithm is added, that is, when traversing in a certain direction, if m points have been traversed but n consecutive 0s have not been continuously traversed, it is determined that the edge area is valid information, the traversal of this row (column) stops, and all elements of this row (column) are assigned 0.

[0098] Finally, the above algorithm is called in the binarization processing result of the improved lung edge information to generate the final CT image that has undergone conventional image preprocessing and image binarization preprocessing.

[0099] (3) Input the preprocessed CT image into a classification convolutional neural network for training;

[0100] Since Grad-CAM is a technology that identifies the feature region with the highest correlation with a specific category during the classification process, we can utilize the fact that the main difference between the patient's CT and the healthy person's CT is the presence or absence of lesions. During the classification process, Grad-CAM is used to identify and distinguish the feature regions of the two, that is, the lesions, so as to achieve the segmentation of chest lesions without pixel-level segmentation labels. Therefore, before using Grad-CAM, the present invention needs to perform binary classification on the patient's CT and the healthy person's CT.

[0101] Specifically, in the present invention, the pre-trained SqueezeNet1.1 is called in PyTorch as the classification neural network used in the present invention, and the value of out_channels of its last convolutional layer is modified to 2. The dataset processed by the conventional image preprocessing and binarization preprocessing methods is loaded, and the model training parameters are set, where epochs = 100, and then the training starts. After training, the SqueezeNet1.1 model with a high classification accuracy is obtained.

[0102] (4) Call Grad-CAM in the trained classification convolutional neural network to generate a heatmap for feature region localization for CT image classification;

[0103] Since Grad-CAM can be applied to any CNN-based architecture, it can be directly used in any classification neural network without changing the network structure to generate a heatmap and perform lesion segmentation. The present invention uses the trained SqueezeNet1.1 to classify CT images, uses Grad-CAM to identify the feature region on the CT image with the classification prediction result being the patient's CT, and generates a heatmap for feature region localization according to the probability size, that is, obtains a heatmap of lesion probability distribution in the CT image with the classification prediction result being the patient's CT. The specific calculation method is as follows:

[0104] To obtain the class discriminative localization map for any class c Grad-CAM first performs forward propagation to calculate the score gradient y of class c before the softmax layer c , where y c is related to the feature map A of the convolutional layer k (k represents the kth channel in the feature layer A); the calculated gradient is backpropagated and global average pooling is performed to obtain the neuron importance weight

[0105]

[0106] where Z is the product of the width and height of the feature layer, is the data at the position of ij in the kth channel of the feature layer A. The weight represents the partial linearization of the deep network downstream of the feature map A and obtains the importance of the kth feature map of the target class c; then Grad-CAM performs a weighted combination of the forward activation map and obtains the class discriminative localization map of class c through ReLU

[0107]

[0108] Finally, through a heat map with the same size as the convolutional feature map is generated to achieve the localization of the lesion area.

[0109] (5) Set a segmentation threshold in the heat map, and obtain the lung lesion segmentation result according to the threshold.

[0110] Since the lesions identified by Grad-CAM are presented by generating a heat map, the present invention needs to process the generated original heat map to determine the final lesion segmentation result. The present invention calls Grad-CAM on a large number of publicly available COVID-19 chest CT datasets to generate a heat map of the lesion probability distribution, sets different thresholds by the bisection method in the generated heat map, and evaluates the lesion segmentation effect at this threshold to continuously approach the optimal segmentation threshold. Through the analysis of a large amount of data, the present invention sets the optimal segmentation threshold to 0.32, and retains the area above the optimal segmentation threshold in the heat map of the lesion probability distribution as the final lesion segmentation result.

[0111] It should be noted that the above embodiments are only the preferred embodiments of the present invention, and do not limit the protection scope of the present invention. Any equivalent replacement or substitution made on the basis of the above technical solutions belongs to the protection scope of the present invention.

Claims

1. A method for segmenting COVID-19 lesions based on Grad-CAM, characterized in that, The method includes the following steps: Step 1: Perform conventional image preprocessing on the CT image; Step 2: Use an image binarization preprocessing method with OSTU as the core to enhance the contrast between the lung lesions and the healthy area, and highlight the edge information of the lesions; Step 3: Input the preprocessed CT image into a classification convolutional neural network for training; Step 4: Call Grad-CAM in the trained classification convolutional neural network to generate a feature region localization heat map for CT image classification; Step 5: Set a segmentation threshold in the heat map, and obtain the lung lesion segmentation result according to the threshold; In the said Step 1, the method for performing conventional image preprocessing on the CT image is as follows: Step 1.1: Denoise the CT image using median filtering; Step 1.2: Crop the CT image to retain the ROI; Step 1.3: Unify the correspondence between the CT image direction and the human body direction; Step 1.4: Unify the size of the CT image; In the said Step 2, it includes the following steps: Step 2.1: Adjust the brightness of the CT image; Step 2.2: Adjust the contrast of the CT image using adaptive histogram equalization; Step 2.3: Use OSTU to determine the binarization threshold of the CT image and perform binarization processing; Step 2.4: Retain the complete lung contour by performing a union operation on the binary image; Step 2.5: Invert the result of the binarization processing; Step 2.6: Use hole filling and dilation to extract the edge information of the lung contour; Step 2.7: Use the algorithm designed in the present invention to remove the invalid information in the image edge area; In the said Step 2, In the said Step 2.3, the best threshold for binarization processing of each CT image is determined by the maximum inter-class variance method, and the threshold calculation method is as follows: In the CT image I, it is assumed that there are L gray levels, where L ≥ 1. The number of pixels with gray level i is s i , 1 ≤ i ≤ L. The total number of pixels in the image I is S. Then the probability P that the number of pixels with all gray levels i appears i is Divide the set of image pixels into the target pixel set C t and the background pixel set C b . Assume that the optimal threshold is k, and the pixels with gray level i in [1, k] are classified into C t , and the pixels with gray level i in [k + 1, L] are classified into C b ; then the proportion ω t of the pixels in the target pixel set C b and the background pixel set C t among all the pixels in the image I b are respectively From the above two equations, the mean value μ of the target pixels can be obtained t and the mean value μ of the background pixels b are respectively Between-class variance σ t and σ b are respectively The total gray mean value μ of the two is Total between-class variance σ 2 is σ 2 = ω t σ t 2 + ω b σ b 2 The best threshold Th is The above formula represents taking the total variance σ between classes 2 The k at the maximum value is used as the optimal threshold; each pixel point in the image is assigned 0 or 1 according to this threshold, thereby completing the binarization process of the CT image; In the said Step 3, before using Grad-CAM, use the classification convolutional neural network to perform binary classification on the patient's CT and the healthy person's CT; In the said Step 4, use the trained classification convolutional neural network to classify the CT image. On the CT image with the classification prediction result of the patient's CT, use Grad-CAM to identify the feature region, and generate a feature region localization heat map according to the probability size, that is, obtain the lesion probability distribution heat map in the CT image with the classification prediction result of the patient's CT; the specific calculation method is: To obtain the class discriminant localization map for any class c Grad-CAM first performs forward propagation to calculate the score gradient y of class c before the softmax layer c , where y c is related to the feature map A of the convolutional layer k , k represents the k-th channel in the feature layer A; the calculated gradient is backpropagated and globally average pooled to obtain the neuron importance weight where Z is the product of the width and height of the feature layer, is the data at the position of coordinates ij in channel k of the feature layer A, and the weight represents the partial linearization of the deep network downstream of the feature map A and obtains the importance of the feature map k for the target class c; then Grad-CAM performs a weighted combination on the forward activation map and obtains the class discriminant localization map for class c through ReLU Finally, through generate a heat map with the same size as the convolutional feature map to achieve the localization of the lesion area.

Images