A fracture false positive screening method based on curriculum learning and spatial attention

Abstract

The present invention belongs to the field of medical image processing technology, and specifically is a false positive screening method based on curriculum learning and spatial attention. In the case of fracture detection, the detection model is prone to generate a large number of false positive prediction results due to the high similarity between bones. The present invention proposes a false positive screening network Rib-FPR Net, which uses a spatial attention mechanism to enhance the network's ability to learn the features of the fracture area, and combines it with a curriculum learning training model to effectively enhance the network's false positive screening capability. The present invention is simple to implement and highly efficient. Combining curriculum learning and spatial attention mechanisms effectively solves the problem of excessive false positive results, thereby greatly enhancing the auxiliary diagnosis effect of fractures.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical image processing, and in particular relates to a method for screening false positive fractures based on curriculum learning and spatial attention. Background Art

[0002] Due to the inherent characteristics of fractures in CT images, such as subtle structure and high similarity in grayscale distribution between bones, existing fracture detection networks are unable to distinguish between true positive fracture candidate areas and false positive fracture candidate areas. On the other hand, from the perspective of the working principle of the fracture detection network, since the input of the detection network is an image block after sliding window cropping, a large number of image blocks only contain partial cropped units of a certain section of bone. The image block will have broken lines due to cropping, and the characteristic information of the lines will affect the network's learning of fracture feature information. These factors result in the output of a large number of false positive prediction results despite the existing fracture detection network being able to achieve a high fracture detection sensitivity. In clinical applications, too many false positive prediction results from the detection network may interfere with the doctor's diagnosis and increase the doctor's diagnostic burden.

[0003] Therefore, the present invention provides a false positive screening method based on curriculum learning and spatial attention, thereby effectively screening out a large number of false positive candidate regions and achieving good clinical auxiliary diagnosis effects.

[0004] The prior art related to the present invention includes the following references:

[0005] [1]Ogul BB,Kosucu P,Ozcam A,et al.Lung Nodule Detection in X-RayImages:A New Feature Set[C].European Conference of the InternationalFederation for Medical and Biological Engineering,2015.

[0006] [2]Hamidian S, Sahiner B, Petrick N, et al. 3D Convolutional NeuralNetwork for Automatic Detection of Lung Nodules in Chest CT[C]. MedicalImaging 2017: Computer-Aided Diagnosis, 2017.

[0007] [3]Gong J,Liu J,Wang L,et al.Automatic detection of pulmonary nodulesin CT images by incorporating 3D tensor filtering with local image featureanalysis[J].Physical Medical European Journal of Medical Physics,2018,46:124–133.

[0008] [4] He Xuecai, Jin Jing, Li Ming, et al. Rib fracture detection method based on fully integrated network candidate frame[J]. Acta Anatomica Sinica, 2022, 53(3): 396-401. Summary of the Invention

[0009] The purpose of the present invention is to propose a fracture false positive screening method based on curriculum learning and spatial attention to screen out a large number of fracture false positive candidate region results.

[0010] The fracture false positive screening method based on curriculum learning and spatial attention proposed in this paper uses the spatial attention mechanism to enhance the feature learning ability of 3D ResNet50 for the fracture area; cooperates with the curriculum learning training model to enhance the classification ability of the network; and achieves efficient false positive screening through effective classification of true and false positive candidate areas.

[0011] In this invention, the spatial attention mechanism (see Figure 2 The design principle of ) is as follows: use global average pooling GloAvg() to process the input feature image X to obtain the feature map M, and then use the adaptive function Adaptive sigmoid() to obtain the spatial attention map S. The feature map X is then weighted to obtain the spatial attention feature map X′, which makes the network pay more attention to the features related to the lesion and achieves better lesion classification effect.

[0012] The false positive screening method based on curriculum learning and spatial attention proposed in the present invention has the following specific steps.

[0013] Step 1: First, preprocess the image, including intensity normalization, histogram equalization, and cropping image blocks.

[0014] Specifically, the rib fracture CT image is divided into many small blocks of pixels and nonlinearly stretched to make the local grayscale histogram uniformly distributed; the third-order spline interpolation method is used to resample all voxels to a uniform size; the nearest neighbor interpolation method is used for corresponding segmentation and annotation; and random transformations including random rotation, shearing, scaling, and flipping operations are used to enhance the data.

[0015] Step 2: Build a network model; the network model includes the 3DResNet50 network and the spatial attention mechanism module SAM (see Figure 1 ), denoted as Rib-FPRNet; the 3DResNet50 network consists of a convolutional layer "7*7*7Conv64", Conv, batch normalization and nonlinear layer BN / Relu, pooling layer Pool, global average pooling GloAvgPool and a fully connected layer FC2; the network model uses convolution, batch normalization and pooling operations to obtain the feature map of the input image, and combines the spatial attention mechanism to obtain the attention feature map of the feature map, and then obtains the classification confidence through global average pooling and full connection operations.

[0016] Among them, the attention feature map of the feature map obtained by the spatial attention mechanism is expressed as:

[0017]

[0018] Where X represents the input feature image, X′ is the attention feature map obtained by processing X through global average pooling GloAvg() and adaptive activation function Adaptive sigmoid();

[0019] The loss function calculation formula is as follows:

[0020]

[0021] L CE is the loss function, y i ∈{0,1}, represents the true category of sample i. When the sample is a true positive rib fracture area, y i is 1, otherwise, y i is 0; represents the probability that sample i is predicted to be a true positive rib fracture area, N represents the total number of samples.

[0022] Step 3: All test results in the training set are classified by difficulty based on confidence. First, the Rib-FPRNet network model is trained on the training set using a 5-fold cross-classification. This involves using a different 20% of the dataset as the validation set. The classification confidence for all data samples in the training set is obtained by performing five predictions on a different 20% of the dataset. Three confidence thresholds are then used to distinguish the difficulty of each sample: 0.8, 0.5, and 0.2. For true-positive fracture candidate regions, a prediction confidence greater than 0.8 is considered very simple; between 0.5 and 0.8 is considered easy; between 0.2 and 0.5 is considered difficult; and less than 0.2 is considered very difficult. For false-positive fracture candidate regions, the difficulty classification criteria are the opposite of the above.

[0023] Among them, the training loss function is defined as follows:

[0024]

[0025] Among them, y i ∈{0,1}, represents the true category of sample i. When the sample is a true positive rib fracture area, y i is 1, otherwise, y i is 0. represents the probability that sample i is predicted to be a true positive rib fracture area, N represents the total number of samples.

[0026] Step 4: Training the network Rib-FPRNet uses a curriculum-based training method. The network first uses all very simple sample image blocks X as the training set. After training and fitting, it supplements with simple samples, difficult samples, and very difficult samples in sequence for training to obtain model parameters. The trained network Rib-FPRNet is used to classify true and false positives in the fracture area.

[0027] Step 5: False positives are screened out using the classification results of the above true and false positive candidate regions, thereby obtaining the final fracture auxiliary diagnosis results, including statistical indicators such as recall rate, false positive screening ratio (False Positive Reduction Rate, FPRR), true false positive candidate region ratio (True False Rate, TFR), and the average number of false positive candidate regions per case (False Positive Rate, FPR). The specific calculation formula is as follows:

[0028]

[0029] Among them, TP (True Positive) represents the number of true positive candidate regions predicted as positive (with fracture), FN (False Negative) represents the number of true positive candidate regions predicted as negative (without fracture), NFP (Number of False Positive) represents the number of false positive candidate regions before screening, NumP (Number of Patient) represents the number of cases, RFP (Remaining False Positive) represents the number of false positive candidate regions remaining after screening using the classification results, and RTP (Remaining True Positive) represents the number of true positive candidate regions remaining after screening using the classification results.

[0030] Corresponding to the above-mentioned false positive screening method, the present invention is a false positive screening system based on curriculum learning and spatial attention, which specifically includes five modules: image preprocessing module, network building module, confidence difficulty classification module, Rib-FPR Net network training module, and false positive screening module; the five modules respectively execute the five steps in the false positive screening method.

[0031] In the present invention, the attention mechanism improves the network's ability to learn the features of fracture candidate areas. At the same time, the curriculum learning training model improves the network's ability to distinguish between true and false positive candidate areas, achieving a good false positive screening effect.

[0032] Compared with the existing technology, the "false positive screening network based on curriculum learning and spatial attention" of the present invention has the following advantages:

[0033] (1) The spatial attention mechanism enables the network to focus more on the characteristics of the fracture area, improving the classification effect;

[0034] (2) The course learning training model improves the network classification ability and thus improves the false positive screening ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 Illustration of the Rib-FPRNet network structure of the present invention.

[0036] Figure 2 Schematic diagram of the spatial attention mechanism.

[0037] Figure 3 Schematic diagram of false positive screening results. White arrows indicate false positive candidate regions that were effectively screened out, and white circles indicate false positive candidate regions that were not effectively screened out. DETAILED DESCRIPTION

[0038] Example 1: Screening for false positives of rib fractures using the method of the present invention

[0039] The specific implementation process of the false positive screening network based on curriculum learning and spatial attention provided by the present invention is as follows:

[0040] Step 1: The training data set consists of 420 cases from the RibFrac dataset. Images are first subjected to intensity normalization and histogram equalization. The entire image is divided into many small blocks of pixels and nonlinearly stretched to uniformly distribute the local grayscale histogram. To enable the network to correctly learn spatial semantics, all CT voxels are resampled to a uniform size using third-order spline interpolation. Nearest neighbor interpolation is used for corresponding segmentation annotation. Each training iteration involves random transformations, including rotation, shearing, scaling, and flipping, to enhance the data.

[0041] Step 2: Build a network model; the network model includes the 3DResNet50 network and the spatial attention mechanism module SAM (see Figure 1 ), denoted as Rib-FPRNet; spatial attention mechanism module SAM (see Figure 2 The 3DResNet50 network consists of a 7x7x7Conv64 convolutional layer, Conv, batch normalization and nonlinear layers (BN / Relu), pooling layers (Pool), global average pooling (GloAvgPool), and a fully connected layer (FC2). The Rib-FPRNet network uses convolution, batch normalization, and pooling to obtain a feature map of the input CT image of the fracture. It then combines this with a spatial attention mechanism to obtain a rib fracture attention feature map within the feature map. It then uses global average pooling and a fully connected layer to determine the rib fracture classification confidence.

[0042] Step 3: Divide all data samples into different levels of difficulty. First, a 64×64×64 image block is cropped around the center point of the prediction result generated by the detection network (the network is the Rib-Net proposed in reference [4]), and the 3DResNet50 (spatial attention) network is cross-trained on the dataset in 5 folds, that is, a different 20% of the dataset is used as the validation set each time. By predicting a different 20% of the dataset 5 times, the classification confidence of all data samples in the dataset can be obtained. We then use three confidence thresholds to distinguish the difficulty of the samples, namely 0.8, 0.5, and 0.2. For true positive fracture candidate regions, when the prediction confidence is greater than 0.8, the sample is a very simple sample; when the confidence is between 0.5 and 0.8, the sample is a simple sample; when the confidence is between 0.2 and 0.5, the sample is a difficult sample; and when the confidence is less than 0.2, it is a very difficult sample. For false positive fracture candidate regions, the difficulty classification criteria are opposite to the above criteria.

[0043] Step 4, train the classification network Rib-FPRNet that distinguishes true and false positive candidate areas, and the training adopts a curriculum-based training method. Rib-FPR Net needs to complete 4 stages of training and learning. First, the Rib-FPR Net network uses all very simple samples as the training set. When the network model reaches the optimal value, simple samples are added to the training set to further improve the network's learning ability for slightly difficult fracture feature information. Subsequently, in the same way, difficult samples and very difficult samples are added in turn until the network has learned all the data and fits to the optimal value. During the training process, the optimization algorithm uses the classic Adam algorithm, and the initial learning rate is set to 0.001. The learning rate decreases with the increase in the number of training times.

[0044] Step 5: Statistically analyze the classification results of true and false positive candidate regions based on the set classification threshold, and finally filter out false positive candidate regions based on the classification results.

[0045] The test set consists of 60 patients. The classification results on the test set are shown in Table 1, the false positive screening results are shown in Table 2, and the false positive screening results are shown in Figure 3 shown.

[0046] Table 1

[0047] AUC ACC 3D ResNet50 0.757 0.712 Rib-FPR Net 0.776 0.731 Rib-FPR Net+ Course Learning Strategies (ours) 0.791 0.742

[0048] Table 2

[0049]

[0050] The experimental results of this embodiment show that the present invention achieves good true and false positive classification effects; the spatial attention mechanism effectively improves the network's ability to learn the features of the fracture area, and the curriculum learning training model improves the network's classification ability. Based on the classification results, a good false positive screening effect is achieved. The Rib-FPR Net network can screen out most of the false positive candidate areas while maintaining a high recall rate. When the recall rate is the same as the recall rate of the joint diagnosis of two doctors (0.831), it is slightly better than the result of the joint diagnosis of two doctors. Moreover, this method is simple to implement and can be applied to other types of false positive screening tasks of fractures.

Claims

1. A false positive screening method based on curriculum learning and spatial attention, characterized in that: Use spatial attention mechanism to improve 3D ResNet50's ability to learn features of fracture areas; The network's classification ability is improved by coordinating with a course-based training model; efficient false positive screening is achieved through effective classification of true and false positive candidate regions; The spatial attention mechanism simulates the way human visual cells observe objects, focusing attention on regions of interest to prevent interference from disruptive information on network performance. This mechanism uses the spatial attention map to weight the feature map, allowing the network to focus more on features related to lesions, achieving better lesion classification results. The specific steps are as follows: Step 1: First, preprocess the fracture CT image, including intensity normalization, histogram equalization, and image block cropping. Step 2: Build a network model. This network model includes a 3DResNet50 network and a spatial attention mechanism module (SAM), denoted as Rib-FPRNet. The 3DResNet50 network consists of a 7*7*7Conv64 convolutional layer, Conv, a batch normalization and nonlinear layer (BN / Relu), a pooling layer (Pool), a global average pooling (GloAvgPool), and a fully connected layer (FC2). This network model uses convolution, batch normalization, and pooling operations to obtain the feature map of the input image, and combines the spatial attention mechanism to obtain the attention feature map of the feature map. It then obtains the classification confidence through global average pooling and fully connected operations. Among them, the attention feature map of the feature map obtained by the spatial attention mechanism is expressed as: Where X represents the input feature image, X′ is the attention feature map obtained by processing X through global average pooling GloAvg() and adaptive activation function Adaptive sigmoid(); Step 3: Divide the difficulty of all test results in the training set according to the confidence level. First, perform five classification cross-training network models Rib-FPRNet on the training set, that is, each time use a different 20% data set as the validation set, and obtain the classification confidence of all data samples in the training set by predicting a different 20% data set five times. Then, three confidence thresholds are used to distinguish the difficulty of the samples, and the three confidence thresholds are 0.8, 0.5, and 0.2 respectively. For true positive fracture candidate areas, when the prediction confidence is greater than 0.8, the sample is a very simple sample; when the confidence is between 0.5 and 0.8, the sample is a simple sample; when the confidence is between 0.2 and 0.5, the sample is a difficult sample; when the confidence is less than 0.2, it is a very difficult sample. For false positive fracture candidate areas, the difficulty classification criteria are opposite to the above criteria. Step 4: Training the network Rib-FPRNet using a curriculum-based training method. First, all very simple sample image blocks X are used as the training set. After training and fitting, simple samples, difficult samples, and very difficult samples are added for training in sequence to obtain model parameters. The trained network Rib-FPRNet is used to classify true and false positive fracture areas. Step 5: Use the classification results of the above true and false positive candidate regions to screen out false positives and obtain the final fracture auxiliary diagnosis results, including the recall rate, false positive screening ratio (FPRR), true and false positive candidate region ratio (TFR), and the average number of false positive candidate regions per case (FPR). The specific calculation formula is as follows: Among them, TP represents the number of true positive candidate regions predicted as positive, FN represents the number of true positive candidate regions predicted as negative, NFP represents the number of false positive candidate regions before screening, NumP represents the number of cases, RFP represents the number of false positive candidate regions remaining after screening using the classification results, and RTP represents the number of true positive candidate regions remaining after screening using the classification results.

2. The fracture false positive screening method based on curriculum learning and spatial attention according to claim 1, characterized in that: The image is preprocessed as described in step 1, specifically including: dividing the fracture CT image into several small blocks of pixels and performing nonlinear stretching to make the local grayscale histogram uniformly distributed; using third-order spline interpolation to resample all voxels to a uniform size; using nearest neighbor interpolation for corresponding segmentation and annotation; and using random transformations including random rotation, shearing, scaling, and flipping operations to enhance the data.

3. The false positive screening method based on curriculum learning and spatial attention according to claim 1, characterized in that: In step 2, the loss function calculation formula involved in the Rib-FPRNet network model is as follows: y i ∈{0,1}, represents the true category of sample i. When the sample is a true positive fracture area, y i is 1, otherwise, y i is 0; represents the probability that sample i is predicted to be a true positive rib fracture area, N represents the total number of samples.

4. A system based on the false positive screening method according to any one of claims 1 to 3, characterized in that: It includes five modules: image preprocessing module, network construction module, confidence difficulty classification module, Rib-FPR Net network training module, and false positive screening module; the five modules respectively perform the five steps in the false positive screening method.

Images