An Improved Self-Supervised Detection Method for Mammography Images

Abstract

This invention belongs to the field of computer-aided medical image technology, specifically relating to an improved self-supervised detection method for mammogram images. Based on Faster R-CNN and combined with improved self-supervised pre-training, this invention can effectively detect lesion regions in mammogram images and classify them as benign or malignant. Using a computer-aided diagnostic system to assist doctors in diagnosis significantly reduces their workload, which is of great significance. Compared to natural images, medical image data is difficult to obtain and annotation is expensive. Therefore, the object detection in this invention combines self-supervised pre-training, allowing the model to achieve better results using unlabeled images.

Description

Technical Field

[0001] This invention belongs to the field of computer-aided medical imaging technology, specifically relating to an improved self-supervised detection method for mammography images. Background Technology

[0002] Cancer is the second leading cause of death worldwide, with breast cancer accounting for 19% of all cancer cases and 30% of all cancers in women, making it one of the most common cancers among women. While breast cancer cannot be effectively prevented, timely diagnosis and treatment can significantly improve the survival rate of breast cancer patients and greatly reduce the cost of cancer treatment. As with most other chronic diseases, early breast cancer detection should consider two very clear clinical situations: early diagnosis in symptomatic patients and screening in asymptomatic individuals.

[0003] Early detection and screening of breast cancer primarily rely on two imaging methods: breast ultrasound and mammography. Ultrasound imaging offers advantages such as high resolution from high-frequency probes and no radiation exposure. However, it cannot clearly visualize extremely small, fine, sand-like or punctate calcifications. While mammography involves some radiation, the carcinogenic risk is close to the natural incidence rate. Mammography is highly sensitive to microcalcifications within the breast, a characteristic feature of breast cancer. It is particularly effective in diagnosing large breasts and fatty breasts, and is not limited by body type. Therefore, mammography results can be considered one of the gold standards for early breast cancer screening.

[0004] Reading mammograms remains challenging for radiologists, and the workload is enormous. With the development of deep learning, computer-aided systems can significantly reduce doctors' workload and prevent missed diagnoses and misdiagnoses caused by doctors' fatigue during image reading. Summary of the Invention

[0005] The purpose of this invention is to provide an improved self-supervised detection method for mammography images.

[0006] An improved self-supervised detection method for mammogram images includes the following steps:

[0007] Step 1: Obtain publicly available mammography datasets;

[0008] Step 2: Perform data augmentation on the molybdenum target images in the dataset;

[0009] Step 3: Self-supervised pre-training;

[0010] The BYOL framework was built. Based on BYOL, the prediction head was removed, and the backbone network was replaced with ResNet18 instead of ResNet50. On this basis, scale-invariant learning rules were added. The feature representation was adjusted to different grid sizes through several average pooling layers with different kernel sizes and strides to generate a set of spatial pyramid clipping for EMD.

[0011] After extracting features from the molybdenum target image through the backbone network, two feature maps W are obtained. The two feature maps are expanded into HW C-dimensional vectors, i.e., X = {x i |i=1,2,…,HW},Y={y j |j=1,2,…,HW}, the distance between two feature maps is defined using EMD distance:

[0012]

[0013] Among them, c ij It is the distance between two feature maps; the similarity between two feature maps is defined by the distance between them.

[0014]

[0015] in, It represents the optimal transformation between two feature maps, that is, treating the two feature maps as the resource and the destination, respectively. It is the optimal conversion that transports resources to their destination at the lowest transportation cost;

[0016] The loss function is defined as follows:

[0017] L = 2 - 2S(X,Y)

[0018] After obtaining the loss, the network weight parameters are updated according to the gradient descent algorithm. The formula for updating the momentum of the weights of the backbone network of the target network is:

[0019] ξ=mξ+(1-m)θ

[0020] Where ξ is the weight value to be updated; m is the given momentum rate, and m∈[0,1];

[0021] Step 4: Train the Faster R-CNN network;

[0022] Step 4.1: Train the Region Proposal Network (RPN); Initialize the RPN network using the pre-trained model from Step 3, and begin end-to-end training of the RPN separately to generate region proposals;

[0023] Step 4.2: Train the R-CNN network; freeze the parameters of the convolutional and fully connected layers unique to the RPN network, initialize the parameters of the pre-shared convolutional network using the pre-trained model in Step 3, and train the parameters of the Fast R-CNN network using the target candidate boxes generated by the RPN network;

[0024] Step 4.3: Optimize the region candidate network; freeze the parameters of the pre-shared convolutional network layers trained using Fast R-CNN, and fine-tune the parameters of the convolutional layers and fully connected layers unique to the RPN network.

[0025] Step 4.4: Optimize the R-CNN network; Freeze the parameters of the convolutional and fully connected layers unique to the RPN network, and perform fine-tuning optimization training on the parameters of the fully connected layers of the R-CNN network;

[0026] Step 5: Input the mammogram image to be detected into the trained Faster R-CNN network to obtain the detection results of the lesion area in the mammogram image.

[0027] Furthermore, the mammography public dataset in step 1 is an updated and standardized version of a digital database used for screening mammography examinations, containing normal, benign, and malignant cases with validated pathological information.

[0028] Furthermore, the data augmentation methods include random horizontal flipping, random cropping, modifying image brightness, modifying image contrast, normalization, and style transfer; the style transfer uses a CycleGAN network to convert between mammogram images of various styles.

[0029] The beneficial effects of this invention are as follows:

[0030] This invention, based on Faster R-CNN and combined with improved self-supervised pre-training, provides an improved self-supervised detection method for mammography images. This invention can effectively detect lesion regions in mammography images and classify them as benign or malignant. It assists doctors in making diagnoses using computer-aided diagnostic systems, significantly reducing their workload, which is of great significance. Medical image data is more difficult to obtain and more expensive to annotate than natural images; therefore, the object detection in this invention combines self-supervised pre-training, allowing the model to achieve better results using unannotated images. Attached Figure Description

[0031] Figure 1 These are three styles of mammogram images.

[0032] Figure 2 This is a comparison chart of the BYOL algorithm and the improved BYOL algorithm.

[0033] Figure 3 This is a flowchart of the Faster R-CNN algorithm. Detailed Implementation

[0034] The present invention will now be further described with reference to the accompanying drawings.

[0035] This invention, based on Faster R-CNN and combined with improved self-supervised pre-training, provides a method for detecting lesion regions in mammography. Mammography images, also known as breast X-rays, are commonly used for breast cancer screening, serving as the imaging standard for breast cancer screening, similar to breast ultrasound. Compared to breast ultrasound images, mammography images are better at revealing small lesions, resulting in larger images and smaller lesions, which increases the difficulty for radiologists in interpreting the images. Manual interpretation is labor-intensive; due to the small size of the lesions, they are difficult to detect, often requiring two or three doctors to review the images and provide a reliable diagnosis to prevent missed or misdiagnosis. Therefore, the difficulty is high. This invention can effectively detect lesion regions in mammography images and classify them as benign or malignant. Using a computer-aided diagnostic system (CAD) to assist doctors in diagnosis significantly reduces their workload, which is of great significance. Medical image data is more difficult to obtain and labeling is more expensive than natural image data. Therefore, the target detection method in this invention combines self-supervised pre-training, allowing the model to achieve better results using unlabeled images.

[0036] The present invention aims to use a computer-aided diagnostic system to detect lesion areas and diagnose benign or malignant conditions in mammograms. The goal is to develop an effective computer-aided diagnostic system to help radiologists reduce their workload and the probability of misdiagnosis and missed diagnosis when interpreting mammograms.

[0037] This invention is based on the Faster R-CNN object detection algorithm and uses an improved BYOL algorithm for model pre-training, which can effectively solve the problem that the limited availability and high cost of annotation in medical datasets make it difficult to train deep learning models. This invention includes the following steps:

[0038] Step 1: Obtaining Datasets. Obtain the publicly available mammography dataset CBIS-DDSM. CBIS-DDSM (a selected breast imaging subset of DDSM) is an updated and standardized version of a digital database for mammography screening, containing 2620 scan film mammography studies. It includes normal, benign, and malignant cases with validated pathological information. Obtain the publicly available mammography dataset INBreast. The INBreast dataset contains 115 cases, with 90 cases having 4 images per case and 25 cases having 2 images per case. Mammography images were also obtained from collaborating hospitals during the development of this invention.

[0039] Step 2: Data Preprocessing. Data augmentation is performed on the mammogram images. Augmentation methods include: random horizontal flipping, random cropping, image brightness modification, image contrast modification, normalization, and style transfer. Style transfer uses a CycleGAN network to convert between different styles of mammogram images. Different styles of mammogram images are shown below. Figure 1 As shown, (a), (b), and (c) are mammograms in common styles, respectively. This invention relates to mammograms in the above three styles. Style transformation involves random transformations between the three styles of mammograms.

[0040] Data augmentation methods include style transfer because mammograms taken by different machines do not have completely consistent styles. This is visually reflected in differences in the average grayscale of breast tissue, which in turn leads to variations in the visibility of lesion areas. Therefore, to improve the model's generalization ability, this invention uses a CycleGAN network for style transfer training during the data preprocessing step.

[0041] Step 3: Self-supervised pre-training. Build the BYOL framework. Based on BYOL, remove the prediction head, extract features from the image through the backbone network, and obtain two feature maps, denoted as X. The two feature maps are expanded into HW C-dimensional vectors, i.e., X = {x i |i=1,2,…,HW},Y={y j |j=1,2,…,HW}, the distance between two feature maps is defined using EMD distance, which is defined by equation (1):

[0042]

[0043] Where c ij It is the distance between two feature maps. The similarity between two feature maps is defined by the distance between them:

[0044]

[0045] in It represents the optimal transformation between two feature maps, that is, treating the two feature maps as the resource and the destination, respectively. This represents the optimal transformation that minimizes the transportation cost of resources to their destination. The similarity between the two feature maps is obtained, and the loss function is defined as follows:

[0046] L=2-2S(X,Y) (3)

[0047] After obtaining the loss, we can update our network weight parameters according to the gradient descent algorithm. The weights of the backbone network of the target network are updated with momentum using equation (4):

[0048] ξ=mξ+(1-m)θ (4)

[0049] Where ξ is the weight value to be updated, m is the given momentum rate, and m∈[0,1].

[0050] Comparison of BYOL processes before and after improvement, for example Figure 2 As shown, the backbone network replaces ResNet50 with ResNet18. This reduces the number of parameters, allows for training on larger mammogram images to minimize data loss, and focuses on the low-level shape features of mammogram images. Building upon this, scale-invariant learning rules are incorporated. Since object detection involves not only classification but also localization, the threat posed by changes in object scale is particularly significant. Therefore, explicit scale-invariant learning rules are designed: the feature representation is adjusted to different grid sizes (i.e., 7×7, 5×5, and 3×3) through several average pooling layers with different kernel sizes and strides, generating a set of spatial pyramid clipping (SPC) for EMD.

[0051] The network was trained using a large subset of unlabeled mammogram images and fine-tuned using a small subset of labeled mammogram images. Furthermore, due to the difference between mammogram images and natural images, random initialization and self-supervised pre-training performed better than networks pre-trained using natural images.

[0052] Step 4: Train the Faster R-CNN network. The principle of Faster R-CNN is as follows: Figure 3 .

[0053] Step 4.1: Train the Region Proposal Network (RPN). Initialize the RPN network using the pre-trained model from Step 3, and begin end-to-end training of the RPN to generate region proposals.

[0054] Step 4.2: Train the R-CNN network. Freeze the parameters of the convolutional and fully connected layers unique to the RPN network, initialize the parameters of the pre-shared convolutional network using the same pre-trained model, and train the Fast R-CNN network parameters using the target candidate boxes generated by the RPN network.

[0055] Step 4.3: Optimize the Region Candidate Network. Freeze the parameters of the pre-shared convolutional network layers trained using Fast R-CNN, and fine-tune the parameters of the convolutional layers and fully connected layers unique to the RPN network.

[0056] Step 4.4: Optimize the R-CNN network. Similar to Step 4.2, freeze the parameters of the convolutional and fully connected layers unique to the RPN network, and perform fine-tuning optimization training on the parameters of the fully connected layers of the R-CNN network.

[0057] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An improved self-supervised detection method for mammography molybdenum target images, characterized in that, The method comprises the following steps: Step 1: obtaining a breast molybdenum target public data set; Step 2: data enhancement is performed on the molybdenum target images in the data set; Step 3: self-supervised pre-training; A BYOL framework is built, and on the basis of the BYOL, the prediction head is removed, and the backbone network is replaced by ResNet18 instead of ResNet50; on this basis, a scale-invariant learning rule is added, and a group of spatial pyramid cropping for EMD is generated by adjusting the feature representation to different grid sizes through several average pooling layers with different kernel sizes and steps; After extracting features from the molybdenum target image through the backbone network, two feature maps were obtained. The two feature maps are expanded into HW C-dimensional vectors, i.e., X = {x i |x=1,2,…,HW},Y={y j |j=1,2,…,HW}, the distance between two feature maps is defined using EMD distance: where c ij is the distance between two feature maps; the similarity between two feature maps is defined by the distance between them: wherein, is an optimal transformation between two feature maps, that is, taking two feature maps as resource and destination respectively, is an optimal transformation of the lowest transportation cost of transporting the resource to the destination; The loss function is defined as follows: L = 2 - 2S(X, Y) After obtaining the loss, the network weight parameters are updated according to the gradient descent algorithm, and the formula for momentum updating of the weight of the backbone network of the target network is as follows: ξ = mξ + (1 - m)θ Wherein, ξ is the weight value to be updated; m is a given momentum rate, and m ∈ [0, 1]; Step 4: training the FasterR-CNN network; Step 4.1: training the region proposal network RPN; the pre-training model in step 3 is used to initialize the RPN network, and the RPN is trained independently to generate region proposals; Step 4.2: training the R-CNN network; the convolutional layer and fully connected layer parameters unique to the RPN network are frozen, the pre-training model in step 3 is used to initialize the front shared convolutional network parameters, and the target candidate frame generated by the RPN network is used to train the Fast R-CNN network parameters; Step 4.3: optimizing the region candidate network; the front shared convolutional network layer parameters trained by the FastR-CNN are frozen, and the convolutional layer and fully connected layer parameters unique to the RPN network are fine-tuned and optimized; Step 4.4: optimizing the R-CNN network; the convolutional layer and fully connected layer parameters unique to the RPN network are frozen, and the fully connected layer parameters of the R-CNN network are fine-tuned and optimized; Step 5: inputting the breast molybdenum target image to be detected into the trained FasterR-CNN network to obtain the detection result of the breast molybdenum target image lesion area.

2. The improved self-supervised detection method of mammography molybdenum target image according to claim 1, characterized in that: The breast molybdenum target public data set in step 1 is an updated and standardized version of a digital database for screening breast molybdenum target examination, containing normal, benign and malignant cases with verified pathological information.

3. The improved self-supervised detection method of mammography molybdenum target image according to claim 1, characterized in that: The data enhancement method includes random horizontal flip, random cropping, modifying image brightness, modifying image contrast, normalization and style transfer; the style transfer uses a CycleGAN network to convert multiple styles of breast molybdenum target images to each other.

Images