An Image Quality Improvement Method Based on Low-Resolution Mammograms

Abstract

The present invention relates to the technical field of medical images, and more specifically, to an image quality improvement method based on low-resolution mammograms, comprising the following steps: S1. Obtain two-dimensional breast tomosynthesis images and a set of low-resolution mammogram images; S2. Correspondingly pair all the two-dimensional breast tomosynthesis images with the images in the set of low-resolution mammogram images, and then divide them into a training set and a test set; S3. Construct a deep learning reconstruction model based on the deep learning method; S4. Input the training set to train the deep learning reconstruction model; S5. Use the trained deep learning reconstruction model to test on the test set, and then select the model corresponding to the synthetic mammogram image with the best performance in the test set as the output model. The present invention can obtain synthetic mammograms of higher quality on the basis of reducing the radiation dose that patients need to receive during imaging screening.

Description

Technical Field

[0001] The present invention relates to the technical field of medical images, and more specifically, to an image quality improvement method based on low-resolution mammograms. Background Art

[0002] Breast cancer is one of the common cancer types in women. Early diagnosis and treatment are crucial for preventing breast cancer. Mammograms are a two-dimensional breast imaging technology and one of the commonly used means for breast clinical examinations and preventive screenings. Digital breast tomosynthesis (DBT) is a three-dimensional breast imaging technology. The generated three-dimensional breast tomographic images can provide tomographic information of the breast and can effectively solve the problem of tissue masking in mammograms. Many clinical studies have shown that compared with using only mammograms, using both mammograms and three-dimensional breast tomographic images can significantly improve sensitivity and reduce the false positive rate. However, using both mammograms and three-dimensional breast tomographic images will double the radiation dose, which will increase the damage to the human body.

[0003] To reduce the radiation dose, current solutions include: using the tissue distribution information contained in three-dimensional breast tomographic images for synthesis to obtain synthetic mammograms, and using the synthetic mammograms instead of mammograms. However, the existing synthetic methods for synthetic mammograms only use the reconstructed breast tomographic data as the only information source for synthesizing synthetic mammograms, and the information of breast tomograms is insufficient. First, although the overall radiation dose of breast tomograms is comparable to that of mammograms, a part of the radiation dose is used to provide tomographic information. Therefore, the two-dimensional spatial resolution of breast tomograms is different from that of mammograms. Second, the breast tomographic imaging system uses a limited projection angle range, which is relatively narrow and incomplete. Therefore, the tomographic resolution of the reconstructed breast tomograms is low and there are many artifacts. The insufficient information in breast tomograms makes it difficult to obtain high-resolution and high-quality synthetic mammograms similar to real mammograms in synthetic mammograms. Summary of the Invention

[0004] An object of the present invention is to overcome the deficiency that there is a gap in the image quality between synthetic mammograms and real mammograms in the prior art, and to provide an image quality improvement method based on low-resolution mammograms, which can obtain higher-quality synthetic mammograms on the basis of reducing the radiation dose that patients need to receive during imaging screening.

[0005] To solve the above technical problems, the technical solution adopted by the present invention is:

[0006] Provide an image quality improvement method based on low-resolution mammograms, including the following steps:

[0007] S1. Obtain three-dimensional breast tomosynthesis images and low-radiation-dose mammography images of multiple projection positions of multiple patients, and respectively obtain two-dimensional breast tomosynthesis images and low-resolution mammography image groups after preprocessing the images;

[0008] S2. Correspondingly pair all the two-dimensional breast tomosynthesis images with the images in the low-resolution mammography image group to obtain a paired image group, and then divide all the obtained paired image groups into a training set and a test set;

[0009] S3. Construct a deep learning reconstruction model based on the deep learning method; the deep learning reconstruction model includes a generation network, a generation network loss function, a high-resolution network, and a high-resolution network loss function;

[0010] S4. Input the training set to train the deep learning reconstruction model; wherein: Step S4 includes the following steps:

[0011] S41. Set a low-resolution discrimination point. If the resolution of the low-resolution mammography image in the current paired image group is equal to the low-resolution discrimination point, execute step S43; otherwise, execute step S42;

[0012] S42. Use the generation network to perform feature extraction and stitching on the current paired image group to obtain a low-resolution synthetic mammography image equal to the low-resolution discrimination point, then use the generation network loss function for evaluation, and then update the evaluated low-resolution synthetic mammography image to the current paired image group, and then execute step S43;

[0013] S43. Use the high-resolution network to perform feature extraction, stitching, and magnification on the current paired image group to obtain a synthetic mammography image, then use the high-resolution network loss function to measure the gap between the synthetic mammography image and the corresponding low-radiation-dose mammography image, and minimize the loss to optimize the parameters of the high-resolution network;

[0014] S5. Use the trained deep learning reconstruction model to test on the test set, then compare according to the obtained synthetic mammography image and the corresponding low-radiation-dose mammography image, and finally select the deep learning reconstruction model corresponding to the synthetic mammography image with the best performance in the test set as the output model.

[0015] The present invention is an image quality improvement method based on low-resolution mammograms, which can use low-resolution mammography images as additional inputs to supplement the spatial information that may be missing in two-dimensional breast tomosynthesis images, eliminate artifact interference, enabling a deep learning reconstruction model to generate synthetic mammogram images approaching the image quality of real mammograms, and can significantly reduce the radiation dose that patients need to receive during imaging screening.

[0016] Further, step S1 includes the following steps:

[0017] S11. Obtain three-dimensional breast tomosynthesis images and low-radiation-dose mammogram images at the projection positions of the left cranio-caudal, left mediolateral oblique, right cranio-caudal, and right mediolateral oblique of the patient;

[0018] S12. Standardize and desensitize the three-dimensional breast tomosynthesis images to obtain the two-dimensional breast tomosynthesis images;

[0019] S13. Recombine the grids of the low-radiation-dose mammogram images to obtain the low-resolution mammogram image group, where the low-resolution mammogram image group includes multiple 1 / N resolution mammogram images; where N≥4;

[0020] Step S2 includes: Pairing each image in the two-dimensional breast tomosynthesis image and the low-resolution mammogram image group at the same projection position of the same patient to obtain a paired image group, and dividing it into a training set S and a test set T according to the patient granularity.

[0021] Further, in step S41, the low-resolution discrimination point is 1 / 4.

[0022] Further, in step S3, the loss function of the generation network is:

[0023]

[0024] In the formula, G L represents the generation network, |*| represents the l1 norm, |S| represents the size of the training set, represents the 1 / 4 resolution mammogram image, represents the 1 / 4 resolution synthetic mammogram image, represents one-time Gaussian filtering smoothing, I T represents the set of two-dimensional breast tomosynthesis images;

[0025] The loss function of the high-resolution network is:

[0026] L(G N× ) = L MS-MSE (G N× ) + LGGGAN (G N× ) + λ Percep L Percep (G N× );

[0027] In the formula, G N× represents the high-resolution network, and L MS-MSE (G N× ) represents the mean squared error, and L GGGAN (G N× ) represents the generative adversarial network loss, and L Percep (G N× ) represents the perceptual loss, and λ Percep represents the weight for balancing the perceptual loss.

[0028] Furthermore, the mean squared error is:

[0029]

[0030] In the formula, G N× represents the high-resolution network, |S| represents the size of the training set, represents the synthetic mammogram image, represents the mammogram image at 1 / N resolution, I T represents the set of two-dimensional breast tomosynthesis images, and I D represents the low-dose mammogram image, represents the composite of two Gaussian filter smoothings;

[0031] The generative adversarial network loss is:

[0032] L GGGAN (G N× ) = L Adver′ (G N× ) + λ FM′ L FM′ (G N× );

[0033] In the formula, L Adver′ (G N× ) represents the loss function of the generative adversarial network, and L FM′ (G N× ) represents the feature matching loss function, and λ FM′ represents the weight for balancing the feature matching loss function;

[0034] The perceptual loss is:

[0035]

[0036] In the formula, V represents the backbone network, and T V represents the number of intermediate convolutional layers, represents the number of elements of the output features of the intermediate convolutional layer of the j-th layer, V j (*) represents the intermediate convolutional layer of the j-th layer of V(*).

[0037] Furthermore, in the loss of the generative adversarial network:

[0038]

[0039] In the formula, x i represents a two-dimensional breast tomosynthesis image, and y i respectively represent 1 / 4 resolution mammogram images, represents the synthetic mammogram image predicted by the high-resolution network, y' i , respectively represent the gradients of y i , , D' represents the discriminant network guided by the gradient, and T D′ represents the total number of intermediate convolutional layers, D' j represents the output of the j-th layer of the discriminant network, N D′j represents the dimension of the output features of the j-th intermediate convolutional layer of D'.

[0040] Furthermore, in step S5, the peak signal-to-noise ratio, structural similarity, Dice coefficient, and F1 score of calcification points are selected to construct an evaluation index function, and the best-performing synthetic mammogram image is selected using the evaluation index function; among them, the evaluation index function is:

[0041] Score = ω PSNR PSNR + ω SsIM SSIM + ω Dice Dice + ω F1 F1 score;

[0042] In the formula, PSNR represents the peak signal-to-noise ratio, SSIM represents the structural similarity, Dice represents the Dice coefficient, and F1 score represents the F1 score of calcification points; ω PSNR , ω SSIM , ω Dice , ω F1 respectively represent the weighting coefficients of the peak signal-to-noise ratio, structural similarity, Dice coefficient, and F1 score of calcification points.

[0043] Furthermore, the peak signal-to-noise ratio is:

[0044]

[0045] In the formula, x represents the low-dose mammogram image, y represents the synthetic mammogram image, MAXI MAX represents the maximum pixel value of the image, and MSE(x,y) represents the mean square error between x and y;

[0046] The structural similarity is as follows:

[0047]

[0048] c1 = (0.01MAX I ) 2 ;

[0049] c2 = (0.03MAX I ) 2 ;

[0050] In the formula, μ x represents the mean value of x, μ y represents the mean value of t, represents the variance of x, represents the variance of y, σ xy represents the covariance of x and y.

[0051] Furthermore, an automatic segmentation algorithm is used to segment the mass in the synthetic mammogram to extract the mass contour, and an artificial delineation method is used to extract the mass contour, and the mass retention is evaluated based on the Dice coefficient; the Dice coefficient is:

[0052]

[0053] In the formula, A represents the mass contour obtained by using the automatic segmentation algorithm, B represents the mass contour obtained by using artificial delineation, |A| represents the area of a, |B| represents the area of B, and |A∩B| represents the intersection area of A and B.

[0054] Furthermore, the F1 score of the calcification points is:

[0055]

[0056] In the formula, Precision represents the precision, and Sensitivity represents the sensitivity;

[0057] Among them: The calcification points in the synthetic mammogram and the low-dose mammogram are detected by using the calcification point detection algorithm; then, the calcification points on the low-dose mammogram are used as the gold standard calcification points; then, for each calcification point detected on the synthetic mammogram, if there is a gold standard calcification point within the radius of the largest microcalcification point around it, it is considered a true positive and set as TP detect , otherwise it is considered a false positive and set as FP detect ; Thus, it can be obtained:

[0058]

[0059] For each gold standard calcification point, if there is a detected calcification point within a fixed pixel size in the synthesized mammogram, it is considered detected and set as TP gt , otherwise it is considered undetected and set as FP gt ; Thus, it can be obtained that:

[0060]

[0061] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0062] The present invention is an image quality improvement method based on low-resolution mammograms. By using low-resolution mammogram images as additional inputs to supplement the spatial information that may be missing in tomosynthesis images and exclude artifact interference, the deep learning reconstruction model can generate synthesized mammogram images approaching the image quality of real mammograms, which can significantly reduce the radiation dose that patients need to receive during imaging screening. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 is a flowchart of an image quality improvement method based on low-resolution mammograms according to the present invention;

[0064] Figure 2 is the network structure diagram of the generation network in step S3 of the present invention;

[0065] Figure 3 is the network structure diagram of the high-resolution network in step S3 of the present invention;

[0066] Figure 4 is the network structure diagram of the adversarial network loss and perceptual loss of the high-resolution network loss function in step S3 of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0067] The present invention will be further described below in conjunction with specific embodiments. In order to enable those skilled in the art to better understand the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0068] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects.

[0069] It should be noted that the terms "including" and "having" in the description and claims of the present invention and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or are inherent to these processes, methods, products, or devices.

[0070] It should be noted that, without conflict, the embodiments in the present invention and the features in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

[0071] Embodiment 1

[0072] As Figures 1 to 4 shown in the first embodiment of an image quality improvement method for low-resolution mammograms of the present invention, the method includes the following steps:

[0073] S1. Obtain three-dimensional mammography images and low-radiation-dose mammogram images of multiple projection positions of multiple patients, and respectively obtain two-dimensional mammography images and a group of low-resolution mammogram images after preprocessing the images;

[0074] S2. Pair the two-dimensional mammography images with all the images in the group of low-resolution mammogram images to obtain a paired image group, and then divide all the obtained paired image groups into a training set and a test set;

[0075] S3. Construct a deep learning reconstruction model based on the deep learning method; the deep learning reconstruction model includes a generation network, a generation network loss function, a high-resolution network, and a high-resolution network loss function;

[0076] S4. Input the training set to train the deep learning reconstruction model; wherein: step S4 includes the following steps:

[0077] S41. Set a low-resolution discrimination point. If the resolution of the low-resolution mammogram image in the current paired image group is equal to the low-resolution discrimination point, execute step S43; otherwise, execute step S42;

[0078] S42. Use the generation network to extract and splice features from the current paired image group to obtain a low-resolution synthetic mammogram equal to the low-resolution discrimination point, then evaluate it using the generation network loss function, and then update the evaluated low-resolution synthetic mammogram to the current paired image group, and then perform step S43;

[0079] S43. Use the high-resolution network to extract, splice, and magnify features from the current paired image group to obtain a synthetic mammogram, then use the high-resolution network loss function to measure the gap between the synthetic mammogram and the corresponding low-dose mammogram, and minimize the loss to optimize the parameters of the high-resolution network;

[0080] S5. Use the trained deep learning reconstruction model to test on the test set, then compare according to the obtained synthetic mammogram and the corresponding low-dose mammogram, and finally select the deep learning reconstruction model corresponding to the synthetic mammogram with the best performance in the test set as the output model.

[0081] The present invention can use low-resolution mammograms as additional inputs to supplement the spatial information that may be missing in tomosynthesis images, eliminate artifact interference, so that the deep learning reconstruction model can generate synthetic mammograms approaching the image quality of real mammograms, and can greatly reduce the radiation dose that patients need to receive during imaging screening. It should be noted that the "low radiation dose" of the low-dose mammogram refers to an image obtained under a non-standard radiation dose.

[0082] Embodiment 2

[0083] This embodiment is the second embodiment of an image quality improvement method based on low-resolution mammograms. This embodiment is similar to Embodiment 1, and the difference lies in:

[0084] Specifically, step S1 includes the following steps:

[0085] S11. Obtain three-dimensional tomosynthesis images and low-dose mammograms of the patient's left cranio-caudal (LCC), left mediolateral oblique (LMLO), right cranio-caudal (RCC), and right mediolateral oblique (RMLO) projection positions;

[0086] It should be noted that the case inclusion criteria are patients who are screened for breast cancer for the first time and have not undergone breast surgery or biopsy before participating in the screening; the gold standard is that after inclusion, through biopsy, surgery or follow-up, it is confirmed that there are breast lesions in at least one breast tissue, including masses, calcifications, and structurally distorted lesions; patients who do not meet the gold standard are excluded. For each case, three-dimensional breast tomosynthesis images and low-dose breast X-ray images of four projection positions, namely left craniocaudal, left mediolateral oblique, right craniocaudal, and right mediolateral oblique, are collected and stored in the original DICOM medical image format. The image data is stored in 16-bit grayscale, the pixel size within the tomogram is 0.086 mm to 0.108 mm (mean ± standard deviation: 0.089 ± 0.005), and the physical interval between adjacent tomograms is 1 mm;

[0087] S12. Standardize and desensitize the three-dimensional breast tomosynthesis images to obtain two-dimensional breast tomosynthesis images;

[0088] S13. Reconstruct the grid of the low-dose breast X-ray images to obtain a set of low-resolution breast X-ray images, where the set of low-resolution breast X-ray images includes multiple 1 / N resolution breast X-ray images; where N≥4 and N = 2 n 。

[0089] Specifically, step S2 includes: corresponding and pairing each image in the two-dimensional breast tomosynthesis images and the set of low-resolution breast X-ray images of the same patient at the same projection position to obtain a set of paired images, and dividing them into a training set S and a test set T according to the patient granularity; that is, for the image content of each projection position of the same patient, all of them need to be assigned to the training set S or all of them need to be assigned to the test set T.

[0090] Specifically, in step S3, the generation network and the high-resolution network are both deep convolutional networks, where:

[0091] The loss function of the generation network is:

[0092]

[0093] In the formula, G L represents the generation network, |*| represents the l1 norm, |S| represents the size of the training set, represents the 1 / 4 resolution breast X-ray image, represents the 1 / 4 resolution synthetic breast X-ray image, represents one-time Gaussian filtering and smoothing, I T represents the set of two-dimensional breast tomosynthesis images; where, the 1 / 4 resolution breast X-ray image is used as the regression target, and the l1 loss function is used to retain more local structures;

[0094] In step S3, the high-resolution network loss function is as follows:

[0095] L(G N× ) = L MS-MSE (G N× ) + L GGGAN (G N× ) + λ Percep L Percep (G N× );

[0096] In the formula, G N× represents the high-resolution network, L MS-MSE (G N× ) represents the mean square error, L GGGAN (G N× ) represents the generative adversarial network loss, L Percep (G N× ) represents the perceptual loss, and λ Percep represents the weight for balancing the perceptual loss; in this embodiment, the high-resolution network uses the high-resolution network loss function composed of the mean square error, the generative adversarial network loss, and the perceptual loss to measure the gap between the synthetic mammogram and the corresponding low-dose mammogram, and minimizes the parameters of G N× .

[0097] Among them, the mean square error is:

[0098]

[0099] In the formula, G N× represents the high-resolution network, |S| represents the size of the training set, represents the synthetic mammogram, represents the mammogram at 1 / N resolution, I T represents the set of two-dimensional breast tomosynthesis images, and I D represents the low-dose mammogram, represents the compounding of two Gaussian filter smoothings;

[0100] Among them, the generative adversarial network loss is:

[0101] L GGGAN (G N× ) = L Adver′ (G N× ) + λ jM′ L FM′ (G N× );

[0102] In the formula, L Adver′ (G N× ) represents the loss function of the generative adversarial network, LFM′ (G N× ) represents the feature matching loss function, and λ FM′ represents the weight of the balanced feature matching loss function;

[0103] In the generative adversarial network loss:

[0104]

[0105] In the formula, x i represents the two-dimensional breast tomosynthesis image, and y i respectively represent the 1 / 4 resolution mammogram images, represents the synthetic mammogram image predicted by the high-resolution network, and y′ i 、 respectively represent the gradients of y i 、 , D′ represents the discriminant network guided by the gradient, and T D′ represents the total number of intermediate convolutional layers, and D′ j represents the output of the j-th layer of the discriminant network, and N D′j represents the dimension of the output features of the j-th intermediate convolutional layer of D′;

[0106] Among them, the perceptual loss is:

[0107]

[0108] In the formula, V represents the backbone network, specifically VGG-16 pre-trained on ImageNet; T V represents the number of intermediate convolutional layers, represents the number of elements of the output features of the j-th intermediate convolutional layer, and V j (λ) represents the j-th intermediate convolutional layer of V(λ).

[0109] Specifically, step S4 includes the following steps:

[0110] S41. Set the low-resolution discrimination point to 1 / 4. If the 1 / N resolution mammogram image in the current paired image group is a 1 / 4 resolution mammogram image, then execute step S43; otherwise, execute step S42;

[0111] S42. Use the generation network to extract the features of the two-dimensional breast tomosynthesis images in the current paired image group, and also extract the features of the 1 / N resolution mammogram images in the current paired image group. Then, splice the extracted features to obtain a 1 / 4 resolution synthetic mammogram image that implicitly combines the image information of both. Next, use the generation network loss function for evaluation. Then, replace the 1 / N resolution mammogram images in the initial paired image group with the evaluated 1 / 4 resolution synthetic mammogram images, that is, pair the evaluated 1 / 4 resolution synthetic mammogram images with the two-dimensional breast tomosynthesis images, and then perform step S43;

[0112] S43. Use the high-resolution network to extract features, splice, and magnify the current paired image group to obtain a synthetic mammogram image. Then, use the high-resolution network loss function to measure the gap between the synthetic mammogram image and the corresponding low-dose mammogram image, and minimize the loss to optimize the parameters of the high-resolution network.

[0113] Example 3

[0114] This example is the third example of an image quality improvement method based on low-resolution mammograms. This example is similar to Example 1 or 2, and the difference lies in:

[0115] Specifically, in step S5, select the peak signal-to-noise ratio, structural similarity, Dice coefficient, and F1 score of calcification points to construct an evaluation index function, and use the evaluation index function to select the synthetic mammogram image with the best performance. Among them, the evaluation index function is:

[0116] Score = ω PSNR PSNR + ω SSIM SSIM + ω Dice Dice + ω F1 F1 score;

[0117] In the formula, PSNR represents the peak signal-to-noise ratio, SSIM represents the structural similarity, Dice represents the Dice coefficient, and F1 score represents the F1 score of calcification points; ω PSNR 、ω SSIM 、ω Dice 、ω F1 respectively represent the weighting coefficients of the peak signal-to-noise ratio, structural similarity, Dice coefficient, and F1 score of calcification points. In this example, take ω PSNR = 0.01, ω SSIM = 1, ω Dice = 1, ω F1 = 1.

[0118] Among them, the peak signal-to-noise ratio is:

[0119]

[0120] In the formula, x represents the low-dose mammogram image, y represents the synthesized mammogram image, MAX I represents the maximum pixel value of the image, and MSE(x, y) represents the mean square error between x and y.

[0121] Among them, the structural similarity is:

[0122]

[0123] c1 = (0.01MAX I ) 2 ;

[0124] c2 = (0.03MAX I ) 2 ;

[0125] In the formula, μ x represents the mean of x, μ y represents the mean of y, represents the variance of x, represents the variance of y, σ xy represents the covariance between x and y; in this embodiment, when calculating, a 11×11 rectangular window is taken, the rectangular window is slid and the structural similarity of each rectangular window is calculated, and the mean value of the structural similarities of all rectangular windows is taken as the global structural similarity of the image.

[0126] Among them, an automatic segmentation algorithm is used to segment and extract the contour of the mass in the synthesized mammogram image, and the method of manual delineation is used to extract the contour of the mass, and the retention of the mass is evaluated based on the Dice coefficient; the Dice coefficient is:

[0127]

[0128] In the formula, A represents the contour of the mass obtained by using the automatic segmentation algorithm, B represents the contour of the mass obtained by using manual delineation, |A| represents the area of A, |B| represents the area of B, and |A∩B| represents the intersection area of A and B.

[0129] Among them, the F1 score of the calcification points is:

[0130]

[0131] In the formula, Precision represents precision, and Sensitivity represents sensitivity;

[0132] Among them: The calcification point detection algorithm is used to detect the calcification points on the synthetic mammogram and the low-radiation-dose mammogram; then, the calcification points on the low-radiation-dose mammogram are used as the gold standard calcification points; then, for each detected calcification point on the synthetic mammogram, if there is a gold standard calcification point within the radius (5 pixels) of the largest microcalcification point around it, it is considered a true positive and set as TP detect , otherwise it is considered a false positive and set as FP detect ; Thus, it can be obtained that:

[0133]

[0134] For each gold standard calcification point, if there is a detected calcification point within 5 pixels around it in the synthetic mammogram, it is considered detected and set as TP gt , otherwise it is considered undetected and set as FP gt ; Thus, it can be obtained that:

[0135]

[0136] In the specific content of the above specific implementation manner, each technical feature can be combined arbitrarily without contradiction. For the sake of concise description, not all possible combinations of the above technical features are described. However, as long as the combinations of these technical features do not conflict, they should all be considered as within the scope described in this specification.

[0137] Obviously, the above embodiments of the present invention are merely examples for clearly explaining the present invention, rather than limiting the implementation manners of the present invention. For those of ordinary skill in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to enumerate all the implementation manners here. Any modifications, equivalent replacements, and improvements made within the spirit and principle of the present invention shall be included within the protection scope of the claims of the present invention.

Claims

1. An image quality improvement method based on low-resolution mammograms, characterized in that, Including the following steps: S1. Obtain three-dimensional breast tomosynthesis images and low-radiation-dose mammography images of multiple projection positions of multiple patients. After preprocessing the images, obtain two-dimensional breast tomosynthesis images and a group of low-resolution mammography images respectively; S2. Pair the two-dimensional breast tomosynthesis images with all the images in the group of low-resolution mammography images to obtain a paired image group, and then divide all the obtained paired image groups into a training set and a test set; S3. Construct a deep learning reconstruction model based on the deep learning method; the deep learning reconstruction model includes a generation network, a generation network loss function, a high-resolution network, and a high-resolution network loss function; S4. Input the training set to train the deep learning reconstruction model; among them: step S4 includes the following steps: S41. Set a low-resolution discrimination point. If the resolution of the low-resolution mammography image in the current paired image group is equal to the low-resolution discrimination point, execute step S43, otherwise execute step S42; S42. Use the generation network to perform feature extraction and splicing on the current paired image group to obtain a low-resolution synthetic mammography image equal to the low-resolution discrimination point, then use the generation network loss function for evaluation, and then update the evaluated low-resolution synthetic mammography image to the current paired image group, and then execute step S43; S43. Use the high-resolution network to perform feature extraction, splicing, and magnification on the current paired image group to obtain a synthetic mammography image, then use the high-resolution network loss function to measure the gap between the synthetic mammography image and the corresponding low-radiation-dose mammography image, and minimize the loss to optimize the parameters of the high-resolution network; S5. Use the trained deep learning reconstruction model to test on the test set, then compare according to the obtained synthetic mammography image and the corresponding low-radiation-dose mammography image, and finally select the deep learning reconstruction model corresponding to the synthetic mammography image with the best performance in the test set as the output model.

2. The method for improving the image quality based on low-resolution mammograms according to claim 1, wherein Step S1 includes the following steps: S11. Obtain three-dimensional breast tomosynthesis images and low-radiation-dose mammography images of the left craniocaudal, left mediolateral oblique, right craniocaudal, and right mediolateral oblique projection positions of the patient; S12. Perform standardization and desensitization processing on the three-dimensional breast tomosynthesis images to obtain the two-dimensional breast tomosynthesis images; S13. Perform grid reorganization on the low-radiation-dose mammography images to obtain the group of low-resolution mammography images, where the group of low-resolution mammography images includes multiple 1 / N-resolution mammography images; where N≥4; Step S2 includes: Pair the two-dimensional breast tomosynthesis images with each image in the group of low-resolution mammography images of the same patient at the same projection position to obtain a paired image group, and divide it into a training set S and a test set T according to the patient granularity.

3. The image quality improvement method based on low-resolution mammograms according to claim 1, characterized in that In step S41, the low-resolution discrimination point is 1 / 4.

4. The method for improving the image quality based on low-resolution mammograms according to claim 3, wherein, In step S3, the generation network loss function is: where G L represents the generation network, |*| represents the l1 norm, |S| represents the size of the training set, represents the 1 / 4 resolution mammogram, represents the 1 / N resolution mammogram, represents the 1 / 4 resolution synthetic mammogram, represents a single Gaussian filter smoothing, I T represents the set of two-dimensional breast tomosynthesis images; The high-resolution network loss function is as follows: L(G N× ) = L MS-MSE (G N× ) + L GGGAN (G N× ) + λ Percep L Percep (G N× ); Where G N× represents the high-resolution network, L MS-MSE (G N× ) represents the mean squared error, L GGGAN (G N× ) represents the generative adversarial network loss, L Percep (G N× ) represents the perceptual loss, and λ Percep represents the weight for balancing the perceptual loss.

5. The image quality improvement method based on low-resolution mammograms according to claim 4, characterized in that The mean squared error is as follows: where G N× represents the high-resolution network, |S| represents the size of the training set, represents the synthetic mammogram, represents the mammogram at 1 / N resolution, I T represents the set of two-dimensional breast tomosynthesis images, I D represents the low-dose mammogram, represents the double Gaussian filtering smoothing composite; The generative adversarial network loss is as follows: L GGGAN (G N× ) = L Adver′ (G N× ) + λ FM′ L FM′ (G N× ); where L Adver′ (G N× ) represents the loss function of the generative adversarial network, and L FM′ (G N× ) represents the feature matching loss function, and λ FM′ represents the weight of the balanced feature matching loss function; The perceptual loss is as follows: In the formula, V represents the backbone network, and T V represents the number of intermediate convolutional layers, represents the number of elements of the output features of the j-th intermediate convolutional layer, and V j (*) represents the j-th intermediate convolutional layer of V(*).

6. The method for improving the image quality based on low-resolution mammograms according to claim 5, wherein In the generative adversarial network loss: where x i represents a two-dimensional breast tomosynthesis image, and y i respectively represent 1 / 4 resolution mammogram images, represents the synthetic mammogram image predicted by the high-resolution network, y' i and respectively represent the gradients of y i and respectively. D' represents the discriminative network guided by the gradient, and T D′ represents the total number of intermediate convolutional layers, and D' j represents the output of the j-th layer of the discriminative network, and N D′ j represents the dimension of the output features of the j-th intermediate convolutional layer of D'.

7. The method for improving image quality based on low-resolution mammograms according to claim 1, characterized in that In step S5, a peak signal-to-noise ratio, structural similarity, Dice coefficient, and F1 score of calcification points are selected to construct an evaluation metric function, and the best-performing synthetic mammogram is selected using the evaluation metric function; wherein, the evaluation metric function is as follows: Score = ω PSNR PSNR + ω SSIM SSIM + ω Dice Dice + ω F1 F1 score; Wherein, PSNR represents the peak signal-to-noise ratio, SSIM represents the structural similarity index, Dice represents the Dice coefficient, and F1 score represents the F1 score of calcification points; ω PSNR 、ω SSIM 、ω dice 、ω F1 respectively represent the weighting coefficients of the peak signal-to-noise ratio, the structural similarity index, the Dice coefficient, and the F1 score of calcification points.

8. The method for improving the image quality based on low-resolution mammograms according to claim 7, wherein The peak signal-to-noise ratio is as follows: Wherein, x represents a low-dose mammogram, y represents a synthetic mammogram, and MAX I represents the maximum pixel value of the image, and MSE(x, y) represents the mean square error between x and y; The structural similarity is as follows: c1 = (0.01MAX I ) 2 ; c2 = (0.03MAX I ) 2 ; where μ x represents the mean of x, μ y represents the mean of y, represents the variance of x, represents the variance of y, σ xy represents the covariance of x and y.

9. The method for improving the image quality based on low-resolution mammograms according to claim 7, wherein The tumor in the synthetic mammogram is segmented using an automatic segmentation algorithm to extract the tumor contour, and the tumor contour is extracted using manual delineation, and the evaluation of tumor retention based on the Dice coefficient is performed; the Dice coefficient is as follows: In the formula, A represents the tumor contour obtained using the automatic segmentation algorithm, B represents the tumor contour obtained using manual delineation, |A| represents the area of A, |B| represents the area of B, and |A∩B| represents the intersection area of A and B.

10. The method for improving the image quality based on low-resolution mammograms according to claim 7, characterized in that, The F1 score of the calcification points is as follows: In the formula, Precision represents precision, and Sensitivity represents sensitivity; Among them: The calcification point detection algorithm is used to detect the calcification points on the synthesized mammogram and the low-dose mammogram; then, the calcification points on the low-dose mammogram are used as the gold standard calcification points; next, for each detected calcification point on the synthesized mammogram, if there is a gold standard calcification point within the radius of the largest microcalcification point around it, it is considered a true positive and set as TP detect , otherwise it is considered a false positive and set as FP detect ; Thus, it can be obtained that: For each gold standard calcification point, if there is a detected calcification point within a fixed pixel size in the synthetic mammogram, it is considered detected and set as TP gt , otherwise it is considered undetected and set as FP gt ; Thus, it can be obtained that:

Images