Image quality improving method based on low-resolution mammary gland X-ray film

Abstract

The invention relates to the technical field of medical images, in particular to an image quality improving method based on a low-resolution mammary gland X-ray film, which comprises the following steps: S1, acquiring a two-dimensional mammary gland tomography image and a low-resolution mammary gland X-ray image group; s2, performing corresponding pairing on the two-dimensional breast tomography image and all images in the low-resolution breast X-ray image group, and then dividing the two-dimensional breast tomography image and the low-resolution breast X-ray image group into a training set and a test set; s3, constructing a deep learning reconstruction model based on a deep learning method; s4, inputting the training set to train the deep learning reconstruction model; and S5, testing on the test set by using the trained deep learning reconstruction model, and then selecting the model corresponding to the synthesized mammary gland X-ray image with the best performance in the test set as an output model. According to the invention, the synthetic mammary X-ray film with higher quality can be obtained on the basis of reducing the radiation dose required to be accepted by a patient during image screening.

Description

Technical Field

[0001] The present invention relates to the technical field of medical images, and more particularly to a method for improving image quality based on low-resolution breast X-ray films. Background Art

[0002] Breast cancer is one of the most common types of cancer in women, and early diagnosis and treatment are crucial to preventing breast cancer. Mammogram is a two-dimensional breast imaging technology and is one of the commonly used methods for clinical breast examination and preventive screening. Digital breast tomosynthesis X-ray imaging technology is a three-dimensional breast imaging technology. The generated three-dimensional breast tomographic images can provide breast tomographic information and effectively solve the tissue shielding problem of mammograms. Many clinical studies have shown that compared with using only mammograms, the simultaneous use of mammograms and three-dimensional breast tomographic images can significantly improve sensitivity and reduce false positive rates. However, the simultaneous use of mammograms and three-dimensional breast tomographic images will double the radiation dose, which will increase damage to the human body.

[0003] In order to reduce the radiation dose, the current solutions include: using the tissue distribution information contained in the three-dimensional breast tomography image for synthesis to obtain a synthetic mammogram, and using the synthetic mammogram to replace the mammogram. However, the existing synthesis method of synthetic mammograms only uses the reconstructed breast tomography data as the only information source for the synthetic mammogram, and the information of the breast tomography is insufficient. First, although the overall radiation dose of the breast tomography is equivalent to that of the mammogram, a part of the radiation dose is used to provide tomography information, so the two-dimensional spatial resolution of the breast tomography is different from that of the mammogram. Secondly, the breast tomography imaging system uses a limited projection angle range, which is relatively narrow and incomplete, so the tomographic resolution of the reconstructed breast tomography is low and there are many artifacts. The insufficient information in the breast tomography makes it difficult to obtain a high-resolution and high-quality synthetic mammogram similar to the real mammogram in the synthetic mammogram. Summary of the invention

[0004] The purpose of the present invention is to overcome the shortcomings of the prior art that there is a gap in image quality between synthetic mammary X-rays and real mammary X-rays, and to provide an image quality improvement method based on low-resolution mammary X-rays, which can obtain higher quality synthetic mammary X-rays on the basis of reducing the radiation dose required for patients during image screening.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0006] A method for improving image quality based on low-resolution mammary X-ray films is provided, comprising the following steps:

[0007] S1. Acquire three-dimensional breast tomography images and low-radiation-dose breast X-ray images of multiple patients at multiple projection positions, and obtain two-dimensional breast tomography images and low-resolution breast X-ray image groups after preprocessing the images;

[0008] S2. Pairing the two-dimensional breast tomography image with all images in the low-resolution breast X-ray image group to obtain a paired image group, and then dividing all the obtained paired image groups into a training set and a test set;

[0009] S3. Constructing a deep learning reconstruction model based on a deep learning method; the deep learning reconstruction model includes a generative network, a generative network loss function, a high-resolution network, and a high-resolution network loss function;

[0010] S4. Input the training set to train the deep learning reconstruction model; wherein: step S4 includes the following steps:

[0011] S41. Set a low-resolution distinction point. If the resolution of the low-resolution mammary X-ray image in the current paired image group is equal to the low-resolution distinction point, execute step S43, otherwise execute step S42;

[0012] S42. Use the generative network to perform feature extraction and splicing on the current paired image group to obtain a low-resolution synthetic breast X-ray image equal to the low-resolution distinction point, and then use the generative network loss function to evaluate, and then update the evaluated low-resolution synthetic breast X-ray image to the current paired image group, and then execute step S43;

[0013] S43. Use the high-resolution network to extract features, stitch and amplify the current paired image group to obtain a synthetic breast X-ray image, and then use the high-resolution network loss function to measure the difference between the synthetic breast X-ray image and the corresponding low-radiation dose breast X-ray image, and minimize the loss to optimize the parameters of the high-resolution network;

[0014] S5. Use the trained deep learning reconstruction model to test on the test set, then compare the obtained synthetic breast X-ray image with the corresponding low-radiation dose breast X-ray image, and finally select the deep learning reconstruction model corresponding to the best performing synthetic breast X-ray image in the test set as the output model.

[0015] The present invention is a method for improving image quality based on low-resolution breast X-ray films. The method can use low-resolution breast X-ray images as additional input to supplement the spatial information that may be missing in two-dimensional breast tomography images and eliminate artifact interference, so that the deep learning reconstruction model can generate synthetic breast X-ray images with image quality close to that of real breast X-ray films, which can significantly reduce the radiation dose that patients need to receive during imaging screening.

[0016] Furthermore, step S1 includes the following steps:

[0017] S11. Obtain three-dimensional breast tomosynthesis images and low-dose breast X-ray images of the patient in the left head-foot position, left medial-lateral oblique position, right head-foot position, and right medial-lateral oblique position;

[0018] S12. Standardizing and desensitizing the three-dimensional breast tomography image to obtain the two-dimensional breast tomography image;

[0019] S13. Regridding the low-radiation-dose breast X-ray images to obtain the low-resolution breast X-ray image group, wherein the low-resolution breast X-ray image group includes a plurality of 1 / N resolution breast X-ray images; wherein N≥4;

[0020] Step S2 includes: pairing the two-dimensional breast tomography image of the same patient and the same projection position with each image in the low-resolution breast X-ray image group to obtain a paired image group, and dividing it into a training set S and a test set T according to the patient granularity.

[0021] Further, in step S41, the low-resolution distinction point is 1 / 4.

[0022] Furthermore, in step S3, the generated network loss function is:

[0023]

[0024] In the formula, G L represents the generating network, |*| represents l 1 norm, |S| represents the size of the training set, represents a 1 / 4 resolution mammographic image, represents a 1 / 4 resolution synthetic mammographic image, represents a Gaussian filter smoothing, I T represents a collection of two-dimensional breast tomosynthesis images;

[0025] The high-resolution network loss function is:

[0026] L(G N× )=L MS-MSE (GN× )+L GGGAN (G N× )+λ Percep L Percep (G N× );

[0027] In the formula, G N× represents the high-resolution network, L MS-MSE (G N× ) represents the mean square error, L GGGAN (G N× ) represents the loss of the generative adversarial network, L Percep (G N× ) represents the perceptual loss, λ Percep represents the weight of the balanced perceptual loss.

[0028] Furthermore, the mean square error is:

[0029]

[0030] In the formula, G N× represents a high-resolution network, |S| represents the training set size, represents a synthetic mammographic image, represents a 1 / N resolution mammographic image, I T represents a set of two-dimensional breast tomosynthesis images, I D Represents a low-radiation-dose mammogram. Represents two Gaussian filter smoothing composites;

[0031] The generative adversarial network loss is:

[0032] L GGGAN (G N× )=L Adver′ (G N× )+λ FM′ L FM′ (G N× );

[0033] Where, L Adver′ (G N× ) represents the loss function of the generative adversarial network, L FM′ (G N× ) represents the feature matching loss function, λ FM′ Represents the weight of the balanced feature matching loss function;

[0034] The perceptual loss is:

[0035]

[0036] In the formula, V represents the backbone network, T Vrepresents the number of intermediate convolutional layers, Represents the number of elements of the output features of the j-th intermediate convolutional layer, V j (*) denotes the j-th intermediate convolutional layer of V(*).

[0037] Furthermore, in the generative adversarial network loss:

[0038]

[0039] In the formula, x i represents a two-dimensional breast tomosynthesis image, y i Respectively represent 1 / 4 resolution mammary X-ray images, represents the synthetic mammogram predicted by the high-resolution network, y′ i , Respectively represent y i , The gradient of , D′ represents the discriminant network guided by the gradient, T D′ Represents the total number of intermediate convolutional layers, D′ j represents the output of the jth layer of the discriminant network, N D′j Represents the dimension of the output features of the jth intermediate convolutional layer of D′.

[0040] Furthermore, in step S5, the peak signal-to-noise ratio, structural similarity, Dice coefficient, and F1score of calcification points are selected to construct an evaluation index function, and the best performing synthetic breast X-ray image is selected using the evaluation index function; wherein the evaluation index function is:

[0041] Score=ω PSNR PSNR+ω SsIM SSIM+ω Dice Dice+ω F1 F1 score;

[0042] In the formula, PSNR represents peak signal-to-noise ratio, SSIM represents structural similarity, Dice represents Dice coefficient, and F1 score represents the F1 score of calcification point; ω PSNR ,ω SSIM ,ω Dice ,ω F1 They represent the weighting coefficients of peak signal-to-noise ratio, structural similarity, Dice coefficient, and F1 score of calcification points respectively.

[0043] Furthermore, the peak signal-to-noise ratio is:

[0044]

[0045] Where x represents the low-dose mammographic image, y represents the synthetic mammographic image, and MAX I Represents the maximum pixel value of the image, and MSE(x,y) represents the mean square error between x and y;

[0046] The structural similarity is:

[0047]

[0048] c 1 =(0.01MAX I ) 2 ;

[0049] c 2 =(0.03MAX I ) 2 ;

[0050] In the formula, μ x represents the mean of x, μ y represents the mean of t, represents the variance of x, represents the variance of y, σ xy represents the covariance of x and y.

[0051] Furthermore, the tumor in the synthetic breast X-ray image is segmented using an automatic segmentation algorithm to extract the tumor contour, and the tumor contour is extracted using a manual delineation method, and the mass retention is evaluated based on the Dice coefficient; the Dice coefficient is:

[0052]

[0053] Where A represents the tumor contour obtained by automatic segmentation algorithm, B represents the tumor contour obtained by manual delineation, |A| represents the area of ​​a, |B| represents the area of ​​B, and |A∩B| represents the intersection area of ​​A and B.

[0054] Furthermore, the F1 score of the calcification point is:

[0055]

[0056] In the formula, Precision represents accuracy, and Sensitivity represents sensitivity;

[0057] The calcification point detection algorithm is used to detect the calcification points on the synthetic breast X-ray image and the low-radiation-dose breast X-ray image; then, the calcification points on the low-radiation-dose breast X-ray image are used as the gold standard calcification points; then, for each calcification point detected on the synthetic breast X-ray image, if there is a gold standard calcification point within the radius of the largest microcalcification point around it, it is considered to be a true positive and is set as TPdetect , otherwise it is considered a false positive and set to FP detect ; From this we can get:

[0058]

[0059] For each gold standard calcification point, if there is a detected calcification point within a fixed pixel size in the synthetic mammographic image, it is considered detected and set as TP gt Otherwise, it is considered undetected and set as FP gt ; From this we can get:

[0060]

[0061] Compared with the prior art, the present invention has the following beneficial effects:

[0062] The present invention is a method for improving image quality based on low-resolution breast X-ray films. The method uses low-resolution breast X-ray images as additional input to supplement the spatial information that may be missing in two-dimensional breast tomography images and eliminate artifact interference, so that the deep learning reconstruction model can generate synthetic breast X-ray images with image quality close to that of real breast X-ray films, which can significantly reduce the radiation dose that patients need to receive during imaging screening. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 A flowchart of a method for improving image quality based on low-resolution mammary X-ray films according to the present invention;

[0064] Figure 2 A network structure diagram of the network generated in step S3 of the present invention;

[0065] Figure 3 A network structure diagram of the high-resolution network in step S3 of the present invention;

[0066] Figure 4 It is a network structure diagram of the adversarial network loss and the perceptual loss of the high-resolution network loss function in step S3 of the present invention. DETAILED DESCRIPTION

[0067] The present invention is further described below in conjunction with specific implementation methods. In order to enable those skilled in the art to better understand the solution of the present invention, the technical solution in the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is only a part of the embodiment of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in the field without creative work should fall within the scope of protection of the present invention.

[0068] Those skilled in the art will appreciate that the embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects.

[0069] It should be noted that the terms "including" and "having" and any variations thereof in the specification and claims of the present invention and the above-mentioned drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or apparatuses.

[0070] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0071] Embodiment 1

[0072] like Figures 1 to 4 The first embodiment of the method for improving image quality based on low-resolution mammary X-ray films of the present invention is shown, comprising the following steps:

[0073] S1. Acquire three-dimensional breast tomography images and low-radiation-dose breast X-ray images of multiple patients at multiple projection positions, and obtain two-dimensional breast tomography images and low-resolution breast X-ray image groups after preprocessing the images;

[0074] S2. Pairing the two-dimensional breast tomography image with all the images in the low-resolution breast X-ray image group to obtain paired image groups, and then dividing all the obtained paired image groups into a training set and a test set;

[0075] S3. Construct a deep learning reconstruction model based on deep learning methods; the deep learning reconstruction model includes a generative network, a generative network loss function, a high-resolution network, and a high-resolution network loss function;

[0076] S4. Input the training set to train the deep learning reconstruction model; wherein: step S4 includes the following steps:

[0077] S41. Set a low-resolution distinction point. If the resolution of the low-resolution mammary X-ray image in the current paired image group is equal to the low-resolution distinction point, execute step S43, otherwise execute step S42;

[0078] S42. Use the generative network to extract features and stitch the current paired image group to obtain a low-resolution synthetic breast X-ray image equal to the low-resolution distinction point, and then use the generative network loss function to evaluate, and then update the evaluated low-resolution synthetic breast X-ray image to the current paired image group, and then execute step S43;

[0079] S43. Use the high-resolution network to extract features, splice and amplify the current paired image group to obtain a synthetic breast X-ray image, and then use the high-resolution network loss function to measure the difference between the synthetic breast X-ray image and the corresponding low-radiation dose breast X-ray image, and minimize the loss to optimize the parameters of the high-resolution network;

[0080] S5. Use the trained deep learning reconstruction model to test on the test set, then compare the obtained synthetic breast X-ray images with the corresponding low-radiation dose breast X-ray images, and finally select the deep learning reconstruction model corresponding to the best performing synthetic breast X-ray image in the test set as the output model.

[0081] The present invention can use low-resolution breast X-ray images as additional input to supplement the spatial information that may be missing from two-dimensional breast tomography images, eliminate artifact interference, and enable the deep learning reconstruction model to generate synthetic breast X-ray images that are close to the quality of real breast X-ray images, which can significantly reduce the radiation dose that patients need to receive during imaging screening. It should be noted that the "low radiation dose" of low-radiation-dose breast X-ray images refers to images obtained under non-standard radiation doses.

[0082] Embodiment 2

[0083] This embodiment is a second embodiment of a method for improving image quality based on low-resolution breast X-ray films. This embodiment is similar to the first embodiment, except that:

[0084] Specifically, step S1 includes the following steps:

[0085] S11. Obtain three-dimensional breast tomosynthesis images and low-dose breast X-ray images of the patient in the left cranio-caulal (LCC), left mediolateral oblique (LMLO), right cranio-caulal (RCC), and right mediolateral oblique (RMLO) projection positions;

[0086] It should be noted that the inclusion criteria for cases were patients who were undergoing breast cancer screening for the first time and had not undergone breast surgery or biopsy before participating in the screening; the gold standard was that after inclusion, biopsy, surgery or follow-up confirmed the presence of breast damage in at least one side of the breast tissue, including masses, calcifications, and lesions with structural distortion; patients who did not meet the gold standard were excluded. Three-dimensional breast tomography images and low-radiation-dose breast X-ray images of four projection positions, namely left head-foot position, left medial-lateral oblique position, right head-foot position, and right medial-lateral oblique position, were collected for each case and stored in the original DICOM medical image format. The image data was stored in 16-bit grayscale, the pixel size within the slice was 0.086 mm to 0.108 mm (mean ± standard deviation: 0.089 ± 0.005), and the physical interval between adjacent slices was 1 mm.

[0087] S12. Standardizing and desensitizing the three-dimensional breast tomography image to obtain a two-dimensional breast tomography image;

[0088] S13. Regridding the low-radiation-dose breast X-ray images to obtain a low-resolution breast X-ray image group, wherein the low-resolution breast X-ray image group includes a plurality of 1 / N resolution breast X-ray images; wherein N≥4 and N=2 n .

[0089] Specifically, step S2 includes: pairing the two-dimensional breast tomography image of the same patient at the same exposure position with each image in the low-resolution breast X-ray image group to obtain a paired image group, and dividing it into a training set S and a test set T according to the patient granularity; that is, the image content of each exposure position of the same patient must all be divided into the training set S, or all into the test set T.

[0090] Specifically, in step S3, the generation network and the high-resolution network are both deep convolutional networks, where:

[0091] The generated network loss function is:

[0092]

[0093] In the formula, G L represents the generating network, |*| represents l 1 norm, |S| represents the size of the training set, represents a 1 / 4 resolution mammographic image, represents a 1 / 4 resolution synthetic mammographic image, represents a Gaussian filter smoothing, I T Represents a set of 2D breast tomosynthesis images; among them, 1 / 4 resolution breast X-ray images are used As the regression target, in order to retain more local structures, we use l 1Loss function;

[0094] In step S3, the high-resolution network loss function is:

[0095] L(G N× )=L MS-MSE (G N× )+L GGGAN (G N× )+λ Percep L Percep (G N× );

[0096] In the formula, G N× represents the high-resolution network, L MS-MSE (G N× ) represents the mean square error, L GGGAN (G N× ) represents the loss of the generative adversarial network, L Percep (G N× ) represents the perceptual loss, λ Percep represents the weight of the balanced perceptual loss; in this embodiment, the high-resolution network uses a high-resolution network loss function composed of mean square error, generative adversarial network loss, and perceptual loss to measure the difference between the synthetic breast X-ray image and its corresponding low-radiation dose breast X-ray image, and minimizes G N× Parameters.

[0097] Among them, the mean square error is:

[0098]

[0099] In the formula, G N× represents a high-resolution network, |S| represents the training set size, represents a synthetic mammographic image, represents a 1 / N resolution mammographic image, I T represents a set of two-dimensional breast tomosynthesis images, I D Represents a low-radiation-dose mammogram. Represents two Gaussian filter smoothing composites;

[0100] Among them, the loss of the generated adversarial network is:

[0101] L GGGAN (G N× )=L Adver′ (G N× )+λ jM′ L FM′ (G N× );

[0102] Where, L Adver′ (G N×) represents the loss function of the generative adversarial network, L FM′ (G N× ) represents the feature matching loss function, λ FM′ Represents the weight of the balanced feature matching loss function;

[0103] In the GAN loss:

[0104]

[0105] In the formula, x i represents a two-dimensional breast tomosynthesis image, y i Respectively represent 1 / 4 resolution mammary X-ray images, represents the synthetic mammogram predicted by the high-resolution network, y′ i , Respectively represent y i , The gradient of , D′ represents the discriminant network guided by the gradient, T D′ Represents the total number of intermediate convolutional layers, D′ j represents the output of the jth layer of the discriminant network, N D′j represents the dimension of the output features of the jth intermediate convolutional layer of D′;

[0106] Among them, the perceptual loss is:

[0107]

[0108] In the formula, V represents the backbone network, specifically VGG-16 pre-trained on ImageNet; T V represents the number of intermediate convolutional layers, Represents the number of elements of the output features of the j-th intermediate convolutional layer, V j (λ) denotes the j-th intermediate convolutional layer of V(λ).

[0109] Specifically, step S4 includes the following steps:

[0110] S41. Set the low-resolution distinction point to 1 / 4. If the 1 / N resolution mammary X-ray image in the current paired image group is a 1 / 4 resolution mammary X-ray image, execute step S43, otherwise execute step S42;

[0111] S42. Use the generative network to extract features of the two-dimensional breast tomography image in the current paired image group, and also extract features of the 1 / N resolution breast X-ray image in the current paired image group, and then splice the extracted features to obtain a 1 / 4 resolution synthetic breast X-ray image that implicitly combines the image information of the two, and then use the generative network loss function to evaluate; then, replace the 1 / N resolution breast X-ray image in the initial paired image group with the evaluated 1 / 4 resolution synthetic breast X-ray image, that is, pair the evaluated 1 / 4 resolution synthetic breast X-ray image with the two-dimensional breast tomography image, and then execute step S43;

[0112] S43. Use a high-resolution network to extract features, stitch and amplify the current paired image group to obtain a synthetic breast X-ray image, and then use a high-resolution network loss function to measure the difference between the synthetic breast X-ray image and the corresponding low-radiation dose breast X-ray image, and minimize the loss to optimize the parameters of the high-resolution network.

[0113] Embodiment 3

[0114] This embodiment is a third embodiment of a method for improving image quality based on a low-resolution breast X-ray film. This embodiment is similar to the first or second embodiment, except that:

[0115] Specifically, in step S5, the peak signal-to-noise ratio, structural similarity, Dice coefficient, and F1 score of calcification points are selected to construct an evaluation index function, and the best performing synthetic breast X-ray image is selected using the evaluation index function; wherein the evaluation index function is:

[0116] Score=ω PSNR PSNR+ω SSIM SSIM+ω Dice Dice+ω F1 F1 score;

[0117] In the formula, PSNR represents peak signal-to-noise ratio, SSIM represents structural similarity, Dice represents Dice coefficient, and F1 score represents the F1 score of calcification point; ω PSNR ,ω SSIM ,ω Dice ,ω F1 They represent the weighting coefficients of peak signal-to-noise ratio, structural similarity, Dice coefficient, and F1 score of calcification points respectively; in this embodiment, ω PSNR =0.01,ω SSIM =1,ω Dice =1,ω F1 =1.

[0118] The peak signal-to-noise ratio is:

[0119]

[0120] Where x represents the low-dose mammographic image, y represents the synthetic mammographic image, and MAX I It represents the maximum pixel value of the image, and MSE(x,y) represents the mean square error between x and y.

[0121] Among them, the structural similarity is:

[0122]

[0123] c 1 =(0.01MAX I ) 2 ;

[0124] c 2 =(0.03MAX I ) 2 ;

[0125] In the formula, μ x represents the mean of x, μ y represents the mean of y, represents the variance of x, represents the variance of y, σ xy represents the covariance of x and y; in this embodiment, a 11×11 rectangular window is taken during the calculation, the rectangular window is slid and the structural similarity of each rectangular window is calculated, and the mean of the structural similarities of all rectangular windows is taken as the global structural similarity of the image.

[0126] Among them, the tumor in the synthetic breast X-ray image was segmented using an automatic segmentation algorithm to extract the tumor contour, and the tumor contour was extracted using a manual delineation method, and the mass retention was evaluated based on the Dice coefficient; the Dice coefficient is:

[0127]

[0128] Where A represents the tumor contour obtained by automatic segmentation algorithm, B represents the tumor contour obtained by manual delineation, |A| represents the area of ​​A, |B| represents the area of ​​B, and |A∩B| represents the intersection area of ​​A and B.

[0129] Among them, the F1 score of the calcification point is:

[0130]

[0131] In the formula, Precision represents accuracy, and Sensitivity represents sensitivity;

[0132] The calcification point detection algorithm is used to detect calcification points on the synthetic breast X-ray image and the low-radiation-dose breast X-ray image. Then, the calcification points on the low-radiation-dose breast X-ray image are used as the gold standard calcification points. Then, for each calcification point detected on the synthetic breast X-ray image, if there is a gold standard calcification point within the radius (5 pixels) of the largest microcalcification point around it, it is considered to be a true positive and is set as TP. detect , otherwise it is considered a false positive and set to FP detect ; From this we can get:

[0133]

[0134] For each gold standard calcification point, if there is a detected calcification point within the surrounding 5 pixels in the synthetic mammographic image, it is considered detected and set as TP gt Otherwise, it is considered undetected and set as FP gt ; From this we can get:

[0135]

[0136] In the specific contents of the above-mentioned specific implementation methods, the various technical features can be combined in any non-contradictory manner. In order to make the description concise, not all possible combinations of the above-mentioned technical features are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0137] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A method for improving image quality based on low-resolution mammary X-ray films, characterized in that: The steps include: S1. Acquire three-dimensional breast tomography images and low-radiation-dose breast X-ray images of multiple patients at multiple projection positions, and obtain two-dimensional breast tomography images and low-resolution breast X-ray image groups after preprocessing the images; S2. Pairing the two-dimensional breast tomography image with all images in the low-resolution breast X-ray image group to obtain a paired image group, and then dividing all the obtained paired image groups into a training set and a test set; S3. Constructing a deep learning reconstruction model based on a deep learning method; the deep learning reconstruction model includes a generative network, a generative network loss function, a high-resolution network, and a high-resolution network loss function; S4. Input the training set to train the deep learning reconstruction model; wherein: step S4 includes the following steps: S41. Set a low-resolution distinction point. If the resolution of the low-resolution mammary X-ray image in the current paired image group is equal to the low-resolution distinction point, execute step S43, otherwise execute step S42; S42. Use the generative network to perform feature extraction and splicing on the current paired image group to obtain a low-resolution synthetic breast X-ray image equal to the low-resolution distinction point, and then use the generative network loss function to evaluate, and then update the evaluated low-resolution synthetic breast X-ray image to the current paired image group, and then execute step S43; S43. Use the high-resolution network to extract features, stitch and amplify the current paired image group to obtain a synthetic breast X-ray image, and then use the high-resolution network loss function to measure the difference between the synthetic breast X-ray image and the corresponding low-radiation dose breast X-ray image, and minimize the loss to optimize the parameters of the high-resolution network; S5. Use the trained deep learning reconstruction model to test on the test set, then compare the obtained synthetic breast X-ray image with the corresponding low-radiation dose breast X-ray image, and finally select the deep learning reconstruction model corresponding to the best performing synthetic breast X-ray image in the test set as the output model.

2. The method for improving image quality based on low-resolution mammary X-ray films according to claim 1, characterized in that: Step S1 includes the following steps: S11. Obtain three-dimensional breast tomosynthesis images and low-dose breast X-ray images of the patient in the left head-foot position, left medial-lateral oblique position, right head-foot position, and right medial-lateral oblique position; S12. Standardizing and desensitizing the three-dimensional breast tomography image to obtain the two-dimensional breast tomography image; S13. Regridding the low-radiation-dose breast X-ray images to obtain the low-resolution breast X-ray image group, wherein the low-resolution breast X-ray image group includes a plurality of 1 / N resolution breast X-ray images; wherein N≥4; Step S2 includes: pairing the two-dimensional breast tomography image of the same patient and the same projection position with each image in the low-resolution breast X-ray image group to obtain a paired image group, and dividing it into a training set S and a test set T according to the patient granularity.

3. The method for improving image quality based on low-resolution mammary X-ray films according to claim 1, characterized in that: In step S41, the low-resolution distinction point is 1 / 4.

4. The method for improving image quality based on low-resolution mammary X-ray films according to claim 3, characterized in that: In step S3, the generated network loss function is: In the formula, G L represents the generating network, |*| represents the l1 norm, |S| represents the training set size, represents a 1 / 4 resolution mammographic image, represents a 1 / N resolution mammographic image, represents a 1 / 4 resolution synthetic mammographic image, represents a Gaussian filter smoothing, I T represents a collection of two-dimensional breast tomosynthesis images; The high-resolution network loss function is: L(G N× )=L MS-MSE (G N× )+L GGGAN (G N× )+λ Percep L Percep (G N× ); In the formula, G N× represents the high-resolution network, L MS-MSE (G N× ) represents the mean square error, L GGGAN (G N× ) represents the loss of the generative adversarial network, L Percep (G N× ) represents the perceptual loss, λ Percep represents the weight of the balanced perceptual loss.

5. The method for improving image quality based on low-resolution mammary X-ray films according to claim 4, characterized in that: The mean square error is: In the formula, G N× represents a high-resolution network, |S| represents the training set size, represents a synthetic mammographic image, represents a 1 / N resolution mammographic image, I T represents a set of two-dimensional breast tomosynthesis images, I D Represents a low-radiation-dose mammogram. Represents two Gaussian filter smoothing composites; The generative adversarial network loss is: L GGGAN (G N× )=L Adver′ (G N× )+λ FM′ L FM′ (G N× ); Where, L Adver′ (G N× ) represents the loss function of the generative adversarial network, L FM′ (G N× ) represents the feature matching loss function, λ FM′ Represents the weight of the balanced feature matching loss function; The perceptual loss is: In the formula, V represents the backbone network, T V represents the number of intermediate convolutional layers, Represents the number of elements of the output features of the j-th intermediate convolutional layer, V j (*) denotes the j-th intermediate convolutional layer of V(*).

6. The method for improving image quality based on low-resolution mammary X-ray films according to claim 5, characterized in that: In the GAN loss: In the formula, x i represents a two-dimensional breast tomosynthesis image, y i Respectively represent 1 / 4 resolution mammary X-ray images, represents the synthetic mammogram predicted by the high-resolution network, y′ i , Respectively represent y i , The gradient of , D′ represents the discriminant network guided by the gradient, T D′ Represents the total number of intermediate convolutional layers, D′ j represents the output of the jth layer of the discriminant network, N D′ j represents the dimension of the output features of the jth intermediate convolutional layer of D′.

7. The method for improving image quality based on low-resolution mammary X-ray films according to claim 1, characterized in that: In step S5, the peak signal-to-noise ratio, structural similarity, Dice coefficient, and F1 score of calcification points are selected to construct an evaluation index function, and the best performing synthetic breast X-ray image is selected using the evaluation index function; wherein the evaluation index function is: Score=ω PSNR PSNR+ω SSIM SSIM+ω Dice Dice+ω F1 F1 score; Where PSNR represents peak signal-to-noise ratio, SSIM represents structural similarity, Dice represents Dice coefficient, and F1 score represents the F1 score of calcification point; ω PSNR ,ω SSIM ,ω dice ,ω F1 They represent the weighting coefficients of peak signal-to-noise ratio, structural similarity, Dice coefficient, and F1 score of calcification points respectively.

8. The method for improving image quality based on low-resolution mammary X-ray films according to claim 7, characterized in that: The peak signal-to-noise ratio is: Where x represents the low-dose mammographic image, y represents the synthetic mammographic image, and MAX I Represents the maximum pixel value of the image, and MSE(x,y) represents the mean square error between x and y; The structural similarity is: c1=(0.01MAX I ) 2 ; c2=(0.03MAX I ) 2 ; In the formula, μ x represents the mean of x, μ y represents the mean of y, represents the variance of x, represents the variance of y, σ xy represents the covariance of x and y.

9. The method for improving image quality based on low-resolution mammary X-ray films according to claim 7, characterized in that: The tumor in the synthetic mammographic image was segmented and its contour was extracted using an automatic segmentation algorithm, and the tumor contour was extracted using a manual delineation method, and the mass retention was evaluated based on the Dice coefficient; the Dice coefficient is: Where A represents the tumor contour obtained by automatic segmentation algorithm, B represents the tumor contour obtained by manual delineation, |A| represents the area of ​​A, |B| represents the area of ​​B, and |A∩B| represents the intersection area of ​​A and B.

10. The method for improving image quality based on low-resolution mammary X-ray films according to claim 7, characterized in that: The F1 score of the calcification point is: In the formula, Precision represents accuracy, and Sensitivity represents sensitivity; The calcification point detection algorithm is used to detect the calcification points on the synthetic breast X-ray image and the low-radiation-dose breast X-ray image; then, the calcification points on the low-radiation-dose breast X-ray image are used as the gold standard calcification points; then, for each calcification point detected on the synthetic breast X-ray image, if there is a gold standard calcification point within the radius of the largest microcalcification point around it, it is considered to be a true positive and is set as TP detect , otherwise it is considered a false positive and set to FP detect ; From this we can get: For each gold standard calcification point, if there is a detected calcification point within a fixed pixel size in the synthetic mammographic image, it is considered detected and set as TP gt Otherwise, it is considered undetected and set as FP gt ; From this we can get:

Images