Method for synthesizing CT angiography image based on generative adversarial network algorithm

Abstract

The invention discloses a method for synthesizing a CT angiography image based on a generative adversarial network algorithm, and the method comprises the steps: constructing an image synthesis model which sequentially comprises a generative network, a registration network, a spatial transformation network and a multi-scale discriminator, and carrying out the generative adversarial training of the image synthesis model; according to the method, image generation and registration are combined to the same network, error accumulation is reduced through end-to-end training, the generated image and the target image are more consistent in geometric structure, and the final generation quality is improved; and a multi-scale discriminator is combined with a weighted mean square error loss function, so that the game performance is enhanced, and more accurate error calculation is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and particularly to a method for synthesizing CT angiography images based on a generative adversarial network algorithm. Background Art

[0002] CTPA, namely Computed Tomography Pulmonary Angiography, is a medical imaging examination technique dedicated to observing the vascular conditions of the pulmonary artery and its branches. However, traditional CTPA examinations require the injection of contrast agents, which increases the radiation exposure and examination complexity of the subjects, and may also cause adverse reactions such as contrast agent allergies. In addition, the injection of contrast agents increases the examination cost and time.

[0003] Non-contrast CT, also known as non-enhanced CT or plain CT scan, is a commonly used medical imaging examination technique that is widely applied to the examinations of various parts of the body. Non-contrast CT examinations do not require the injection of contrast agents (such as iodine agents), so they are relatively simple and fast, and the subjects do not need to bear the risks such as allergies that contrast agents may bring.

[0004] Therefore, a method for synthesizing CTPA images based on non-contrast CT images is explored, with the expectation of improving the safety and efficiency of the examination without affecting the diagnostic accuracy. Summary of the Invention

[0005] Aiming at the deficiencies of the above-mentioned prior art, the present invention aims to provide a method for synthesizing CT angiography images based on a generative adversarial network algorithm, so as to synthesize CTPA images from non-contrast CT images and improve the examination safety and efficiency.

[0006] To solve the above problems, the present invention adopts the following technical solutions:

[0007] A method for synthesizing CT angiography images based on a generative adversarial network algorithm includes the following steps:

[0008] Step 1: Obtain multiple pairs of original non-contrast CT images and original CTPA images, preprocess the original non-contrast CT images and original CTPA images to obtain corresponding enhanced non-contrast CT images and corresponding enhanced CTPA images, and take each pair of enhanced non-contrast CT images and corresponding enhanced CTPA images as a group of sample pairs. The enhanced non-contrast CT image is denoted as the INCCT image, and the enhanced CTPA image is denoted as the ICTPA image;

[0009] Step 2: Construct an image synthesis model and a loss function, input the sample pairs into the image synthesis model and perform generative adversarial training on the image synthesis model to obtain a trained image synthesis model;

[0010] The image synthesis model sequentially includes a generation network, a registration network, a spatial transformation network, a blood vessel segmentation model, and a multi-scale discriminator;

[0011] The loss function includes the loss function \(L\) of the image synthesis model s and the loss function \(L\) of the multi-scale discriminator b , and the loss function \(L\) of the image synthesis model s is used to train and optimize the generation network and the registration network, and the loss function \(L\) of the multi-scale discriminator b is used to train and optimize the multi-scale discriminator;

[0012] Step 3: Load the anatomical structure registration information matrix and the spatial transformation network output by the trained generation network and registration network in the image synthesis model, input the to-be-processed INCCT image into the generation network, and generate the corresponding registered synthetic CTPA image after passing through the generation network and the spatial transformation network in sequence.

[0013] As described above, the generation network inputs the INCCT image and the corresponding ICTPA image in the sample pair to generate a synthetic CTPA image;

[0014] The registration network inputs the synthetic CTPA image and the corresponding ICTPA image to obtain an anatomical structure registration information matrix;

[0015] The spatial transformation network inputs the synthetic CTPA image and the anatomical structure registration information matrix to obtain a registered synthetic CTPA image;

[0016] The blood vessel segmentation model inputs the registered synthetic CTPA image and the corresponding ICTPA image to obtain a synthetic CTPA blood vessel segmentation image and the corresponding ICTPA blood vessel segmentation image; the blood vessel segmentation model performs segmentation at corresponding scales on each pair of registered synthetic CTPA images and the corresponding ICTPA images based on different segmentation scales;

[0017] When the multi-scale discriminator performs forward propagation to train and optimize the multi-scale discriminator, the synthetic CTPA blood vessel segmentation image or the ICTPA blood vessel segmentation image at different segmentation scales is used as the input image, and the multi-scale discriminator discriminates the probability that the input image is not a real ICTPA blood vessel segmentation image or the probability that the input image is a real ICTPA blood vessel segmentation image; when performing backpropagation to train and optimize the generation network and the registration network, the synthetic CTPA blood vessel segmentation image at different segmentation scales is used as the input image, and the multi-scale discriminator discriminates the probability that the input image is a real ICTPA blood vessel segmentation image.

[0018] As described above, the loss function \(L\) of the image synthesis model s :

[0019] L s = L sm + L sr + L adv + L snr

[0020] L sm is the generation network loss, L sr is the registration network loss, L adv is the generation network adversarial loss, L snr is the generation network vascular pixel loss;

[0021] The generation network loss L sm is:

[0022]

[0023] where Tr is the anatomical structure registration information matrix, E(·) is the operator for calculating the mean, and are the horizontal gradient and vertical gradient, and k sm is the generation network loss coefficient;

[0024] The registration network loss L sr is:

[0025]

[0026] where N is the number of sample pairs in each batch for training, is the registered synthetic CTPA image corresponding to the i-th sample pair, y i is the ICTPA image of the i-th sample pair, and k sr is the registration network loss coefficient;

[0027] The generation network adversarial loss L adv is:

[0028]

[0029] where p i is the probability that the synthetic CTPA vascular segmentation image at the first segmentation scale corresponding to the i-th sample pair is the real ICTPA vascular segmentation image; q i is the probability that the synthetic CTPA vascular segmentation image at the second segmentation scale corresponding to the i-th sample pair is the real ICTPA vascular segmentation image; k adv1 is the first weight coefficient of the generation network adversarial loss, and k adv2 is the second weight coefficient of the generation network adversarial loss;

[0030] The generation network vascular pixel loss L nsr is:

[0031]

[0032] where y i ′ _1 is the ICTPA vascular segmentation image at the first segmentation scale corresponding to the i-th sample pair; is the synthetic CTPA vascular segmentation image at the first segmentation scale corresponding to the i-th sample; y i ′ _2 is the ICTPA vascular segmentation image at the second segmentation scale corresponding to the i-th sample pair; is the synthetic CTPA vascular segmentation image at the second segmentation scale corresponding to the i-th sample; k snr is the vascular pixel loss coefficient of the generation network.

[0033] The loss function L of the multi-scale discriminator as described above b is:

[0034] L b =(L SynCTPA +L ICTPA ) / 2

[0035] where L SynCTPA is the loss for the discriminator to determine that the synthetic CTPA vascular segmentation image is forged, and its formula is defined as follows:

[0036]

[0037] where m i is the probability that the synthetic CTPA vascular segmentation image at the first segmentation scale corresponding to the i-th sample pair is not a real ICTPA vascular segmentation image; n i is the probability that the synthetic CTPA vascular segmentation image at the second segmentation scale corresponding to the i-th sample pair is not a real ICTPA vascular segmentation image; k SynCTPA1 is the first weight coefficient of the multi-scale discriminator loss, k SynCTPA2 is the second weight coefficient of the multi-scale discriminator loss;

[0038] L ICTPA is the loss for the discriminator to determine that the ICTPA vascular segmentation image is real, and its formula is defined as follows:

[0039]

[0040] where s i is the probability that the ICTPA vascular segmentation image at the first segmentation scale corresponding to the i-th sample pair is a real ICTPA vascular segmentation image; t iis the probability that the ICTPA vascular segmentation image corresponding to the i-th sample pair is the true ICTPA vascular segmentation image at the second segmentation scale; k ICTPA1 is the third weight coefficient of the multi-scale discriminator loss, k ICTPA1 is the fourth weight coefficient of the multi-scale discriminator loss.

[0041] Input the sample pairs described above into the image synthesis model for generative adversarial training of the image synthesis model. Each round of iterative training specifically includes the following steps:

[0042] A. Input the INCCT image and the corresponding ICTPA image in the sample pair into the generation network to generate a synthetic CTPA image;

[0043] B. Input the synthetic CTPA image and the corresponding ICTPA image into the registration network to obtain an anatomical structure registration information matrix;

[0044] C. Input the synthetic CTPA image and the anatomical structure registration information matrix into the spatial transformation network to obtain the registered synthetic CTPA image;

[0045] D. Input the registered synthetic CTPA image and the corresponding ICTPA image into the vascular segmentation model to obtain the synthetic CTPA vascular segmentation image and the corresponding ICTPA vascular segmentation image;

[0046] E. Keep the parameters of the multi-scale discriminator unchanged. Input the synthetic CTPA vascular segmentation image into the multi-scale discriminator. The multi-scale discriminator calculates the probability that the synthetic CTPA vascular segmentation image is the true ICTPA vascular segmentation image, calculates the value of the loss function Ls of the minimized image synthesis model, and updates the parameters of the generation network and the registration network through backpropagation;

[0047] F. Keep the parameters of the generation network and the registration network unchanged. Input the synthetic CTPA vascular segmentation image and the corresponding ICTPA vascular segmentation image into the multi-scale discriminator respectively. The multi-scale discriminator calculates the probability that the synthetic CTPA vascular segmentation image is not the true ICTPA vascular segmentation image and calculates the probability that the ICTPA vascular segmentation image is the true ICTPA vascular segmentation image respectively, and then calculates the value of the loss function L b of the minimized multi-scale discriminator and updates the parameters of the multi-scale discriminator.

[0048] Step 3 described above further includes the following steps:

[0049] The registered synthetic CTPA image undergoes denormalization, smoothing, and artifact removal, and is saved in DICOM format to obtain the final synthetic CTPA image.

[0050] A computer device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements steps 2-3 of the method for synthesizing a CT angiography image as described above in any item.

[0051] A computer-readable storage medium stores a computer program thereon. When the computer program is executed by a processor, it implements steps 2-3 of the method for synthesizing a CT angiography image as described above in any item.

[0052] A computer program product includes a computer program. When the computer program is executed by a processor, it implements steps 2-3 of the method for synthesizing a CT angiography image as described above in any item.

[0053] A system for synthesizing a CT angiography image based on a generative adversarial network algorithm includes an image synthesis model construction and training module and an image synthesis module:

[0054] The image synthesis model construction and training module is used to implement step 2 of the method for synthesizing a CT angiography image as described above in any item.

[0055] The image synthesis module is used to implement step 3 of the method for synthesizing a CT angiography image as described above in any item.

[0056] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0057] The present invention combines registration and generation, and performs end-to-end optimization, reducing deformation distortion and maintaining the consistency of anatomical structures. Compared with the traditional two-step method (first generating using a generative adversarial network GAN and then performing registration), the present invention combines image generation and registration into the same network, reducing error accumulation through end-to-end training, making the generated image more consistent with the target image in geometric structure and improving the final generation quality.

[0058] Using a single-scale discriminator usually can only focus on a fixed resolution or feature range, while the present invention uses a multi-scale discriminator that can discriminate features at different scales, improving the overall robustness: global scale (large range): focusing on the overall structure, such as organ shape and anatomical consistency; local scale (small range): focusing on details, such as edge sharpness and noise processing; in addition, a single-scale discriminator is prone to mode collapse, that is, the generator (G) may only learn local patterns during training and ignore global information. The multi-scale discriminator can: evaluate the quality of the generated image at different scales, making the generator have to deceive the discriminator at multiple scales, thereby generating more natural and comprehensive images; forcing the generator to learn global and local information more evenly and avoiding the problem of mode collapse.

[0059] The multi-scale discriminator combined with the weighted mean square error loss function enhances the game performance, thus achieving more accurate error calculation. In addition, the present invention integrates multi-dimensional information by introducing a 3D generation network, a 3D registration network, a 3D spatial transformation network, and a 3D multi-scale discriminator, significantly improving the expression ability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 It is a flowchart of the method for synthesizing CT angiography images based on the generative adversarial network algorithm of the present invention.

[0061] Figure 2 It is the original plain CT image of the lungs of the same subject of the present invention.

[0062] Figure 3 It is the original CTPA image of the lungs of the same subject of the present invention.

[0063] Figure 4 It is the CTPA image synthesized by the method of the present invention.

[0064] Figure 5 It is a schematic diagram of the system for synthesizing CT angiography images based on the generative adversarial network algorithm of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0065] To facilitate the understanding and use of the present invention by those skilled in the art, the present invention will be further described in detail below with reference to the embodiments. It should be noted that these embodiments are only used to illustrate the present invention, rather than limiting the present invention. Any simple improvement of the method under the premise of the concept of the present invention falls within the scope of protection required by the present invention.

[0066] Embodiment 1:

[0067] The method for synthesizing CT angiography images based on the generative adversarial network algorithm, as Figure 1 shown, includes the following steps:

[0068] Step 1: Obtain multiple pairs of original plain CT images and original CTPA images. After preprocessing the original plain CT images and original CTPA images, the corresponding enhanced plain CT images and corresponding enhanced CTPA images are obtained. Each pair of enhanced plain CT images and corresponding enhanced CTPA images is used as a group of sample pairs. The enhanced plain CT image is denoted as the INCCT image, and the enhanced CTPA image is denoted as the ICTPA image.

[0069] The original plain CT images and original CTPA images of the same subject at the same location (the lungs in this embodiment) collected are used as the original image data pairs, and the original image data pairs that meet the inclusion and exclusion rules are screened. The screened original plain CT images in the screened image data pairs are as Figure 2As shown, the filtered original CTPA images are as Figure 3 shown;

[0070] Preprocess the filtered original plain scan CT images and the filtered original CTPA images in the filtered image data pairs. The preprocessing includes, but is not limited to, cropping, rotation, flipping, contrast enhancement, and normalization to obtain INCCT images and corresponding ICTPA images.

[0071] Take each pair of INCCT images and corresponding ICTPA images as a group of sample pairs.

[0072] Step 2: Construct an image synthesis model and a loss function, and input the sample pairs into the image synthesis model for adversarial training of the image synthesis model. The specific steps are as follows:

[0073] Step 2.1: Construct an image synthesis model:

[0074] The image synthesis model sequentially includes a generation network, a registration network, a spatial transformation network, a vessel segmentation model, and a multi-scale discriminator;

[0075] The generation network is used to input the INCCT image and the corresponding ICTPA image in the sample pair, extract the anatomical structure features in the INCCT image and the vascular enhancement features in the ICTPA image, and generate a synthesized CTPA image; the synthesized CTPA image retains both the anatomical content of the INCCT image and the vascular enhancement information of the ICTPA image.

[0076] During training, the registration network is used to input the synthesized CTPA image and the corresponding ICTPA image, with the ICTPA image as the training label, to obtain an anatomical structure registration information matrix; so that the subsequent spatial transformation network can use the anatomical structure registration information matrix to register the synthesized CTPA image to obtain a registered synthesized CTPA image, thereby ensuring the consistency of the registered synthesized CTPA image and the real ICTPA image in the anatomical position.

[0077] The spatial transformation network is used to input the synthesized CTPA image and the anatomical structure registration information matrix to obtain a registered synthesized CTPA image; so that the registered synthesized CTPA image is aligned with the corresponding ICTPA image in the spatial position.

[0078] The vascular segmentation model is used to input the registered synthetic CTPA image and the corresponding ICTPA image, extract the vascular regions in the registered synthetic CTPA image and the corresponding ICTPA image, obtain the synthetic CTPA vascular segmentation image and the corresponding ICTPA vascular segmentation image, and the synthetic CTPA vascular segmentation image and the corresponding ICTPA vascular segmentation image are used as a pair of vascular segmentation images. Based on different segmentation scales, the vascular segmentation model performs segmentation at corresponding scales on each pair of registered synthetic CTPA images and the corresponding ICTPA images, that is, each pair of registered synthetic CTPA images and the corresponding ICTPA images are segmented according to the first segmentation scale, and then segmented according to the second segmentation scale until all segmentation scales are traversed for the registered synthetic CTPA images and the corresponding ICTPA images.

[0079] The multi-scale discriminator is used to optimize the generation network and the registration network through backpropagation training during the adversarial training process, and optimize the multi-scale discriminator through forward propagation training. When performing forward propagation training to optimize the multi-scale discriminator, the synthetic CTPA vascular segmentation image or ICTPA vascular segmentation image at different segmentation scales is used as the input image, and the multi-scale discriminator discriminates the probability that the input image is not a real ICTPA vascular segmentation image or the probability that the input image is a real ICTPA vascular segmentation image; when performing backpropagation training to optimize the generation network and the registration network, the synthetic CTPA vascular segmentation image at different segmentation scales is used as the input image, and the multi-scale discriminator discriminates the probability that the input image is a real ICTPA vascular segmentation image.

[0080] In this embodiment, the above-mentioned generation network, registration network, spatial transformation network, vascular segmentation model, and multi-scale discriminator all adopt a 3D network structure.

[0081] Step 2.2, construct a loss function

[0082] The loss function includes the loss function \(L\) of the image synthesis model s and the loss function \(L\) of the multi-scale discriminator b , and the loss function \(L\) of the image synthesis model s is used to train and optimize the generation network and the registration network, and the loss function \(L\) of the multi-scale discriminator b is used to train and optimize the multi-scale discriminator.

[0083] (1) Construct the loss function \(L\) of the image synthesis model s :

[0084] \(L\) s = \(L\) sm + \(L\) sr + \(L\) adv + \(L\) snr

[0085] L sm For generating the network loss, L sr For the registration network loss, L adv For the adversarial loss of the generation network, L snr For the vascular pixel loss of the generation network.

[0086] Among them, the generation network loss L sm is:

[0087]

[0088] Among them, Tr is the anatomical structure registration information matrix, and E(·) is the operator for calculating the mean, and are the horizontal gradient and the vertical gradient. k sm is the generation network loss coefficient. In this embodiment, k sm = 10.

[0089] The registration network loss L sr is:

[0090]

[0091] Among them, N is the number of each batch of sample pairs used for training, is the registered synthetic CTPA image corresponding to the i-th sample pair, y i is the ICTPA image of the i-th sample pair, k sr is the registration network loss coefficient. In this embodiment, k sr = 20.

[0092] The adversarial loss of the generation network L adv is:

[0093]

[0094] Among them, p i is the probability that the synthetic CTPA vascular segmentation image at the first segmentation scale corresponding to the i-th sample pair is the real ICTPA vascular segmentation image; q i is the probability that the synthetic CTPA vascular segmentation image at the second segmentation scale corresponding to the i-th sample pair is the real ICTPA vascular segmentation image; p i and q i belong to the probabilities that the synthetic CTPA vascular segmentation image is the real ICTPA vascular segmentation image obtained by inputting the synthetic CTPA image after different segmentation scales into the multi-scale discriminator. k adv1 is the first weight coefficient of the adversarial loss of the generation network, k adv2 is the second weight coefficient of the adversarial loss of the generation network. In this embodiment, kadv1 = 1.8, k adv2 = 0.2.

[0095] Generate the network vascular pixel loss L nsr as:

[0096]

[0097] where y i ′ _1 is the ICTPA vascular segmentation image at the first segmentation scale corresponding to the i-th sample pair; is the synthetic CTPA vascular segmentation image at the first segmentation scale corresponding to the i-th sample; y i ′ _2 is the ICTPA vascular segmentation image at the second segmentation scale corresponding to the i-th sample pair; is the synthetic CTPA vascular segmentation image at the second segmentation scale corresponding to the i-th sample; k snr is the network vascular pixel loss coefficient, and in this embodiment, k snr = 20.

[0098] (2) Construct the loss function L of the multi-scale discriminator b :

[0099] L b = (L SynCTPA + L ICTPA ) / 2

[0100] where L SynCTPA is the loss that the discriminator discriminates the synthetic CTPA vascular segmentation image as forged, and its formula is defined as follows:

[0101]

[0102] where m i is the probability that the synthetic CTPA vascular segmentation image at the first segmentation scale corresponding to the i-th sample pair is not a real ICTPA vascular segmentation image; n i is the probability that the synthetic CTPA vascular segmentation image at the second segmentation scale corresponding to the i-th sample pair is not a real ICTPA vascular segmentation image; m i and n i belong to the same registered synthetic CTPA image after being segmented at different segmentation scales and are respectively input into the multi-scale discriminator to obtain the probability that the synthetic CTPA vascular segmentation image is not a real ICTPA vascular segmentation image. k SynCTPA1 is the first weight coefficient of the multi-scale discriminator loss, k SynCTPA2 is the second weight coefficient of the multi-scale discriminator loss, and in this embodiment, kSynCTPA1 = 1.8, k SynCTPA2 = 0.2.

[0103] L ICTPA is the loss for the discriminator to judge that the ICTPA vascular segmentation image is real, and its formula is defined as follows:

[0104]

[0105] where s i is the probability that the ICTPA vascular segmentation image at the first segmentation scale corresponding to the i-th sample pair is a real ICTPA vascular segmentation image; t i is the probability that the ICTPA vascular segmentation image at the second segmentation scale corresponding to the i-th sample pair is a real ICTPA vascular segmentation image; k ICTPA1 is the third weight coefficient of the multi-scale discriminator loss, k ICTPA1 is the fourth weight coefficient of the multi-scale discriminator loss. In this embodiment, k ICTPA1 = 1.8, k ICTPA2 = 0.2.

[0106] Step 2.3. Perform model training. Each round of iteration includes the following processes:

[0107] A. Input the INCCT image and the corresponding ICTPA image in the sample pair into the generation network to generate a synthetic CTPA image.

[0108] B. Input the synthetic CTPA image and the corresponding ICTPA image into the registration network to obtain the anatomical structure registration information matrix.

[0109] C. Input the synthetic CTPA image and the anatomical structure registration information matrix into the spatial transformation network to obtain the registered synthetic CTPA image.

[0110] D. Input the registered synthetic CTPA image and the corresponding ICTPA image into the vascular segmentation model to obtain the synthetic CTPA vascular segmentation image and the corresponding ICTPA vascular segmentation image.

[0111] E. Keep the parameters of the multi-scale discriminator unchanged. Input the synthetic CTPA vascular segmentation image into the multi-scale discriminator. The multi-scale discriminator calculates the probability that the synthetic CTPA vascular segmentation image is a real ICTPA vascular segmentation image, calculates the value of the loss function Ls of the minimized image synthesis model, and updates the parameters of the generation network and the registration network through backpropagation, thereby guiding the generation network to reduce the difference between the synthetic CTPA image and the corresponding ICTPA image.

[0112] F. The parameters of the generation network and the registration network are fixed. The synthesized CTPA vascular segmentation image and the corresponding ICTPA vascular segmentation image are respectively input into the multi-scale discriminator. The multi-scale discriminator calculates the probability that the synthesized CTPA vascular segmentation image is not a real ICTPA vascular segmentation image (i.e., the forgery probability) and the probability that the ICTPA vascular segmentation image is a real ICTPA vascular segmentation image (i.e., the true probability), and then calculates the loss function L b of the multi-scale discriminator, thereby updating the parameters of the multi-scale discriminator and further optimizing the recognition ability of the multi-scale discriminator.

[0113] In this embodiment, the Adam optimizer is used to update the parameters of the generation network, the registration network, and the multi-scale discriminator.

[0114] Step 3. Load the anatomical structure registration information matrix and the spatial transformation network output by the generation network and the registration network in the trained image synthesis model. The anatomical structure registration information matrix output by the registration network is used to input the spatial transformation network. The INCCT image to be processed is input into the generation network, and after passing through the generation network and the spatial transformation network in sequence, the corresponding registered synthesized CTPA image is generated;

[0115] The registered synthesized CTPA image undergoes inverse normalization, smoothing processing, and artifact removal, and is saved as an image in DICOM format, which is the final synthesized CTPA image, as Figure 4 shown.

[0116] Embodiment 2

[0117] A system for synthesizing CT angiography images based on the generative adversarial network algorithm, as Figure 5 shown, implements the method for synthesizing CT angiography images based on the generative adversarial network algorithm described in Embodiment 1, including an image synthesis model construction and training module and an image synthesis module:

[0118] The image synthesis model construction and training module is used to implement Step 2 described in Embodiment 1,

[0119] The image synthesis module is used to implement Step 3 described in Embodiment 1.

[0120] Embodiment 3

[0121] A computer device includes a memory and a processor. A computer program is stored in the memory. When the processor executes the computer program, it implements Steps 2-3 in the above Embodiment 1.

[0122] Embodiment 4

[0123] A computer-readable storage medium has a computer program stored thereon, and when the computer program is executed by a processor, the steps 2-3 in the above-mentioned Embodiment 1 are implemented.

[0124] Embodiment 5

[0125] A computer program product includes a computer program, and when the computer program is executed by a processor, the steps 2-3 in the above-mentioned Embodiment 1 are implemented.

[0126] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described by referring to the preferred embodiments of the present invention, those of ordinary skill in the art should understand that various changes can be made to it in form and details without departing from the spirit and scope of the present invention defined by the appended claims.

Claims

1. A method for synthesizing CT angiography images based on a generative adversarial network algorithm, characterized in that: The steps include: Step 1, obtaining multiple groups of paired original plain scan CT images and original CTPA images, preprocessing the original plain scan CT images and the original CTPA images to obtain corresponding enhanced plain scan CT images and corresponding enhanced CTPA images, taking each group of paired enhanced plain scan CT images and corresponding enhanced CTPA images as a group of sample pairs, the enhanced plain scan CT images are recorded as INCCT images, and the enhanced CTPA images are recorded as ICTPA images; Step 2: construct an image synthesis model and a loss function, input the sample pairs into the image synthesis model, and perform generative adversarial training on the image synthesis model to obtain a trained image synthesis model; The image synthesis model includes a generation network, a registration network, a space transformation network, a blood vessel segmentation model and a multi-scale discriminator in sequence; The loss function includes the loss function L of the image synthesis model s and the loss function L of the multi-scale discriminator b , the loss function L of the image synthesis model s Used to train and optimize the generation network and the registration network, the loss function L of the multi-scale discriminator b Used to train and optimize multi-scale discriminators; Step 3: Load the generative network in the trained image synthesis model, the anatomical structure registration information matrix output by the registration network, and the spatial transformation network, input the INCCT image to be processed into the generative network, and generate the corresponding registered synthetic CTPA image after passing through the generative network and the spatial transformation network in turn.

2. The method for synthesizing CT angiography images based on a generative adversarial network algorithm according to claim 1, characterized in that: The generating network inputs the INCCT image and the corresponding ICTPA image in the sample pair to generate a synthesized CTPA image; The registration network inputs the synthesized CTPA image and the corresponding ICTPA image to obtain the anatomical structure registration information matrix; The spatial transformation network inputs the synthesized CTPA image and the anatomical structure registration information matrix to obtain a registered synthesized CTPA image; The vascular segmentation model inputs the registered synthetic CTPA image and the corresponding ICTPA image to obtain the synthetic CTPA vascular segmentation image and the corresponding ICTPA vascular segmentation image; the vascular segmentation model performs segmentation of each pair of registered synthetic CTPA images and corresponding ICTPA images at corresponding scales based on different segmentation scales; When the multi-scale discriminator is performing forward propagation training to optimize the multi-scale discriminator, the synthetic CTPA vascular segmentation image or the ICTPA vascular segmentation image at different segmentation scales is used as the input image, and the multi-scale discriminator determines the probability that the input image is not a true ICTPA vascular segmentation image or the probability that the input image is a true ICTPA vascular segmentation image; when the back-propagation training is performing to optimize the generation network and the registration network, the synthetic CTPA vascular segmentation image at different segmentation scales is used as the input image, and the multi-scale discriminator determines the probability that the input image is a true ICTPA vascular segmentation image.

3. The method for synthesizing CT angiography images based on a generative adversarial network algorithm according to claim 2, characterized in that: The loss function L of the image synthesis model is s : L s =L sm +L sr +L adv +L snr L sm To generate the network loss, L sr is the registration network loss, L adv To generate the network adversarial loss, L snr To generate the network blood vessel pixel loss; Generate network loss L sm for: Among them, Tr is the anatomical structure registration information matrix, E(·) is the mean operator, and is the horizontal gradient and vertical gradient, k sm To generate the network loss coefficient; Registration network loss L sr for: Where N is the number of sample pairs in each batch used for training, is the registered synthetic CTPA image corresponding to the i-th sample pair, y i is the ICTPA image of the i-th sample pair, k sr is the registration network loss coefficient; Generate network adversarial loss L adv for: Among them, p i is the probability that the synthetic CTPA vessel segmentation image under the first segmentation scale corresponding to the i-th sample pair is the true ICTPA vessel segmentation image; q i is the probability that the synthetic CTPA vessel segmentation image under the second segmentation scale corresponding to the i-th sample pair is the true ICTPA vessel segmentation image; k adv1 To generate the first weight coefficient of the network adversarial loss, k adv2 To generate the second weight coefficient of network adversarial loss; Generate network blood vessel pixel loss L nsr for: Among them, y i ′ _1 is the ICTPA blood vessel segmentation image at the first segmentation scale corresponding to the i-th sample pair; The synthetic CTPA vascular segmentation image at the first segmentation scale corresponding to the i-th sample; y i ′ _2 is the ICTPA blood vessel segmentation image at the second segmentation scale corresponding to the i-th sample pair; k is the synthesized CTPA vascular segmentation image at the second segmentation scale corresponding to the i-th sample; snr To generate the network vessel pixel loss coefficient.

4. The method for synthesizing CT angiography images based on a generative adversarial network algorithm according to claim 2, characterized in that: The loss function L of the multi-scale discriminator b for: L b =(L SynCTPA +L ICTPA ) / 2 Among them, L SynCTPA is the loss of the discriminator in distinguishing the synthetic CTPA vascular segmentation image as forged, and its formula is defined as follows: Among them, m i is the probability that the synthetic CTPA vessel segmentation image at the first segmentation scale corresponding to the i-th sample pair is not the true ICTPA vessel segmentation image; n i is the probability that the synthetic CTPA vessel segmentation image at the second segmentation scale corresponding to the i-th sample pair is not the true ICTPA vessel segmentation image; k SynCTPA1 is the first weight coefficient of the multi-scale discriminator loss, k SynCTPA2 is the second weight coefficient of the multi-scale discriminator loss; L ICTPA The loss for the discriminator to judge the ICTPA blood vessel segmentation image as true is defined as follows: Among them, s i is the probability that the ICTPA vessel segmentation image at the first segmentation scale corresponding to the i-th sample pair is the true ICTPA vessel segmentation image; t i is the probability that the ICTPA vessel segmentation image at the second segmentation scale corresponding to the i-th sample pair is the true ICTPA vessel segmentation image; k ICTPA1 is the third weight coefficient of the multi-scale discriminator loss, k ICTPA1 It is the fourth weight coefficient of the multi-scale discriminator loss.

5. The method for synthesizing CT angiography images based on a generative adversarial network algorithm according to claim 2, characterized in that: The sample is input to the image synthesis model to perform generative adversarial training on the image synthesis model. Each round of iterative training specifically includes the following steps: A. Input the INCCT image and the corresponding ICTPA image in the sample pair into the generation network to generate a synthetic CTPA image; B. The synthesized CTPA image and the corresponding ICTPA image are input into the registration network to obtain the anatomical structure registration information matrix; C. The synthesized CTPA image and the anatomical structure registration information matrix are input into a spatial transformation network to obtain a registered synthesized CTPA image; D. The registered synthetic CTPA image and the corresponding ICTPA image are input into the vascular segmentation model to obtain a synthetic CTPA vascular segmentation image and a corresponding ICTPA vascular segmentation image; E. The parameters of the multi-scale discriminator remain unchanged, and the synthesized CTPA vascular segmentation image is input into the multi-scale discriminator. The multi-scale discriminator calculates the probability that the synthesized CTPA vascular segmentation image is the real ICTPA vascular segmentation image, calculates the value of the loss function Ls of the minimized image synthesis model, and updates the parameters of the generation network and the registration network through back propagation; F. The parameters of the generation network and the registration network are fixed. The synthetic CTPA vascular segmentation image and the corresponding ICTPA vascular segmentation image are respectively input into the multi-scale discriminator. The multi-scale discriminator calculates the probability that the synthetic CTPA vascular segmentation image is not the real ICTPA vascular segmentation image and the probability that the ICTPA vascular segmentation image is the real ICTPA vascular segmentation image, and then calculates the loss function L that minimizes the multi-scale discriminator. b The value of , updates the parameters of the multi-scale discriminator.

6. The method for synthesizing CT angiography images based on a generative adversarial network algorithm according to claim 1, characterized in that: The step 3 also includes the following steps: The registered synthetic CTPA images were denormalized, smoothed, and artifacts removed, and saved in DICOM format to obtain the final synthetic CTPA images.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, steps 2-3 of the method for synthesizing CT angiography images according to any one of claims 1 to 6 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, steps 2-3 of the method for synthesizing CT angiography images according to any one of claims 1 to 6 are implemented.

9. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, steps 2-3 of the method for synthesizing CT angiography images according to any one of claims 1 to 6 are implemented.

10. A system for synthesizing CT angiography images based on a generative adversarial network algorithm, characterized in that: Including image synthesis model construction and training module and image synthesis module: The image synthesis model construction and training module is used to implement step 2 of the method for synthesizing CT angiography images according to any one of claims 1 to 6, The image synthesis module is used to implement step 3 of the method for synthesizing CT angiography images according to any one of claims 1 to 6.

Images