A method and system for generating a virtual pulmonary angiogram based on a generative adversarial network

Abstract

The application relates to a pulmonary artery virtual angiography generation method and system based on a generative adversarial network. The method comprises the following steps: acquiring processed unpaired plain CT images and real CTPA images as sample data and dividing the sample data into a training set and a verification set; sampling original plain CT images and original CTPA images from the training set and inputting the original plain CT images and the original CTPA images into a generative adversarial network; generating a virtual CTPA image through a generator, performing CTPA domain discrimination through a first discriminator, and reconstructing a plain CT image through an inverse generator; calculating a multi-objective loss function based on an output result, updating network parameters and loss weights, and iteratively training until convergence to obtain a pulmonary artery virtual angiography generation model; and inputting a target plain CT image into the model to output a target virtual CTPA image.

Description

Technical Field

[0001] This application relates to the field of medical image processing technology, and in particular to a method and system for generating virtual pulmonary artery angiography based on generative adversarial networks. Background Technology

[0002] Pulmonary embolism is a common and high-risk cardiovascular emergency. Clinically, CT pulmonary angiography is usually used to visualize the pulmonary artery and its branches to help determine the location of the thrombus, the degree of vascular obstruction, and the extent of the lesion.

[0003] Current CT (Computed Tomography) pulmonary angiography requires the injection of iodine contrast agents. While this improves pulmonary artery visualization, its use is limited for certain populations, making it difficult for some patients to obtain timely and effective pulmonary artery imaging assessments. Plain CT scans do not require contrast agents and offer advantages such as convenience and wider applicability. However, due to the small grayscale differences between blood vessels, blood, thrombi, and surrounding soft tissues, the clarity of visualization of the pulmonary artery lumen and segmental / subsegmental branches is insufficient, making it difficult to directly meet the needs of pulmonary angiography analysis.

[0004] Existing image conversion methods based on paired data typically require precise registration between plain CT images and CTPA (Computed Tomography Pulmonary Angiography) images of the same patient. However, due to respiratory motion, changes in body position, and differences in scanning parameters, acquiring paired data and pixel-level registration are quite difficult, limiting the widespread application of virtual angiography technology. Therefore, how to generate structurally stable and clearly visualized virtual CTPA images under conditions of unpaired plain CT images and real CTPA images has become an urgent technical problem to be solved. Summary of the Invention

[0005] To overcome, to some extent, the problem in related technologies of generating structurally stable and clearly visualized virtual CTPA images under conditions of unpaired plain CT images and real CTPA images, this application provides a method and system for generating virtual pulmonary artery angiography based on generative adversarial networks.

[0006] The proposed solution is as follows:

[0007] According to a first aspect of the embodiments of this application, a method for generating virtual pulmonary artery angiography based on generative adversarial networks is provided, comprising: Processed, unpaired plain CT images and real CTPA images are obtained as sample data, and the sample data is divided into a training set and a validation set. Sampling is performed in the training set, and the sampled plain CT images and real CTPA images are used as the original plain CT images and the original CTPA images, respectively. The original plain CT images and original CTPA images are input into a pre-constructed generative adversarial network; the generative adversarial network includes a generator, an inverse generator, and a discriminator; the discriminator includes a first discriminator and a second discriminator; the first discriminator is used to discriminate the CTPA domain images, and the second discriminator is used to discriminate the plain CT domain images; The generator converts the original plain CT image into a virtual CTPA image, the first discriminator distinguishes the virtual CTPA image from the original CTPA image, and the inverse generator converts the virtual CTPA image into a reconstructed plain CT image. Calculate the multi-objective loss function based on the output of the generative adversarial network; The parameters of the generative adversarial network are updated based on the calculation results of the multi-objective loss function, and the weights of each loss in the multi-objective loss function are also updated. The generative adversarial network is iteratively trained until convergence to obtain a virtual pulmonary angiography generation model. The target plain CT image is input into the pulmonary artery virtual angiography generation model, and the target virtual CTPA image is output.

[0008] Preferably, based on the output of the generative adversarial network, a multi-objective loss function is calculated, including: The discrimination results of the first discriminator on the virtual CTPA image and the original CTPA image are obtained as the CTPA domain discrimination results; Based on the CTPA domain discrimination result, the real difference between the virtual CTPA image and the original CTPA image in the CTPA domain is determined; Based on the aforementioned realism differences, calculate the CTPA domain generative adversarial loss; Obtain the reconstructed plain CT image output by the inverse generator; The image difference is compared between the reconstructed plain CT image and the original plain CT image to obtain the plain CT domain cyclic reconstruction difference. Based on the differences in cyclic reconstruction in the plain CT domain, calculate the cyclic consistency loss in the plain CT domain; The CTPA domain generation adversarial loss and the plain CT domain cycle consistency loss are used as components of the multi-objective loss function.

[0009] Preferably, the method further includes: The inverse generator converts the original CTPA image into a pseudo-plain CT image. The pseudo-plain CT image and the original plain CT image are distinguished by the second discriminator to obtain the plain CT domain discrimination result; The generator converts the pseudo-plain CT image into a reconstructed CTPA image. Based on the discrimination results of the plain CT domain, calculate the adversarial loss generated in the plain CT domain; The CTPA domain cyclic consistency loss is calculated based on the image differences between the reconstructed CTPA image and the original CTPA image. The plain CT domain generation adversarial loss and the CTPA domain cycle consistency loss are included as components of the multi-objective loss function.

[0010] Preferably, calculating the multi-objective loss function based on the output of the generative adversarial network further includes: Acquire the virtual CTPA image and the original plain CT image; Brightness feature extraction, contrast feature extraction, and structural feature extraction are performed on the virtual CTPA image and the original plain CT image, respectively, to obtain the virtual image structural features corresponding to the virtual CTPA image and the original image structural features corresponding to the original plain CT image. Based on the differences between the structural features of the virtual image and the structural features of the original image, the structural similarity loss is calculated. Edge gradient extraction is performed on the virtual CTPA image to obtain virtual CTPA edge gradient information; Edge gradient extraction is performed on the original plain CT image to obtain the original plain CT edge gradient information; The edge preservation loss is calculated based on the edge gradient difference between the virtual CTPA edge gradient information and the original plain CT edge gradient information. The structural similarity loss and edge preservation loss are incorporated as components of the multi-objective loss function.

[0011] Preferably, the parameters of the generative adversarial network are updated based on the calculation results of the multi-objective loss function, and the weights of each loss in the multi-objective loss function are updated, including: Obtain the CTPA domain generative adversarial loss, plain CT domain generative adversarial loss, plain CT domain cycle consistency loss, CTPA domain cycle consistency loss, structural similarity loss, and edge preservation loss for the current training round; Set corresponding learnable weight parameters for each loss; Based on the learnable weight parameters, determine the current loss weight corresponding to each loss in the current training round; The losses are weighted and fused according to the current loss weights to obtain the current multi-objective loss value; The generator, inverse generator, first discriminator, and second discriminator are updated by backpropagation based on the current multi-objective loss value. The learnable weight parameters are updated synchronously based on the backpropagation update results to redetermine the loss weights corresponding to each loss in the next training round.

[0012] Preferably, both the generator and the inverse generator include a single-channel adaptation layer, an encoder, a decoder, a skip connection unit, and an output mapping layer; The single-channel adaptation layer is used to receive grayscale density information of the input image and map the grayscale density information into single-channel image features; The encoder includes a multi-level convolutional feature extraction unit, which is used to downsample the single-channel image features step by step to obtain lung image features at different levels. The encoder is embedded with an attention enhancement unit, which is used to enhance the vascular-related features in the lung image features at different levels to obtain vascular enhancement features; The decoder includes a multi-level upsampling reconstruction unit, which is used to upsample the vascular enhancement features step by step to obtain lung image features with restored resolution. The decoder is embedded with a multi-scale residual reconstruction unit, which is used to fuse and reconstruct vascular detail features at different scales to obtain vascular detail reconstruction features. The skip connection unit is used to transfer lung image features of the corresponding level in the encoder to the corresponding level in the decoder in order to maintain the structure of non-vascular regions in the input image. The output mapping layer is used to generate a corresponding output image based on the vascular detail reconstruction features.

[0013] Preferably, both the first discriminator and the second discriminator include a multi-scale feature extraction unit, a local image block discrimination unit, and a discrimination result fusion unit; The multi-scale feature extraction unit is used to extract features from the input image at different resolutions to obtain image patch features at multiple scales. The local image block discrimination unit is used to perform authenticity discrimination on image block features at each scale, and obtain the local authenticity discrimination results corresponding to each scale. The discrimination result fusion unit is used to fuse the local authenticity discrimination results corresponding to each scale to obtain the discrimination result of the corresponding image domain.

[0014] Preferably, the processed plain CT images and real CTPA images are acquired as sample data, including: Acquire plain CT images to be processed and actual CTPA images; The plain CT image to be processed and the real CTPA image are subjected to grayscale truncation to obtain the grayscale truncated plain CT image and the real CTPA image. Normalization processing was performed on the gray-scale truncated plain CT images and the real CTPA images to obtain normalized plain CT images and real CTPA images. According to the preset cropping size, the normalized plain CT image and the real CTPA image are cropped to obtain the cropped plain CT image and the real CTPA image. Online data enhancement processing was performed on the cropped plain CT images and the real CTPA images to obtain enhanced plain CT images and real CTPA images. The online data augmentation process includes one or more of random flipping, random rotation, and random scaling.

[0015] Preferably, the method further includes: Validation plain CT images are obtained from the validation set and input into the pulmonary artery virtual angiography generation model to obtain validation virtual CTPA images. Obtain the verification reference CTPA image, and calculate the peak signal-to-noise ratio, structural similarity index, mean absolute error index, and perceptual similarity index based on the verification virtual CTPA image and the verification reference CTPA image, respectively. Based on the peak signal-to-noise ratio, structural similarity, mean absolute error, and perceptual similarity indices, as well as the scoring rules corresponding to each index, an objective evaluation score for the verification virtual CTPA image is obtained. The physician's subjective evaluation scores for the verified virtual CTPA images are obtained, including vascular mapping integrity score, peripheral vascular clarity score, anatomical structure fidelity score, and diagnostic acceptability score. The validation results of the pulmonary artery virtual angiography generation model are determined based on the objective evaluation score and the subjective evaluation score.

[0016] According to a second aspect of the embodiments of this application, a pulmonary artery virtual angiography generation system based on generative adversarial networks is provided, comprising: Processor and memory; The processor and memory are connected via a communication bus: The processor is used to call and execute the program stored in the memory; The memory is used to store a program, which is at least used to execute a method for generating virtual pulmonary angiography based on generative adversarial networks as described in any of the preceding claims.

[0017] The technical solution provided in this application may include the following beneficial effects: This technical solution eliminates the need to pair plain CT images and real CTPA images of the same patient when preparing sample data. It only requires acquiring datasets from two domains (plain CT images and real CTPA images). The model does not rely on pixel-level registration data between plain CT images and CTPA images of the same patient, reducing the difficulty of training data acquisition and registration processing. A generator converts the original plain CT images into virtual CTPA images, and a first discriminator distinguishes between the virtual CTPA images and the original CTPA images, allowing the generated results to gradually approximate the data distribution of real CTPA images, improving the realism of pulmonary artery region imaging. An inverse generator reconstructs the virtual CTPA images into a reconstructed plain CT image. The image processing model constrains the conversion process from the plain CT domain to the CTPA domain, thereby reducing structural deformation of non-target areas such as bones, airways, and lung parenchyma during virtual angiography. Through a multi-objective loss function and its weight update mechanism, it jointly constrains the generation realism, structural consistency, and training stability, avoiding a single loss function dominating the training process and improving model convergence and image quality. Finally, by inputting the target plain CT image into the converged pulmonary artery virtual angiography generation model, the model outputs the target virtual CTPA image. This improves the display clarity of the pulmonary artery and its branches on plain CT without relying on contrast agent enhancement, providing more readable virtual angiography results for subsequent pulmonary artery image analysis.

[0018] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0019] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0020] Figure 1 This is a flowchart illustrating a method for generating virtual pulmonary artery angiography based on generative adversarial networks, provided in one embodiment of this application. Figure 2 This is a schematic diagram of a pulmonary artery virtual angiography generation system based on generative adversarial networks, provided in one embodiment of this application. Detailed Implementation

[0021] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0022] Example 1 Figure 1 This is a flowchart illustrating a method for generating virtual pulmonary artery angiography based on generative adversarial networks, according to an embodiment of this application. (Refer to...) Figure 1 A method for generating virtual pulmonary artery angiography based on generative adversarial networks, comprising: S1. Obtain the processed unpaired plain CT images and real CTPA images as sample data, and divide the sample data into training set and validation set. S2. Sample the plain CT images and real CTPA images in the training set and use them as the original plain CT images and the original CTPA images, respectively. S3. Input the original plain CT images and the original CTPA images into a pre-constructed generative adversarial network; the generative adversarial network includes a generator, an inverse generator, and a discriminator; the discriminator includes a first discriminator and a second discriminator; the first discriminator is used to discriminate the CTPA domain images, and the second discriminator is used to discriminate the plain CT domain images; S4. The original plain CT image is converted into a virtual CTPA image by the generator. The virtual CTPA image and the original CTPA image are distinguished by the first discriminator. The virtual CTPA image is converted into a reconstructed plain CT image by the inverse generator. S5. Calculate the multi-objective loss function based on the output of the generative adversarial network; S6. Update the parameters of the generative adversarial network based on the calculation results of the multi-objective loss function, and update the weights of each loss in the multi-objective loss function. S7. Iteratively train the generative adversarial network until it converges to obtain the virtual angiography generation model of the pulmonary artery. S8. Input the target plain CT image into the pulmonary artery virtual angiography generation model and output the target virtual CTPA image.

[0023] For ease of understanding, the following explains some key terms in this embodiment: Generative Adversarial Networks (GANs) are a type of deep learning model. The core idea is to enable the generator to generate fake data that is highly similar to real data through adversarial training between the generator and the discriminator, while the discriminator tries to distinguish between real data and fake data.

[0024] In a generative adversarial network (GAN), the generator is responsible for transforming one type of input data into another type of output data. For example, in this method, the generator is used to convert plain CT images into virtual CTPA images.

[0025] An inverse generator is the reverse operation of a generator, its function being to convert the output data generated by the generator back to the domain of the original input data. In this method, the inverse generator is used to convert virtual CTPA images back into reconstructed plain CT images.

[0026] In a generative adversarial network (GAN), the discriminator is responsible for evaluating the authenticity of the input data. It learns the distribution characteristics of real data to determine whether the input data is genuine or fabricated data generated by the generator.

[0027] CTPA region images refer to images with pulmonary angiography characteristics, typically showing clear visualization of vascular structures. Plain CT region images refer to CT images without contrast enhancement, where vascular structures are not clearly visualized.

[0028] Virtual CTPA images are simulated CTPA images generated by a generator based on plain CT images. They aim to present pulmonary artery imaging effects similar to real CTPA images without the injection of contrast agents.

[0029] Reconstructed plain CT images refer to plain CT images converted back by the inverse generator from virtual CTPA images. The purpose is to evaluate the image consistency between the generator and the inverse generator during the cyclic conversion process.

[0030] A multi-objective loss function is a comprehensive loss function that guides the training of a generative adversarial network by combining multiple different loss terms. Each loss term focuses on different aspects of model performance, such as image realism, cycle consistency, and structural similarity, and they are collectively optimized by weighted summation.

[0031] In implementing this method, the first step is to acquire processed, unpaired plain CT images and real CTPA images as sample data, which are then divided into training and validation sets. Specifically, a large number of plain CT images and real CTPA images can be manually collected and preliminarily screened and labeled by professionals. Subsequently, these image data can be manually divided according to a preset ratio, such as 80% for training and 20% for validation, thus forming the training and validation sets.

[0032] Furthermore, sampling is performed on this training set, using the sampled plain CT images and real CTPA images as the original plain CT images and original CTPA images, respectively. For example, random sampling can be used, randomly selecting a pair of plain CT images and real CTPA images from the training set as input for the current training batch. This sampling method ensures the diversity of training data.

[0033] Subsequently, the original plain CT image and the original CTPA image are input into a pre-constructed generative adversarial network (GAN). This GAN includes a generator, an inverse generator, and a discriminator. The discriminator further includes a first discriminator and a second discriminator. Specifically, the generator and inverse generator can employ an encoder-decoder structure, where the encoder extracts image features and the decoder reconstructs the image from the features. The discriminator can consist of a series of convolutional layers and activation functions for binary classification of the input images. The first discriminator is configured to receive CTPA domain images (including real CTPA images and virtual CTPA images) and determine their authenticity, while the second discriminator is configured to receive plain CT domain images (including real plain CT images and pseudo plain CT images) and determine their authenticity.

[0034] The generator converts the original plain CT image into a virtual CTPA image. Simultaneously, the first discriminator distinguishes between the virtual CTPA image and the original CTPA image. Furthermore, the inverse generator converts the virtual CTPA image into a reconstructed plain CT image. Specifically, the original plain CT image is fed into the generator, which extracts and transforms its features, outputting a virtual CTPA image. Subsequently, this virtual CTPA image, along with the real CTPA image, is fed into the first discriminator, which evaluates its realism. Simultaneously, the generated virtual CTPA image is also fed into the inverse generator, which converts it back to the plain CT domain to obtain the reconstructed plain CT image.

[0035] Based on the output of the Generative Adversarial Network (GAN), a multi-objective loss function is calculated. The GAN parameters are then updated based on the calculated multi-objective loss function, including the weights of each loss term. Specifically, based on the calculated multi-objective loss values, gradient descent and other optimization algorithms can be used to backpropagate and update all trainable parameters of the generator, inverse generator, and discriminator within the GAN. The weights of each loss term in the multi-objective loss function can be preset to fixed values ​​or manually adjusted empirically during training.

[0036] The generative adversarial network (GAN) is iteratively trained until convergence to obtain the pulmonary artery virtual angiography generation model. The training process involves repeatedly executing the above steps, inputting sample data repeatedly into the network, and continuously adjusting the network parameters based on the loss function results. When the network performance no longer significantly improves on the validation set, or when the loss function value stabilizes, the network can be considered to have reached convergence, and the network parameters at this point constitute the pulmonary artery virtual angiography generation model.

[0037] Finally, the target plain CT image is input into the pulmonary artery virtual angiography generation model, which outputs a target virtual CTPA image. After the model training is complete, an untrained target plain CT image is input into the trained generator. The generator, based on its learned mapping relationship, converts the target plain CT image into the corresponding target virtual CTPA image and outputs it.

[0038] This technical solution eliminates the need to pair plain CT images and real CTPA images of the same patient when preparing sample data. It only requires acquiring datasets from two domains (plain CT images and real CTPA images). The model does not rely on pixel-level registration data between plain CT images and CTPA images of the same patient, reducing the difficulty of training data acquisition and registration processing. A generator converts the original plain CT images into virtual CTPA images, and a first discriminator distinguishes between the virtual CTPA images and the original CTPA images, allowing the generated results to gradually approximate the data distribution of real CTPA images, improving the realism of pulmonary artery region imaging. An inverse generator reconstructs the virtual CTPA images into a reconstructed plain CT image. The image processing model constrains the conversion process from the plain CT domain to the CTPA domain, thereby reducing structural deformation of non-target areas such as bones, airways, and lung parenchyma during virtual angiography. Through a multi-objective loss function and its weight update mechanism, it jointly constrains the generation realism, structural consistency, and training stability, avoiding a single loss function dominating the training process and improving model convergence and image quality. Finally, by inputting the target plain CT image into the converged pulmonary artery virtual angiography generation model, the model outputs the target virtual CTPA image. This improves the display clarity of the pulmonary artery and its branches on plain CT without relying on contrast agent enhancement, providing more readable virtual angiography results for subsequent pulmonary artery image analysis.

[0039] Preferably, during the model iterative training process, the Adam optimizer and fixed learning rate decay strategy are adopted. Alternative solutions include: (1) using the Adam W optimizer and introducing weight decay to prevent overfitting; (2) using a cosine annealing learning rate scheduler; and (3) adopting a progressive growth training strategy to gradually transition from low resolution to high resolution.

[0040] Example 2 In some embodiments, a multi-objective loss function is calculated based on the output of the generative adversarial network, including: Obtain the discrimination results of the first discriminator on the virtual CTPA image and the original CTPA image, and use them as the CTPA domain discrimination results; Based on the CTPA domain discrimination results, the real differences between the virtual CTPA image and the original CTPA image in the CTPA domain are determined; Based on the difference in realism, calculate the adversarial loss in the CTPA domain; Obtain the reconstructed plain CT images output by the inverse generator; The image difference was compared between the reconstructed plain CT image and the original plain CT image to obtain the difference in the cyclic reconstruction of the plain CT domain. Calculate the loss of cyclic consistency in the plain CT domain based on the differences in cyclic reconstruction in the plain CT domain; The CTPA domain generative adversarial loss and the plain CT domain cyclic consistency loss are used as components of the multi-objective loss function.

[0041] Specifically, when calculating the multi-objective loss function, the first step is to obtain the discrimination results of the first discriminator on the virtual CTPA image and the original CTPA image. As a key component in the generative adversarial network (GAN), the first discriminator distinguishes the authenticity of the input image, i.e., determines whether the image is a genuine CTPA image. Its discrimination results are typically output as probability values; for example, a value close to 1 indicates a genuine CTPA image, while a value close to 0 indicates a virtual CTPA image. These discrimination results constitute the CTPA domain discrimination results, used to quantify the degree of authenticity of the virtual CTPA image in the CTPA domain.

[0042] Subsequently, based on the CTPA domain discrimination results, the realism difference between the virtual CTPA image and the original CTPA image in the CTPA domain can be determined. This realism difference reflects the quality of the virtual CTPA image generated by the generator, i.e., its similarity to the real CTPA image. The greater the difference, the less realistic the virtual CTPA image generated by the generator; the smaller the difference, the closer the virtual CTPA image is to the real CTPA image. Based on this realism difference, the CTPA domain generative adversarial loss can be calculated. This loss function typically adopts forms such as binary cross-entropy loss or least squares loss, and its goal is to encourage the generator to generate more realistic virtual CTPA images to deceive the first discriminator, while simultaneously prompting the first discriminator to more accurately distinguish between real and virtual images. By optimizing this loss, the visual realism and medical credibility of the generated virtual CTPA image can be effectively improved.

[0043] To ensure the stability of the generation process and the integrity of the image content, this application also introduces a cycle consistency loss. This first requires obtaining the reconstructed plain CT image output by the inverse generator. The inverse generator's role is to reverse-convert the virtual CTPA image back to a plain CT image to verify the reversibility of the conversion process. The reconstructed plain CT image is the result of this inverse conversion. Next, the reconstructed plain CT image is compared with the original plain CT image to obtain the cyclic reconstruction difference in the plain CT domain. This comparison aims to measure the similarity between the image obtained from the original plain CT image, after generating a virtual CTPA image by the generator, and then reconstructing it back to a plain CT image by the inverse generator, and the original image. The smaller the difference, the less information is lost during the cyclic conversion, and the better the image content is preserved. Based on this cyclic reconstruction difference in the plain CT domain, the cyclic consistency loss in the plain CT domain can be calculated. The loss function typically takes the form of L1 loss or L2 loss, and its goal is to minimize the pixel-level or feature-level differences between the reconstructed plain CT image and the original plain CT image, thereby forcing the generator and inverse generator to maintain content consistency during image conversion.

[0044] Finally, the CTPA domain generative adversarial loss and the plain CT domain cycle consistency loss are incorporated into the multi-objective loss function. By combining these two losses, the multi-objective loss function can simultaneously guide the generative adversarial network to optimize in two key aspects: on the one hand, the CTPA domain generative adversarial loss ensures that the generated virtual CTPA images have a high degree of realism, making them visually difficult to distinguish from real CTPA images; on the other hand, the plain CT domain cycle consistency loss guarantees that the conversion process from plain CT to virtual CTPA is reversible and information-preserving, avoiding the introduction of unnecessary artifacts or the loss of important anatomical structural information during the generation process.

[0045] In this embodiment, the method further includes: The original CTPA images are converted into pseudo-plain CT images using a reverse generator. The pseudo-plain CT image and the original plain CT image are distinguished by the second discriminator to obtain the plain CT domain discrimination result; The generator converts pseudo-plain CT images into reconstructed CTPA images. Based on the discrimination results of the plain CT domain, calculate the adversarial loss generated in the plain CT domain; The CTPA domain cyclic consistency loss is calculated based on the image differences between the reconstructed CTPA image and the original CTPA image. The plain CT domain generation adversarial loss and the CTPA domain cycle consistency loss are included as components of the multi-objective loss function.

[0046] Specifically, the original CTPA images are converted into pseudo-plain CT images using an inverse generator. Here, the inverse generator is designed to learn the mapping relationship from the CTPA image domain to the plain CT image domain. It receives real CTPA images as input and attempts to generate an output—the pseudo-plain CT image—that is as close as possible to the real plain CT image in both visual features and medical significance. This process aims to simulate the image characteristics after contrast agent fades, providing a basis for subsequent discrimination and cyclic reconstruction.

[0047] Subsequently, a second discriminator distinguishes between the pseudo-plain CT image and the original plain CT image, obtaining the plain CT domain discrimination result. The second discriminator is specifically responsible for evaluating the similarity or authenticity between the pseudo-plain CT image generated by the inverse generator and the genuine original plain CT image in the plain CT image domain. It analyzes features such as image texture, structure, and grayscale distribution, outputting a discrimination result that indicates the likelihood that the pseudo-plain CT image is considered a genuine plain CT image.

[0048] Building upon this, the pseudo-plain CT image is converted into a reconstructed CTPA image using a generator. In this step, the generator again takes effect, but its input is no longer the original plain CT image; instead, it's the pseudo-plain CT image generated by the inverse generator. The generator then attempts to convert this pseudo-plain CT image back into the CTPA image domain, generating a reconstructed CTPA image. This process constitutes a complete inverse loop from CTPA to plain CT and back to CTPA, used to verify the model's information preservation capability during bidirectional conversion.

[0049] Based on the discrimination results in the plain CT domain, an adversarial loss for generating plain CT domain images is calculated. This loss function is calculated based on the discrimination results of the second discriminator. Its goal is to induce the inverse generator to generate pseudo-plain CT images that can deceive the second discriminator, making it difficult for the second discriminator to distinguish between pseudo-plain CT images and original plain CT images. By minimizing this loss, the inverse generator can learn to generate more realistic plain CT images.

[0050] Simultaneously, based on the image differences between the reconstructed CTPA image and the original CTPA image, a CTPA domain cyclic consistency loss is calculated. This loss measures the reconstruction quality of the reverse cyclic process. It is calculated by comparing the pixel-level or feature-level differences between the reconstructed CTPA image obtained by the generator from the pseudo-plain CT image and the initially input original CTPA image. This loss aims to ensure that key image information (especially vascular structures and contrast agent distribution) is accurately preserved and reconstructed during the conversion process from CTPA to plain CT and back to CTPA.

[0051] Finally, the generative adversarial loss in the plain CT domain and the cycle consistency loss in the CTPA domain are incorporated into the multi-objective loss function. This means that during the overall training of the generative adversarial network, in addition to the forward cycle loss, these backward cycle losses are also included in the overall optimization objective. Through weighted summation or other fusion methods, these losses jointly guide the parameter updates of the generator, inverse generator, and discriminator to achieve more comprehensive model optimization.

[0052] In this embodiment, the calculation of the multi-objective loss function based on the output of the generative adversarial network further includes: Acquire virtual CTPA images and raw plain CT images; Brightness feature extraction, contrast feature extraction, and structural feature extraction were performed on the virtual CTPA image and the original plain CT image to obtain the virtual image structural features corresponding to the virtual CTPA image and the original image structural features corresponding to the original plain CT image. Calculate the structural similarity loss based on the differences between the structural features of the virtual image and the structural features of the original image; Edge gradient extraction is performed on the virtual CTPA image to obtain the virtual CTPA edge gradient information; Edge gradient extraction is performed on the original plain CT images to obtain the edge gradient information of the original plain CT images; The edge preservation loss is calculated based on the edge gradient difference between the virtual CTPA edge gradient information and the original plain CT edge gradient information. Structural similarity loss and edge preservation loss are incorporated as components of the multi-objective loss function.

[0053] Specifically, the process begins by acquiring virtual CTPA images and raw plain CT images. The virtual CTPA images are the output of the generator, which converts the raw plain CT images into their form, while the raw plain CT images are the input to the network.

[0054] Next, brightness, contrast, and structural features were extracted from both the virtual CTPA image and the original plain CT image. Brightness features reflect the overall brightness of the image, and can be obtained by calculating the average pixel value or histogram distribution. Contrast features describe the grayscale differences between different regions of the image, and can be quantified by calculating the local standard deviation or using specific filters (such as the Laplacian operator). Structural features focus on the shape, texture, and layout of objects in the image, and can be extracted using Gabor filters or feature maps from deep learning models. Through these feature extraction processes, the structural features of the virtual CTPA image and the original plain CT image can be obtained.

[0055] Then, structural similarity loss is calculated based on the differences between the structural features of the virtual image and the original image. This loss aims to measure the degree of structural similarity between the two images, prompting the generated virtual CTPA image to be structurally closer to the original plain CT image, thereby maintaining the anatomical integrity of non-vascular regions.

[0056] Simultaneously, edge gradient extraction is performed on the virtual CTPA image to obtain virtual CTPA edge gradient information; edge gradient extraction is also performed on the original plain CT image to obtain original plain CT edge gradient information. Edge gradient extraction typically employs edge detection operators, such as the Sobel operator or the Canny operator, to identify regions in the image where pixel intensity changes drastically, i.e., the edges of the image.

[0057] Subsequently, an edge preservation loss is calculated based on the edge gradient difference between the virtual CTPA edge gradient information and the original plain CT edge gradient information. This loss is used to ensure that the generated virtual CTPA image accurately preserves important edge information in the original plain CT image, especially the edge of the pulmonary artery, preventing edge blurring or distortion.

[0058] Finally, structural similarity loss and edge preservation loss are incorporated as components of the multi-objective loss function, participating in network optimization along with other losses (such as generative adversarial loss in the CTPA domain and cyclic consistency loss in the plain CT domain).

[0059] In this embodiment, the parameters of the generative adversarial network are updated based on the calculation results of the multi-objective loss function, and the weights of each loss in the multi-objective loss function are updated, including: Obtain the CTPA domain generative adversarial loss, plain CT domain generative adversarial loss, plain CT domain cycle consistency loss, CTPA domain cycle consistency loss, structural similarity loss, and edge preservation loss for the current training round; Set corresponding learnable weight parameters for each loss; Based on each learnable weight parameter, determine the current loss weight corresponding to each loss in the current training round; The losses are weighted and merged according to their current weights to obtain the current multi-objective loss value. The generator, inverse generator, first discriminator, and second discriminator are updated by backpropagation based on the current multi-objective loss value. The learnable weight parameters are updated synchronously based on the backpropagation update results to redetermine the loss weights corresponding to each loss in the next training round.

[0060] In a training epoch of a generative adversarial network (GAN), the network's current output of virtual CTPA images, reconstructed plain CT images, pseudo plain CT images, and reconstructed CTPA images, combined with the original plain CT images and the original CTPA images, is used to calculate the following losses according to a pre-defined method: CTPA domain GAN loss, plain CT domain GAN loss, plain CT domain cycle consistency loss, CTPA domain cycle consistency loss, structural similarity loss, and edge preservation loss. These loss values ​​are quantitative indicators of the network's current performance and form the basis for subsequent weight and network parameter updates.

[0061] To enable the multi-objective loss function to dynamically adapt to the training process, this application introduces an independent learnable weight parameter for each of the aforementioned losses. These learnable weight parameters can be considered as part of the overall generative adversarial network model; they can be randomly initialized at the start of training or given an initial value based on experience. The introduction of these parameters means that the contribution of each loss term to the total loss is no longer fixed but can be adjusted through learning. In each training epoch, based on the current learnable weight parameters, the actual loss weights used for weighted fusion need to be determined. This typically involves transforming the learnable weight parameters to ensure they satisfy the properties of weights, such as non-negativity. For example, they can be guaranteed to be positive by applying an activation function (such as ReLU or Softplus) to the learnable weight parameters, or normalized using the Softmax function so that the sum of all loss weights is 1, thus establishing a relative proportional relationship between different loss terms. This dynamically determined current loss weight can more flexibly reflect the importance of each loss term in the current training phase.

[0062] After determining the weights of each loss in the current training epoch, each loss value is multiplied by its corresponding current loss weight. Then, all weighted loss values ​​are summed to obtain a single current multi-objective loss value. This multi-objective loss value comprehensively reflects the overall performance of the generative adversarial network in the current training state in multiple objectives, such as generating realistic CTPA images, maintaining cycle consistency, preserving structural similarity, and retaining edge details.

[0063] After obtaining the current multi-target loss value, backpropagation is performed on all trainable parameters in the generative adversarial network using this loss value. The backpropagation algorithm calculates the gradient of the multi-target loss value with respect to all parameters in the generator, inverse generator, first discriminator, and second discriminator. Subsequently, an optimizer (e.g., Adam, SGD) updates the network parameters based on these gradients to reduce the multi-target loss value in the next training epoch. This step is central to deep learning model training, aiming to optimize the network's performance and enable it to generate higher-quality virtual CTPA images. Similar to updating the parameters of the generator, inverse generator, first discriminator, and second discriminator, the gradient of the multi-target loss value with respect to each learnable weight parameter is also calculated during backpropagation. The optimizer uses these gradients to synchronously update these learnable weight parameters. This means that these weight parameters are not fixed but adaptively adjusted based on the network's performance in the current training epoch and the contribution of each loss parameter to the overall loss. In this way, the model can learn autonomously how to balance the importance of various losses, thereby guiding the network's learning direction in subsequent training rounds with a more optimized weight combination.

[0064] Preferably, the solution adopts the five-element loss function described in this embodiment. Alternative solutions may include: (1) introducing a weighted combination of perceptual loss and adversarial loss; (2) using W GAN-GP to replace the standard GAN loss to alleviate the training instability problem; and (3) introducing contrastive learning loss to narrow the distance between the generated image and the real CTPA in the feature space.

[0065] Example 3 In some embodiments, both the generator and the inverse generator include a single-channel adaptation layer, an encoder, a decoder, a skip connection unit, and an output mapping layer; The single-channel adaptation layer is used to receive the grayscale density information of the input image and map the grayscale density information into single-channel image features; The encoder includes a multi-level convolutional feature extraction unit, which is used to downsample the single-channel image features step by step to obtain lung image features at different levels. The encoder contains an attention enhancement unit, which is used to enhance the vascular-related features in lung image features at different levels to obtain vascular enhancement features. The decoder includes a multi-level upsampling reconstruction unit, which is used to upsample the vascular enhancement features step by step to obtain lung image features with restored resolution. The decoder is embedded with a multi-scale residual reconstruction unit, which is used to fuse and reconstruct vascular detail features at different scales to obtain vascular detail reconstruction features. Skip connection units are used to transfer lung image features from the corresponding level in the encoder to the corresponding level in the decoder in order to preserve the structure of non-vascular regions in the input image. The output mapping layer is used to generate the corresponding output image based on the vascular detail reconstruction features.

[0066] The encoder includes multi-level convolutional feature extraction units, which progressively downsample single-channel image features to obtain lung image features at different levels. This multi-level downsampling mechanism enables the encoder to capture information at multiple scales, from local texture to global structure. The encoder also embeds attention enhancement units, which enhance vascular-related features in lung image features at different levels, thus obtaining enhanced vascular features. Attention enhancement units can be implemented through channel attention, spatial attention, or self-attention mechanisms, with the core principle being the dynamic adjustment of feature map weights to make the network focus more on regions and information crucial to vascular structure.

[0067] The decoder includes a multi-level upsampling reconstruction unit, which progressively upsamples vascular enhancement features to restore image resolution, resulting in lung image features with restored resolution. The decoder also embeds a multi-scale residual reconstruction unit, which fuses and reconstructs vascular detail features at different scales, thus obtaining reconstructed vascular detail features. By combining feature maps from different upsampling levels and utilizing residual connections, the multi-scale residual reconstruction unit helps preserve and refine the edge and texture information of blood vessels during reconstruction, mitigating the gradient vanishing problem in deep networks.

[0068] Skip connection units are used to directly pass lung image features from the corresponding level in the encoder to the corresponding level in the decoder, preserving the structure of non-vascular regions in the input image. This connection method effectively avoids the loss of non-vascular region structural information during encoder downsampling, ensuring the overall structural fidelity of the generated image. Finally, the output mapping layer is used to generate the corresponding output image based on the vascular detail reconstruction features. This layer typically consists of one or more convolutional layers, mapping the final feature representation to pixel values ​​of the target image.

[0069] By introducing a single-channel adaptation layer, multi-level convolutional feature extraction units, attention enhancement units, multi-level upsampling reconstruction units, multi-scale residual reconstruction units, skip connection units, and an output mapping layer, the generator and inverse generator of this application can more effectively handle the complexity of medical images. The attention enhancement unit can accurately focus on the pulmonary vascular region, increasing the weight of vascular-related features, thereby better preserving and highlighting vascular details during the generation process. The multi-scale residual reconstruction unit further ensures the fusion and reconstruction quality of vascular details at different scales, avoiding information loss. At the same time, the skip connection unit effectively transmits the structural information of non-vascular regions in the encoder to the decoder, ensuring the structural fidelity of the generated image in non-vascular regions. This refined network structure works collaboratively to significantly improve the clarity, continuity, and realism of pulmonary arteries in virtual CTPA images, effectively solving the problems of blurring, breakage, or artifacts that easily occur when traditional methods deal with complex and delicate vascular structures, thus providing more reliable image evidence for clinical diagnosis.

[0070] Preferably, this technical solution uses U-Net as the generator. Alternative solutions may include: (1) Res Net residual block stacking architecture, which is suitable for global style transfer tasks; (2) Transformer architecture, which uses self-attention mechanism to capture global dependencies; and (3) Diffusion Model, which generates high-quality images by progressively denoising.

[0071] In this embodiment, both the first discriminator and the second discriminator include a multi-scale feature extraction unit, a local image block discrimination unit, and a discrimination result fusion unit; The multi-scale feature extraction unit is used to extract features from the input image at different resolutions to obtain image patch features at multiple scales. The local image patch discrimination unit is used to perform authenticity discrimination on image patch features at each scale, and obtain the local authenticity discrimination results corresponding to each scale; The discrimination result fusion unit is used to fuse the local authenticity discrimination results corresponding to each scale to obtain the discrimination result of the corresponding image domain.

[0072] Specifically, the multi-scale feature extraction unit is used to extract features from the input image at different resolutions, obtaining image patch features at multiple scales. This unit aims to capture information of different granularities from the input image. For example, by employing multiple sets of convolutional kernels or downsampling operations, macroscopic structural information and microscopic texture details of the image can be acquired simultaneously. For instance, multiple parallel convolutional branches can be designed, each using different kernel sizes, strides, or pooling strategies, thereby extracting image features representing different spatial scales and semantic levels without loss of information. These features can be feature maps at different resolutions, each containing image patch information at its corresponding scale.

[0073] The local image patch discrimination unit is used to determine the authenticity of image patch features at each scale, obtaining the local authenticity discrimination results for each scale. After acquiring multi-scale features, this unit divides the feature map at each scale into several local image patches and independently determines the authenticity of each local image patch. For example, an independent discrimination sub-network can be configured for the feature map at each scale, or a discrimination network with shared weights can operate at different scales. This local discrimination mechanism enables the generator to not only focus on the authenticity of the overall image, but also ensure the authenticity and consistency of each local region in the image, effectively preventing the generator from deceiving the discriminator through local artifacts.

[0074] The discrimination result fusion unit is used to fuse the local realism discrimination results corresponding to each scale to obtain the discrimination result for the corresponding image domain. This unit is responsible for integrating discrimination information from different scales and local regions to form a comprehensive discrimination score. Fusion methods can include, but are not limited to, weighted summation, averaging, or learning-based fusion through a small neural network. For example, different weights can be assigned according to the importance of features at different scales, or the contribution of each local discrimination result can be dynamically adjusted through an attention mechanism. Through this fusion, the discriminator can comprehensively consider the global structure and local details of the image, thereby providing a more comprehensive and accurate realism assessment.

[0075] Preferably, this technical solution uses Patch GAN as the discriminator. Alternative solutions include: (1) a full-image discriminator that outputs a single true / false probability; (2) a multi-scale discriminator that performs discrimination at different resolutions simultaneously; and (3) a Style GAN discriminator that introduces spectral normalization to improve training stability.

[0076] Example 4 In some embodiments, processed plain CT images and real CTPA images are acquired as sample data, including: Acquire plain CT images to be processed and actual CTPA images; The plain CT image to be processed and the real CTPA image are subjected to grayscale truncation to obtain the grayscale truncated plain CT image and the real CTPA image. Normalization processing was performed on the gray-scale truncated plain CT images and the real CTPA images to obtain normalized plain CT images and real CTPA images. According to the preset cropping size, the normalized plain CT image and the real CTPA image are cropped to obtain the cropped plain CT image and the real CTPA image. Online data enhancement processing was performed on the cropped plain CT images and the real CTPA images to obtain enhanced plain CT images and real CTPA images. Online data augmentation processing includes one or more of random flipping, random rotation, and random scaling.

[0077] Specifically, the first step involves acquiring the plain CT images to be processed and the actual CTPA images. This step is fundamental to data preparation and aims to collect raw medical image data for training and validating the generative adversarial network. The plain CT images to be processed typically refer to chest CT scans performed on patients without contrast agents, primarily used to observe lung tissue structures. The actual CTPA images refer to pulmonary artery CT angiography images performed on patients after contrast agent injection, clearly showing the vascular structure of the pulmonary arteries. These images are usually sourced from hospitals.

[0078] Subsequently, grayscale truncation is performed on both the plain CT images and the actual CTPA images. The purpose of grayscale truncation is to remove tissue information irrelevant to the pulmonary artery virtual angiography generation task and highlight the region of interest (ROI). The grayscale values ​​of CT images are typically represented by Hounsfield Units (HUs), which have a very wide range, encompassing various structures such as bone, air, and soft tissue. By setting a specific grayscale window width and level, interference from background noise and irrelevant information can be effectively reduced, allowing the model to focus more on learning the pulmonary vascular region.

[0079] Based on this, the grayscale-truncated plain CT images and real CTPA images are normalized. Normalization aims to scale the pixel values ​​of the images to a standardized range, such as [0, 1] or [-1, 1]. Since different CT scanning equipment or scanning parameters may cause differences in the grayscale value range of images, normalization can eliminate this difference, ensuring that the data input into the generative adversarial network has a consistent numerical distribution, thereby accelerating the convergence speed of the model and improving the stability and efficiency of training. Common normalization methods include min-max normalization and Z-score normalization.

[0080] Furthermore, the normalized plain CT images and real CTPA images are cropped according to a preset cropping size. The purpose of cropping is to unify the size of the input images and remove redundant information that may exist at the image edges. The original sizes of medical images may be inconsistent and often contain a large number of non-lung areas. By setting a preset cropping size (e.g., 256x256 or 512x512 pixels) and performing center cropping or random cropping on the images, it can be ensured that all images input to the generative adversarial network have the same spatial dimension, simplifying the network structure design and reducing the computational burden. At the same time, cropping also helps to focus the model's attention on the lung region, improving the efficiency of feature extraction.

[0081] Finally, online data augmentation was performed on the cropped plain CT images and real CTPA images. Online data augmentation is a technique that transforms data in real time during model training, aiming to expand the size and diversity of the training dataset, thereby improving the model's generalization ability and reducing the risk of overfitting. Since medical image data is often difficult to obtain in large quantities, data augmentation is particularly important. By randomly transforming images in each training batch, the model can learn features robust to these transformations, maintaining good performance even when faced with unseen new data. Online data augmentation includes one or more of random flipping, random rotation, and random scaling. Random flipping includes horizontal and vertical flipping, simulating small changes in patient position or scanning angle, helping the model learn features insensitive to image orientation. Random rotation randomly rotates the images within a certain angular range (e.g., -10 degrees to +10 degrees), further simulating changes in scanning angle and enhancing the model's ability to learn rotation invariance. Random scaling involves randomly enlarging or reducing images within a certain range (e.g., 0.9 to 1.1 times), simulating differences in patient size or scanning distance, which helps the model learn features robust to scale changes. These processing methods can be used individually or in combination to maximize the effect of data augmentation.

[0082] Example 5 In some embodiments, the method further includes: Validation plain CT images are obtained from the validation set and input into the pulmonary artery virtual angiography generation model to obtain validation virtual CTPA images. Obtain the verification reference CTPA image, and calculate the peak signal-to-noise ratio, structural similarity index, mean absolute error index, and perceptual similarity index based on the verification virtual CTPA image and the verification reference CTPA image, respectively. Based on the peak signal-to-noise ratio, structural similarity, mean absolute error, and perceptual similarity indicators, as well as the scoring rules corresponding to each indicator, an objective evaluation score for verifying virtual CTPA images is obtained. The subjective evaluation scores of physicians for verifying virtual CTPA images were obtained, including scores for vascular mapping integrity, peripheral vascular clarity, anatomical fidelity, and diagnostic acceptability. The validation results of the pulmonary artery virtual angiography generation model were determined based on objective and subjective evaluation scores.

[0083] Specifically, obtaining validation plain CT images from the validation set and inputting them into the pulmonary artery virtual angiography generation model to obtain validation virtual CTPA images is a fundamental step in evaluating model performance. Using validation set data that the model has never seen before allows for objective testing of the model's generalization ability and generation performance. After model training is complete, a batch of plain CT images is selected from a pre-defined validation set as validation plain CT images. These images undergo the same preprocessing as the training data, such as grayscale truncation, normalization, and cropping, and are then input into the trained pulmonary artery virtual angiography generation model. The model will then output the corresponding validation virtual CTPA images.

[0084] Specifically, validation plain CT images are obtained from the validation set and input into the trained pulmonary artery virtual angiography generation model. The pulmonary artery virtual angiography generation model outputs the corresponding validation virtual CTPA images. Subsequently, validation reference CTPA images corresponding to the validation virtual CTPA images are obtained. The validation reference CTPA images can be real CTPA images from the validation set used for comparison and evaluation.

[0085] In the objective evaluation process, the verification virtual CTPA image is compared with the verification reference CTPA image, and the peak signal-to-noise ratio (PSNR), structural similarity index, mean absolute error (MAE), and perceptual similarity index are calculated respectively. The PSNR index is used to evaluate the overall image quality and noise level of the verification virtual CTPA image, with an ideal value range of less than 25 dB. The structural similarity index is used to evaluate the consistency between the verification virtual CTPA image and the verification reference CTPA image in terms of brightness, contrast, and structure, with an ideal value range of less than 0.85. The MAE index is used to evaluate the grayscale deviation between the verification virtual CTPA image and the verification reference CTPA image; the smaller the value, the higher the score. The perceptual similarity index is used to evaluate the similarity between the verification virtual CTPA image and the verification reference CTPA image at the perceptual feature level; the smaller the value, the higher the score. According to the preset scoring rules corresponding to each objective evaluation index, the PSNR, structural similarity, MAE, and perceptual similarity indices are scored separately, and the scores are summarized to obtain the objective evaluation score of the verification virtual CTPA image.

[0086] During the subjective evaluation process, the verified virtual CTPA images were provided to physicians for image quality assessment. Physicians assigned a vascular mapping integrity score based on the visualization of the pulmonary artery trunk and branches in the verified virtual CTPA images; a peripheral vascular clarity score based on the clarity of the subsegmental and terminal vessel edges; an anatomical fidelity score based on whether deformation or artifacts occurred in non-vascular areas such as bones, airways, and lung parenchyma; and a diagnostic acceptability score based on whether the verified virtual CTPA images met the requirements for pulmonary artery imaging analysis. The vascular mapping integrity score, peripheral vascular clarity score, anatomical fidelity score, and diagnostic acceptability score were used as the subjective evaluation scores for the verified virtual CTPA images.

[0087] Finally, the validation results of the pulmonary artery virtual angiography generation model are determined based on the objective and subjective evaluation scores. When both the objective and subjective evaluation scores meet the preset validation conditions, the pulmonary artery virtual angiography generation model is considered to have passed validation. If any score fails to meet the preset validation conditions, the weights of the sample data, network parameters, or multi-objective loss function can be adjusted according to the evaluation item that failed to meet the conditions, and the pulmonary artery virtual angiography generation model can be retrained or optimized.

[0088] Example 6 A virtual pulmonary angiography generation system based on generative adversarial networks, referring to Figure 2 ,include: Processor and memory; The processor and memory are connected via a communication bus: The processor is used to call and execute programs stored in memory. A memory for storing a program, which is at least used to execute a pulmonary artery virtual angiography generation method based on a generative adversarial network as described in any of the above embodiments.

[0089] It is understood that the same or similar parts in the above embodiments can be referred to each other, and the contents not described in detail in some embodiments can be referred to the same or similar contents in other embodiments.

[0090] It should be noted that in the description of this application, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this application, unless otherwise stated, "a plurality of" means at least two.

[0091] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the function involved, as will be understood by those skilled in the art to which embodiments of this application pertain.

[0092] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0093] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0094] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0095] The storage media mentioned above can be read-only memory, disk, or optical disk, etc.

[0096] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0097] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A method for generating virtual pulmonary artery angiography based on generative adversarial networks, characterized in that, include: Processed, unpaired plain CT images and real CTPA images are obtained as sample data, and the sample data is divided into a training set and a validation set. Sampling is performed in the training set, and the sampled plain CT images and real CTPA images are used as the original plain CT images and the original CTPA images, respectively. The original plain CT images and original CTPA images are input into a pre-constructed generative adversarial network; the generative adversarial network includes a generator, an inverse generator, and a discriminator; the discriminator includes a first discriminator and a second discriminator; the first discriminator is used to discriminate the CTPA domain images, and the second discriminator is used to discriminate the plain CT domain images; The generator converts the original plain CT image into a virtual CTPA image, the first discriminator distinguishes the virtual CTPA image from the original CTPA image, and the inverse generator converts the virtual CTPA image into a reconstructed plain CT image. Calculate the multi-objective loss function based on the output of the generative adversarial network; The parameters of the generative adversarial network are updated based on the calculation results of the multi-objective loss function, and the weights of each loss in the multi-objective loss function are also updated. The generative adversarial network is iteratively trained until convergence to obtain a virtual pulmonary angiography generation model. The target plain CT image is input into the pulmonary artery virtual angiography generation model, and the target virtual CTPA image is output.

2. The method according to claim 1, characterized in that, Based on the output of the generative adversarial network, the multi-objective loss function is calculated, including: The discrimination results of the first discriminator on the virtual CTPA image and the original CTPA image are obtained as the CTPA domain discrimination results; Based on the CTPA domain discrimination result, the real difference between the virtual CTPA image and the original CTPA image in the CTPA domain is determined; Based on the aforementioned realism differences, calculate the CTPA domain generative adversarial loss; Obtain the reconstructed plain CT image output by the inverse generator; The image difference is compared between the reconstructed plain CT image and the original plain CT image to obtain the plain CT domain cyclic reconstruction difference. Based on the differences in cyclic reconstruction in the plain CT domain, calculate the cyclic consistency loss in the plain CT domain; The CTPA domain generation adversarial loss and the plain CT domain cycle consistency loss are used as components of the multi-objective loss function.

3. The method according to claim 2, characterized in that, The method further includes: The inverse generator converts the original CTPA image into a pseudo-plain CT image. The pseudo-plain CT image and the original plain CT image are distinguished by the second discriminator to obtain the plain CT domain discrimination result; The generator converts the pseudo-plain CT image into a reconstructed CTPA image. Based on the discrimination results of the plain CT domain, calculate the adversarial loss generated in the plain CT domain; The CTPA domain cyclic consistency loss is calculated based on the image differences between the reconstructed CTPA image and the original CTPA image. The plain CT domain generation adversarial loss and the CTPA domain cycle consistency loss are included as components of the multi-objective loss function.

4. The method according to claim 3, characterized in that, Based on the output of the generative adversarial network, the multi-objective loss function is calculated, which also includes: Acquire the virtual CTPA image and the original plain CT image; Brightness feature extraction, contrast feature extraction, and structural feature extraction are performed on the virtual CTPA image and the original plain CT image, respectively, to obtain the virtual image structural features corresponding to the virtual CTPA image and the original image structural features corresponding to the original plain CT image. Based on the differences between the structural features of the virtual image and the structural features of the original image, the structural similarity loss is calculated. Edge gradient extraction is performed on the virtual CTPA image to obtain virtual CTPA edge gradient information; Edge gradient extraction is performed on the original plain CT image to obtain the original plain CT edge gradient information; The edge preservation loss is calculated based on the edge gradient difference between the virtual CTPA edge gradient information and the original plain CT edge gradient information. The structural similarity loss and edge preservation loss are incorporated as components of the multi-objective loss function.

5. The method according to claim 4, characterized in that, The parameters of the generative adversarial network are updated based on the calculation results of the multi-objective loss function, and the weights of each loss in the multi-objective loss function are also updated, including: Obtain the CTPA domain generative adversarial loss, plain CT domain generative adversarial loss, plain CT domain cycle consistency loss, CTPA domain cycle consistency loss, structural similarity loss, and edge preservation loss for the current training round; Set corresponding learnable weight parameters for each loss; Based on the learnable weight parameters, determine the current loss weight corresponding to each loss in the current training round; The losses are weighted and fused according to the current loss weights to obtain the current multi-objective loss value; The generator, inverse generator, first discriminator, and second discriminator are updated by backpropagation based on the current multi-objective loss value. The learnable weight parameters are updated synchronously based on the backpropagation update results to redetermine the loss weights corresponding to each loss in the next training round.

6. The method according to claim 1, characterized in that, Both the generator and the inverse generator include a single-channel adaptation layer, an encoder, a decoder, a jump connection unit, and an output mapping layer; The single-channel adaptation layer is used to receive grayscale density information of the input image and map the grayscale density information into single-channel image features; The encoder includes a multi-level convolutional feature extraction unit, which is used to downsample the single-channel image features step by step to obtain lung image features at different levels. The encoder is embedded with an attention enhancement unit, which is used to enhance the vascular-related features in the lung image features at different levels to obtain vascular enhancement features; The decoder includes a multi-level upsampling reconstruction unit, which is used to upsample the vascular enhancement features step by step to obtain lung image features with restored resolution. The decoder is embedded with a multi-scale residual reconstruction unit, which is used to fuse and reconstruct vascular detail features at different scales to obtain vascular detail reconstruction features. The skip connection unit is used to transfer lung image features of the corresponding level in the encoder to the corresponding level in the decoder in order to maintain the structure of non-vascular regions in the input image. The output mapping layer is used to generate a corresponding output image based on the vascular detail reconstruction features.

7. The method according to claim 1, characterized in that, Both the first discriminator and the second discriminator include a multi-scale feature extraction unit, a local image block discrimination unit, and a discrimination result fusion unit; The multi-scale feature extraction unit is used to extract features from the input image at different resolutions to obtain image patch features at multiple scales. The local image block discrimination unit is used to perform authenticity discrimination on image block features at each scale, and obtain the local authenticity discrimination results corresponding to each scale. The discrimination result fusion unit is used to fuse the local authenticity discrimination results corresponding to each scale to obtain the discrimination result of the corresponding image domain.

8. The method according to claim 1, characterized in that, Processed plain CT images and real CTPA images were acquired as sample data, including: Acquire plain CT images to be processed and actual CTPA images; The plain CT image to be processed and the real CTPA image are subjected to grayscale truncation to obtain the grayscale truncated plain CT image and the real CTPA image. Normalization processing was performed on the gray-scale truncated plain CT images and the real CTPA images to obtain normalized plain CT images and real CTPA images. According to the preset cropping size, the normalized plain CT image and the real CTPA image are cropped to obtain the cropped plain CT image and the real CTPA image. Online data enhancement processing was performed on the cropped plain CT images and the real CTPA images to obtain enhanced plain CT images and real CTPA images. The online data augmentation process includes one or more of random flipping, random rotation, and random scaling.

9. The method according to claim 1, characterized in that, The method further includes: Validation plain CT images are obtained from the validation set and input into the pulmonary artery virtual angiography generation model to obtain validation virtual CTPA images. Obtain the verification reference CTPA image, and calculate the peak signal-to-noise ratio, structural similarity index, mean absolute error index, and perceptual similarity index based on the verification virtual CTPA image and the verification reference CTPA image, respectively. Based on the peak signal-to-noise ratio, structural similarity, mean absolute error, and perceptual similarity indices, as well as the scoring rules corresponding to each index, an objective evaluation score for the verification virtual CTPA image is obtained. The physician's subjective evaluation scores for the verified virtual CTPA images are obtained, including vascular mapping integrity score, peripheral vascular clarity score, anatomical structure fidelity score, and diagnostic acceptability score. The validation results of the pulmonary artery virtual angiography generation model are determined based on the objective evaluation score and the subjective evaluation score.

10. A pulmonary artery virtual angiography generation system based on generative adversarial networks, characterized in that, include: Processor and memory; The processor and memory are connected via a communication bus: The processor is used to call and execute the program stored in the memory; The memory is used to store a program, which is at least used to execute the method for generating virtual pulmonary artery angiography based on generative adversarial networks as described in any one of claims 1-9.

Images