CECT automatic synthesis method and device

Abstract

The invention provides a CECT automatic synthesis method and device, and the method comprises the steps: constructing a multi-path cyclic consistent confrontation vision Transform architecture model SPM-CyVIT which can achieve the self-supervision and pre-training, and enabling a plain-scan CT image to be converted into a high-quality virtual CECT image through the model; according to the SPM-CyVIT, a convolutional neural network (CNN) and a visual Transform (VIT) are integrated through a two-way hybrid coding strategy, unique advantages of the CNN and the VIT are fused in the coding stage, and the purpose of realizing contrast agent uptake areas such as blood vessels, tumors and blood flow rich tissues is achieved while key features such as anatomical structures and focus forms, positions and boundaries in original CT slices are kept unchanged. Bidirectional conversion of native phase (NCCT) and arterial phase (CECT) effects; and meanwhile, a high-quality virtual CECT is synthesized from a common NCCT, so that the dependence on ICAs is eliminated.

Description

Technical Field

[0001] This invention relates to machine learning technology, specifically to the field of machine learning supporting neural network models, and more specifically to the technology of CECT automatic synthesis method. Background Technology

[0002] Contrast-enhanced computed tomography (CECT) provides rich information on vascular anatomy and pathology, playing a crucial role in the diagnosis, staging, treatment response assessment, and follow-up care of related diseases. CECT typically utilizes iodine contrast agents (ICAs) to enhance the visualization of cardiovascular structures. For complex lesions such as aortic aneurysms and aortic dissections, the high-resolution images from CECT can accurately display the extent and nature of lesion features, thus providing a reliable basis for developing personalized treatment plans. However, the use of iodine contrast agents is accompanied by a series of adverse reactions, including potentially life-threatening events such as anaphylactic shock, contrast-induced nephropathy, and acute kidney injury, especially in patients with renal insufficiency or iodine allergy, where this risk is significantly increased. Furthermore, the administration of contrast agents introduces considerable procedural complexity, requiring meticulous patient monitoring throughout the process, preparation for additional medical interventions, and consideration of factors such as scan misalignment caused by patient movement during image acquisition at different phases. All of these factors significantly increase the time consumed in the diagnostic process.

[0003] Plain CT, also known as non-contrast enhanced CT (NCCT), is a widely used imaging technique for initial clinical screening and assessment. Its advantages, such as ease of operation, speed, and low cost, make it suitable for various emergency situations including trauma and chest pain. However, NCCT does not require the injection of iodine contrast agents, and parameters such as slice thickness and radiation dose differ somewhat from CECT. This limits its ability to highlight vascular structures and detect minute lesions, resulting in blurred boundaries of various shapes and easily leading to missed or misdiagnosed cases, thus affecting subsequent treatment decisions. In recent years, deep generative models, represented by GANs, have achieved breakthroughs in many medical imaging fields, including MRI enhancement and multi-source ultrasound standardization. Therefore, developing a fully ICA-free virtual CECT automatic synthesis method based on a large amount of unpaired NCCT-CECT data and applying it to clinical auxiliary diagnosis is very promising. Summary of the Invention

[0004] In view of the shortcomings of existing technologies, this invention proposes an automatic CECT synthesis method and apparatus, which constructs a self-supervised pretrained multi-path cycle-consistent generative adversarial vision transformer (SPM-CyVIT) architecture model (hereinafter referred to as SPM-CyVIT). SPM-CyVIT integrates convolutional neural networks (CNN) and visual transformers (VIT) through a dual-path hybrid encoding strategy, fully utilizing their complementary advantages at different scales. It combines the unique advantages of CNN and VIT in the encoding stage, aiming to preserve key features such as anatomical structures, lesion morphology, location, and boundaries in the original CT slices while achieving bidirectional conversion between native phase (NCCT) and arterial phase (CECT) effects in contrast agent uptake areas such as blood vessels, tumors, and blood-rich tissues. The aim is to synthesize high-quality virtual CECTs from NCCT, thereby eliminating dependence on ICAs.

[0005] In a first aspect, the present invention proposes an automated CECT synthesis method, the method comprising the following steps:

[0006] S1. Acquire plain CT images;

[0007] S2. Input the plain CT image into SPM-CyViT and convert it to obtain a CECT image.

[0008] Furthermore, the SPM-CyViT network model establishes a conversion mapping relationship between NCCT images and CECT images.

[0009] Specifically, the SPM-CyVIT employs a dual-path hybrid coding design, including a generator G. NCCT→CECT and pre-trained ViT encoder (ε VIT Generator G NCCT→CECT Including local encoders (ε CNN ) and decoder; wherein, step S2 specifically includes:

[0010] VIT global encoder (ε VIT It extracts global semantic features to obtain the global feature sequence U of the plain CT image; through the encoder (ε) CNNThe convolutional layer downsamples the plain CT image to obtain a CNN local detail feature map, and the global feature sequence U and the CNN local detail feature map are fused by deep feature fusion (FF) to obtain a hybrid coded feature map T″. The hybrid coded feature map T″ is fed into the decoder, and after passing through multiple convolutional upsampling layers in the decoder, a CECT image is obtained.

[0011] Specifically, the SPM-CyVIT includes a generator G. NCCT→CECT Generator F CECT→NCCT Convolutional Decoder Three tag-specific discriminators D CECT D NCCT D MonoE Generator G NCCT→CECT The structure specifically includes a convolutional encoder ε CNN Global encoder ε ViT Dynamic Aggregation (DA), Feature Fusion (FF), and Convolutional Decoder Generator F CECT→NCCT The structure specifically includes a convolutional encoder ε CNN Global encoder ε ViT Dynamic Aggregation (DA), Feature Fusion (FF), and Convolutional Decoder

[0012] Specifically, the convolutional encoder ε CNN Responsible for extracting local detail feature maps T from CT images; global encoder ε ViT The module DA is responsible for modeling the global feature sequence U of the high-resolution input CT image; the dynamic aggregation module DA is responsible for enhancing ε. CNN The global encoded feature map T′ is generated within the encoded feature map; the feature fusion module FF is responsible for deeply fusing the two parallel encoded results T′ and U to obtain a two-dimensional feature map T″ with global context information and local details; the convolutional decoder... It is responsible for restoring the two-dimensional feature map T″ to a CECT image with the same shape and resolution as the image before encoding.

[0013] Further, step S2 specifically includes: the plain CT image X input generator C NCCT→CECT In the generator, X needs to be passed in parallel through the convolutional encoder ε. CNN and global encoder ε ViTEncoding is performed to obtain local feature maps T and global feature sequences U; T is then processed by a dynamic aggregation module (DA) to obtain a feature map T′ that incorporates global context information; T and U are input into a feature fusion module (FF) to calculate a two-dimensional feature map T″; T″ is then simultaneously fed into... and Two convolutional decoders synthesize virtual CECT images. and virtual MonoE images

[0014] Secondly, the present invention proposes an automated CECT synthesis device, comprising:

[0015] The input module is used to acquire plain CT images;

[0016] The synthesis module inputs the plain CT image into SPM-CyViT and converts it into a CECT image.

[0017] Thirdly, the present invention provides an electronic device, characterized in that it includes a memory and one or more programs, wherein one or more programs are stored in the memory and configured to be executed by one or more processors, the one or more programs including the method described above.

[0018] Fourthly, the present invention provides a computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and the computer program, when executed by a processor, implements the steps of the above-described method.

[0019] In summary, compared with the prior art, the CECT automatic synthesis method and apparatus based on the present invention achieves the following beneficial technical effects:

[0020] This invention proposes an automatic CECT synthesis method and apparatus. It constructs a self-supervised pre-trained multi-path Cyclic Consistent Adversarial Visual Transformer architecture model, SPM-CyVIT. SPM-CyVIT integrates Convolutional Neural Networks (CNN) and Visual Transformers (VIT) through a dual-path hybrid encoding strategy, fully utilizing their complementary advantages at different scales. In the encoding stage, it combines the unique advantages of CNN and VIT, aiming to preserve key features such as anatomical structures, lesion morphology, location, and boundaries in the original CT slices while achieving bidirectional conversion between native phase (NCCT) and arterial phase (CECT) effects in contrast agent uptake areas such as blood vessels, tumors, and blood-rich tissues. Simultaneously, it synthesizes high-quality virtual CECTs from ordinary NCCT, thereby eliminating dependence on ICAs (Integrated Circuits Assembled ...

[0021] Meanwhile, to accommodate the requirement of using MonoE as an additional label, this invention divides the generator into an encoder and a decoder, and combines the decoder with the discriminator to build a separate decoding-discrimination branch for each label. This improvement enables stable end-to-end learning of SPM-CyVIT across multiple domains, effectively avoiding the mode collapse problem that occurs during training. By introducing 40keV monolevel images (MonoE) as auxiliary labels and designing a multi-branch decoding-discrimination path, feature learning in contrast agent-sensitive regions is optimized, significantly improving the visualization of blood vessel edges and fine structures. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a schematic diagram of the framework of the method provided in the embodiments of the present invention.

[0024] Figure 2 This is a schematic diagram illustrating the visualization of the predicted plane and actual position provided in an embodiment of the present invention.

[0025] Figure 3 This is a schematic diagram of the overall structure of the MAE of the pre-trained VIT encoder provided in an embodiment of the present invention.

[0026] Figure 4 This is a schematic diagram of the dynamic aggregation module DA module provided in an embodiment of the present invention.

[0027] Figure 5 This is a schematic diagram comparing the synthetic CECT effects of different models provided in the embodiments of the present invention. Detailed Implementation

[0028] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, unless otherwise specified, the various embodiments of the following methods, apparatus, and systems, and the features in those embodiments, can be combined with each other.

[0029] While data-driven NCCT-CECT conversion methods can effectively avoid the side effects of ICAs and simplify clinical procedures, they also present several challenges. First, there's the spatial registration issue between NCCT and CECT. Influenced by the contrast agent injection interval, rigid body movements, intestinal peristalsis, and random respiratory artifacts within the same patient can lead to misalignment in the imaging results. These subtle differences are easily amplified by pixel-by-pixel supervised secondary nonlinear calculations, causing the generated virtual CECT to deviate from the actual target domain. Furthermore, clinically acquired CECT data typically refers to the superimposed mixed-level imaging results. Compared to the single-level enhanced images (MonoE) provided by dual-energy spectral CT, this results in some loss of detail, especially in structures like the cardiovascular system that are highly sensitive to iodine contrast agents. Additionally, another significant issue is that general convolutional neural networks (CNNs) are inefficient at capturing global anatomical information. They are typically only suitable for fitting high-quality medical image data from a single center. When faced with completely unpaired, multi-source domain, high dynamic range big data, their generalization and scalability are relatively poor.

[0030] In view of the shortcomings of existing technologies, this invention proposes an automatic CECT synthesis method and apparatus, which constructs a self-supervised pretrained multi-path cycle-consistent generative adversarial vision transformer (SPM-CyVIT) architecture model (hereinafter referred to as SPM-CyVIT). SPM-CyVIT integrates convolutional neural networks (CNN) and visual transformers (VIT) through a dual-path hybrid encoding strategy, fully utilizing their complementary advantages at different scales. It combines the unique advantages of CNN and VIT in the encoding stage, aiming to preserve key features such as anatomical structures, lesion morphology, location, and boundaries in the original CT slices while achieving bidirectional conversion between native phase (NCCT) and arterial phase (CECT) effects in contrast agent uptake areas such as blood vessels, tumors, and blood-rich tissues. The aim is to synthesize high-quality virtual CECTs from NCCT, thereby eliminating dependence on ICAs.

[0031] Furthermore, to accommodate the requirement of MonoE as an additional label, this invention divides the generator into an encoder and a decoder, and combines the decoder with the discriminator to build a separate decoding-discrimination branch for each label. This improvement enables stable end-to-end learning of SPM-CyVIT across multiple domains, effectively avoiding the mode collapse problem that occurs during training. By introducing 40keV monolevel images (MonoE) as auxiliary labels and designing a multi-branch decoding-discrimination path, feature learning in contrast agent-sensitive regions is optimized, significantly improving the visualization of blood vessel edges and fine structures.

[0032] This invention selects 2200 cases from FLARE2023 as a pre-training dataset, which comes from multiple medical centers around the world, covers all stages of CT, and includes common CT manufacturers, including GE, Philips, Siemens and Toshiba, allowing for use for research purposes and sharing.

[0033] Most of the paired data required for this invention was provided by the Department of Radiology, West China Hospital, Sichuan University. Data was acquired using Siemens dual-energy spectral CT (Somatom Force Dual Source), with a scan range from the skull base to the upper edge of the pelvis. This private dataset contains 56 cases, each with both native phase (NCCT) and arterial phase (CECT) images. Eighteen cases have CT images with enhanced 40keV MonoE labels. In addition, we supplemented this dataset with the publicly available Coltea-Lung-CT-100W chest dataset, which consists of 100 complete three-phase lung CT scans. Except for the absence of low-energy labels, it shares similar properties with the West China dataset, such as a resolution of 512*512, the same number of paired data slices for the same case, and approximate spatial alignment.

[0034] The actual partitioning of the paired dataset is shown in Table 1. The validation set (15 cases) and test set (15 cases) for the chest region were randomly selected from Coltea-Lung-CT-100W, and the remaining cases were included in the training set (126 cases) along with the West China dataset. The three subsets for the abdomen were partitioned from the West China dataset in a ratio of approximately 9:1:1, and all cases with MonoE labels were included in the training set.

[0035] Tab 1. Composition of Chest and Abdominal CT Dataset

[0036]

[0037] In the following embodiments of the present invention, the specific exemplary technical solutions are for illustrative purposes only, and the implementation of the technical solutions is not limited thereto.

[0038] First Embodiment

[0039] In one embodiment, the present invention proposes an automatic CECT synthesis method, the method steps of which specifically include the following steps.

[0040] Step S100: Obtain plain CT images.

[0041] In one embodiment, exemplarily, plain CT, also known as non-contrast-enhanced CT (NCCT), is an imaging technique widely used for preliminary clinical screening and assessment, and is a conventional method of acquiring CT images. Plain CT scans do not require the injection of iodine contrast agents, and parameters such as slice thickness and radiation dose differ somewhat from CECT, which limits its ability to highlight vascular structures and detect minute lesions. Intuitively, various shapes and boundaries are blurred, easily leading to missed or misdiagnosed lesions, thus affecting subsequent treatment decisions. Therefore, in one embodiment, a plain CT image is a typical conventional CT image, which does not possess the effect of CECT images in revealing minute details through iodine contrast agents.

[0042] Step S200: Input the plain CT image into SPM-CyViT and convert it into a CECT image.

[0043] In one embodiment, SPM-CyViT may be an adversarial generative architecture derived from CycleGAN, but the invention is not limited thereto, and it may be other adversarial network generative architectures. The function of SPM-CyViT is to synthesize high-fidelity virtual contrast-enhanced CT (CECT) from real non-contrast CT (NCCT) without using iodine contrast agent (ICM), thereby enhancing the visual contrast of cardiovascular regions and aiding in clinical diagnosis.

[0044] Specifically, in conjunction with the appendix Figure 1 and 2 To illustrate SPM-CyVIT, in one embodiment, SPM-CyVIT employs a dual-path hybrid coding design, including at least a generator G. NCCT→CECT and pre-trained ViT encoder (ε VIT The pre-trained ViT encoder is a type of ViT global encoder (ε). VIT The generator extracts global semantic features to obtain the VIT feature sequence U of the plain CT image. In one embodiment, the pre-trained ViT encoder may be part of the generator, but the invention is not limited thereto. Generator G NCCT→CECT Including local encoders (ε CNN ) and decoder, used to transmit through encoder (ε CNNThe convolutional layer downsamples the plain CT image to obtain a CNN local detail feature map. The VIT feature sequence U is then fused with the CNN local detail feature map using deep feature fusion (FF) to obtain a hybrid encoded feature map T″. This hybrid encoded feature map T″ is fed into a convolutional decoder, where it undergoes multiple convolutional upsampling layers to restore the original single-channel original resolution image. This yields various semantically consistent high-fidelity virtual CT images and CECT images. Both local detail features and global semantic features are universally known features obtained through image information convolution, and no specific limitations are imposed here.

[0045] In one embodiment, the SPM-CyViT network model can be understood as establishing a transformation mapping relationship between NCCT images and CECT images. The SPM-CyViT network model is trained using a pre-constructed (NCCT image - CECT image) image dataset. In this embodiment, a private dataset of 56 examples is used, each containing both native (NCCT) and arterial (CECT) images. During training, features are extracted from the SPM-CyViT network model by inputting NCCT images, including the pre-trained ViT encoder and the local encoder (ε). CNN Feature maps are extracted separately, and a single-channel raw resolution image is obtained through a decoder. This single-channel raw resolution image is then compared with the CECT images in the dataset to establish a difference iteration. Iteration conditions are set for model training, and the iteration continues until the single-channel raw resolution images obtained by the trained model are as close as possible to each other. The current training steps are limited to the process of this embodiment, and the training process of this invention is not limited to this.

[0046] Furthermore, in one embodiment, specifically, the pre-trained VIT encoder is implemented based on the MAE network framework, but the invention is not limited to this, and it can be other network generation architectures. MAE selects a moderately sized VIT-L as the backbone, and its core contributions are mainly twofold: first, it sets a high image mask ratio, the purpose of which is to construct a more challenging reconstruction task, prompting the model to learn strong latent feature representations; second, it adopts an asymmetric Encoder-Decoder design, specifically, both attention layers have 16 heads, but the Decoder's depth is only 1 / 3 of the Encoder's.

[0047] Specifically, in one embodiment, the overall structure of the MAE of the pre-trained VIT encoder is as follows: Figure 3 As shown, specifically, this invention selects 2200 cases from FLARE2023 as a pre-training dataset, which comes from multiple medical centers around the world, covers all stages of CT, and includes common CT manufacturers, including GE, Philips, Siemens and Toshiba, allowing it to be used for research purposes and shared.

[0048] Further, in one embodiment, a 3-channel sample is stitched together from the target CT slice and adjacent CT slices in the pre-training dataset as input. The original resolution is 512*512, downsampled to 224*224 resolution, and after patching, 196 patches are generated. Each patch is 16*16 pixels in size, and its initial sequence position is recorded in order from left to right and top to bottom. Each patch is flattened into a one-dimensional vector and then mapped to a fixed dimension through a linear projection layer (LinearProj) to obtain patch embeddings with a shape of 196*1024. In addition, to ensure that the Transformer layer can retain spatial information in the serialized input, the positional embeddings of each patch need to be calculated separately and added to the corresponding patch embeddings.

[0049] During each training process, this invention shuffles the relative positions of all patches and performs random masking on 75% of the patches. This means that only 49 patch embeddings and the additional CLS tokens are input into the encoder, which greatly reduces the computational cost of the model.

[0050] Encoder blocks utilize multi-head self-attention (MHSA) to model global dependencies between patches. This is followed by a feed-forward neural network (FFN) with powerful nonlinear mapping capabilities. FFN consists of two fully connected layers sandwiching a rectified linear unit (ReLU) activation function. To accelerate model convergence, both FFN and MHSA introduce residual connections and use layer normalization (LN) to normalize the output of each layer, preventing gradient vanishing or gradient exploding.

[0051] The encoder outputs 50 out-of-order vectors, which are channel-reduced to 512 dimensions and concatenated with 147 zero vectors of the same dimension to obtain patch embeddings of length 197. These are then unshuffled back to their initial relative positions before being input into the decoder. The decoder blocks have the same internal structure as the encoder blocks, but their output needs to be channel-expanded to 768 dimensions to facilitate reshaping back into 16*16 3-channel patches. After unpatchification, a 224*224 resolution predicted CT slice is obtained.

[0052] The pre-training of CT images in this invention retains the self-supervised nature of MAE, that is, the loss function only calculates its reconstruction mean square error on the occluded patch, and the calculation method is as shown in formula (1).

[0053]

[0054] XiX i It is the i-th masked patch in the original CT slice. It is the i-th patch reconstructed by the decoder, N masked The total number of patches that are obscured.

[0055] The resulting global encoder VIT has powerful feature extraction capabilities. After being transferred to SPM-CyVIT, it can effectively avoid the overfitting problem on small-scale CT datasets.

[0056] Specifically, in one embodiment, the training process of the global encoder VIT may further include setting training parameters. Specifically, to fully utilize computational resources and reduce time costs, pre-loaded VIT-L open-source weights that have been fully trained on the ImageNet dataset. Considering that the shallow part of these weights has already learned common low-level features, only the parameters of the last 5 blocks of the encoder and the channel compression layer are updated, while the decoder blocks undergo layer-wise freezing to ensure the robustness of the process. Each pre-trained batch contains 4 samples, and the Adam optimizer uses the parameters recommended by the original project. The learning rate adopts linear decay, with an initial value set to 2e-5, and a total of 10 epochs are trained.

[0057] In another embodiment, SPM-CyVIT may include a generator G. NCCT→CECT (abbreviated as generator G), generator F CECT→NCCT (Generator F for short), three tag-specific discriminators D CECT DNCCT D MonoE Combined with the appendix Figure 1-2 As shown, where ε is the encoder, G is the decoder, and D is the discriminator. Generator G NCCT→CECT Generator F is responsible for converting NCCT to CECT. CECT→NCCT This is responsible for converting CECT back to NCCT. Specifically, SPM-CyVIT incorporates single-level contrast-enhanced CT (MonoE) as... NCCT→CECT An auxiliary label for direction conversion was used, and a corresponding decoder was designed. With the corresponding discriminator D MonoE Furthermore, the decoder-discriminator branches of MonoE and CECT share the same encoded feature map T″. G As input.

[0058] Specifically, as can be understood, Mono-level Enhanced CT (MonoE) utilizes the different attenuation characteristics of low-energy and high-energy X-rays to distinguish tissues and materials. In clinical practice, the voltage of one X-ray tube can be switched or two X-ray tubes with different voltages can be run to generate the required spectral information. Existing research has confirmed that when vascular contrast is used as the primary evaluation criterion, the image quality of low-energy MonoE is generally higher than that of common 120-kVp multicolor images. MonoE in the 40-70keV range performs better in conventional indicators such as contrast signal-to-noise ratio. Furthermore, due to the increased attenuation value of iodine-containing structures, the resolution of vascular walls and tumor boundaries is significantly enhanced, which also means that it may contain richer details. In the current embodiment, Mono-level Enhanced CT (MonoE) images are CT images acquired through Mono-level Enhanced CT (MonoE). Therefore, introducing Mono-level Enhanced CT (MonoE) as an auxiliary label for the NCCT→CECT conversion direction in this invention's SPM-CyVIT will help obtain richer and more accurate CECT images.

[0059] Therefore, this invention also used a private dataset of 56 examples, each containing both native phase (NCCT) and arterial phase (CECT) images, with 18 examples including CT images with enhanced 40keV MonoE labels. The aforementioned dataset was used in the subsequent end-to-end model training process of this invention.

[0060] Specifically, such as Figure 1 , 2As shown, specifically, both generators G and F employ VIT-CNN dual-channel hybrid encoding, similar to the previous embodiment. The generator consists of an encoder and a decoder, with the encoder fusing a local convolutional encoder (CNN) and a pre-trained VIT global encoder. The pre-trained VIT encoders of generators G and F are input to NCCT and CECT images respectively, yielding their respective global feature sequences U. The training process for the pre-trained VIT encoders of generators G and F involves inputting NCCT and CECT images from the pre-training dataset.

[0061] Specifically, generator G NCCT→CECT The structure can specifically include a convolutional encoder ε CNN Global encoder ε ViT Dynamic Aggregation (DA), Feature Fusion (FF), and Convolutional Decoder The internal structure of the generator is as follows: Figure 2 As shown, the convolutional encoder ε CNN Responsible for extracting local detail features T of the image; global encoder ε VIT The module DA is responsible for modeling the global feature sequence U of the high-resolution input image; the dynamic aggregation module DA is responsible for enhancing ε. CNN The global encoded feature map T′ is part of the encoded feature map; the feature fusion module FF is responsible for deeply fusing the two parallel encoded results to obtain a two-dimensional feature map T″ with global context information and local details; the convolutional decoder... The generator F is responsible for restoring the image to the same shape and resolution as the image before encoding; similarly, in one embodiment, the generator F... CECT→NCCT Structure and Generator G NCCT-CECT The structures are exactly the same.

[0062] Specifically, in one embodiment, the CNN local encoder ε CNN The system consists of five convolutional layers. The first convolutional input layer has a large receptive field of 7x7, with 32 kernels, a stride of 1, and 3 pixels of padding to maintain the feature map spatial size. This is followed by four single-convolutional downsampling layers with kernels of 32, 64, 128, and 256 respectively, all with a spatial scale of 3x3, a stride of 2, and 1-pixel padding, which progressively reduce the feature map spatial size and expand the number of channels. Furthermore, each convolutional layer is followed by a batch normalization (BN) layer and a ReLU activation function. Let ε... CNN The obtained local feature map is When the input resolution is 512*512, the size of T is h=w=32, and the number of channels is c=256.

[0063] Specifically, in one embodiment, the VIT global encoder ε VIT Consistent with the aforementioned MAE structure, in another embodiment, the 224*224 resolution image can be divided into 196 patches. Without performing Randing Mask and Shuffle operations, the global sequence features can be obtained by sequentially passing the LinearProj layer, 24 VIT Encoder Blocks, and 2 ChannelReduction layers. n = 196 and d = 256. In one embodiment, during the end-to-end training of SPM-CyVIT, given that the large-scale pre-trained weights already possess rich feature representation capabilities, the VIT global encoder ε will be... VIT Perform a seamless migration, then completely freeze ε. VIT Each layer. The advantage of this approach is not only reflected in the sharp reduction of the number of trainable parameters, but also in the hierarchical separation of subsequent feature fusion, which reduces mutual interference between features, drives the generator to focus on detailed texture features, and generates more realistic and visually superior images.

[0064] Specifically, in one embodiment, MHSA in Dynamic Aggregation (DA module) endows the convolutional encoder with context-aware capabilities by automatically adjusting the degree of attention given to different features. Each head i∈{1,...,n} of MHSA... h Both contain a convolutional projection layer with positional embedding (ConvProj) and a self-attention mechanism, where n h =8 represents the number of heads. The feature map T can be viewed as a set of overlapping visual tokens of size 32*32. To compensate for the lack of absolute positional relationships between the tokens, this invention flattens T pixel by pixel and adds positional encoding, then reshapes it back to its original shape to obtain T. pos T pos This represents a two-dimensional feature map after adding positional encoding. Convolutional projection is used to compute the feature map query, key, and value, such as... Figure 4 As shown.

[0065] Specifically, the convolutional projection consists of three parameter-independent projection blocks, each of which is a depthwise separable convolution containing two convolutional layers and one batch normalization (BN) layer. The first depthwise convolutional layer uses 256 3x3 convolutional kernels with padding of 1. The stride of the Query projection block is 1 (generating a 32x32 activation map), while the stride of the Key and Value layers is 2 (generating a 16x16 activation map). The output passes through the BN layer and then enters a pointwise convolutional layer with 1x1 kernels, no padding, a stride of 1, and all channels adjusted to 64. Finally, a flattening operation is performed to obtain the values ​​of each head. and Where n q =n=1024, n k =n v =256 and d q =d k =d v =64. To integrate the outputs of each head, the results of Self-Attention need to be concatenated along the channel dimension to obtain... And d = 512, the relevant calculation process is as shown in formula (2).

[0066]

[0067] T′=LN(Reshape(LinearProj(U′)))+T (3)

[0068] Represents the transpose of Ki, the dot product divided by Softmax normalization is used to ensure numerical stability of the computation and effective gradient propagation. As shown in Equation (3), UT′ undergoes channel compression through the LinearProj layer and is reshaped into a 32*32*256 two-dimensional feature map. After layer normalization, it is added to the input tensor T to achieve residual connection, thus obtaining the output tensor of MHSA.

[0069] Specifically, in one embodiment, Feature Fusion (FF) includes a Multi-Head Cross-Attention (MHCA) mechanism. The dynamically aggregated CNN feature map T′ and VIT feature sequence U are further input into the Multi-Head Cross-Attention (MHCA) mechanism. By constructing the correlation between Q, K, and V, dynamic weighting of information interaction is achieved. This mechanism can deeply fuse the two features in terms of feature distribution and semantic expression. The specific implementation process is as follows: Equations (4)-(5) are used. T′ is mapped by a convolutional projection with a stride of 1. and It originates from U, which is sampled from a convolutional projection layer with a stride of 1.

[0070] Q=Flatten(ConvProj(Reshape(T′ pos S = 1

[0071] K,V=(W K,V S=1)U (5)

[0072] Where S is the stride length, and T′ is the stride length. pos To incorporate the positional embedding T′, W K，V The three are learnable independent projection matrices. They all represent the integration of eight 64-dimensional independent tensors in the channel dimension d. After being evenly divided, they can be used as the inputs of each head in MHCA. Therefore, the feature fusion process can be simplified as formulas (6)-(7).

[0073]

[0074] T″=LN(PointWise 2 (T″ cross ))+T″ cross (7)

[0075] Among them, T″ cross This indicates that the cross-attention tensor after adding residual connections needs to be fed into a dual convolutional module consisting of two pointwise convolutional layers. This dual convolutional module executes on 512 and 256 convolutional kernels respectively, inserting a GELU activation function in between. The layer normalization result is added to the input tensor to obtain the final fused feature map.

[0076] Therefore, in one embodiment, step S200, inputting the plain CT image into SPM-CyVIT to obtain a CECT image, specifically includes:

[0077] Input plain CT images into SPM-CyVIT, where T is the convolution encoder ε. CNNThe encoded feature map, U is the attention encoder ε ViT The encoded feature sequence is T; T is input into the dynamic aggregation module, where it undergoes multi-head self-attention volume calculation and residual connection to obtain a feature map T with certain global feature information; then, U and T′ are sent to the feature fusion module, where deep feature fusion is performed by calculating their multi-head cross attention, and the final encoded feature map T′ is output after passing through a double convolutional layer and residual connection; T′ is sent to the convolutional decoder, and after passing through multiple convolutional upsampling layers, it is restored to the original single-channel original resolution image, which is the CECT image.

[0078] In one embodiment, in conjunction with the appendix Figure 1 and attached Figure 2 To illustrate, the SPM-CyVIT of this invention is equipped with three tag-specific discriminators (D... CECT D NCCT D MonoE SPM-CyVIT decouples the dual encoder from the tag-specific decoding-discrimination branch, aiming to empower the generator G with shared hybrid coding features. NCCT→CECT The multi-label generation capability balances the differential representation between CECT and 40keV MonoE. Decoders for all labels are provided. (correspond Figure 1 The three label decoders (CECT, MonoE, and NCCT) all start from T″. Specifically, upsampling is implemented by four deconvolutional layers, with the number of channels from bottom to top being 256, 128, 64, and 32, respectively. Each layer first goes through a 3*3 deconvolution with a stride of 2 and padding of 1. The output is then residually connected to a downsampled feature map of the same size. This is followed by two 3*3 convolutions with a stride of 1 and padding of 1, both with additional BN layers and ReLU activation functions. Finally, this invention also uses a 7*7 convolutional output mapping layer with a stride of 1 and padding of 3, with only 3 convolutional kernels, to reconstruct a 512*512*1 target virtual CT. Discriminator (D) (corresponding to Figure 1 The discriminator (corresponding to the three labels CECT, MonoE, and NCCT) specifically adopts a structure derived from the classic PatchGAN network, containing five 4*4 convolutional layers padded with 1s. The first three layers have a stride of 2, and the last two have a stride of 1. Except for the last layer, each convolutional layer is followed by an instance normalization layer and a LeakyReLU operation. The number of channels is progressively expanded to 256 and then compressed back to 1, ultimately resulting in a 62*62*1 probability matrix. The local receptive field corresponding to each output value is 94*94. The average probability of each region is calculated to distinguish between virtual CT and real CT.

[0079] Therefore, in another embodiment, as an example, step S200, inputting the plain CT image into SPM-CyVIT to obtain a CECT image specifically includes: first, inputting a real NCCT image X with a shape of 512*512*1 into the generator G. NCCT→CECT In the generator, X needs to be passed in parallel through the convolutional encoder ε. CNN (consisting of one convolutional input layer and four convolutional downsampling layers) and attention encoder ε ViT The core structure is a multi-head self-attention block. Encoding is performed to obtain a local feature map T (32*32*1024 shape) and a global feature sequence U (1024*512 shape). Then, T passes through a dynamic aggregation module (DA) to obtain a feature map T (also 32*32*1024 shape) that incorporates some global context information. T and U are then input into a feature fusion module (FF) to calculate multi-head cross-attention, resulting in the final encoded feature map T″ (32*32*1024 shape) that integrates local and global information. Since there are two labels, CECT and MonoE, T″ needs to be input simultaneously. and Two convolutional decoders (each consisting of four convolutional upsampling layers and one output convolutional layer) synthesize virtual CECT images. and virtual MonoE images (All shapes are 512*512*1), and each is identified using its own discriminator; subsequently... and The components are stitched together along the channel dimension (each component is 512*512*2) and then fed into generator F. CECT→NCCT After undergoing the same internal process, the output is a cyclically reconstructed virtual NCCT image. Using discriminator D NCCT Identify it; ultimately achieve the overall data flow for this direction within a training step;

[0080] In one embodiment, before step S200, the method may further include: step S300, training the SPM-CyVIT model.

[0081] Specifically, after establishing the SPM-CyVIT model, the end-to-end training of SPM-CyVIT involves three data domains: non-contrast CT (NCCT) in the primary phase, mixed-energy CECT in the arterial phase, and 40keV MonoE. These will be referred to as symbols thereafter. To simplify the representation, let (X, Y, Z) be any of the true paired samples. Combined Figure 1 The model architecture, generator G ~ (G Y G Z )~(DY (ε G )) and discriminator D Y and D Z This forms the transition pathway from the primary phase to the arterial phase. Generators F to D X (ε F ) and discriminator D X This represents the opposite path.

[0082] Similarly, the generated virtual pairing samples are recorded as The generators G and F attempt to map the real sample distribution, while the discriminator mainly distinguishes between the two. This balance is maintained during training until the authenticity of the generated result tends to be optimal. Thus, the adversarial loss in both directions is derived as shown in formulas (8)-(9).

[0083]

[0084] The expected value is represented by the subscript, which indicates the source of the data sample. Note that the input of the generator F is the result of concatenating samples Y and Z along the channel dimension. In addition, to enhance the intra-domain fidelity of the model, it is necessary to calculate the identity loss between the output and input of the generator when the input is a target domain image. The specific calculation process is shown in formula (10).

[0085]

[0086] ||·||1 is the L1 norm, and the input to the generator F is the result of concatenating the two X samples along the channel dimension. Of course, the key to achieving unsupervised transformation is to introduce a multi-layer cycle consistency loss, which ensures that the image remains unchanged after bidirectional cycle mapping. The original resolution level is calculated using formula (11).

[0087]

[0088] (G Y (X), G Z (X)) represents different decoding results Y and Z under the same X input. Similarly, for intermediate high-level features that exist in both generators G and F, namely T before cross-fusion and T″ after cross-fusion, we add an additional cycle-consistent perceptual loss, calculated as formulas (12)-(13).

[0089]

[0090] ||·||2 is the L2 norm, This indicates that a pre-trained VGG-19 network is used for mapping. Virtual pairing samples As an intermediate result when inputting into the generator. Summarizing all the loss terms above, the complete loss function of SPM-CyVIT can be obtained as Equation (14).

[0091]

[0092] α, β, and γ are adjustable hyperparameter weights, and their values ​​during actual training can be 5, 10, and 1, respectively.

[0093] In one embodiment, specifically, the model training process may further include preprocessing the CT images in the dataset. Specifically, preprocessing may include intercepting the data within [-1024, 1024] Hounsfield Units (HU), then uniformly dividing by 1024, and standardizing to [-1, 1]. Specifically, all CT images are uniformly displayed using a 400HU window width and a 30HU window level to enhance the visual contrast of blood vessels.

[0094] Specifically, this invention proposes an automatic CECT synthesis method by constructing a self-supervised pre-trained multi-path Cyclic Consistent Adversarial Visual Transformer architecture model, SPM-CyVIT. SPM-CyVIT integrates Convolutional Neural Networks (CNN) and Visual Transformers (VIT) through a dual-path hybrid encoding strategy, fully utilizing their complementary advantages at different scales. In the encoding stage, it combines the unique advantages of CNN and VIT, aiming to preserve key features such as anatomical structures, lesion morphology, location, and boundaries in the original CT slices while achieving bidirectional conversion between native phase (NCCT) and arterial phase (CECT) effects in contrast agent uptake areas such as blood vessels, tumors, and blood-rich tissues. Simultaneously, it synthesizes high-quality virtual CECTs from ordinary NCCT, thereby eliminating dependence on ICAs (Integrated Circuits Assembled ...

[0095] Specifically, the experimental results of this invention demonstrate its specific effectiveness. In one embodiment, it can be combined with... Figure 5 As shown, the conversion results of different models are illustrated. In another embodiment, to comprehensively and objectively evaluate model performance, this invention calculates four quantitative indicators: Mean Absolute Error (MAE) represents image intensity deviation, Peak Signal-to-Noise Ratio (PSNR) measures pixel-level reconstruction accuracy, Structural Similarity (SSIM) reflects the consistency of image structure and texture, and Normalized Mean Square Error (RMSE) quantifies the overall difference between the generated image and the target image.

[0096] Table 2-3 details the data results for the chest and abdomen, respectively, showing that SPM-CyVIT achieved optimal values ​​on almost all metrics. Among all evaluated models, Pix2Pix is ​​limited by the inherent limitations of paired training, which restricts its generalization ability on unpaired datasets. On the other hand, CycleGAN's performance may be affected by the original ResNet9 generator, which may not be well-suited for handling high-resolution images. Therefore, neither of these models performed optimally, consistent with our initial hypothesis.

[0097] Table 2. Quantitative analysis data of chest cavity test

[0098]

[0099] While AttentionGAN performs well on the MAE metric for the chest region, it still lags behind our proposed model in the overall evaluation. This gap is particularly pronounced on the RMSE metric, where SPM-CyVIT significantly outperforms AttentionGAN. SPM-CyVIT's superior performance is attributed to its innovative dual-path hybrid encoding design and the effective integration of global and local features, which enhance the model's ability to capture complex details and maintain structural consistency.

[0100] Table 3. Quantitative analysis data from abdominal experiments

[0101]

[0102] In summary, the experimental results fully validate the superior performance of SPM-CyVIT in unpaired CT domain translation. The model's ability to achieve state-of-the-art results on multiple quantitative metrics highlights its potential to advance medical image analysis and facilitate more accurate diagnosis.

[0103] Example 2

[0104] The present invention also provides another embodiment, which proposes a CECT automated synthesis device, comprising:

[0105] The input module is used to acquire plain CT images;

[0106] The synthesis module inputs the plain CT image into SPM-CyViT and converts it into a CECT image.

[0107] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the steps, processing methods, devices, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0108] Example 3

[0109] The present invention also provides another embodiment, which proposes an electronic device including: a processor 1 and a memory 2.

[0110] The memory 2 is used to store computer programs.

[0111] The memory 2 includes various media capable of storing program code, such as ROM, RAM, magnetic disk, USB flash drive, memory card, or optical disk.

[0112] The processor 1 is connected to the memory 2 and is used to execute the computer program stored in the memory 2 so that the CECT automatic synthesis device performs the above-described CECT automatic synthesis method.

[0113] Preferably, the processor 1 may be a central processing unit (CPU); or it may be an application-specific integrated circuit (ASIC).

[0114] Example 4

[0115] The present invention also provides another embodiment, namely, a computer-readable storage medium storing a computer program that can be executed by at least one processor to cause the at least one processor to perform the steps of the CECT automatic synthesis method as described above.

[0116] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0117] Those skilled in the art will recognize that the modules, units, and / or method steps of the various embodiments described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0118] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another device or system, or some features may be ignored or not executed.

[0119] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated unit described above can be implemented in hardware or as a software functional unit.

[0120] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An automated CECT synthesis method, characterized in that, The method includes the following steps: S1. Acquire plain CT images; S2. Input the plain CT image into SPM-CyViT and convert it to obtain a CECT image.

2. The CECT automatic synthesis method according to claim 1, characterized in that, The SPM-CyViT network model establishes a conversion mapping relationship between NCCT images and CECT images.

3. The CECT automatic synthesis method according to claim 1 or 2, characterized in that, The SPM-CyVIT employs a dual-path hybrid coding design, including a generator G. NCCT→CECT and pre-trained ViT encoder (ε VIT Generator G NCCT→CECT Including local encoders (ε CNN ) and decoder; wherein, step S2 specifically includes: VIT global encoder (ε VIT It extracts global semantic features to obtain the global feature sequence U of the plain CT image; through the encoder (ε) CNN The convolutional layer downsamples the plain CT image to obtain a CNN local detail feature map, and the global feature sequence U and the CNN local detail feature map are fused by deep feature fusion (FF) to obtain a hybrid coded feature map T″. The hybrid coded feature map T″ is fed into the decoder, and after passing through multiple convolutional upsampling layers in the decoder, a CECT image is obtained.

4. The CECT automatic synthesis method according to claim 1, characterized in that, The SPM-CyVIT includes a generator G NCCT→CECT Generator F CECT→NCCT Convolutional Decoder Three tag-specific discriminators D CECT D NCCT D MonoE Generator G NCCT→CECT The structure specifically includes a convolutional encoder ε CNN Global encoder ε ViT Dynamic Aggregation (DA), Feature Fusion (FF), and Convolutional Decoder Generator F CECT→NCCT The structure specifically includes a convolutional encoder ε CNN Global encoder ε ViT Dynamic Aggregation (DA), Feature Fusion (FF), and Convolutional Decoder 5. The CECT automatic synthesis method according to claim 4, characterized in that, in, Convolutional encoder ε CNN Responsible for extracting local detail feature maps T from CT images; global encoder ε ViT The module DA is responsible for modeling the global feature sequence U of the high-resolution input CT image; the dynamic aggregation module DA is responsible for enhancing ε. CNN The global encoded feature map T′ is generated within the encoded feature map; the feature fusion module FF is responsible for deeply fusing the two parallel encoded results T′ and U to obtain a two-dimensional feature map T″ with global context information and local details; the convolutional decoder... It is responsible for restoring the two-dimensional feature map T″ to a CECT image with the same shape and resolution as the image before encoding.

6. The CECT automatic synthesis method according to claim 4 or 5, characterized in that, Step S2 specifically includes: the plain CT image X input generator G NCCT→CECT In the generator, X needs to be passed in parallel through the convolutional encoder ε. CNN and global encoder ε ViT Encoding is performed to obtain local feature maps T and global feature sequences U; T is then processed by a dynamic aggregation module (DA) to obtain a feature map T′ that incorporates global context information; T and U are input into a feature fusion module (FF) to calculate a two-dimensional feature map T″; T″ is then simultaneously fed into... and Two convolutional decoders synthesize virtual CECT images. and virtual MonoE images 7. A CECT automated synthesis device, comprising: The input module is used to acquire plain CT images; The synthesis module inputs the plain CT image into SPM-CyViT and converts it into a CECT image.

8. An electronic device, characterized in that, It includes a memory and one or more programs, wherein one or more programs are stored in the memory and configured to be executed by one or more processors, wherein the one or more programs include steps for performing the CECT automatic synthesis method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the CECT automatic synthesis method as described in any one of claims 1-6.

Images