A method for automatic colorization of CT images
By combining CT image preprocessing with a generative discriminant network, the problems of unrealistic colors and the need for manual adjustment in existing medical image coloring methods are solved, generating color CT images that conform to human visual preferences, thus improving image quality and diagnostic effectiveness.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- NORTHEASTERN UNIV CHINA
- Filing Date
- 2022-12-08
- Publication Date
- 2026-04-17
AI Technical Summary
Existing medical image coloring methods do not produce colors that are realistic and reliable enough, and they deviate to some extent from human visual preferences, requiring a great deal of manual intervention and adjustment.
We employ CT image preprocessing, establish a first network for texture enhancement and a generative discriminant network, improve image quality through threshold segmentation and morphological processing, and combine PET metabolic information and self-attention mechanism to construct a generative discriminant network to generate colorized results that conform to human visual preferences.
It improves the realism and consistency of colorization results, reduces human intervention, and generates realistic color CT images that meet medical diagnostic requirements.
Smart Images

Figure CN115953490B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of medical imaging technology, and more particularly to an automatic colorization method for CT images. Background Technology
[0002] Medical imaging plays an increasingly important role in medical research, diagnosis, and treatment planning. Due to the unique nature of its imaging mechanisms, current medical imaging rarely considers color information. With the widespread application of 3D visualization and multimodal imaging, the use of color has grown rapidly, and researchers have begun to explore colorizing medical images using scientific visualization techniques. Image colorization has been widely applied in many fields, and numerous solutions exist. However, since there are no truly colored medical images in the real world, and medical images differ significantly from natural images, directly applying natural image coloring techniques to medical images yields poor-quality color results. Traditional medical image colorization is based on manual intervention, using interactive methods to propagate initial strokes or color points throughout the grayscale image, typically based on low-level similarity methods. With technological advancements, deep learning-based image colorization methods have gradually become mainstream, primarily including reference-based colorization and automatic image colorization.
[0003] First, medical image colorization methods based on handcrafted features can achieve colorization relatively easily. However, these methods require manual color input, and the colorization results depend on human experience, which is time-consuming and labor-intensive. Another problem to be addressed is that pseudo-color colorization results obtained through region color propagation are stiff and unnatural, easily ignoring the pathological features of the original medical image. Second, some existing deep learning-based medical image colorization methods perform color transfer through analogy between the colored medical image and the image to be colored, or train a colorization network for colorization. While these methods can reduce user input and improve colorization efficiency to some extent, and also improve generalization ability, the colors are not realistic and reliable enough, and color bleeding may occur, still requiring manual intervention to adjust the colors. Furthermore, the networks predicting chromaticity information do not consider visual metrics, resulting in colorization results that deviate to some extent from human visual preferences. Summary of the Invention
[0004] (a) Technical problems to be solved
[0005] In view of the above-mentioned shortcomings and deficiencies of the prior art, the present invention provides an automatic colorization method for CT images, which solves the technical problem that the coloring of medical images in the prior art is not realistic and reliable and deviates to a certain extent from human visual preferences.
[0006] (II) Technical Solution
[0007] To achieve the above objectives, the main technical solutions adopted by the present invention include:
[0008] This invention provides an automatic colorization method for CT images, the method comprising:
[0009] Step 1: Obtain contrast-enhanced CT images by preprocessing the CT images;
[0010] Step 2: Establish a first network for acquiring texture-enhanced CT images, and train the first network based on a pre-acquired training dataset to obtain a trained first network;
[0011] The training dataset includes multiple pairs of images; each pair of images includes: a training CT image and a training PET image.
[0012] Step 3: Establish a generative discriminant network for colorization of CT images, and train the generative discriminant network based on the pre-acquired initial training set to obtain the trained generative discriminant network;
[0013] Step 4: Based on the contrast-enhanced CT image, the trained first network, and the trained generative discriminative network, the final CT color visualization result is obtained.
[0014] Preferably, step 1 specifically includes:
[0015] Step 1.1: Set the fusion thresholds t1 and t2 and the visualization window value W for the CT images. Lung W Mediastinum W Bone ;
[0016] Based on preset fusion thresholds t1 and t2, the CT image is automatically segmented using the original HU value to obtain the initial lung region mask, the initial mask for other regions excluding the lung and bone, and the initial bone region mask.
[0017] Connectivity and morphological processing are applied to the initial lung region mask, the initial mask for regions other than lung and bone, and the initial bone region mask to obtain the complete lung region mask R. L Complete masking of other areas besides the lungs and bones. M and complete bone region mask R B ;
[0018] Step 1.2: Visualize the original CT image according to formulas (1), (2), and (3) respectively, and obtain the lung visualization area I. L (x, y), visualization area excluding lungs and bones I M (x, y), Visualization area of the bone I B (x, y);
[0019] IL (x, y) = W Lung (I0(x,y)×R) L , I0(x,y)<t1 (1);
[0020] I M (x, y) = MW Mediastinum (I0(x, y))×RM, t1≤I0(x, y)≤t2, (2);
[0021] I B (x, y) = W Bone (I0(x,y))×R B , t2<I0(x,y) (3);
[0022] Where I0(x,y) is the original CT image; M is the maximum intensity value of I0(x,y);
[0023] Formula (4) is used to fuse lung visualization regions I L (x, y), visualization area excluding lungs and bones I M (x, y), Visualization area of the bone I B (x, y) yields the grayscale image I of the CT image. ct (x, y);
[0024] I' ct (x, y) = I L (x, y) + I M (x, y) + I B (x, y) (4);
[0025] Step 1.3: Use the Sobel edge detection operator to extract the lung visualization region I. L (x, y), visualization area excluding lungs and bones I M (x, y), Visualization area of the bone I B The boundary (x, y) is used to perform a pre-defined convolution kernel on the grayscale image I′ of the CT image. ct The boundaries of the three visualization regions in (x, y) are convolved to obtain a contrast-enhanced CT image;
[0026] Wherein, the size of the convolution kernel is m×n; m=n=3; where, the convolution kernel
[0027] Preferably, step 2 specifically includes:
[0028] Step 2.1: Obtain the training dataset; preprocess the CT images in each pair of images to obtain the corresponding contrast-enhanced CT images;
[0029] Obtain the SUV value of the PET image trained in the pair of images, and truncate the SUV value of the trained PET image according to two preset thresholds to obtain the PET image after metabolic suppression.
[0030] The PET images after metabolic suppression were then resampled to the same isotropic resolution as the contrast-enhanced CT images, resulting in resampled PET images.
[0031] Gaussian filtering is applied to the resampled PET image to reduce noise, resulting in a denoised PET image.
[0032] The weight map I for each pair of images is obtained using formula (6). weight ;
[0033] Formula (6) is:
[0034]
[0035] Where I ct It is a contrast-enhanced CT image normalized to [0, 1].
[0036] I Pet This is a normalized and denoised PET image;
[0037] α is a preset influence factor of the denoised PET image in the weight map;
[0038] Step 2.2: Based on the weight map I of each pair of images in the training dataset weight Train the pre-established first network to obtain the trained first network;
[0039] The first network is a Zero-DCE network suitable for single-channel medical images;
[0040] The input to the first network is the weight graph I. weight And contrast-enhanced CT images, weighted map I weight Multiply the image by all intermediate feature maps of the first network and the output image to output the corresponding parameter map;
[0041] The input contrast-enhanced CT image is multiplied by the parametric map to obtain the final texture-enhanced CT image.
[0042] Preferably,
[0043] The first network comprises a combination of seven symmetrically connected convolutional layers and activation function layers, with the first six convolutional layers having 64 channels and the last convolutional layer having 4 channels.
[0044] Preferably, step 3 includes:
[0045] Step 3.1: Obtain the initial training set, which includes multiple color cryosection images of the human body;
[0046] The initial training set is then subjected to a second processing step to obtain the final training set.
[0047] Step 3.2: Establish a generator-discriminator network, which includes a generator and a discriminator;
[0048] Step 3.3: Construct a colorization objective function, and train the system to minimize the colorization objective function;
[0049] G * =argminmaxL GAN (G,D)+αL color +βL per +γL tv ;
[0050] G * Colorization objective function;
[0051] x represents the input to the generator discriminant network;
[0052] y represents the gold standard;
[0053] G represents a generator;
[0054] D represents the discriminator;
[0055] α, β, and γ are preset weights;
[0056] L GAN (G, D) = log (D (x, y)) + log (1-D (x, G (x)));
[0057] G(x) represents the color image generated by the generator;
[0058] L color =|yG(x)|;
[0059] L per Perceived loss;
[0060] L tv (G(x)) represents the TV loss;
[0061] Step 3.4: Convert each image in the final training set from RGB color space to Lab color space, normalize it, and extract L channel and ab channel data pairs. Use the L luminance channel as input to the generator-discriminator network to predict the ab chrominance channels. Use the original ab channels as labels for supervised training. According to the preset generator-discriminator network training parameters, train the generator and discriminator in the generator-discriminator network alternately until the adversarial relationship between the generator and discriminator reaches a balance, and obtain the trained generator-discriminator network.
[0062] Preferably,
[0063] The second process includes:
[0064] The color of the cryosol in each human body color cryosection image in the initial training set is set as the background value, and the foreground of each human body color cryosection image is extracted by the labeling method.
[0065] Then, the brightness and saturation of the human body color cryosection image after the foreground was extracted were enhanced to obtain an enhanced human body color cryosection image;
[0066] Then, the Region of Interest (ROI) in the enhanced human body color cryosection image is cropped with the foreground as the center, and the cropped ROI in the enhanced human body color cryosection image is scaled to the preset image resolution.
[0067] Preferably,
[0068] The generator is used to extract image features from the input image in the generative discriminant network;
[0069] The generator performs a downsampling operation, where the output feature maps of the third, fourth, and fifth convolutional blocks are connected to the self-attention module; each convolutional block contains batch normalization, activation functions, and convolutional layers;
[0070] In the upsampling part of the generator, bilinear upsampling layers and convolutional layers are used to replace the standard deconvolutional layers, and shallow feature reuse is achieved through skip connections. Self-attention modules are added to the second, third, and fourth layers of the discriminator.
[0071] Preferably, step 4 specifically includes:
[0072] The contrast-enhanced CT image is input into the trained first network to obtain the final texture-enhanced CT image. Then, the final texture-enhanced CT image is input into the trained generative discriminative network to obtain the final CT color visualization result.
[0073] (III) Beneficial Effects
[0074] The beneficial effects of this invention are as follows: The automatic colorization method for CT images uses threshold segmentation and morphological methods, and adaptively fuses lung windows, bone windows, and mediastinal windows based on HU values, overcoming the problem of low contrast when multiple organs and tissues with significantly different values are displayed simultaneously. A first network embedding PET metabolic information prompts the network to focus on dark areas and high-metabolic regions, further improving the brightness and texture details of the CT image. A generative discriminant network introducing self-attention mechanisms and perceptual metrics improves color consistency within the same organ and tissue and color contrast between organs and tissues, making the colorization result more in line with human visual preferences. In summary, this method improves color visualization quality through two stages: improving image quality before colorization and constructing a task-specific generative discriminant network. The resulting color CT is in line with human visual preferences, realistic, and retains medical diagnostic significance. It solves the problems of existing colorization methods, such as requiring extensive manual intervention, low image quality, and unreliable color accuracy. Attached Figure Description
[0075] Figure 1 This is a flowchart of an automatic colorization method for CT images according to the present invention;
[0076] Figure 2 This is a schematic diagram of automatic colorization of CT images in an embodiment of the present invention;
[0077] Figure 3 This is a schematic diagram of the first network structure in an embodiment of the present invention;
[0078] Figure 4 This is a schematic diagram of the generation and discrimination network structure in an embodiment of the present invention. Detailed Implementation
[0079] To better explain and facilitate understanding of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.
[0080] To better understand the above technical solutions, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present invention can be understood more clearly and thoroughly, and that the scope of the present invention can be fully conveyed to those skilled in the art.
[0081] See Figure 1 This embodiment provides an automatic colorization method for CT images, the method comprising:
[0082] Step 1: Preprocess the CT images to obtain contrast-enhanced CT images.
[0083] In this embodiment, step 1 specifically includes:
[0084] Step 1.1: Set the fusion thresholds t1 and t2 and the visualization window value W for the CT images. Lung W Mediastinum W Bone .
[0085] Based on preset fusion thresholds t1 and t2, the CT image is automatically segmented using the original HU value to obtain the initial lung region mask, the initial mask for other regions excluding the lung and bone, and the initial bone region mask.
[0086] Connectivity and morphological processing are applied to the initial lung region mask, the initial mask for regions other than lung and bone, and the initial bone region mask to obtain the complete lung region mask R. L Complete masking of other areas besides the lungs and bones. M and complete bone region mask R B .
[0087] Step 1.2: Visualize the original CT image according to formulas (1), (2), and (3) respectively, and obtain the lung visualization area I. L (x, y), visualization area excluding lungs and bones I M (x, y), Visualization area of the bone I B (x, y).
[0088] I L (x, y) = W Lung (I0(x,y))×R L , I0(x,y)<t1 (1);
[0089] I M (x, y) = MW Mediastinun (I0(x,y))×R M , t1≤I0(x, y)≤t2, (2);
[0090] I B (x, y) = W Bone (I0(x,y))×R B , t2<I0(x,y) (3);
[0091] Where I0(x,y) is the original CT image; M is the maximum intensity value of I0(x,y).
[0092] Formula (4) is used to fuse lung visualization regions I L (x, y), visualization area excluding lungs and bones I M (x, y), Visualization area of the bone I B(x, y) yields the grayscale image I of the CT image. ct (x, y).
[0093] I' ct (x, y) = I L (x, y) + I M (x, y) + I B (x, y) (4);
[0094] Step 1.3: Use the Sobel edge detection operator to extract the lung visualization region I. L (x, y), visualization area excluding lungs and bones I M (x, y), Visualization area of the bone I B The boundary (x, y) is used to perform a pre-defined convolution kernel on the grayscale image I′ of the CT image. ct Convolution is performed on the boundaries of the three visualization regions in (x, y) to obtain a contrast-enhanced CT image.
[0095] Wherein, the size of the convolution kernel is m×n; m=n=3; where, the convolution kernel
[0096] Step 2: Establish a first network for acquiring texture-enhanced CT images, and train the first network based on a pre-acquired training dataset to obtain a trained first network.
[0097] The training dataset includes multiple pairs of images; each pair of images includes: a training CT image and a training PET image.
[0098] Step 2 specifically includes:
[0099] Step 2.1: Obtain the training dataset; preprocess the CT images in each pair of images to obtain the corresponding contrast-enhanced CT images.
[0100] Obtain the SUV value of the PET image trained in the pair of images, and truncate the SUV value of the trained PET image according to two preset thresholds to obtain the PET image after metabolic suppression.
[0101] The PET images after metabolic suppression were then resampled to the same isotropic resolution as the contrast-enhanced CT images, resulting in resampled PET images.
[0102] Gaussian filtering is applied to the resampled PET image to reduce noise, resulting in a denoised PET image.
[0103] The weight map I for each pair of images is obtained using formula (6). weight .
[0104] Formula (6) is:
[0105]
[0106] Where I ct It is a contrast-enhanced CT image normalized to [0, 1].
[0107] I pet This is a normalized and denoised PET image.
[0108] α is a preset influence factor of the denoised PET image in the weight map.
[0109] Step 2.2: Based on the weight map I of each pair of images in the training dataset weight Train the pre-established first network to obtain the trained first network.
[0110] The first network is a Zero-DCE network suitable for single-channel medical images.
[0111] The input to the first network is the weight graph I. weight And contrast-enhanced CT images, weighted map I weight Multiply the image by all intermediate feature maps of the first network and the output image to output the corresponding parameter map.
[0112] The input contrast-enhanced CT image is multiplied by the parametric map to obtain the final texture-enhanced CT image.
[0113] Step 3: Establish a generative discriminant network for colorizing CT images, and train the generative discriminant network based on the pre-acquired initial training set to obtain the trained generative discriminant network.
[0114] Step 4: Based on the contrast-enhanced CT image, the trained first network, and the trained generative discriminative network, the final CT color visualization result is obtained.
[0115] See Figure 3 In this embodiment, the first network includes a combination of 7 convolutional layers and activation function layers with symmetrical connections, wherein the first six convolutional layers have 64 channels and the last convolutional layer has 4 channels.
[0116] In the practical application of this embodiment, step 3 includes:
[0117] Step 3.1: Obtain the initial training set, which includes multiple human color cryosection images.
[0118] The initial training set is then subjected to a second processing step to obtain the final training set.
[0119] Step 3.2: Establish a generative discriminant network, see [link to documentation]. Figure 4 The generator-discriminator network includes a generator and a discriminator.
[0120] Step 3.3: Construct a colorization objective function, and train the system to minimize the colorization objective function;
[0121] G * =argminmaxL GAN (G,D)+αL color +βL per +γL tv ;
[0122] C * Colorization objective function.
[0123] x represents the input to the generator discriminant network.
[0124] y represents the gold standard.
[0125] G represents a generator.
[0126] D represents the discriminator.
[0127] α, β, and γ are preset weights.
[0128] L GAN (G,D)=log(D(x,y))+log(1-D(x,G(x))).
[0129] G(x) represents the color image generated by the generator.
[0130] L color =|yG(x)|.
[0131] L per Perceived loss.
[0132] L tv (G(x)) represents the TV loss.
[0133] Step 3.4: Convert each image in the final training set from RGB color space to Lab color space, normalize it, and extract L channel and ab channel data pairs. Use the L luminance channel as input to the generator-discriminator network to predict the ab chrominance channels. Use the original ab channels as labels for supervised training. According to the preset generator-discriminator network training parameters, train the generator and discriminator in the generator-discriminator network alternately until the adversarial relationship between the generator and discriminator reaches a balance, and obtain the trained generator-discriminator network.
[0134] In this embodiment, the second process includes:
[0135] The cryotherapy color in each human body cryosection image in the initial training set is set as the background value, and the foreground of each human body cryosection image is extracted by labeling.
[0136] Then, the brightness and saturation of the extracted foreground human body color cryosection image are enhanced to obtain an enhanced human body color cryosection image.
[0137] Then, the Region of Interest (ROI) in the enhanced human body color cryosection image is cropped with the foreground as the center, and the cropped ROI in the enhanced human body color cryosection image is scaled to the preset image resolution.
[0138] See Figure 4 The generator is used to extract image features from the input image in the discriminant network.
[0139] The generator performs a downsampling operation, where the output feature maps of the third, fourth, and fifth convolutional blocks are connected to the self-attention module; each convolutional block contains batch normalization, activation functions, and convolutional layers.
[0140] In the upsampling part of the generator, bilinear upsampling layers and convolutional layers are used to replace the standard deconvolutional layers, and shallow feature reuse is achieved through skip connections. Self-attention modules are added to the second, third, and fourth layers of the discriminator.
[0141] Specifically, step 4 includes: (See below) Figure 2 The contrast-enhanced CT image is input into the trained first network to obtain the final texture-enhanced CT image. Then, the final texture-enhanced CT image is input into the trained generative discriminative network to obtain the final CT color visualization result.
[0142] This embodiment presents an automatic colorization method for CT images, which fuses lung windows, bone windows, and mediastinal windows through threshold segmentation and morphological post-processing. Compared with traditional medical single-window techniques, this method can simultaneously present multiple density structures such as soft tissue, bone, and lung tissue, expanding the grayscale display level of the region of interest. Key points include: First, the visualization of multiple CT windows does not rely on additional segmentation but is directly defined based on thresholds in the CT HU space. This ensures the medical diagnostic characteristics of CT and allows for setting different fusion thresholds and visualization window values according to specific datasets and the reader's habits. Second, the inversion of the mediastinal window enhances the intensity of organ edges and information about adjacent tissue structures, making the visualized distribution closer to the distribution of real human organs and tissues. Since the intensity values of some tissues and organs in the real human body are negatively correlated with the HU value in computed tomography (CT), for example, fat is represented as a bright color in real human slices, and its histogram distribution is located in the high-brightness region. However, in CT imaging, due to its lower density, fat is distributed in the low-brightness region. Third, processing the region boundaries can adjust abnormal discontinuous grayscale differences to a range suitable for visual analysis.
[0143] The first network, which embeds PET metabolic information, can improve the brightness and enhance the texture details of CT images. Furthermore, combining anatomical and metabolic information helps to enhance tumors. The proposed self-regularized weight map computation, specific to the CT image enhancement task, prompts the network to focus on enhancing dark areas and high-metabolic regions, as these areas are often also lesion sites. This step provides the foundation for generating color medical images with higher overall quality.
[0144] A generative-discriminative network incorporating a self-attention mechanism is constructed. The self-attention mechanism utilizes information from distant regions, combining information from similar or related regions at each location. Using weighted feature summation, it captures geometric or structural features, modeling dependencies across large-scale, multi-level image regions, ensuring regional consistency in the generated images. This overcomes the color bleeding problem often caused by traditional convolutional generative networks, which, due to kernel size limitations, capture more local region relationships and cannot model long-range dependencies. Therefore, introducing a self-attention module into the generator improves image feature extraction, while introducing it into the discriminator improves global image structure judgment, resulting in better detail and improved reconstruction performance. Consequently, the colorization network incorporating the self-attention mechanism improves color consistency within the same organ or tissue and color contrast between organs and tissues, generating more accurate and vibrant colors.
[0145] This embodiment presents an automatic colorization method for CT images. It employs threshold segmentation and morphological methods, adaptively fusing lung, bone, and mediastinal windows based on HU values to overcome the low contrast problem when multiple organs and tissues with significantly different HU values are displayed simultaneously. A first network embedding PET metabolic information prompts it to focus on dark and high-metabolic areas, further improving the brightness and texture detail of the CT image. A generative discriminative network incorporating self-attention mechanisms and perceptual metrics improves color consistency within the same organ and tissue and color contrast between organs and tissues, making the colorization result more in line with human visual preferences. In summary, this method improves color visualization quality through two stages: enhancing image quality before colorization and constructing a task-specific generative discriminative network. The resulting color CT is in line with human visual preferences, realistic, and retains its medical diagnostic significance. It solves the problems of existing colorization methods, such as requiring extensive manual intervention, low image quality, and unreliable color accuracy.
[0146] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0147] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions.
[0148] It should be noted that any reference numerals placed between parentheses in the claims should not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention can be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In claims that enumerate several means, several of these means may be embodied by the same hardware. The use of the terms first, second, third, etc., is merely for convenience of expression and does not indicate any order. These terms can be understood as part of the component names.
[0149] Furthermore, it should be noted that in the description of this specification, the terms "one embodiment," "some embodiments," "embodiment," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.
[0150] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the claims should be interpreted to include both the preferred embodiments and all changes and modifications falling within the scope of the invention.
[0151] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, then this invention should also include these modifications and variations.
Claims
1. An automatic colorization method for CT images, characterized in that, The method includes: Step 1: Obtain contrast-enhanced CT images by preprocessing the CT images; Step 2: Establish a first network for acquiring texture-enhanced CT images, and train the first network based on a pre-acquired training dataset to obtain a trained first network; The training dataset includes multiple pairs of images; each pair of images includes: a training CT image and a training PET image. Step 2 specifically includes: Step 2.1: Obtain the training dataset; preprocess the CT images in each pair of images to obtain the corresponding contrast-enhanced CT images; Obtain the SUV value of the PET image trained in the pair of images, and truncate the SUV value of the trained PET image according to two preset thresholds to obtain the PET image after metabolic suppression. The PET images after metabolic suppression were then resampled to the same isotropic resolution as the contrast-enhanced CT images, resulting in resampled PET images. Gaussian filtering is applied to the resampled PET image to reduce noise, resulting in a denoised PET image. The weight map for each pair of images is obtained using formula (6). ; Formula (6) is: (6); in It is a contrast-enhanced CT image normalized to [0,1]. This is a normalized and denoised PET image; It is the preset influence factor of the denoised PET image in the weight map; Step 2.2: Based on the weight map of each pair of images in the training dataset Train the pre-established first network to obtain the trained first network; The first network is a Zero-DCE network suitable for single-channel medical images; The input to the first network is a weight graph. And contrast-enhanced CT images, weighted maps Multiply the image by all intermediate feature maps of the first network and the output image to output the corresponding parameter map; The input contrast-enhanced CT image is multiplied by the parametric map to obtain the final texture-enhanced CT image; Step 3: Establish a generative discriminant network for colorization of CT images, and train the generative discriminant network based on the pre-acquired initial training set to obtain the trained generative discriminant network; Step 4: Based on the contrast-enhanced CT image, the trained first network, and the trained generative discriminative network, the final CT color visualization result is obtained; Step 4 specifically includes: The contrast-enhanced CT image is input into the trained first network to obtain the final texture-enhanced CT image. Then, the final texture-enhanced CT image is input into the trained generative discriminative network to obtain the final CT color visualization result.
2. The method according to claim 1, characterized in that, Step 1 specifically includes: Step 1.1: Set the fusion threshold for the CT images. and visual window values , , ; Based on a preset fusion threshold The initial lung region mask, the initial mask for other regions excluding the lung and bone, and the initial bone region mask are automatically obtained by segmenting the CT image based on the original HU value. Connectivity and morphological processing are applied to the initial lung region mask, the initial mask for regions other than lungs and bones, and the initial bone region mask to obtain the complete lung region mask. Complete masking of areas other than the lungs and bones and complete bone region mask ; Step 1.2: Visualize the original CT images according to formulas (1), (2), and (3) respectively, and obtain the visualized lung regions. Visual areas excluding lungs and bones Visualized areas of bones ; (1); (2); (3); in It is the original CT image; ; Formula (4) is used to fuse lung visualization regions. Visual areas excluding lungs and bones Visualized areas of bones Obtain the grayscale image of the CT image. ; (4); Step 1.3: Extract the lung visualization region using the Sobel edge detection operator. Visual areas excluding lungs and bones Visualized areas of bones The boundaries are defined by using a preset convolution kernel on the grayscale image of the CT image. The boundaries of the three visualized regions in the image are convolved to obtain a contrast-enhanced CT image; Wherein, the size of the convolution kernel is m×n; m=n=3; where, the convolution kernel .
3. The method according to claim 2, characterized in that, The first network comprises a combination of seven symmetrically connected convolutional layers and activation function layers, with the first six convolutional layers having 64 channels and the last convolutional layer having 4 channels.
4. The method according to claim 3, characterized in that, Step 3 includes: Step 3.1: Obtain the initial training set, which includes multiple color cryosection images of the human body; The initial training set is then subjected to a second processing step to obtain the final training set. Step 3.2: Establish a generator-discriminator network, which includes a generator and a discriminator; Step 3.3: Construct a colorization objective function, and train the system to minimize the colorization objective function; ; Colorization objective function; This represents the input to the discriminant network; Indicates the gold standard; Indicates a generator; Indicates the discriminator; Preset weights; ; This represents the color image generated by the generator; ; Perceived loss; Losses to TV; Step 3.4: Convert each image in the final training set from RGB color space to Lab color space, normalize it, and extract L channel and ab channel data pairs. Use the L luminance channel as input to the generator-discriminator network to predict the ab chrominance channels. Use the original ab channels as labels for supervised training. According to the preset generator-discriminator network training parameters, train the generator and discriminator in the generator-discriminator network alternately until the adversarial relationship between the generator and discriminator reaches a balance, and obtain the trained generator-discriminator network.
5. The method according to claim 4, characterized in that, The second process includes: The color of the cryosol in each human body color cryosection image in the initial training set is set as the background value, and the foreground of each human body color cryosection image is extracted by the labeling method. Then, the brightness and saturation of the human body color cryosection image after the foreground was extracted were enhanced to obtain an enhanced human body color cryosection image; Then, the Region of Interest (ROI) in the enhanced human body color cryosection image is cropped with the foreground as the center, and the cropped ROI in the enhanced human body color cryosection image is scaled to the preset image resolution.
6. The method according to claim 5, characterized in that, The generator is used to extract image features from the input image in the generative discriminant network; The generator performs a downsampling operation, where the output feature maps of the third, fourth, and fifth convolutional blocks are connected to the self-attention module; each convolutional block contains batch normalization, activation functions, and convolutional layers; In the upsampling part of the generator, bilinear upsampling layers and convolutional layers are used to replace the standard deconvolutional layers, and shallow feature reuse is achieved through skip connections. Self-attention modules are added to the second, third, and fourth layers of the discriminator.
Citation Information
Patent Citations
An image coloring method based on a self-attention generative adversarial network
CN109712203A
CT image automatic coloring method based on adversarial learning and adaptive chrominance correction
CN112330790A