A method for texture and detail enhancement of CT images

Abstract

The present invention proposes a method for enhancing the texture and details of CT images, and relates to the technical field of image enhancement. The present invention proposes a CT image data enhancement processing flow, including CT image data collection, CT image data preprocessing, constructing a dynamic convolution extraction module, constructing a dynamic feature enhancement module, constructing a dynamic feature mixing module, constructing a CT image data enhancement model, and training and detecting a CT image data enhancement model; at the same time, a CT image data enhancement model is proposed, wherein the dynamic convolution extraction module is used to extract features in the CT image that are helpful for texture and detail enhancement, the dynamic feature enhancement module is used to enhance significant texture and detail information in a feature map output by the dynamic convolution extraction module, and the dynamic feature mixing module is used to fuse feature maps of different stages output by the dynamic feature enhancement module to enhance the final texture and detail features.

Description

Technical Field

[0001] The invention belongs to the technical field of image enhancement, and in particular relates to a method for enhancing texture and details of CT images. Background Art

[0002] CT imaging technology uses X-rays to penetrate the human body and obtains tomographic images through computer processing, which can provide high-resolution anatomical information. CT images have unique advantages in displaying bones and certain tissue structures, but have limitations in soft tissue contrast, which limits its effectiveness in certain clinical applications. In clinical practice, doctors often need to identify tiny lesions and structural details from CT images; however, due to problems such as noise, artifacts and insufficient contrast, these details are often difficult to identify; therefore, improving the texture and detail information of CT images is of great significance to improving the accuracy and reliability of diagnosis.

[0003] Traditional image enhancement methods, such as filtering, contrast stretching, and histogram equalization, although they improve image quality to a certain extent, often come at the expense of other important information; with the development of deep learning technology, learning-based methods, such as convolutional neural networks and generative adversarial networks, have begun to be applied to the field of image enhancement, but these methods still face challenges when processing medical images, including dependence on labeled data, consumption of computing resources, and limitations on model generalization capabilities.

[0004] With the increasing application of deep learning technology in the field of image enhancement, especially network structures such as attention mechanism and state-space model, new ideas are provided for processing long-distance dependencies and global context information. These technologies can improve the texture and detail information of CT images; in addition, multimodal image fusion technology can improve the accuracy of diagnosis by fusing information from different imaging modalities. Summary of the invention

[0005] The present invention provides a method for enhancing texture and details of CT images, aiming to propose a CT image data enhancement model, comprising a dynamic convolution extraction module DCEM, which effectively extracts features that are conducive to texture and detail enhancement from CT images through a state-space model and a long short-term memory network; a dynamic feature enhancement module DACM, which enhances significant texture and detail information in a feature map output by the dynamic convolution extraction module through a dual-channel attention mechanism; and a dynamic feature mixing module DFUM, which fuses feature maps of different stages output by the dynamic feature enhancement module through deep convolution and self-attention, so as to enhance the final texture and detail features.

[0006] The present invention aims to provide a method for CT image texture and detail enhancement, comprising the following steps.

[0007] S1. CT image data collection: collecting high-resolution CT image data, performing preprocessing operations on the CT images, including image conversion to grayscale, image resolution adjustment to a uniform size, image normalization, image data enhancement, image contrast adjustment, and dividing the preprocessed CT images into a training set and a validation set.

[0008] S2. Construct a dynamic convolution extraction module DCEM, which includes a normalization layer, a state space model, and a long short-term memory network.

[0009] S3. Build a dynamic feature enhancement module DACM, which includes attention operation, pooling layer and ReLU activation function.

[0010] S4. Construct a dynamic feature mixing module DFUM, which includes residual connection, depth convolution and point-by-point convolution.

[0011] S5. Construct a CT image data enhancement model, which includes input, dynamic convolution extraction module DCEM, dynamic feature enhancement module DACM, dynamic feature mixing module DFUM and output.

[0012] S6. Training and testing of CT image data enhancement model. Use the CT image training set to train the CT image data enhancement model. After the training is completed, input the CT image verification set into the CT image data enhancement model to obtain the processing result.

[0013] Preferably, in step S1, a preprocessing operation is performed on the CT image, and the image is converted into a grayscale image to reduce the amount of data processed and focus on the texture and shape information of the CT image; the resolution of the image is adjusted to a uniform size to ensure that all CT image data have the same dimension to facilitate subsequent processing and analysis; the image normalization operation helps the CT image data enhancement model to learn and extract features more stably by scaling the CT image pixel values ​​to a uniform range; the image data enhancement operation improves the generalization ability of the CT image data enhancement model to different CT image changes and increases the robustness of the CT image data enhancement model; the image contrast adjustment operation optimizes the visual effect of the CT image, making the details and structures in the CT image clearer and facilitating feature recognition.

[0014] Preferably, in step S2, the dynamic convolution extraction module DCEM is input with the CT image preprocessed in step S1. , , and are the height and width of the preprocessed CT image, respectively, to ensure that the CT image data is suitable for subsequent image enhancement tasks. Input to the adaptive normalization layer to calculate the entire CT image The mean and standard deviation , , ,in It is a CT image In Location The gray value at each pixel is calculated Adaptive normalization of , get the normalized feature map , ensuring that the pixel values ​​of the CT image are distributed in a uniform range, which helps the CT image data enhancement model to better learn and identify image features. Input into the state space model SSM to get the feature map , which helps to extract more comprehensive features of CT images. Input into the long short-term memory network LSTM to obtain the feature map of the final output of DCEM , which can effectively capture the long-term feature changes in CT images and improve the depth and quality of feature extraction.

[0015] Preferably, in step S3, for the dynamic feature enhancement module DACM, the feature map finally output by the DCEM in step S2 is Defined as , the feature map Input into the dual-channel attention, , through weighted operations, important features are highlighted and unimportant features are suppressed, which can more accurately capture the importance of different features. Through the global average pooling layer, , reducing the spatial dimension of the feature while retaining important feature information to obtain the pooled feature map , the pooled feature map Through a fully connected layer, , is the weight of the fully connected layer, It is the bias of the fully connected layer, providing a linear transformation so that the dynamic feature enhancement module DACM can learn the complex relationship between features and obtain the compressed feature map , the compressed feature map Through the ReLU activation function, , nonlinearity is introduced to help the dynamic feature enhancement module DACM capture nonlinear features and obtain the feature map , calculate the feature map The weight of , and It is the parameter of the weight layer, using the softmax function to calculate the weight Normalize it, , the normalized attention weight Weighted, , It is element-by-element multiplication to obtain the feature map of the final output of DACM .

[0016] Preferably, in step S4, the dynamic feature mixing module DFUM is used to convert the feature map finally output by the DACM in step S3 into Defined as , and It is a feature map The feature maps of different processing stages in and Through the average pooling operation, the spatial size of the feature is reduced, the complexity of the calculation is reduced, and the feature map of the same dimension is obtained. and , the feature maps of the same dimension and Through deep convolution operations, by capturing deeper feature relationships, richer and more abstract feature representations can be extracted. , and obtain the mixed features , the mixed feature Through the self-attention mechanism, the expressiveness of features is improved by strengthening important features and ignoring unimportant features. , and get the enhanced fusion features .

[0017] Preferably, in step S5, for the CT image data enhancement model, the CT image preprocessed in step S1 is input To the dynamic convolution extraction module DCEM, , and are the height and width of the preprocessed CT image, respectively. Through the state space model and long short-term memory network, features that are helpful for texture and detail enhancement are effectively extracted from the CT image to obtain the feature map of the final DCEM output. , the feature map finally output by DCEM in step S2 Defined as , input feature map To the dynamic feature enhancement module DACM, through the dual-channel attention mechanism, the significant texture and detail information in the feature map output by the dynamic convolution extraction module is enhanced to obtain the feature map finally output by DACM , the feature map finally output by DACM in step S3 Defined as , and It is a feature map Feature maps at different processing stages, input feature maps and To the dynamic feature mixing module DFUM, through deep convolution and self-attention, the feature maps of different stages output by the dynamic feature enhancement module are fused to enhance the final texture and detail features to obtain the enhanced fused features .

[0018] Compared with the prior art, the present invention has the following technical effects:

[0019] The technical solution provided by the present invention proposes a CT image data enhancement model, including a dynamic convolution extraction module DCEM, which effectively extracts features that are conducive to texture and detail enhancement from CT images through a state-space model and a long short-term memory network; a dynamic feature enhancement module DACM, which enhances the significant texture and detail information in the feature map output by the dynamic convolution extraction module through a dual-channel attention mechanism; and a dynamic feature mixing module DFUM, which fuses the feature maps of different stages output by the dynamic feature enhancement module through deep convolution and self-attention to enhance the final texture and detail features. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 The present invention provides a CT image enhancement processing flow chart.

[0021] Figure 2 This is a structural diagram of the dynamic convolution extraction module DCEM provided by the present invention.

[0022] Figure 3 It is a structural diagram of the dynamic feature enhancement module DACM provided by the present invention.

[0023] Figure 4 This is a structural diagram of the dynamic feature mixing module DFUM provided by the present invention.

[0024] Figure 5 This is a structural diagram of the CT image data enhancement model provided by the present invention.

[0025] Figure 6 This is an effect diagram of the CT image texture and detail before enhancement provided by the present invention.

[0026] Figure 7 This is the effect diagram of the CT image texture and detail enhancement provided by the present invention. DETAILED DESCRIPTION

[0027] The present invention provides a method for enhancing texture and details of CT images, aiming to propose a CT image data enhancement model, comprising a dynamic convolution extraction module DCEM, which effectively extracts features that are conducive to texture and detail enhancement from CT images through a state-space model and a long short-term memory network; a dynamic feature enhancement module DACM, which enhances significant texture and detail information in a feature map output by the dynamic convolution extraction module through a dual-channel attention mechanism; and a dynamic feature mixing module DFUM, which fuses feature maps of different stages output by the dynamic feature enhancement module through deep convolution and self-attention, so as to enhance the final texture and detail features.

[0028] See also Figure 1 As shown, a method for enhancing texture and details of CT images in an embodiment of the present application.

[0029] S1. CT image data collection: collecting high-resolution CT image data, performing preprocessing operations on the CT images, including image conversion to grayscale, image resolution adjustment to a uniform size, image normalization, image data enhancement, image contrast adjustment, and dividing the preprocessed CT images into a training set and a validation set.

[0030] Furthermore, in step S1, the CT image is preprocessed, the image is converted into a grayscale image using a grayscale algorithm; the image resolution is adjusted to a uniform size using a bilinear interpolation method; the image is normalized using a minimum and maximum normalization method; the image data is enhanced using a data enhancement library provided by PyTorch; and the image contrast is adjusted using an adaptive histogram equalization method.

[0031] S2. Construct a dynamic convolution extraction module DCEM, which includes a normalization layer, a state space model, and a long short-term memory network.

[0032] Furthermore, in step S2, for the dynamic convolution extraction module DCEM, its structure is as follows Figure 2 As shown, the CT image preprocessed in step S1 is input. , , 256 and 256 are the height and width of the preprocessed CT image respectively. Input to the adaptive normalization layer to calculate the entire CT image The mean and standard deviation , , ,in It is a CT image In Location The grayscale value at the position is from 0 to 1. Calculate the grayscale value of each pixel. Adaptive normalization of , get the normalized feature map , the feature map Input into the state space model SSM, the state dimension of the SSM is 256, the observation dimension is 1, and the feature map First, convert it into a feature vector by flattening , the feature vector Input into SSM to obtain the state vector, and the state vector is subjected to inverse flattening operation to obtain the feature map , the feature map Input into the long short-term memory network LSTM, the learning rate of the LSTM is 0.001, the dimension of the hidden layer is 256, and the feature map of the final output of DCEM is obtained .

[0033] S3. Build a dynamic feature enhancement module DACM, which includes attention operation, pooling layer and ReLU activation function.

[0034] Furthermore, in step S3, for the dynamic feature enhancement module DACM, its structure is as follows Figure 3 As shown, the feature map finally output by DCEM in step S2 is Defined as , the feature map Input to the dual-channel attention, the dimension of the attention layer is 64, Through the global average pooling layer, the pooling window size is , , get the feature map after pooling , the pooled feature map Through a fully connected layer, , is the weight of the fully connected layer, is the bias of the fully connected layer, and the compressed feature map is obtained , the compressed feature map Through the ReLU activation function, , get the feature map , calculate the feature map The weight of , and It is the parameter of the weight layer, using the softmax function to calculate the weight Normalize it, , the normalized attention weight Weighted, , It is element-by-element multiplication to obtain the feature map of the final output of DACM .

[0035] S4. Construct a dynamic feature mixing module DFUM, which includes residual connection, depth convolution and point-by-point convolution.

[0036] Furthermore, in step S4, for the dynamic feature mixing module DFUM, its structure is as follows Figure 4 As shown, the characteristic map finally output by DACM in step S3 is Defined as , and It is a feature map The feature maps of different processing stages in and Through the average pooling operation, we get the feature map of the same dimension and , the feature maps of the same dimension and Through the depth convolution operation, , and obtain the mixed features , the mixed feature Through the self-attention mechanism, , and get the enhanced fusion features .

[0037] S5. Construct a CT image data enhancement model, which includes input, dynamic convolution extraction module DCEM, dynamic feature enhancement module DACM, dynamic feature mixing module DFUM and output.

[0038] Furthermore, in step S5, for the CT image data enhancement model, its structure is as follows: Figure 5 As shown, the CT image preprocessed in step S1 is input. To the dynamic convolution extraction module DCEM, , 256 and 256 are the height and width of the preprocessed CT image, respectively, and the feature map of the final DCEM output is obtained. , the feature map finally output by DCEM in step S2 Defined as , input feature map Go to the dynamic feature enhancement module DACM to get the feature map of the final output of DACM , the feature map finally output by DACM in step S3 Defined as , and It is a feature map Feature maps at different processing stages, input feature maps and To the dynamic feature mixing module DFUM, get the enhanced fusion features .

[0039] Furthermore, in step S5, for the CT image data enhancement model, it is written in Python language, using the Pytorch framework, using the cross entropy loss function, using the stochastic gradient descent optimizer, with a learning rate of 0.001, a batch size of 32, and 100 iterations.

[0040] S6. Training and testing of CT image data enhancement model. Use the CT image training set to train the CT image data enhancement model. After the training is completed, input the CT image verification set into the CT image data enhancement model to obtain the processing result.

[0041] Furthermore, in step S6, for CT image data enhancement model training and detection, the CT image data set preprocessed in step S1 is used as a training set, input into the constructed CT image data enhancement model, the loss function is calculated, and the network weights are adjusted according to the loss to minimize the prediction error of the model; after the training is completed, the model is evaluated using the validation set divided in step S1, and the CT images in the validation set are input into the trained CT image data enhancement model. The model will output the enhanced image. The effect diagram of the CT image texture and detail before enhancement is as shown in the figure. Figure 6 As shown in the figure, the effect of CT image texture and detail enhancement is as follows Figure 7 shown.

[0042] The above are only preferred embodiments of the present invention. It should be pointed out that a person skilled in the art can make several modifications and improvements without departing from the inventive concept of the present invention, and these all fall within the protection scope of the present invention.

Claims

1. A method for CT image texture and detail enhancement, characterized in that: The following steps are involved: S1. CT image data collection: collecting high-resolution CT image data, performing preprocessing operations on the CT images, including image conversion to grayscale, image resolution adjustment to a uniform size, image normalization, image data enhancement, image contrast adjustment, and dividing the preprocessed CT images into a training set and a validation set; S2, construct a dynamic convolution extraction module DCEM, including a normalization layer, a state space model and a long short-term memory network. For the dynamic convolution extraction module DCEM, input the CT image I preprocessed in step S1, I∈R H×W×1 , H and W are the height and width of the preprocessed CT image, respectively. The CT image I is input into the adaptive normalization layer to calculate the mean μ and standard deviation σ of the entire CT image I. Where I(x, y) is the grayscale value of the CT image I at position (x, y), and the adaptive normalization of each pixel I(x, y) is calculated. The normalized feature map X is obtained, and the feature map X is input into the state space model SSM to obtain the feature map X SSM , the feature map X SSM Input into the long short-term memory network LSTM to obtain the feature map X of the final output of DCEM out ; S3, construct a dynamic feature enhancement module DACM, including an attention operation, a pooling layer and a ReLU activation function. For the dynamic feature enhancement module DACM, the feature map X finally output by the DCEM in step S2 is converted into out Defined as F, the feature map F is input into the dual-channel attention, F passes through the global average pooling layer, P = GAP (F), and the pooled feature map P is obtained. The pooled feature map P passes through a fully connected layer, S = W s P+b,W S is the weight of the fully connected layer, b is the bias of the fully connected layer, and the compressed feature map S is obtained. The compressed feature map S is activated by the ReLU function, A = ReLU (S), and the feature map A is obtained. The weight of the feature map A is calculated, W′ = W w ·A+b w , W w and b w is the parameter of the weight layer. The weight W′ is normalized using the softmax function, α=softmax(W′). The normalized attention weight α is weighted, F attn =Fθα, θ is the element-by-element multiplication, and the feature map F of the final output of DACM is obtained attn ; S4, construct a dynamic feature mixing module DFUM, including residual connection, depth convolution and point-by-point convolution. For the dynamic feature mixing module DFUM, the feature map F finally output by DACM in step S3 is converted into attn Defined as N, N1 and N2 are feature maps at different processing stages in feature map N, the feature maps N1 and N2 are subjected to average pooling operation to obtain feature maps N1 and N2 of the same dimension, the feature maps N1 and N2 of the same dimension are subjected to deep convolution operation, M = DwConv(N1) + DwConv(N2), to obtain a mixed feature M, the mixed feature M is subjected to a self-attention mechanism, M′ = SelfAttention(M), to obtain an enhanced fused feature M′; S5, constructing a CT image data enhancement model, including input, dynamic convolution extraction module DCEM, dynamic feature enhancement module DACM, dynamic feature mixing module DFUM and output; S6. Training and testing of CT image data enhancement model. Use the CT image training set to train the CT image data enhancement model. After the training is completed, input the CT image verification set into the CT image data enhancement model to obtain the processing result.

2. A method for CT image texture and detail enhancement according to claim 1, characterized in that: In step S5, for the CT image data enhancement model, the CT image I preprocessed in step S1 is input into the dynamic convolution extraction module DCEM, I∈R H×W×1 , H and W are the height and width of the preprocessed CT image, respectively, and the feature map X of the final DCEM output is obtained out , the feature map X finally output by DCEM in step S2 out Defined as F, input feature map F to the dynamic feature enhancement module DACM, and obtain the feature map F of the final output of DACM attn , the feature map F finally output by DACM in step S3 attn Defined as N, N1 and N2 are feature maps at different processing stages in feature map N. Feature maps N1 and N2 are input to the dynamic feature mixing module DFUM to obtain the enhanced fused feature M′.

Images