A method for deblurring of digital core CT images

Abstract

The present application relates to the technical field of image processing, in particular to a kind of digital core CT image deblurring method, by setting a two-stage network based on deep learning, the first stage of network can obtain rich context information, and by setting with transition supervision module, inhibit the irrelevant information extracted, only useful information is transitioned to next stage, to improve the performance of network, the second stage of network is concerned with important feature channel while paying attention to spatial detail information, make up the problem that network generalization ability is insufficient, cannot handle the task that depends on fine spatial information, by global feature fusion network fusion the features extracted in two stages, so that the context features and channel and spatial detail information extracted by two stages respectively can be better fused together, the present application effectively improves the peak signal-to-noise ratio after image deblurring, the image quality obtained is good, close to original image.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method for deblurring digital core CT images. Background Technology

[0002] Microscopic CT scanning has been applied to study various geological problems, allowing the generation of two-dimensional representations of the internal microstructure of materials without damaging the sample, simulating physical processes within rocks, and thus calculating their physical properties. Despite significant advancements in imaging capabilities and image processing algorithms, acquiring high-quality data from images remains challenging and time-consuming. Rock samples can take days to weeks to acquire. Image artifacts such as blurring further complicate the determination of rock properties using imaging methods. Therefore, image deblurring techniques are needed to remove blur artifacts from digital core images and reconstruct high-quality digital core CT scan images.

[0003] Most current deep learning-based methods use encoder-decoder structures to remove blur from images. Encoder-decoder methods acquire a larger receptive field through multiple downsampling operations, thus obtaining broader contextual information and enhancing image processing and recognition performance. However, multiple downsampling operations inevitably lead to poor handling of spatial details in the image. Some methods employ multi-stage networks, decomposing the difficult image restoration task into multiple smaller, easier sub-tasks, progressively reconstructing a high-quality image through each sub-task. However, when extracted features propagate between multiple stages, suboptimal results are inevitable. Furthermore, multi-stage methods generally have excessively large network models, consuming significant computational resources. In recent years, the Transformer has seen rapid development, with more and more researchers applying it to low-level computer vision tasks like image restoration. Transformer-based models have demonstrated remarkably powerful performance in image deblurring. As an architecture based on a self-attention mechanism, the Transformer, through key features such as global perception, capturing long-range dependencies, multi-layered stacked encoder-decoder structures, and positional encoding, can more finely reconstruct detailed information in images while maintaining the overall structural integrity of the image, thereby improving visual quality and detail restoration capabilities. However, the scale dot product attention in Transformers typically requires complex matrix multiplication, with both space and time complexity being quadratic. That is, state-of-the-art results rely on a large number of parameters, heavy computation, and high MAC (matrix accuracy), which is undesirable for resource-constrained devices. Summary of the Invention

[0004] In view of this, the purpose of this invention is to propose a method for deblurring digital core CT images to solve the problem that existing methods do not handle spatial details well.

[0005] To achieve the above objectives, the present invention provides a method for deblurring digital core CT images, comprising the following steps:

[0006] The acquired digital core image set was preprocessed, and the preprocessed digital core image set was divided into training set, validation set and test set.

[0007] The training set was used to train the created digital core CT image deblurring network;

[0008] The digital core CT image to be processed is input into a trained digital core CT image deblurring network for deblurring to obtain a deblurred digital core CT image.

[0009] The process of the digital core CT image deblurring network processing includes:

[0010] Extract broad semantic information from the image to obtain a semantic feature map;

[0011] Extract the channel features and spatial detail information of the image to obtain the channel and spatial feature maps;

[0012] The channel feature map is concatenated with the spatial feature map and the semantic feature map along the channel dimension;

[0013] The stitched feature information is then subjected to global feature fusion to output a deblurred digital core CT image.

[0014] Preferably, the digital core CT image deblurring network includes a first stage and a second stage. The first stage includes an encoder-decoder subnetwork and a transition supervision module. The encoder-decoder subnetwork is used to extract broad semantic information of the image to obtain a semantic feature map. The transition supervision module is used to transfer the features extracted in the first stage to the second stage and suppress irrelevant information.

[0015] Preferably, the encoder-decoder subnetwork includes an encoder, a decoder, and intermediate blocks. The encoder is used to extract broad semantic features, and the decoder is used to recover detailed information of the image. Each encoder and decoder module internally includes layer normalization, convolutional layers, gating mechanisms, and channel attention. The gating mechanism is used to capture long-term dependencies, and the channel attention is used to capture broad contextual semantic features.

[0016] Preferably, the second stage includes a channel and spatial attention module and a global feature fusion network. The channel and spatial attention module includes a parallel channel attention module and a spatial attention module, which obtain channel attention and spatial attention respectively, and fuse them together to obtain a channel and spatial feature map. The obtained channel and spatial feature map is then stitched together with the semantic feature map obtained in the first stage. The global feature fusion network is used to fuse the stitched channel and spatial feature map and the semantic feature map to obtain a deblurred digital core CT image.

[0017] Preferably, the process by which the attention module obtains spatial attention includes:

[0018] The corresponding features are obtained by performing average pooling and max pooling operations respectively: and The two obtained features are concatenated to obtain the spatial information F of the feature map. s Spatial attention is:

[0019] F s =S(Conv 7×7 ([f Avg ;f Max ]))

[0020] Among them, F s For spatial characteristics, S represents the Sigmoid function, Conv 7×7 This indicates that the kernel size for the convolution operation is 7×7, f Avg f is the feature map obtained after average pooling. Max For the feature map obtained after max pooling, [f Avg ;f Max The symbol ] indicates that the two are being joined together. The feature map is represented by H, where H represents height and W represents width.

[0021] Preferably, the process by which the channel attention module obtains channel attention includes:

[0022] Average pooling is used to obtain channel attention:

[0023] F c =S(MLP(f) Avg ))

[0024] Among them, F c For channel features, MLP stands for Multilayer Perceptron, f Avg The feature map is obtained after average pooling;

[0025] The process of obtaining channel and spatial feature maps includes:

[0026] By setting spatial attention and channel attention in parallel, the resulting Fs and F c The concatenated features, after convolution, are added to the initial feature map to obtain the channel and spatial feature maps:

[0027] F CASB =F+Conv([f Avg ;f Max ])

[0028] Among them, F CASB F is the feature map obtained after passing through the channel and spatial attention branches, and F is the feature map of the initial input. Avg f is the feature map obtained after average pooling. Max This is the feature map obtained after max pooling.

[0029] Preferably, the processing procedure of the transition supervision module is as follows:

[0030]

[0031] Among them, F T X is the feature map obtained after passing through the transition supervision module. u X represents the feature map output by the encoder-decoder subnetwork, where X is the original feature map, Conv is the convolution operation, and S is the sigmoid function. This is matrix multiplication.

[0032] Preferably, the global feature fusion network includes two stages: channel attention and spatial attention. The features from the two stages are concatenated along the channel dimension and first pass through the channel attention module, which focuses more on the relationship between channels to achieve better feature fusion. Then, it passes through the spatial attention module to further generate high-resolution features with rich spatial details.

[0033] Preferably, the AdamW optimizer is used to train the created digital core CT image deblurring network. The relevant hyperparameters of the optimizer are set to β1 = 0.9, β2 = 0.99, the initial learning rate is set to 1e-3, and a cosine annealing strategy is used to stably decay to 1e-7.

[0034] The beneficial effects of this invention are:

[0035] This invention employs a deep learning-based two-stage network. The first stage of the network acquires rich contextual information and, through a transition supervision module, suppresses irrelevant information, ensuring that only useful information is passed to the next stage, thereby improving network performance. The second stage focuses on both important feature channels and spatial detail information, compensating for the network's insufficient generalization ability and inability to handle tasks dependent on fine spatial information. A global feature fusion network integrates the features extracted from both stages, enabling better fusion of the contextual features and channels extracted in each stage with spatial detail information. This invention effectively improves the peak signal-to-noise ratio of the deblurred image, resulting in a high-quality image that closely resembles the original image. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only for this invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0037] Figure 1 This is a schematic diagram of the digital core CT image deblurring process according to an embodiment of the present invention;

[0038] Figure 2 This is a diagram of the digital core CT image deblurring network architecture according to an embodiment of the present invention;

[0039] Figure 3 This is a diagram of the encoding / decoding sub-network structure according to an embodiment of the present invention;

[0040] Figure 4 This is a structural diagram of the channel and spatial attention module according to an embodiment of the present invention;

[0041] Figure 5 This is a structural diagram of the transition supervision module according to an embodiment of the present invention;

[0042] Figure 6 This is a diagram of the global feature fusion network structure according to an embodiment of the present invention. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments.

[0044] It should be noted that, unless otherwise defined, the technical or scientific terms used in this invention should have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0045] like Figure 1 As shown in the embodiments of this specification, a method for deblurring digital core CT images is provided, including the following steps:

[0046] The acquired digital core image set was preprocessed. The clear images in the digital core dataset have a resolution of 500*500. Blur was added to these images to create corresponding low-quality digital core images with blurriness. Then, the images were cropped to 256*256 pixels, and the cropped images were randomly rotated by 90°, 180°, and 270° to enhance the robustness of the network.

[0047] The preprocessed digital core image set was divided into a training set, a validation set, and a test set.

[0048] The pre-built digital core CT image deblurring network was trained using a training set. The network is end-to-end trainable and employs the AdamW optimizer with hyperparameters set to β1 = 0.9 and β2 = 0.99. Compared to the Adam optimizer, which adds L2 regularization to the gradient before updating weight decay, potentially affecting gradient accuracy, the AdamW optimizer applies weight decay after parameter updates, achieving more accurate decay. Therefore, the AdamW optimizer solves the weight decay problem, improving the model's generalization ability. The initial learning rate was set to 1e-3, and cosine annealing was used to stably decay it to 1e-7.

[0049] The digital core CT image to be processed is input into a trained digital core CT image deblurring network for deblurring to obtain a deblurred digital core CT image.

[0050] The process of the digital core CT image deblurring network processing includes:

[0051] Extract broad semantic information from the image to obtain a semantic feature map;

[0052] Extract the channel features and spatial detail information of the image to obtain the channel and spatial feature maps;

[0053] The channel feature map is concatenated with the spatial feature map and the semantic feature map along the channel dimension;

[0054] The stitched feature information is then subjected to global feature fusion to output a deblurred digital core CT image.

[0055] like Figure 2 As shown, the network is divided into two stages. The first stage consists of an encoding / decoding network and a transition supervision module. The encoding / decoding network is as follows: Figure 3 As shown, the network consists of an encoder, a decoder, and intermediate blocks, with the encoder and corresponding decoder connected by skip connections. The transition supervision module, located behind the encoder-decoder network, is responsible for better transferring the features extracted by the encoder-decoder network to the second stage. The second stage of the network consists of a channel and spatial attention module and a global feature fusion network. The second stage first extracts channel features and spatial detail information from the original image, then concatenates them along the channel dimension with the features from the first stage passed from the transition supervision module, and then feeds them together into the global feature fusion network to further extract spatial and channel information, resulting in better feature fusion.

[0056] The processing of the channel and spatial attention module: The first-stage encoding / decoding structure uses downsampling operations multiple times, which can obtain broad semantic features, but it is not good at handling spatial details. Therefore, the second stage focuses on both spatial and channel information, and concatenates and fuses the acquired features with the broad contextual features extracted in the first stage to compensate for the shortcomings of the first stage. If only channel information is focused on without attention to spatial details, the network's generalization ability will be insufficient, and it will be unable to handle tasks that depend on fine spatial information. However, focusing on spatial information while ignoring channel information will limit the extracted features, and the model can only consider local information, lacking flexibility when processing multi-channel data, making it difficult for the network to understand the relationships between different channels. The structure of the channel and spatial attention module is as follows: Figure 4 As shown. The upper part of the module focuses more on spatial details, that is, the relationship between different spatial locations within the same channel. We perform average pooling and max pooling operations respectively to obtain the corresponding features: and Then, the two features are concatenated together to obtain the spatial information F of the feature map. s Spatial attention can be represented as:

[0057] F s =S(Conv 7×7 ([fAvg ;f Max ]))

[0058] Among them, F s For spatial characteristics, S represents the Sigmoid function, Conv 7×7 This indicates that the kernel size for the convolution operation is 7×7, f Avg f is the feature map obtained after average pooling. Max For the feature map obtained after max pooling, [f Avg ;f Max The symbol ] indicates a concatenation operation between the two. By focusing on spatial details, our model can assign different weights to each pixel in the feature map, thus paying more attention to regions that are highly relevant to the task and ignoring unimportant regions.

[0059] The lower half of the module focuses more on the relationships between different channels. By assigning different weights to each channel, the model can concentrate on information that is more critical to the current task. We use average pooling to obtain our channel attention, which can be represented as:

[0060] F c =S(MLP(f) Avg ))

[0061] Among them, F c For channel features, MLP stands for Multilayer Perceptron, f Avg This is the feature map obtained after average pooling.

[0062] We set up spatial attention and channel attention in parallel, and obtained F s and F c The concatenations are then performed, and the resulting data is added to the initial feature map to obtain our channel and spatial attention branches:

[0063] F CASB =F+Conv([f Avg ;f Max ])

[0064] Among them, F CASB F is the feature map obtained after passing through the channel and spatial attention branches, and F is the feature map of the initial input. Avg f is the feature map obtained after average pooling. Max This is the feature map obtained after max pooling.

[0065] The processing procedure of the transition supervision module:

[0066] In multi-stage networks, simply passing semantic features from the previous stage directly to the next stage without additional processing does not significantly improve network performance. We designed a transition supervision module after the encoder-decoder network. This module takes both the features output from the encoder-decoder network and the original features as input, thereby enhancing feature extraction capabilities. Afterwards, a sigmoid function is applied to add non-linear expression. The transition supervision module can be represented as:

[0067]

[0068] Among them, F T X is the feature map obtained after TSB. u Here, X represents the feature map output by U-Net, Conv is the convolution operation, and S is the sigmoid function. This involves matrix multiplication. Through a transitional supervision module, semantic features that are unimportant to image restoration can be suppressed, ensuring that only useful information is passed to the next stage.

[0069] The processing procedure of the global feature fusion network: Directly concatenating the broad semantic information and channel and spatial detail features obtained from the first and second stages respectively does not significantly improve network performance. To fuse the features extracted from the two stages, we designed a global feature fusion network. The network consists of a global feature fusion module, which internally also has two stages: channel attention and spatial attention, but these are connected in series. Its specific structure is as follows: Figure 6 As shown, the features from the two stages are concatenated along the channel dimension and first pass through a channel attention module, which focuses more on the relationships between channels to better fuse the features. Then, they pass through a spatial attention module to further generate high-resolution features rich in spatial details. The global feature fusion network effectively preserves the integrity of the features; contextual features, channels, and spatial details are better integrated, making the network less susceptible to information loss.

[0070] Digital core images primarily comprise two sub-image categories: carbonate rocks and sandstone. We used 4,000 pairs of carbonate rock images and 4,000 pairs of sandstone images as our dataset. These images are all CT scan images, with each image pair consisting of a clear digital core CT scan image and a corresponding low-quality image containing blur. 80% of the image pairs were used as the training set, 10% as the validation set, and 10% as the test set. The validation set effectively prevents overfitting, while the test set effectively measures the generalization ability of the network model. We used Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity (SSIM) to evaluate the effectiveness of our approach.

[0071] 1. To verify the effectiveness of our proposed solution, we first used a baseline model as a starting point and then gradually introduced improvement measures. We first used the encoder-decoder network as the baseline model, and then added components such as the proposed channel and spatial attention module, transition supervision module, and global feature fusion network separately. The PSNR was improved by 0.0243, 0.0999, and 0.1091, respectively. The results are shown in the table below:

[0072] Table 1

[0073] Baseline model 28.9481 0.8219 Add channel and space attention module 28.9724 0.8204 Add transition monitoring module 29.0480 0.8236 Add a global feature fusion network 29.0572 0.8240

[0074] 2. To more clearly demonstrate the importance of each component in our solution, we added all modules to the baseline model to obtain the effect of the complete solution. Then, we added the channel and spatial attention module combined with the transition supervision module, and the channel and spatial attention module combined with the global feature fusion network, as well as the transition supervision module combined with the global feature fusion network, to the baseline model. In other words, compared to the complete solution, we reduced only one component before testing the effect. The results are shown in Table 2. Adding the global feature fusion network is better than simply concatenating the context features and channel and spatial details obtained from the two stages, improving the PSNR by 0.1423. Adding the transition supervision module is better than directly inputting the features extracted in the first stage into the second stage, improving the PSNR by 0.2006.

[0075] Table 2

[0076]

[0077]

[0078] 3. To measure the advancement of our proposed solution, we trained and tested our network using both carbonate rock and sandstone datasets. To evaluate the performance of our network, we trained and tested it with state-of-the-art image restoration networks such as NAFNet using the same configuration, and compared our network with their performance. The comparison results are shown in Table 3. Our model achieves a PSNR 0.1405 dB higher than NAFNet on the carbonate rock dataset and a PSNR 0.0102 dB higher than NAFNet on the sandstone dataset.

[0079] Table 3

[0080] MIMO-UNet carbonate rocks 28.4343 0.8024 MPRNet carbonate rocks 28.7002 0.8146 NAFNet carbonate rocks 29.0581 0.8239 This invention network carbonate rocks 29.1986 0.8297 MIMO-UNet sandstone 30.8642 0.7904 MPRNet sandstone 31.4357 0.8603 NAFNet sandstone 31.8333 0.8147 This invention network sandstone 31.8435 0.8157

[0081] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of the invention (including the claims) is limited to these examples; within the framework of the invention, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of the different aspects of the invention as described above, which are not provided in the details for the sake of brevity.

[0082] This invention is intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for deblurring digital core CT images, characterized in that, The method includes the following steps: The acquired digital core image set was preprocessed, and the preprocessed digital core image set was divided into training set, validation set and test set. The training set was used to train the created digital core CT image deblurring network; The digital core CT image to be processed is input into a trained digital core CT image deblurring network for deblurring to obtain a deblurred digital core CT image. The process of the digital core CT image deblurring network processing includes: Extract broad semantic information from the image to obtain a semantic feature map; Extract the channel features and spatial detail information of the image to obtain the channel and spatial feature maps; The channel feature map is concatenated with the spatial feature map and the semantic feature map along the channel dimension; The stitched feature information is then subjected to global feature fusion to output a deblurred digital core CT image. The digital core CT image deblurring network includes a first stage and a second stage. The first stage includes an encoder-decoder subnetwork and a transition supervision module. The encoder-decoder subnetwork is used to extract broad semantic information of the image to obtain a semantic feature map. The transition supervision module is used to transfer the features extracted in the first stage to the second stage and suppress irrelevant information. The encoder-decoder subnetwork includes an encoder, a decoder, and intermediate blocks. The encoder is used to extract broad semantic features, and the decoder is used to recover detailed information of the image. Each encoder and decoder module internally includes layer normalization, convolutional layers, gating mechanisms, and channel attention. The gating mechanism is used to capture long-term dependencies, and the channel attention is used to capture broad contextual semantic features. The second stage includes a channel and spatial attention module and a global feature fusion network. The channel and spatial attention module includes a parallel channel attention module and a spatial attention module, which obtain channel attention and spatial attention respectively, and then fuse them together to obtain a channel and spatial feature map. The obtained channel and spatial feature map is then spliced ​​with the semantic feature map obtained in the first stage. The global feature fusion network is used to fuse the spliced ​​channel and spatial feature map and the semantic feature map to obtain a deblurred digital core CT image.

2. The method for deblurring digital core CT images according to claim 1, characterized in that, The process by which the spatial attention module obtains spatial attention includes: The corresponding features are obtained by performing average pooling and max pooling operations respectively: and The two features obtained are concatenated to obtain the spatial information of the feature map. Spatial attention is: in, For spatial characteristics, S represents the Sigmoid function. The kernel size of the convolution operation is . , The feature map is obtained after average pooling. The feature map obtained after max pooling. This indicates that a splicing operation is being performed on the two. Representing feature maps, Indicates high, It indicates width.

3. The method for deblurring digital core CT images according to claim 2, characterized in that, The process by which the channel attention module obtains channel attention includes: Average pooling is used to obtain channel attention: in, For channel features, MLP stands for Multilayer Perceptron. The feature map is obtained after average pooling; The process of obtaining channel and spatial feature maps includes: Setting spatial attention and channel attention in parallel will yield the following results. and The concatenated features, after convolution, are added to the initial feature map to obtain the channel and spatial feature maps: in, F represents the feature map obtained after passing through the channel and spatial attention branches, and F represents the initial input feature map. The feature map is obtained after average pooling. This is the feature map obtained after max pooling.

4. The method for deblurring digital core CT images according to claim 1, characterized in that, The processing procedure of the transition supervision module is as follows: in, The feature map obtained after passing through the transition supervision module. The feature map output by the encoding / decoding sub-network. This is the original feature map, where Conv is the convolution operation and S is the sigmoid function. This is matrix multiplication.

5. The method for deblurring digital core CT images according to claim 1, characterized in that, The global feature fusion network includes two stages: channel attention and spatial attention. The features from the two stages are concatenated along the channel dimension and first pass through the channel attention module. By paying more attention to the relationship between channels, the features are better fused. Then, they pass through the spatial attention module to further generate high-resolution features with rich spatial details.

6. The method for deblurring digital core CT images according to claim 1, characterized in that, The AdamW optimizer was used to train the created digital core CT image deblurring network. The relevant hyperparameters of the optimizer were set to β1 = 0.9, β2 = 0.99, the initial learning rate was set to 1e-3, and a cosine annealing strategy was used to stably decay to 1e-7.

Images