A terahertz image super-resolution reconstruction method based on an improved residual network

Abstract

This invention relates to a terahertz image super-resolution reconstruction method based on an improved residual network, comprising: acquiring original terahertz image data; inputting the original terahertz image data into a trained reconstruction model to obtain a super-resolution processed image; the reconstruction model extracting features from the original terahertz image data using an improved residual network, mapping the extracted feature maps to a target-scale feature space, and introducing an attention mechanism to calibrate feature weights in the target-scale feature space; performing feature fusion on the calibrated feature maps; and generating a target-scale terahertz super-resolution image based on the fused feature maps. This invention can effectively recover high-frequency details and texture information in terahertz images, improve image resolution, and reduce computational resource consumption.

Description

Technical Field

[0001] This invention relates to the field of terahertz imaging technology, and in particular to a terahertz image super-resolution reconstruction method based on an improved residual network. Background Technology

[0002] Terahertz (THz) imaging technology, with its unique physical properties, has shown great potential in fields such as biomedical imaging and security inspection. However, due to limitations such as long wavelength, diffraction effects, and system noise, the image quality of terahertz images is generally low, making resolution improvement a key challenge.

[0003] To address this problem, existing optimization methods mainly fall into two categories: improvements based on optical systems and enhancements based on numerical algorithms. Optical system optimization, represented by near-field imaging technology, breaks through the diffraction limit through methods such as probe scanning and dynamic aperture, while further improving imaging performance with the help of high-quality detectors, novel lens designs, and metamaterial devices. However, these methods often face limitations such as high cost and slow speed.

[0004] In contrast, optimization methods based on numerical algorithms are more flexible and economical, mainly including traditional interpolation methods and reconstruction methods. Interpolation methods achieve resolution improvement through resampling, while reconstruction methods are divided into two categories: multi-image super-resolution (MISR) and single-image super-resolution (SISR). Among them, MISR methods such as POCS and IBP, as well as SISR techniques such as Lucy-Richardson deconvolution, have achieved certain results. However, traditional algorithms generally suffer from problems such as high computational cost and strong parameter dependence.

[0005] In recent years, the rapid development of deep learning has provided new methods and ideas for terahertz super-resolution imaging. In particular, residual networks can effectively solve the gradient vanishing problem in deep network training and perform well in high-level computer vision tasks such as image classification and object detection. However, when directly applied to terahertz image super-resolution, the results are not outstanding, and there are problems such as low computational efficiency. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to provide a terahertz image super-resolution reconstruction method based on an improved residual network, which can effectively recover high-frequency details and texture information in terahertz images, improve image resolution, and reduce the consumption of computing resources.

[0007] The technical solution adopted by this invention to solve its technical problem is: to provide a terahertz image super-resolution reconstruction method based on an improved residual network, comprising:

[0008] Obtain the raw terahertz image;

[0009] The original terahertz image is input into the trained reconstruction model to obtain the corresponding super-resolution image. The reconstruction model extracts features from the original terahertz image through an improved residual network, maps the extracted feature map to the target scale feature space, and introduces an attention mechanism to calibrate the feature weights in the target scale feature space. After feature fusion of the calibrated feature map, the target scale super-resolution image is generated based on the fused feature map.

[0010] Furthermore, the improved residual network is achieved by removing the batch normalization layer from the basic residual network and removing the ReLU activation function outside the residual block.

[0011] Furthermore, in the improved residual network, a constant scaling layer is placed after the last convolutional layer of each residual block.

[0012] Furthermore, the scaling factor of the constant scaling layer is 0.1.

[0013] Furthermore, in the improved residual network, the number of residual blocks is set to 32, and the number of filters is set to 256.

[0014] Furthermore, the feature weight calibration in the target scale feature space includes:

[0015] After performing average pooling and max pooling operations on the extracted feature map in the spatial dimension, one-dimensional vectors are extracted from each of them.

[0016] Based on the extracted one-dimensional vector, the shared network is used to sum the elements of each feature descriptor, and the resulting feature vector is used to generate a channel weight vector through the sigmoid function.

[0017] Furthermore, the mapping of the extracted feature map to the target scale feature space is achieved through an upsampling layer.

[0018] Furthermore, the reconstruction model is trained using the following method:

[0019] Obtain high-resolution image datasets;

[0020] After performing convolution and downsampling operations on the high-definition images in the high-definition image dataset using a Gaussian blur kernel, Gaussian noise of a set intensity is added to obtain the corresponding degraded image.

[0021] The degraded image is input into the reconstruction model, and the VGG19 network is used to extract the image features of the super-resolution image output by the model and the corresponding high-resolution image to form image feature pairs.

[0022] The reconstruction model is tuned based on the mean square error between image feature pairs.

[0023] Furthermore, during training, the reconstruction model is first pre-trained for a reconstruction task at 2x scale of the original terahertz image. After the model converges, the obtained weights are transferred to the initialization process for training on reconstruction tasks at 3x and 4x scale.

[0024] Furthermore, the reconstructed model was tested using the following methods:

[0025] The super-resolution image output by the reconstructed model is converted to the YCbCr color space;

[0026] The PSNR and SSIM of the transformed image are calculated for the Y channel to evaluate the quality of the super-resolution image output by the model.

[0027] Beneficial effects

[0028] By adopting the above-mentioned technical solutions, this invention has the following advantages and positive effects compared with the prior art: By combining the adaptive residual network with the channel attention mechanism, this invention can effectively recover high-frequency details and texture information in terahertz images, improve image resolution, and reduce the consumption of computing resources. In addition, by optimizing the classic residual network for the characteristics of terahertz images, it can accurately adapt to the imaging characteristics of low signal-to-noise ratio and easy loss of texture in dark areas of terahertz images, and enhance the super-resolution reconstruction effect. Attached Figure Description

[0029] Figure 1 This is a schematic diagram of a terahertz image acquisition device according to an embodiment of the present invention;

[0030] Figure 2 This is a schematic diagram of the deep learning model structure according to an embodiment of the present invention;

[0031] Figure 3 This is a schematic diagram of the residual block and upsampling module structure according to an embodiment of the present invention;

[0032] Figure 4 This is a schematic diagram of the channel attention mechanism according to an embodiment of the present invention;

[0033] Figure 5 This is a schematic diagram of the deep learning model training process according to an embodiment of the present invention;

[0034] Figure 6 This is a comparison image of the original terahertz image and the reconstructed terahertz image according to an embodiment of the present invention. Detailed Implementation

[0035] The present invention will be further illustrated below with reference to specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. Furthermore, it should be understood that after reading the teachings of this invention, those skilled in the art can make various alterations or modifications to the invention, and these equivalent forms also fall within the scope defined by the appended claims.

[0036] The embodiments of the present invention relate to a terahertz image super-resolution reconstruction method based on an improved residual network, comprising:

[0037] Obtain the raw terahertz image;

[0038] The original terahertz image is input into the trained deep learning model to obtain the corresponding super-resolution image.

[0039] like Figure 1 As shown, the terahertz image acquisition device used in this embodiment consists of a TTL array detector, a terahertz lens, a sample stage, a terahertz light source, and an imaging camera. It is used to acquire initial low-resolution terahertz images as the input data source for subsequent super-resolution networks.

[0040] like Figure 2 As shown, the deep learning model mainly consists of an adaptive residual network, an upsampling module, an attention module, and a post-processing module. The adaptive residual network primarily addresses the information loss and efficiency issues in terahertz image super-resolution processing, enabling the recovery of high-frequency details, suppression of noise interference, and reduction of computational resource consumption under low signal-to-noise ratio conditions. The channel attention module focuses on the features of important regions in the terahertz image along the channel dimension. Through the inter-channel dependencies, it dynamically calibrates the importance of each channel, enhances the response of key feature channels, and appropriately strengthens them.

[0041] The adaptive residual network infrastructure makes the following improvements to residual networks:

[0042] 1. Remove the batch normalization layer from the network. Since the batch normalization layer globally standardizes network features, removing it can preserve the flexibility of the feature range, which is crucial for maintaining the natural texture and edge contrast of dark areas in terahertz images.

[0043] 2. Remove the ReLU activation function outside the residual block to remove the output constraint of the model, so as to better preserve the contrast and texture features of the dark parts of the image. At the same time, using a linear activation function can enable the network to learn a direct mapping from low resolution to high resolution.

[0044] 3. In convolutional neural networks, model performance can be improved by increasing the network depth (B) or the number of filters (F). When computational resources are limited, increasing F is more effective in maximizing model capacity. However, excessively increasing the number of feature maps can lead to unstable training values. In the extended baseline model, setting B=32, F=256, and inserting a constant scaling layer (scaling factor 0.1) after the last convolutional layer of each residual block significantly improved training stability.

[0045] Specifically, such as Figure 3 As shown, where, Figure 3 (a) represents the original residual block. Figure 3 (b) is an adaptive residual block. Figure 3 (c) An adaptive residual block with a scaling layer added. Figure 3 (d) is the upsampling module.

[0046] The structure of the attention module is shown in Figure 4. The channel attention mechanism first compresses the feature map in the spatial dimension, extracting a one-dimensional vector. Specifically, for an input feature map of shape (B, C, H, W), average pooling and max pooling operations are performed simultaneously in the spatial dimension, thereby compressing the spatial information of each channel (C), ultimately obtaining an output of shape (B, C, 1, 1). In this process, the spatial information of each channel is transformed into an independent value by max pooling or average pooling, achieving the extraction and compression of global spatial information. Unlike the traditional SENet, average pooling provides feedback to all pixels in the feature map, while max pooling, during gradient backpropagation, only generates gradient feedback to the region with the strongest response in the feature map, making it an effective complement to average pooling.

[0047]

[0048] =

[0049] The compressed feature maps are input into a shared network, which consists of two convolutional layers and a ReLU activation function. This network employs a dimensionality reduction-enlargement structure to reduce parameter overhead and improve computational efficiency. The weights in the MLP are shared; all inputs use the same weights. and The weight matrix is ​​calculated, significantly reducing the number of parameters. After processing by the shared network, each feature descriptor is summed element-wise to merge and output a feature vector, which is then processed by the sigmoid function. Generate a channel weight vector, ensuring its values ​​are within a reasonable range. Multiply this channel weight vector element-wise with the input feature map to obtain the final output feature map.

[0050] The feature map output by the channel attention module is fused by the Fusion module, and then the fused feature map is converted into a pixel space output that conforms to the characteristics of terahertz imaging by the Head module, resulting in a reconstructed high-resolution terahertz image.

[0051] The entire training process is as follows Figure 5 As shown, it includes:

[0052] 1. The open-source DIV2K dataset was used as the training data. This dataset contains 900 high-resolution (HR) images, of which 800 were used for training and 100 were used for testing.

[0053] 2. The DIV2K dataset is preprocessed. The HR image is convolved using a Gaussian blur kernel to simulate the blurring effect of the terahertz imaging system. The degraded LR image is obtained by downsampling and adding Gaussian noise of a specific intensity.

[0054] 3. Using the HR images from the DIV2K dataset as ground truth and the LR images generated by the degradation model as input data, a deep learning model that can be used for terahertz image super-resolution is obtained through training.

[0055] 4. Acquire raw terahertz image data

[0056] 5. Input the original terahertz image data into the trained model, and the output is the super-resolution processed image.

[0057] The image cropping size was set to 192, ResBlock 32, and the ADAM optimizer was used. Its first-order moment estimate decay rate (β1) was set to 0.9, its second-order moment estimate decay rate (β2) to 0.999, and its numerical stability parameter (ε) to 10⁻. 8 The batch size is 32, and the learning rate is initialized to 5 × 10⁻ 4 .

[0058] During training, the model with a magnification of ×2 is trained from scratch first. After the model converges, it is used as a pre-trained model for the ×3 and ×4 magnification tasks to accelerate convergence and improve performance.

[0059] The network is trained using L2 loss, and the loss function is obtained by calculating the mean squared error (MSE) between the features of the super-resolution image (SR) and the ground truth image (HR) in the VGG19 network to adjust the model.

[0060] When testing the model, given the high sensitivity of the human eye to brightness information in an image, the reconstructed image was converted to the YCbCr color space, and PSNR and SSIM were calculated for the Y channel to evaluate the THz image quality.

[0061] To verify the effectiveness of the proposed model, comparative experiments were designed, selecting several classic and advanced super-resolution methods for comparison, including traditional bicubic interpolation and deep learning methods: SRCNN, VDSR, EDSR, CRNN, ResNet, and U-Ne. As shown in Table 1 below, the experimental results demonstrate that the proposed model outperforms other comparative models in super-resolution tasks, further validating its effectiveness.

[0062] The raw terahertz images include typical samples such as biological cancer smears. For example... Figure 6 As shown, the comparison between the original terahertz image and the reconstructed image clearly demonstrates that the super-resolution model effectively improves the image's detail preservation capabilities, making edges clearer and texture information richer, while reducing the impact of blur and artifacts.

[0063] Table 1 Comparison of results from different methods

[0064] Method Bicubic SRCNN VDSR EDSR CRNN ResNet U-Net Ours PSNR 30.68 30.78 30.94 31.58 30.70 31.04 31.67 31.58 SSIM 0.81 0.82 0.83 0.89 0.81 0.83 0.86 0.92

Claims

1. A terahertz image super-resolution reconstruction method based on an improved residual network, characterized in that, include: Obtain the raw terahertz image; The original terahertz image is input into the trained reconstruction model to obtain the corresponding super-resolution image; The reconstruction model extracts features from the original terahertz image using an improved residual network, maps the extracted feature maps to the target-scale feature space, and introduces an attention mechanism to calibrate feature weights in the target-scale feature space. After feature fusion of the calibrated feature maps, a super-resolution image at the target scale is generated based on the fused feature maps.

2. The reconstruction method of claim 1, wherein, The improved residual network is achieved by removing batch normalization from the basic residual network and removing the ReLU activation function outside the residual block.

3. The reconstruction method of claim 2, wherein, In the improved residual network, a constant scaling layer is placed after the last convolutional layer of each residual block.

4. The reconstruction method of claim 3, wherein, The scaling factor of the constant scaling layer is 0.

1.

5. The reconstruction method of claim 3, wherein, In the improved residual network, the number of residual blocks is set to 32, and the number of filters is set to 256.

6. The reconstruction method of claim 1, wherein, The feature weight calibration in the target scale feature space includes: After performing average pooling and max pooling operations on the extracted feature map in the spatial dimension, one-dimensional vectors are extracted from each of them. Based on the extracted one-dimensional vector, the shared network is used to sum the elements of each feature descriptor, and the resulting feature vector is used to generate a channel weight vector through the sigmoid function.

7. The reconstruction method of claim 1, wherein, The process of mapping the extracted feature map to the target scale feature space is achieved through an upsampling layer.

8. The reconstruction method of claim 1, wherein, The reconstruction model was trained using the following method: Obtain high-resolution image datasets; After performing convolution and downsampling operations on the high-definition images in the high-definition image dataset using a Gaussian blur kernel, Gaussian noise of a set intensity is added to obtain the corresponding degraded image. The degraded image is input into the reconstruction model, and the VGG19 network is used to extract the image features of the super-resolution image output by the model and the corresponding high-resolution image to form image feature pairs. The reconstruction model is tuned based on the mean square error between image feature pairs.

9. The reconstruction method of claim 8, wherein, During training, the reconstruction model is first pre-trained for a reconstruction task at 2x scale of the original terahertz image. After the model converges, the obtained weights are transferred to the initialization process for training on reconstruction tasks at 3x and 4x scales.

10. The reconstruction method of claim 1, wherein, The reconstruction model was tested using the following methods: The super-resolution image output by the reconstructed model is converted to the YCbCr color space; The PSNR and SSIM of the transformed image are calculated for the Y channel to evaluate the quality of the super-resolution image output by the model.

Images