A method and system for terahertz image super-resolution reconstruction based on multidimensional attention fusion network

Abstract

This invention discloses a method and system for terahertz image super-resolution reconstruction based on a multi-dimensional attention fusion network. The method includes: Step S1, acquiring terahertz images of the person being detected using a terahertz image acquisition device to form a terahertz dataset; Step S2, performing feature degradation on the terahertz dataset using a degradation model to generate a low-resolution image training dataset with extensive degradation features; Step S3, constructing a multi-dimensional attention fusion network based on multi-scale fusion and channel attention mechanisms, and training the multi-dimensional attention fusion network using the low-resolution image training dataset with extensive degradation features to obtain a trained multi-dimensional attention fusion network; Step S4, inputting the real-time acquired terahertz images into the trained multi-dimensional attention fusion network to obtain a terahertz image super-resolution reconstructed image.

Description

Technical Field

[0001] This invention relates to the field of image super-resolution technology, specifically to a terahertz image super-resolution reconstruction method and system based on a multidimensional attention fusion network. Background Technology

[0002] Terahertz imaging technology, with its unique penetrating and non-ionizing properties, has demonstrated significant application value in fields such as security inspection and biomedicine. However, due to limitations in the diffraction limit of hardware systems and environmental noise interference, the raw terahertz images generally suffer from low resolution and blurred textures, severely affecting the accuracy of critical tasks such as hazardous material identification and pathological diagnosis.

[0003] In recent years, deep learning-based image super-resolution methods have made significant progress in the visible light domain. However, traditional interpolation-based super-resolution methods produce artifacts when reconstructing high-frequency details, and existing deep learning solutions have certain limitations. Existing methods rely on single-scale convolutional kernels, which struggle to capture multi-band features, leading to the loss of crucial details such as the edges of metal tools. They also fail to adequately model complex degradations in real-world imaging (such as blurring, noise, and downsampling aliasing), causing a sharp drop in PSNR exceeding 3dB when the degradation factor σ > 0.3. While directly using large-size convolutional kernels can expand the receptive field, it results in a quadratic increase in the number of model parameters, making it difficult to meet the real-time requirements of security inspection equipment. Therefore, there is an urgent need to construct a dedicated reconstruction framework that balances multi-scale feature extraction capabilities, adaptability to complex degradations, and lightweight computation. Summary of the Invention

[0004] To address the above technical problems, this invention provides a terahertz image super-resolution reconstruction method based on a multi-dimensional attention fusion network. The method constructs a network using an ESRGAN-based super-resolution network, a multi-scale fusion module based on large kernel convolution decomposition, and a decomposed channel attention mechanism. Terahertz images of the subject are captured by a terahertz imaging device, and the target image undergoes super-resolution processing at a corresponding magnification. The method includes:

[0005] Step S1: Use a terahertz image acquisition device to acquire terahertz images of the person being tested to form a terahertz dataset;

[0006] Step S2: Use a degradation model to perform feature degradation on the terahertz dataset to generate a low-resolution image training dataset with extensive degradation features;

[0007] Step S3: Construct a multi-dimensional attention fusion network based on multi-scale fusion and channel attention mechanism, and train the multi-dimensional attention fusion network using the low-resolution image training dataset with extensive degradation features to obtain the trained multi-dimensional attention fusion network.

[0008] Step S4: Input the real-time acquired terahertz image into the trained multidimensional attention fusion network to obtain the terahertz image super-resolution reconstruction image.

[0009] Optionally, step S2 specifically includes:

[0010] The terahertz image I from the terahertz dataset acquired in step S1 HR Spatial degradation processing is performed on the input fuzzy convolution to obtain a spatially degraded image;

[0011] The spatial degradation image and the terahertz image I HR Perform convolution to obtain a blurred feature map;

[0012] A noisy blurred feature map is obtained by superimposing composite noise onto the blurred feature map.

[0013] The fuzzy feature map is downsampled using bicubic interpolation to obtain a sampled fuzzy feature map.

[0014] The noise-blurred feature map is added to the sampled blurred feature map to obtain a low-resolution image with extensive degradation features;

[0015] By generating low-resolution images from all terahertz images in the terahertz dataset, a low-resolution image training dataset with extensive degradation features is obtained.

[0016] Optionally, the content of the spatially degraded image specifically includes:

[0017]

[0018] Where, σ a These represent the standard deviation of the Gaussian kernel controlling the x-direction, σ. b denoted as Gaussian kernel standard deviation controlling the y-direction, and (a, b) as kernel coordinates.

[0019] Optionally, in step S3, the specific content of the multidimensional attention fusion network includes:

[0020] The low-resolution terahertz images I in the low-resolution image training dataset LR Perform shallow feature extraction to obtain a shallow feature map;

[0021] After expanding the channel dimension of the shallow feature map using a convolutional layer, convolution and channel segmentation are performed to obtain segmented feature maps κ1 and κ2.

[0022] Point convolution is used to process the segmented feature maps κ1 and κ2 to obtain locally enhanced feature maps;

[0023] The local enhanced feature maps are concatenated along the channel dimension and the number of channels is adjusted to obtain the output feature map.

[0024] Shallow and mid-layer features are obtained based on the output feature map;

[0025] Based on the shallow and medium layer features input into the multi-scale fusion block, a multi-scale fusion output map is obtained.

[0026] Optionally, the process of obtaining the local enhanced feature map is as follows:

[0027] κ(κ1, κ2) = split(f c (f c (κ) 3×3 ) 1×1 );

[0028] Where split(·) is the feature segmentation function, used to segment the feature map according to the channel dimension; κ1, κ2∈R c×h×w The segmented feature map; f c (·) 1×1 It is a convolutional layer with a kernel size of 1×1.

[0029] Optionally, the process of obtaining the shallow and intermediate layer features is as follows:

[0030] The output feature map is used to generate channel statistics through global average pooling, and any element in the channel statistics is used to obtain the channel attention output feature map of that element through a gating unit.

[0031] The shallow and mid-level features of this element are obtained by using long-skip connections to output feature maps and channel attention output feature maps.

[0032] The feature maps are added element by element to obtain the shallow and medium-level features.

[0033] Optionally, the content of the multi-scale fusion block specifically includes:

[0034] The shallow and mid-layer features are input into two convolutional blocks to obtain a feature map;

[0035] Multidimensional feature information χ is extracted from the feature map using a large kernel self-attention structure. i1 , χ i2 ;

[0036] The multidimensional feature information χ i1 The output result χ of LKA is obtained through a large kernel attention function. iΔ1 ;

[0037] Output result χ iΔ1 With multidimensional feature information χ i2 Element-wise multiplication yields the output feature map of LSA;

[0038] The output feature map of the LSA is convolved, the number of channels is adjusted, and then the elements are added together to obtain the output result of the scale fusion block.

[0039] This invention also discloses a terahertz image super-resolution reconstruction system based on a multi-dimensional attention fusion network, the system comprising:

[0040] The data acquisition module is used to acquire terahertz images of the person being tested using terahertz image acquisition equipment to form a terahertz dataset.

[0041] A training dataset construction module is used to perform feature degradation on the terahertz dataset using a degradation model to generate a low-resolution image training dataset with extensive degradation features.

[0042] The network construction module is used to construct a multidimensional attention fusion network based on multi-scale fusion and channel attention mechanism, and to train the multidimensional attention fusion network using the low-resolution image training dataset with extensive degradation features to obtain the trained multidimensional attention fusion network.

[0043] The image reconstruction module is used to input the real-time acquired terahertz image into the trained multidimensional attention fusion network to obtain a terahertz image super-resolution reconstructed image.

[0044] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0045] This invention provides a method and system for terahertz image super-resolution reconstruction based on a multi-dimensional attention fusion network. Conventional super-resolution algorithms struggle to balance reconstruction speed and accuracy. This invention effectively captures detailed features in terahertz images by introducing a multi-scale fusion module (MSFM) and a decomposed channel attention module (DCAM), thereby improving the accuracy and robustness of super-resolution reconstruction. Simultaneously, the designed MSFM utilizes decomposed large kernel convolution, and the DCAM utilizes channel attention, which can improve computational efficiency while reducing network complexity. This method offers advantages such as speed, lightweight design, and accuracy, making it suitable for real-time applications and various scenarios in terahertz image super-resolution. It has broad application prospects in terahertz imaging, security inspection, and non-destructive testing. Attached Figure Description

[0046] To more clearly illustrate the technical solution of the present invention, the drawings used in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1This is a schematic diagram illustrating the acquisition of a low-resolution terahertz image dataset according to an embodiment of the present invention;

[0048] Figure 2 This is a schematic diagram of a terahertz image degradation model according to an embodiment of the present invention;

[0049] Figure 3 This is a schematic diagram of the overall architecture of the super-resolution network according to an embodiment of the present invention;

[0050] Figure 4 This is a schematic diagram of the channel attention mechanism in the overall architecture of the super-resolution network according to an embodiment of the present invention;

[0051] Figure 5 This is a schematic diagram of the multi-scale fusion module network architecture in the overall super-resolution network architecture of this invention.

[0052] Figure 6 This is a schematic diagram of a large kernel self-attention structure in the multi-scale fusion module network architecture of this invention.

[0053] Figure 7 This diagram illustrates the steps of a terahertz image super-resolution reconstruction method based on a multidimensional attention fusion network, as described in an embodiment of the present invention. Detailed Implementation

[0054] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0055] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0056] Example 1

[0057] A terahertz image super-resolution reconstruction method based on a multi-dimensional attention fusion network, such as... Figure 7 As shown, the method includes:

[0058] Step S1: Use a terahertz image acquisition device to acquire terahertz images of the person being tested to form a terahertz dataset.

[0059] The terahertz image acquisition device includes a computer 103, a terahertz imaging device 102, a terahertz transceiver, and a person carrying items to be inspected. The computer 103 is connected to the terahertz imaging device 102 and also to the terahertz transceiver to acquire a terahertz image dataset. The dataset images are then processed by an enhanced super-resolution generative adversarial network incorporating novel modules to acquire a high-resolution, detailed terahertz image 104 of the person being inspected 101, enabling clearer and more convenient detection of hazardous materials.

[0060] like Figure 1 As shown, images of the examinee 101 are captured by the terahertz imaging device 102, resulting in terahertz images 104. These 104 terahertz images are then combined into a dataset on the computer 103. [Image 1] Thz Image2 Thz ImageK Thz ], where the dataset ImageK Thz The total number of images is K, and the original image size is H×W, where H is the height of the original image and W is the width of the original image.

[0061] Step S2: Use a degradation model to perform feature degradation on the terahertz dataset to generate a low-resolution image training dataset with extensive degradation features.

[0062] Build as Figure 2 The degradation model shown consists of a fuzzy kernel convolution module, a noise superposition module, and a downsampling module, which drive the simulation of the degradation process of the terahertz imaging system through a physical mechanism.

[0063] High-resolution terahertz image I with size H×W HR As input, the input image first undergoes spatial degradation processing in a blur kernel convolution module. An anisotropic Gaussian kernel k is used to simulate the diffraction effect of terahertz beams; its mathematical expression is:

[0064]

[0065] Where, σ a These represent the standard deviation of the Gaussian kernel controlling the x-direction, σ. b ...

[0066]

[0067] in, The convolution operation is represented by a kernel size of K×K. Experiments show that when K≥15, it can effectively simulate the point spread function characteristics of a terahertz system. Composite noise N is superimposed on the blurred feature map B. s Its expression is:

[0068] N s =α g N g +α t N t +α p N p ;

[0069] Where, N g For Gaussian noise, N t For thermal noise, N p For Poisson noise, α g α t α p The noise intensity coefficient is f. Bicubic interpolation is used. s (·) Downsampling of the degenerate feature map:

[0070] I LR =f s (B)+N s ;

[0071] Generate a low-resolution image training dataset with extensive degradation characteristics [Image1] Thz1 Image1 Thz2 Image1 Thzn ImageK Thzn ].

[0072] Step S3: Construct a multi-dimensional attention fusion network based on multi-scale fusion and channel attention mechanisms, and train the multi-dimensional attention fusion network using the low-resolution image training dataset with extensive degradation features to obtain the trained multi-dimensional attention fusion network.

[0073] Building such Figure 3 The MAFNet network model shown includes a shallow feature extraction module, residual dense block group, multi-scale fusion module (MSFM), decomposed channel attention module (DCAM), and super-resolution reconstruction module.

[0074] Using a low-resolution terahertz image of size h×w I LRThe shallow feature extraction module, which serves as input to the network model, sequentially passes through one 3×3 convolutional layer, four residual dense blocks (RRDB), and 15 multi-scale fusion modules (MSFM), outputting multi-level feature maps F1, F2, and F3 with 64, 128, and 256 channels, respectively. By reducing the number of original RRDB modules from six to four and employing a channel decomposition strategy, the number of network parameters is reduced by 37.5%. In the deep feature fusion stage of the network model, a multi-scale fusion module (MSFM) is designed to achieve cross-channel and cross-spatial information interaction between feature maps by decomposing large-kernel convolutions and cross-scale attention mechanisms.

[0075] Decompose channel attention module, such as Figure 4 As shown, firstly, given the input features κ∈R c×h×w Using convolutional layers f c (·) 3×3 Channel dimension expansion is performed. Then, the feature map is segmented according to the channel dimension using a 3×3 convolutional layer and channel segmentation. The segmented feature map is then processed by point convolution f. c (·) 1×1 The point convolution is processed by including a 1×1 kernel layer to enhance the learning of local features. This process can be represented as:

[0076] κ(κ1, κ2) = split(f c (f c (κ) 3×3 ) 1×1 );

[0077] Here, `split(·)` represents the feature segmentation function, used to segment the feature map according to the channel dimension. κ1, κ2∈R c×h×w These are the segmented feature maps. Then, the resulting feature maps κ1 and κ2 are concatenated along the channel dimension. Finally, they are passed through a convolutional layer f with a kernel size of 1×1. c (·) 1×1 Adjust the number of channels. This process can be represented as:

[0078] κ out1 =f c (Concate(κ1, κ2)) 1×1 ;

[0079] Where Concate(·) is the feature information fusion function, κ out1 ∈R c×h×w This is the output feature map of the convolutional layer. (The feature map κ is used as an example.) out1 =[κ 1 κ 2 ,…κ C As input, the channel statistics κ are first generated through global average pooling. z∈R C , where κ z The Cth element can be obtained using the following formula:

[0080]

[0081] Among them, κ C (i, j) is the Cth feature κ C The value at position (i,j), F GP (·) represents the global average pooling function. Then The channel statistics obtained through a simple gating unit are used to scale the input. This process can be represented as:

[0082]

[0083] Among them, W d (·) represents the set of weights for the convolutional layer, used to reduce the channel scale. u (·) indicates a convolutional layer, used to increase the channel scale. The output feature map represents the channel attention. The input and output feature maps are connected via long hops. The feature maps are then summed element-wise to generate shallow and medium-level feature maps κ. out This process can be represented as:

[0084] κ out =κ+κ out2 ;

[0085] Multi-scale fusion module such as Figure 5 As shown, assuming a given input feature map κ out ∈R C×H×W First, input feature map κ. out Two convolutional blocks are used to obtain the feature map χ1. Each convolutional block consists of convolution (Conv), batch normalization (BN), and ReLU. A short skip connection between the head and tail allows the element-wise sum of the two feature maps to obtain the feature map χ. α The process can be described as follows:

[0086] χ α =ConvBlock(κ) out )+ConvBlock(κ out )+κ out ;

[0087] Here, ConvBlock(·) represents a convolutional block composed of Conv, BN, and ReLU. Secondly, as... Figure 6 As shown, the feature map χ αA large kernel self-attention (LSA) structure is used to further extract multidimensional feature information. In LSA, the feature map χ α It is segmented into multiple small feature maps χ along the channel dimension i1 , χ i2 Where i = 1, 2, 3, and the partition ratio is 1:1. This process can be described as follows:

[0088] χ i1 ,χ i2 =split(χ) a );

[0089] Subsequently, these segmented feature maps χ i1 χ is obtained by processing the data using the Large Kernel Attention (LKA) function. iΔ1 , i.e., χ iΔ1 =Λ LKA (χ i1 The specific operation of the LKA function is as follows:

[0090] Λ LKA (·)f c [DW_D_Conv(DW_Conv(·))] 1×1 ;

[0091] This includes depthwise dilated convolution (DW_Conv), depthwise convolution (DW_D_Conv), and pointwise convolution (f). c (·) 1×1 Then, χ iΔ1 With the corresponding χ i2 Perform element-wise multiplication to obtain the output feature map of LSA. This process can be described as follows:

[0092]

[0093] Finally, the three The number of channels is adjusted using 1×1 convolutional layers, and then element-wise summed to obtain the final output feature map Out of MSFM. This process can be described as follows:

[0094]

[0095] The training method includes using the l1(Θ) loss function to guide MAFNet training, which can be expressed as:

[0096]

[0097] In the above formula, F MAFNet(·) represents the network function, and Θ represents the network parameters. Stochastic gradient descent is used to optimize the loss function. Network training parameters are set as follows: learning rate (lr), batch size, training and validation set partitioning, optimizer, and training epochs.

[0098] Step S4: Input the real-time acquired terahertz image into the trained multidimensional attention fusion network to obtain the terahertz image super-resolution reconstruction image.

[0099] The trained network is used for prediction, taking a low-resolution (LR) terahertz image as input and outputting a super-resolution (SR) terahertz image. First, the low-resolution image to be tested is... LR The input image, with a size of H×W, is fed into the network. After network inference, an SR image I is obtained. SR The output feature map has a scale of H×W. MAFNet effectively extracts detailed texture features from terahertz images through a multi-scale fusion module (MSFM) and a decomposed channel attention module (DCAM), improving the accuracy and robustness of super-resolution reconstruction. Furthermore, by adjusting the parameters of the upsampling module (PixelShuffle), it supports ×2, ×4, and ×8 super-resolution tasks.

[0100] Example 2

[0101] A terahertz image super-resolution reconstruction system based on a multidimensional attention fusion network, the system comprising:

[0102] The data acquisition module is used to acquire terahertz images of the person being tested using terahertz image acquisition equipment to form a terahertz dataset.

[0103] The terahertz image acquisition device includes a computer 103, a terahertz imaging device 102, a terahertz transceiver, and a person carrying items to be inspected. The computer 103 is connected to the terahertz imaging device 102 and also to the terahertz transceiver to acquire a terahertz image dataset. The dataset images are then processed by an enhanced super-resolution generative adversarial network incorporating novel modules to acquire a high-resolution, detailed terahertz image 104 of the person being inspected 101, enabling clearer and more convenient detection of hazardous materials.

[0104] Images of the examinee 101 are captured by the terahertz imaging device 102, resulting in terahertz images 104. These 104 terahertz images are then compiled into a dataset on the computer 103. [Image 1] Thz Image2 Thz ImageK Thz ], where the dataset ImageK Thz The total number of images is K, and the original image size is H×W, where H is the height of the original image and W is the width of the original image.

[0105] The training dataset construction module is used to perform feature degradation on the terahertz dataset using a degradation model to generate a low-resolution image training dataset with extensive degradation features.

[0106] Build as Figure 2 The degradation model shown consists of a fuzzy kernel convolution module, a noise superposition module, and a downsampling module, which drive the simulation of the degradation process of the terahertz imaging system through a physical mechanism.

[0107] High-resolution terahertz image I with size H×W HR As input, the input image first undergoes spatial degradation processing in a blur kernel convolution module. An anisotropic Gaussian kernel k is used to simulate the diffraction effect of terahertz beams; its mathematical expression is:

[0108]

[0109] Where, σ a These represent the standard deviation of the Gaussian kernel controlling the x-direction, σ. b ...

[0110]

[0111] in, The convolution operation is represented by a kernel size of K×K. Experiments show that when K≥15, it can effectively simulate the point spread function characteristics of a terahertz system. Composite noise N is superimposed on the blurred feature map B. s Its expression is:

[0112] N s =α g N g +α t N t +α p N p ;

[0113] Where, N g For Gaussian noise, N t For thermal noise, N p For Poisson noise, α g α t α p The noise intensity coefficient is f. Bicubic interpolation is used. s (·) Downsampling of the degenerate feature map:

[0114] I LR =f s (B)+N s ;

[0115] Generate a low-resolution image training dataset with extensive degradation characteristics [Image1] Thz1 Image1 Thz2 Image1 Thzn ImageK Thzn ].

[0116] The network construction module is used to construct a multidimensional attention fusion network based on multi-scale fusion and channel attention mechanisms, and to train the multidimensional attention fusion network using the low-resolution image training dataset with extensive degradation features to obtain the trained multidimensional attention fusion network.

[0117] Building such Figure 3 The MAFNet network model shown includes a shallow feature extraction module, residual dense block group, multi-scale fusion module (MSFM), decomposed channel attention module (DCAM), and super-resolution reconstruction module.

[0118] Using a low-resolution terahertz image of size h×w I LR The shallow feature extraction module, which serves as input to the network model, sequentially passes through one 3×3 convolutional layer, four residual dense blocks (RRDB), and 15 multi-scale fusion modules (MSFM), outputting multi-level feature maps F1, F2, and F3 with 64, 128, and 256 channels, respectively. By reducing the number of original RRDB modules from six to four and employing a channel decomposition strategy, the number of network parameters is reduced by 37.5%. In the deep feature fusion stage of the network model, a multi-scale fusion module (MSFM) is designed to achieve cross-channel and cross-spatial information interaction between feature maps by decomposing large-kernel convolutions and cross-scale attention mechanisms.

[0119] Decompose channel attention module, such as Figure 4 As shown, firstly, given the input features κ∈R c×h×w Using convolutional layers f c (·) 3×3 Channel dimension expansion is performed. Then, the feature map is segmented according to the channel dimension using a 3×3 convolutional layer and channel segmentation. The segmented feature map is then processed by point convolution f. c (·) 1×1 The point convolution is processed by including a 1×1 kernel layer to enhance the learning of local features. This process can be represented as:

[0120] κ(κ1, κ2) = split(f c (f c (κ) 3×3 ) 1×1 );

[0121] Here, `split(·)` represents the feature segmentation function, used to segment the feature map according to the channel dimension. κ1, κ2∈R c×h×w These are the segmented feature maps. Then, the resulting feature maps κ1 and κ2 are concatenated along the channel dimension. Finally, they are passed through a convolutional layer f with a kernel size of 1×1. c (·) 1×1 Adjust the number of channels. This process can be represented as:

[0122] κ out1 =f c (Concate(κ1, κ2)) 1×1 ;

[0123] Where Concate(·) is the feature information fusion function, κ out1 ∈R c×h×w This is the output feature map of the convolutional layer. (The feature map κ is used as an example.) out1 =[κ 1 κ 2 ,…κ C As input, the channel statistics κ are first generated through global average pooling. z ∈R C , where κ z The Cth element can be obtained using the following formula:

[0124]

[0125] Among them, κ C (i, j) is the Cth feature κ C The value at position (i,j), F GP (·) represents the global average pooling function. Then The channel statistics obtained through a simple gating unit are used to scale the input. This process can be represented as:

[0126]

[0127] Among them, W d (·) represents the set of weights for the convolutional layer, used to reduce the channel scale. u (·) indicates a convolutional layer, used to increase the channel scale. The output feature map represents channel attention. The input and output feature maps are connected via long hops. The feature maps are then element-wise summed to generate shallow and mid-level feature maps κ. out This process can be represented as:

[0128] κ out =κ+κ out2 ;

[0129] Multi-scale fusion module such as Figure 5 As shown, assuming a given input feature map κ out ∈R C×H×W First, input feature map κ. out Two convolutional blocks are used to obtain the feature map χ1. Each convolutional block consists of convolution (Conv), batch normalization (BN), and ReLU. A short skip connection between the head and tail allows the element-wise sum of the two feature maps to obtain the feature map χ. α The process can be described as follows:

[0130] χ α =ConvBlock(κ) out )+ConvBlock(κ out )+κ out ;

[0131] Here, ConvBlock(·) represents a convolutional block composed of Conv, BN, and ReLU. Secondly, as... Figure 6 As shown, the feature map χ α A large kernel self-attention (LSA) structure is used to further extract multidimensional feature information. In LSA, the feature map χ α It is segmented into multiple small feature maps χ along the channel dimension i1 , χ i2 Where i = 1, 2, 3, and the partition ratio is 1:1. This process can be described as follows:

[0132] χ i1 ,χ i2 =split(χ) a );

[0133] Subsequently, these segmented feature maps χ i1 χ is obtained by processing the data using the Large Kernel Attention (LKA) function. iΔ1 , i.e., χ iΔ1 =Λ LKA (χ i1 The specific operation of the LKA function is as follows:

[0134] Λ LKA (·)=f c [DW_D_Conv(DW_Conv(·))] 1×1 ;

[0135] This includes depthwise dilated convolution (DW_Conv), depthwise convolution (DW_D_Conv), and pointwise convolution (f). c (·) 1×1 Then, χ iΔ1 With the corresponding χ i2 Perform element-wise multiplication to obtain the output feature map of LSA. This process can be described as follows:

[0136]

[0137] Finally, the three The number of channels is adjusted using 1×1 convolutional layers, and then element-wise summed to obtain the final output feature map Out of MSFM. This process can be described as follows:

[0138]

[0139] The training method includes using the l1(Θ) loss function to guide MAFNet training, which can be expressed as:

[0140]

[0141] In the above formula, F MAFNet (·) represents the network function, and Θ represents the network parameters. Stochastic gradient descent is used to optimize the loss function. Network training parameters are set as follows: learning rate (lr), batch size, training and validation set partitioning, optimizer, and training epochs.

[0142] The image reconstruction module is used to input the real-time acquired terahertz image into the trained multidimensional attention fusion network to obtain a terahertz image super-resolution reconstructed image.

[0143] The trained network is used for prediction, taking a low-resolution (LR) terahertz image as input and outputting a super-resolution (SR) terahertz image. First, the low-resolution image to be tested is... LR The input image, with a size of H×W, is fed into the network. After network inference, an SR image I is obtained. SR The output feature map has a scale of H×W. MAFNet effectively extracts detailed texture features from terahertz images through a multi-scale fusion module (MSFM) and a decomposed channel attention module (DCAM), improving the accuracy and robustness of super-resolution reconstruction. Furthermore, by adjusting the parameters of the upsampling module (PixelShuffle), it supports ×2, ×4, and ×8 super-resolution tasks.

[0144] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A terahertz image super-resolution reconstruction method based on a multi-dimensional attention fusion network, characterized in that, The method includes: Step S1: Use a terahertz image acquisition device to acquire terahertz images of the person being tested to form a terahertz dataset; Step S2: Use a degradation model to perform feature degradation on the terahertz dataset to generate a low-resolution image training dataset with extensive degradation features; Step S3: Construct a multi-dimensional attention fusion network based on multi-scale fusion and channel attention mechanism, and train the multi-dimensional attention fusion network using the low-resolution image training dataset with extensive degradation features to obtain the trained multi-dimensional attention fusion network. The specific content of the multidimensional attention fusion network includes: Low-resolution terahertz images in the low-resolution image training dataset Shallow feature extraction is performed by sequentially passing through one 3×3 convolutional layer, four residual dense blocks (RRDB), and 15 multi-scale fusion modules to obtain a shallow feature map; After expanding the channel dimension of the shallow feature map using a convolutional layer, convolution and channel segmentation are performed to obtain the segmented feature map. , ; The segmented feature map is processed using point convolution. , The point convolution is processed by a convolutional layer containing a kernel size of 1×1 to obtain a local enhanced feature map. The local enhanced feature maps are concatenated along the channel dimension and the number of channels is adjusted to obtain the output feature map. Shallow and mid-level features are obtained by connecting the shallow feature map and the output feature map with long skip connections and adding the feature maps element by element. Based on the shallow and medium layer features, a multi-scale fusion block is input to obtain a multi-scale fusion output map; The content of the multi-scale fusion block specifically includes: The shallow and mid-layer features are input into two convolutional blocks to obtain a feature map; Multidimensional feature information is extracted from the feature map using a large kernel self-attention structure. , ; The multidimensional feature information The output of LKA is obtained through a large kernel attention function. ; Output results With multidimensional feature information Element-wise multiplication yields the output feature map of LSA; After convolving the output feature map of the LSA and adjusting the number of channels, the elements are added together to obtain the output result of the scale fusion block; Step S4: Input the real-time acquired terahertz image into the trained multidimensional attention fusion network to obtain the terahertz image super-resolution reconstruction image.

2. The terahertz image super-resolution reconstruction method based on a multi-dimensional attention fusion network according to claim 1, characterized in that, The specific content of step S2 includes: The terahertz images from the terahertz dataset acquired in step S1 Spatial degradation processing is performed on the input fuzzy convolution to obtain a spatially degraded image; The spatial degradation image and the terahertz image are compared. Perform convolution to obtain a blurred feature map; A noise-blurred feature map is obtained by superimposing composite noise onto the blurred feature map. The fuzzy feature map is downsampled using bicubic interpolation to obtain a sampled fuzzy feature map. The noise-blurred feature map is added to the sampled blurred feature map to obtain a low-resolution image with extensive degradation features; By generating low-resolution images from all terahertz images in the terahertz dataset, a low-resolution image training dataset with extensive degradation features is obtained.

3. The terahertz image super-resolution reconstruction method based on a multi-dimensional attention fusion network according to claim 2, characterized in that, The content of the spatially degraded image is specifically as follows: ； in, Control x The Gaussian kernel standard deviation of the direction Control y The Gaussian kernel standard deviation of the direction For kernel coordinates.

4. The terahertz image super-resolution reconstruction method based on a multi-dimensional attention fusion network according to claim 1, characterized in that, The process of obtaining the local enhanced feature map is as follows: ； in, This is a feature segmentation function used to segment feature maps according to channel dimensions; The segmented feature map; for Convolutional layers with kernel size.

5. The terahertz image super-resolution reconstruction method based on a multi-dimensional attention fusion network according to claim 4, characterized in that, The process of obtaining the shallow and intermediate layer features is as follows: The output feature map is used to generate channel statistics through global average pooling, and any element in the channel statistics is used to obtain the channel attention output feature map of that element through a gating unit. The shallow and mid-level features of this element are obtained by using long skip connections to output feature maps and channel attention output feature maps. The feature maps are added element by element to obtain the shallow and medium-level features.

6. A terahertz image super-resolution reconstruction system based on a multidimensional attention fusion network, the system being used to implement the terahertz image super-resolution reconstruction method according to any one of claims 1-5, characterized in that the system... include: The data acquisition module is used to acquire terahertz images of the person being tested using terahertz image acquisition equipment to form a terahertz dataset. A training dataset construction module is used to perform feature degradation on the terahertz dataset using a degradation model to generate a low-resolution image training dataset with extensive degradation features. The network construction module is used to construct a multidimensional attention fusion network based on multi-scale fusion and channel attention mechanism, and to train the multidimensional attention fusion network using the low-resolution image training dataset with extensive degradation features to obtain the trained multidimensional attention fusion network. The image reconstruction module is used to input the real-time acquired terahertz image into the trained multidimensional attention fusion network to obtain a terahertz image super-resolution reconstructed image.

Images