A method, apparatus and storage medium for ghost imaging image reconstruction
By using the lightweight LD-GIRNet network, combined with CSA-JEM and DA-MSRM modules, the problems of high computational resource consumption, insufficient noise robustness, and limited detail recovery capability in ghost imaging technology are solved, achieving efficient and robust image reconstruction results, suitable for resource-constrained platforms and diverse imaging scenarios.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- JILIN AGRICULTURAL UNIV
- Filing Date
- 2026-01-21
- Publication Date
- 2026-07-03
AI Technical Summary
Existing deep learning models in ghost imaging technology suffer from high computational resource consumption, insufficient noise robustness, and limited detail recovery capabilities, making it difficult to achieve high-precision real-time imaging under low sampling and high noise conditions.
A lightweight LD-GIRNet network is adopted, which includes a lightweight decoding and reconstruction unit LDRU, a channel-spatial attention joint enhancement module CSA-JEM, and a detail enhancement multi-scale residual module DA-MSRM. Shallow and deep features are fused through a residual compensation fusion module RCFM to achieve efficient image reconstruction.
While maintaining a lightweight architecture, it significantly improves reconstruction quality and speed, possesses strong noise robustness and detail recovery capabilities, can be efficiently deployed on resource-constrained platforms, and exhibits excellent generalization capabilities in different imaging scenarios.
Smart Images
Figure CN121582115B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of deep learning technology, and in particular to a ghost imaging image reconstruction method, device and storage medium. Background Technology
[0002] Ghost imaging, a revolutionary computational imaging technique, reconstructs target information through computation rather than direct optical imaging by utilizing the high-order correlation properties of the light field. This technique recovers the image by measuring the intensity correlation function between a known speckle field and the signal from a barrel detector after reflection or transmission from the target object. It possesses unique advantages such as lensless operation, resistance to turbulence interference, and non-local imaging, demonstrating significant application potential in remote sensing, biomedical imaging, security monitoring, and underwater imaging.
[0003] However, the practical application of ghost imaging technology still faces significant challenges. Under low sampling rates (i.e., fewer measurements), the reconstruction process is significantly affected by correlated noise due to insufficient correlation data, resulting in severe background noise, blurred texture details, and degraded structural contours. Traditional reconstruction algorithms (such as second-order correlation algorithms, differential ghost imaging, and compressed sensing algorithms) generally suffer from low reconstruction quality and poor computational efficiency when dealing with this type of noise, making it difficult to meet the demands for high-precision real-time imaging in practical applications.
[0004] In recent years, deep learning-based reconstruction methods have significantly improved the reconstruction performance of ghost imaging by learning the complex mapping relationship between speckle and target images. However, existing deep learning models still have several obvious limitations when applied to low-sampling, high-noise ghost imaging scenarios: (1) High model complexity and computational cost: Existing networks are usually large in structure and have many parameters, resulting in high computational resource consumption and slow inference speed, making it difficult to deploy on resource-constrained actual platforms such as mobile terminals and embedded devices; (2) Insufficient noise robustness: Model training is mostly based on simulation data under ideal or limited noise conditions, which has poor adaptability to the strong correlation noise that is common in actual systems. The generalization ability decreases significantly under low-sampling, high-noise conditions, and the reconstruction quality is unstable; (3) Limited detail recovery ability: Deep networks are prone to high-frequency information attenuation during feature transmission, resulting in blurred edges and loss of texture details in the reconstructed image, making it difficult to maintain the structural integrity and visual clarity of the target while suppressing noise.
[0005] The aforementioned problems collectively hinder the practical application of deep learning in ghost imaging. Therefore, developing novel intelligent reconstruction algorithms that combine lightweight architecture, strong noise resistance, and high detail fidelity has become a key technological challenge in advancing ghost imaging from theoretical research to practical applications. Summary of the Invention
[0006] The technical solution of the present invention to solve the above-mentioned technical problems is to provide a ghost image reconstruction method, comprising the following steps:
[0007] Obtain low-quality reconstructed images of target images generated by ghost imaging systems;
[0008] The low-quality reconstructed image is input into the trained LD-GIRNet network for processing to obtain a high-quality target reconstructed image.
[0009] The LD-GIRNet network includes an input layer, an encoder, a decoder, and an output layer.
[0010] The encoder and decoder are embedded with multiple lightweight decoding and reconstruction units (LDRUs);
[0011] The lightweight decoding and reconstruction unit LDRU integrates either the channel-spatial attention joint enhancement module CSA-JEM or the detail enhancement multi-scale residual module DA-MSRM.
[0012] The decoder and encoder are connected via a residual compensation fusion module (RCFM).
[0013] The RCFM is used to fuse shallow features from the encoder with deep features from the decoder.
[0014] Furthermore, the structure of the LD-GIRNet network is as follows:
[0015] The input layer is followed by the first LDRU;
[0016] The encoder includes a first downsampling layer, a second LDRU, a second downsampling layer, and a third LDRU connected in sequence.
[0017] The decoder includes a third LDRU, a first upsampling layer, a fourth LDRU, a second upsampling layer, and a fifth LDRU connected in sequence.
[0018] The RCFM is provided between the first downsampling layer and the second upsampling layer, and between the second downsampling layer and the first upsampling layer, respectively, for performing feature fusion;
[0019] The end of the encoder is connected to the beginning of the decoder via a bottleneck layer;
[0020] The output layer is connected after the fifth LDRU.
[0021] Furthermore, the lightweight decoding and reconstruction unit LDRU includes a main branch and a shortcut branch;
[0022] The main branch sequentially includes a first depthwise separable convolutional layer, a Dropout2d layer, a second depthwise separable convolutional layer, and an enhancement module, wherein the enhancement module is CSA-JEM or DA-MSRM;
[0023] The shortcut branch is an identity mapping when the number of input and output channels is the same; otherwise, it is a projection mapping containing a 1×1 convolution.
[0024] The output of the main branch is added to the output of the shortcut branch, and then the final output of LDRU is obtained by passing the ReLU activation function.
[0025] Furthermore, the processing procedure of the Channel-Spatial Attention Joint Enhancement Module (CSA-JEM) includes:
[0026] For the input feature map F, global average pooling and global max pooling are performed respectively. The two pooled features are then fed into a shared two-layer mapping network to generate channel attention weights Mc(F).
[0027] The channel attention weight Mc(F) is multiplied element-wise with the input feature map F to obtain the channel enhancement feature F';
[0028] For the channel enhancement feature F', average pooling and max pooling are performed along the channel dimension respectively, and the two spatial feature maps obtained are concatenated along the channel dimension;
[0029] The concatenated feature maps are passed through a 7×7 convolutional layer and then activated by a Sigmoid activation function to generate spatial attention weights Ms(F').
[0030] The spatial attention weight Ms(F') is multiplied element-wise with the channel enhancement feature F' to obtain the final output feature F''.
[0031] Furthermore, the processing procedure of the residual compensation fusion module RCFM includes:
[0032] Receive shallow features Fs from the encoder and deep features Fd from the decoder;
[0033] The shallow feature Fs and the deep feature Fd are concatenated along the channel dimension to obtain the concatenated feature Fcat;
[0034] The spliced feature Fcat is linearly compressed through a 1×1 convolutional layer to obtain the fused feature Ffuse;
[0035] The fused feature Ffuse is added element-wise to the deep feature Fd to obtain the final output Fout of the module.
[0036] Furthermore, the processing procedure of the Detail Enhancement Multi-Scale Residual Module (DA-MSRM) includes:
[0037] For the input feature map X, four convolutional branches with kernel sizes of 1×1, 3×3, 5×5 and 7×7 are passed in parallel. The number of output channels of each branch is one-quarter of the number of input channels, resulting in four multi-scale features F1, F3, F5 and F7.
[0038] The four multi-scale features are concatenated along the channel dimension to obtain the concatenated feature F. cat ;
[0039] The splicing feature F cat The fused feature F is obtained by passing a 1×1 convolutional layer and performing batch normalization. fuse ;
[0040] The fusion feature F fuse The input feature map X is added element-wise, and the final output Y of the module is obtained by passing it through the ReLU activation function.
[0041] To address the aforementioned technical problems, the present invention also proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the ghost imaging image reconstruction method as described above.
[0042] To address the aforementioned technical problems, the present invention also proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the ghost imaging image reconstruction method described above.
[0043] Compared with the prior art, the present invention has the following beneficial effects:
[0044] 1. While maintaining a lightweight architecture, this invention significantly improves reconstruction quality and speed. The proposed LD-GIRNet network uses depthwise separable convolutions to construct the basic LDRU units and employs efficient attention and multi-scale modules, greatly reducing the number of model parameters while ensuring powerful feature representation capabilities. As shown in Table 1, the model parameter count of this invention is only 5.845M, far lower than most comparative models, yet the PSNR (30.09 dB) and SSIM (0.9400) of the reconstructed image are both optimal. Simultaneously, its single-image inference time is controlled at 8.7ms. While achieving optimal reconstruction quality, it maintains efficient computational performance matching the lightweight model, solving the problems of high computational cost and difficulty in deployment on resource-constrained platforms of existing deep learning models.
[0045] 2. It possesses strong noise robustness and detail recovery capabilities, effectively overcoming correlated noise interference at low sampling rates. The core LDRU unit integrates a Channel-Spatial Attention Joint Enhancement Module (CSA-JEM) and a Detail Enhancement Multi-Scale Residual Module (DA-MSRM). CSA-JEM adaptively enhances key target features and suppresses irrelevant background noise through a cascaded channel and spatial attention mechanism; DA-MSRM, through parallel multi-scale convolution, collaboratively extracts features from local texture to global contour, effectively recovering edges and details. Figure 7 and Figure 8 As shown, under strong noise background, the image reconstructed by the method of the present invention has clear edges and complete texture, and the background noise is effectively suppressed, which is significantly better than the structural blurring, loss of detail or artifact problems that often occur in the comparison method.
[0046] 3. By employing cross-layer feature residual compensation fusion, effective complementarity between shallow details and deep semantics is achieved, improving structural fidelity. The residual compensation fusion module (RCFM) designed in this invention splices and fuses shallow features (rich in details and texture) from the encoder path with deep features (rich in semantics and contours) from the decoder path, and compensates the decoder with residuals. This mechanism ensures that high-frequency detail information is not lost during feature transfer and upsampling, resulting in a continuous and complete structure of the reconstructed target object. This advantage is particularly pronounced when processing images with complex strokes or fine structures (such as the bends in handwritten characters or the outline of an airplane).
[0047] 4. Excellent generalization ability and practicality. The method of this invention not only performs excellently on the handwritten character category of the training set, but also shows good generalization ability on natural image categories such as airplanes that were not included in the training set. Figure 8 As shown, it still maintains the highest PSNR and SSIM values, as well as the best visualization results. This indicates that the network architecture and module design proposed in this invention are not overfitted to a specific dataset, but rather have learned general priors and mapping rules for recovering high-quality images from low-quality ghost imaging reconstruction images. It has the potential to cope with different real-world imaging scenarios and is highly practical. Attached Figure Description
[0048] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.
[0049] Figure 1This is an example image of sample image data described in this invention (a represents a portion of letter samples in the dataset, and b represents a portion of number samples in the dataset).
[0050] Figure 2 This is a schematic diagram of the LDRU lightweight noise-resistant residual unit described in this invention;
[0051] Figure 3 This is a schematic diagram of the CSA-JEM module described in this invention.
[0052] Figure 4 This is a schematic diagram of the RCFM module described in this invention;
[0053] Figure 5 This is a schematic diagram of the DA-MSRM module described in this invention;
[0054] Figure 6 This is a schematic diagram of the LD-GIRNet network framework described in this invention;
[0055] Figure 7 This is a visual comparison of the image reconstruction results based on different networks described in this invention.
[0056] Figure 8 The images shown are visualization results of the generalization test of the aircraft target images described in this invention (a, b, c, d, and e represent visualization results of the generalization test of different types of aircraft target images, respectively). Detailed Implementation
[0057] This invention proposes a ghost image reconstruction method, device, and storage medium, aiming to design a ghost image reconstruction method that significantly improves the reconstruction quality and robustness of ghost images affected by correlated noise at a low number of measurements while maintaining a lightweight network.
[0058] The ghost imaging image reconstruction method, device, and storage medium proposed in this invention will be described below in specific embodiments:
[0059] Example 1:
[0060] In the technical solution of this embodiment, such as Figure 1 As shown, a ghost image reconstruction method includes the following steps:
[0061] Obtain low-quality reconstructed images of target images generated by ghost imaging systems;
[0062] The low-quality reconstructed image is input into the trained LD-GIRNet network for processing to obtain a high-quality target reconstructed image.
[0063] The LD-GIRNet network includes an input layer, an encoder, a decoder, and an output layer.
[0064] The encoder and decoder are embedded with multiple lightweight decoding and reconstruction units (LDRUs);
[0065] The lightweight decoding and reconstruction unit LDRU integrates either the channel-spatial attention joint enhancement module CSA-JEM or the detail enhancement multi-scale residual module DA-MSRM.
[0066] The decoder and encoder are connected via a residual compensation fusion module (RCFM).
[0067] The RCFM is used to fuse shallow features from the encoder with deep features from the decoder.
[0068] Furthermore, the structure of the LD-GIRNet network is as follows:
[0069] The input layer is followed by the first LDRU;
[0070] The encoder includes a first downsampling layer, a second LDRU, a second downsampling layer, and a third LDRU connected in sequence.
[0071] The decoder includes a third LDRU, a first upsampling layer, a fourth LDRU, a second upsampling layer, and a fifth LDRU connected in sequence.
[0072] The RCFM is provided between the first downsampling layer and the second upsampling layer, and between the second downsampling layer and the first upsampling layer, respectively, for performing feature fusion;
[0073] The end of the encoder is connected to the beginning of the decoder via a bottleneck layer;
[0074] The output layer is connected after the fifth LDRU.
[0075] Furthermore, the lightweight decoding and reconstruction unit LDRU includes a main branch and a shortcut branch;
[0076] The main branch sequentially includes a first depthwise separable convolutional layer, a Dropout2d layer, a second depthwise separable convolutional layer, and an enhancement module, wherein the enhancement module is CSA-JEM or DA-MSRM;
[0077] The shortcut branch is an identity mapping when the number of input and output channels is the same; otherwise, it is a projection mapping containing a 1×1 convolution.
[0078] The output of the main branch is added to the output of the shortcut branch, and then the final output of LDRU is obtained by passing the ReLU activation function.
[0079] Furthermore, the processing procedure of the Channel-Spatial Attention Joint Enhancement Module (CSA-JEM) includes:
[0080] For the input feature map F, global average pooling and global max pooling are performed respectively. The two pooled features are then fed into a shared two-layer mapping network to generate channel attention weights Mc(F).
[0081] The channel attention weight Mc(F) is multiplied element-wise with the input feature map F to obtain the channel enhancement feature F';
[0082] For the channel enhancement feature F', average pooling and max pooling are performed along the channel dimension respectively, and the two spatial feature maps obtained are concatenated along the channel dimension;
[0083] The concatenated feature maps are passed through a 7×7 convolutional layer and then activated by a Sigmoid activation function to generate spatial attention weights Ms(F').
[0084] The spatial attention weight Ms(F') is multiplied element-wise with the channel enhancement feature F' to obtain the final output feature F''.
[0085] Furthermore, the processing procedure of the residual compensation fusion module RCFM includes:
[0086] Receive shallow features Fs from the encoder and deep features Fd from the decoder;
[0087] The shallow feature Fs and the deep feature Fd are concatenated along the channel dimension to obtain the concatenated feature Fcat;
[0088] The spliced feature Fcat is linearly compressed through a 1×1 convolutional layer to obtain the fused feature Ffuse;
[0089] The fused feature Ffuse is added element-wise to the deep feature Fd to obtain the final output Fout of the module.
[0090] Furthermore, the processing procedure of the Detail Enhancement Multi-Scale Residual Module (DA-MSRM) includes:
[0091] For the input feature map X, four convolutional branches with kernel sizes of 1×1, 3×3, 5×5 and 7×7 are passed in parallel. The number of output channels of each branch is one-quarter of the number of input channels, resulting in four multi-scale features F1, F3, F5 and F7.
[0092] The four multi-scale features are concatenated along the channel dimension to obtain the concatenated feature F. cat ;
[0093] The splicing feature Fcat The fused feature F is obtained by passing a 1×1 convolutional layer and performing batch normalization. fuse ;
[0094] The fusion feature F fuse The input feature map X is added element-wise, and the final output Y of the module is obtained by passing it through the ReLU activation function.
[0095] Example 2:
[0096] A method for reconstructing ghost images includes the following steps:
[0097] Obtain low-quality reconstructed images of target images generated by ghost imaging systems;
[0098] like Figure 1 The image shows a sample dataset used to train LD-GIRNet. This dataset is built based on the publicly available handwritten character libraries MNIST and EMNIST, covering 36 character classes including digits 0-9 and the 26 English letters. By filtering the original samples and introducing diverse transformations, including using different font styles and randomly perturbing character size and position, all samples were uniformly processed into 64×64 pixel binary images. The entire dataset contains 1400 images, with 100 for digits and 1300 for letters, and divided into 1260 training samples and 140 test samples in a 9:1 ratio. To simulate the actual ghost imaging process based on digital micromirror devices, a speckle pattern was generated using a random Gaussian matrix to encode each original target image, obtaining the corresponding observation values. With 1000 measurements performed, the observation values were correlated with the measurement matrix to obtain a low-quality reconstructed image under traditional GI methods. This processing flow effectively simulates imaging degradation in a real GI system. The generated low-quality images are used as input for network training to learn an end-to-end mapping from GI reconstruction results to high-quality target images.
[0099] The low-quality reconstructed image is input into the trained LD-GIRNet network for processing to obtain a high-quality target reconstructed image.
[0100] Among them, the LD-GIRNet network, such as Figure 6 As shown, the LD-GIRNet network includes an input layer, an encoder, a decoder, and an output layer; the encoder and decoder embed multiple lightweight decoding and reconstruction units (LDRUs); the lightweight decoding and reconstruction units (LDRUs) integrate a channel-spatial attention joint enhancement module (CSA-JEM) or a detail enhancement multi-scale residual module (DA-MSRM); the decoder and encoder are connected through a residual compensation fusion module (RCFM), which is used to fuse shallow features from the encoder with deep features from the decoder.
[0101] LDRU module, such as Figure 2 As shown, depthwise separable convolution is used as the backbone operator, Dropout2d is introduced for regularization, and attention mechanism or multi-scale enhancement module is combined at the end of the main branch to highlight the target structure and suppress noise interference.
[0102] In LDRU, let the input feature map be... ,in For batch size, and These represent the height and width of the feature map, respectively. The input channel number is given. Depthwise separable convolution consists of two steps: depthwise convolution and pointwise convolution, followed by batch normalization and ReLU activation after each step. Its mathematical expression is:
[0103] (1);
[0104] in, Represents depthwise convolution, achieved by setting... Implement channel-wise convolution; express Pointwise convolution is used for channel blending and mapping the number of channels to... ; and This indicates a batch normalization operation; This represents the ReLU activation function. The final output feature is... .
[0105] The basic unit of LDRU is a lightweight residual block; let the input be... The output is The main branch consists of two depthwise separable convolutional layers, with Dropout2d regularization introduced after the first layer, and an attention or multi-scale enhancement module connected at the end. Its output format is as follows:
[0106] (2);
[0107] in, and This indicates that two depthwise separable convolutional units are executed in the order of “convolution + BN + ReLU” as defined in equation (1); This represents a two-dimensional Dropout (Dropout2d) operation used to randomly deactivate a portion of channels to suppress overfitting and noise accumulation. In the design, the Dropout probability is set to... ; The enhancement modules are as follows: CBAM (Channel and Spatial Attention Module) is used when a channel-spatial attention mechanism is employed; MSB (Multi-Scale Enhancement Module) is used when a multi-scale enhancement mechanism is employed. Indicates a shortcut branch. When hour, (Identity mapping); when In this case, channel alignment is achieved through projection mapping, and the common form is as follows: Depthwise separable convolution or Convolution with batch normalization; "+" indicates element-wise addition; This is the ReLU activation function.
[0108] By combining lightweight convolution, residual connection, regularization, and feature enhancement, LDRU effectively improves feature stability and structural discriminability in noisy environments while reducing computational redundancy.
[0109] CSA-JEM module, such as Figure 3 As shown, a CBAM (Continuous Enhancement Model) approach combining channel attention and spatial attention is adopted. First, importance is recalibrated in the channel dimension, and then key regions are focused in the spatial dimension, thereby achieving adaptive enhancement of the target structure and suppression of background noise. The figure shows the proposed joint channel and spatial attention enhancement module.
[0110] First, let the input features be... The channel attention branch uses global average pooling and global max pooling to construct channel descriptors, and generates channel weights through a shared two-layer mapping:
[0111] (3);
[0112] (4);
[0113] in, This indicates global average pooling, with an output size of ; This indicates global max pooling (implementation is equivalent to taking the maximum value of the spatial dimension), and the output size is the same as above; For a two-layer mapping with shared weights, the following is adopted: Convolution achieves channel compression and restoration. Therefore, the channel attention weights are generated through a shared two-layer mapping network as follows:
[0114] (5);
[0115] in, For the first The weight matrix of the convolutional layer is used to change the channel dimension from Down to ; It is the ReLU activation function; The second The weight matrix of the convolutional layer is used to change the channel dimension from Restore to the original channel .here The preset channel compression rate, This represents the number of channels in the input feature map. The above operations enable the network to learn the non-linear dependencies between channels with a relatively low number of parameters, thereby achieving feature recalibration for different channels.
[0116] After channel recalibration, the spatial attention branch further locates salient structural regions in the spatial dimension. First, it examines the input features... Perform average pooling and max pooling along the channel dimension respectively:
[0117] (6);
[0118] (7);
[0119] The two feature maps are concatenated along the channel dimension to obtain the spatial context descriptor:
[0120] (8);
[0121] Among them, spatial feature map It is achieved by using two feature maps and It is constructed by concatenating features along the channel dimension. This operation allows the model to simultaneously utilize features extracted by average pooling and max pooling, thereby enhancing its understanding and representation of the input data. The final feature map is obtained. Having two channels allows the model to capture richer information, which is then passed through a convolutional kernel with a size of [size missing]. The convolutional layers fuse spatial information and then generate a spatial attention weight map using a sigmoid activation function:
[0122] (9);
[0123] in Indicates the kernel size as A convolution operation with 1 output channel. This is the Sigmoid function. Finally, the recalibrated features of the channels are spatially modulated using this weight map:
[0124] (10);
[0125] In the formula This represents element-wise multiplication. This process enables the network to adaptively enhance spatially salient regions while suppressing irrelevant background noise.
[0126] RCFM module, such as Figure 4 As shown, through cross-layer residual connections and This module fuses convolutions to integrate shallow and deep features. It first concatenates the two to form a joint feature, then reduces the number of channels through linear compression convolutions, and finally combines it with the deep features to enhance stability. Assuming the shallow features are... Deep features are The module first concatenates the two along the channel dimension to form a joint feature representation:
[0127] (11);
[0128] To avoid channel expansion and improve fusion efficiency, the concatenated features are integrated using a linear compression convolution, as follows:
[0129] (12);
[0130] in To ensure consistency between the channel dimensions of the fused network and subsequent networks, the fusion result is residually added to the deep features while simultaneously incorporating compensation from the shallow texture.
[0131] (13);
[0132] This residual structure not only enhances the stability of deep semantic features but also allows shallow texture information to directly participate in the reconstruction of deep structures, effectively preventing information loss during convolution. By fusing shallow and deep features, RCFM ensures the synergistic effect of details and global semantics, enabling the model to better preserve image edges and local details in noisy environments, thereby improving the quality and consistency of reconstructed images. This mechanism achieves effective transfer and optimization of feature information through residual connections, thus enhancing the network's image restoration capabilities when removing severe background noise interference.
[0133] DA-MSRM module, such as Figure 5 As shown, parallel multi-scale convolutional branches cover the receptive field from local texture to global contour, and feature fusion and residual connections are used to achieve complementary enhancement of multi-scale information. The mechanism lies in the collaborative extraction of fine-grained features and structural context, thereby significantly improving the edge sharpness, texture realism, and contour integrity of the reconstructed image in a noisy background.
[0134] In DA-MSRM, the input features are represented as ,in These represent batch size, number of channels, height, and width, respectively. To effectively address multi-scale degradation, DA-MSRM designs four parallel convolutional branches with kernel sizes of [sizes to be filled in]. , , and The number of output channels for each branch is set to [number]. The output after the ReLU activation function can be expressed as:
[0135] (14);
[0136] in Indicates from the first Features obtained from each convolutional branch.
[0137] Next, the outputs of the four branches are concatenated along the channel dimension to obtain a feature set. :
[0138] (15);
[0139] Subsequently, through Convolutional processing fuses and compresses the concatenated features, and then incorporates Batch Normalization (BN) to stabilize the feature distribution, generating the final fused features. :
[0140] (16);
[0141] To preserve the original input features and enhance the stable propagation of gradients, DA-MSRM ultimately uses residual connections for output, defined as:
[0142] (17);
[0143] in, The symbol represents the final output characteristic of DA-MSRM. This represents element-wise addition. Through parallel multi-scale feature extraction and residual connections, this design enhances receptive field coverage while maintaining the stability and integrity of feature representation, helping the model to more effectively address multi-scale structural degradation issues in ghost imaging reconstruction tasks.
[0144] By fusing supervision signals at three levels—pixel, structure, and semantics—a multi-scale perceptual alignment loss space is constructed, thereby achieving synergistic optimization between numerical accuracy, structural integrity, and visual realism.
[0145] Let the real image be The reconstruction result is Pixel-level supervision employs mean square error to constrain the global intensity distribution.
[0146] (18)
[0147] Here, mean squared error loss It is used to measure the difference between a real image and a reconstructed image. It works by calculating the difference in location between the real image and the reconstructed image. pixel values Pixel values at the same location as the reconstructed image The squared difference between the pixels is summed over all pixels and then averaged to reflect the accuracy of the model reconstruction. Here, This represents the height of the image (number of rows), while Indicates the width of the image (number of columns).
[0148] Structure-level supervision guides texture and contour reconstruction based on the Structural Similarity Index (SSIM).
[0149] (19)
[0150] The SSIM function evaluates structural similarity by comparing the brightness, contrast, and structural components within a local window of an image. Its value range is [0,1], with higher values indicating greater structural similarity.
[0151] The model is guided to recover texture and contours from three aspects: brightness, contrast, and structural pattern; perceptual constraints are constructed based on differences in high-level semantic features.
[0152] (20)
[0153] in This is a deep network feature extractor used to measure distances in the visual feature space. Finally, the total loss is obtained through weighted fusion:
[0154] (twenty one)
[0155] in Controlling the influence intensity of different supervision dimensions allows the training process to maintain an appropriate balance between accuracy, structure, and vision.
[0156] This joint supervision mechanism effectively corrects various problems in ghost imaging, including pixel shifts caused by noise, structural blurring due to speckle degradation, and semantic inconsistencies caused by weak textures. This allows the model to maintain stable reconstruction performance when faced with different noise intensities, sampling ratios, and scene complexities.
[0157] To verify the advancement and practicality of the proposed method, the LD-GIRNet model was compared with commonly used methods in current image restoration tasks, such as ResNet, U-Net, DnCNN, RIDNet, and NAFNet. All comparison models were tested on the same test set and under a unified evaluation process. Reconstruction quality was measured using PSNR and SSIM, and the number of parameters (M) and single-frame inference time (ms) were simultaneously analyzed to reflect model complexity and inference efficiency, ensuring fair and comparable comparison results. The experimental results are shown in Table 1.
[0158] Table 1 Comparative Experiments:
[0159]
[0160] As shown in Table 1, LD-GIRNet achieves the best reconstruction quality, with a PSNR of 30.09 dB and an SSIM of 0.9400. Among the comparative methods, DnCNN (PSNR 24.59 dB, SSIM 0.8400), which performs relatively well, is still significantly lower than LD-GIRNet, demonstrating that the LD-GIRNet model has stronger robustness in noise suppression, structure preservation, and detail recovery. ResNet and U-Net have PSNRs of 22.26 dB and 28.08 dB, respectively, and SSIMs of 0.7590 and 0.8710, respectively; RIDNet and NAFNet show further declines, with PSNRs of 23.16 dB and 21.90 dB, respectively. Overall, the method of this invention improves the PSNR by 7.83dB, 2.01dB, 6.93dB and 8.19dB compared with ResNet, U-Net, RIDNet and NAFNet respectively, verifying that it can achieve higher fidelity reconstruction output in the current task.
[0161] In terms of model size, LD-GIRNet has 5.845M parameters, significantly smaller than ResNet (25.6M), U-Net (21.639M), RIDNet (13.32M), and NAFNet (26.9M), demonstrating good lightweight characteristics. It should be noted that DnCNN has the smallest number of parameters (0.556M), but its inference time is the longest (18.53ms), indicating that inference overhead is not only related to the number of parameters but also affected by factors such as the form of network operators and the way the network structure is stacked.
[0162] In terms of inference time, LD-GIRNet's single-image inference time is 8.7ms, maintaining a relatively fast overall speed. Although slightly slower than ResNet (5.4ms), U-Net (5.34ms), and NAFNet (5.62ms), LD-GIRNet significantly improves PSNR and SSIM while keeping the inference time below 10ms, making it feasible for practical deployment. Furthermore, LD-GIRNet is significantly faster than DnCNN (18.53ms) and RIDNet (10.2ms), reducing inference time by 9.83ms (approximately 53.05%) compared to DnCNN and by 1.5ms compared to RIDNet. Therefore, LD-GIRNet significantly improves image quality while maintaining reasonable computational overhead.
[0163] On the test set, Figure 7 The visualization results of the reconstruction of handwritten digits and letters (A, S, C, 1, 0, 6) are presented. From the visualization results, LD-GIRNet can more stably recover the continuity of thin strokes and boundaries, such as the slant and horizontal bar of "A", the curved boundary of "S", the open outline of "C", and the internal hollow structures of "0" and "6". Its reconstruction results show clearer edges and a more complete subject, while retaining less background noise. In comparison, while U-Net can recover the subject outline, local edges still exhibit blurring and loss of detail; ResNet is prone to structural breaks and background noise; DnCNN tends to over-smooth, leading to weakened details; RIDNet easily introduces granular artifacts under strong noise conditions; and NAFNet is insufficient in noise suppression and structure preservation, exhibiting severe background noise interference and insufficiently sharp target outlines.
[0164] Figure 8 This section presents visualizations of the generalization test results using aircraft target images. The purpose of using aircraft target images for generalization testing, compared to the handwritten text images used in the training and testing phases, is to evaluate the model's adaptability and generalization performance when handling different types of input. By introducing aircraft images as a novel category, the aim is to verify that the model not only performs well on its training data but also efficiently processes images from other scenes. This process helps verify the model's robustness and effectiveness, ensuring its accurate reconstruction of diverse targets in real-world applications.
[0165] LD-GIRNet demonstrates superior performance in image detail restoration and denoising, producing sharp image edges and effectively suppressing background noise. While other models offer improvements in some areas, they significantly fall short of LD-GIRNet in terms of detail and structure preservation. This indicates that LD-GIRNet exhibits stronger generalization ability and superior performance in image denoising tasks, maintaining excellent performance in various noisy environments. Figure 8 (a) Taking this as an example, LD-GIRNet significantly outperforms other models in terms of image quality after denoising, with a PSNR of 30.31 and an SSIM of 0.9213, demonstrating good restoration performance and effectively removing noise while preserving image details. In contrast, U-Net has a PSNR of 22.00 dB and an SSIM of 0.4850. Although it removes some noise, it suffers significant loss of image details, resulting in a less effective restoration than LD-GIRNet. ResNet has a PSNR of 21.58 dB and an SSIM of 0.5521, performing similarly to U-Net, but its denoising effect is also insufficient. DnCNN performs slightly better after denoising, with a PSNR of 22.31 dB and an SSIM of 0.8889, but it still cannot reach the level of LD-GIRNet. RIDNet and NAFNet have poor denoising performance, with both PSNR and SSIM being low.
[0166] Example 3:
[0167] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the ghost imaging image reconstruction method described in Embodiment 1.
[0168] Example 4:
[0169] A computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the ghost imaging image reconstruction method described in Example 1.
[0170] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.
Claims
1. A method for reconstructing ghost images, characterized in that, Includes the following steps: Obtain low-quality reconstructed images of target images generated by ghost imaging systems; The low-quality reconstructed image is input into the trained LD-GIRNet network for processing to obtain a high-quality target reconstructed image. The LD-GIRNet network includes an input layer, an encoder, a decoder, and an output layer. The encoder and decoder are embedded with multiple lightweight decoding and reconstruction units (LDRUs); The lightweight decoding and reconstruction unit LDRU integrates either the channel-spatial attention joint enhancement module CSA-JEM or the detail enhancement multi-scale residual module DA-MSRM. The decoder and encoder are connected via a residual compensation fusion module (RCFM). The RCFM is used to fuse shallow features from the encoder with deep features from the decoder. The lightweight decoding and reconstruction unit LDRU includes a main branch and a shortcut branch; The main branch sequentially includes a first depthwise separable convolutional layer, a Dropout2d layer, a second depthwise separable convolutional layer, and an enhancement module, wherein the enhancement module is CSA-JEM or DA-MSRM; The shortcut branch is an identity mapping when the number of input and output channels is the same; otherwise, it is a projection mapping containing a 1×1 convolution. The output of the main branch is added to the output of the shortcut branch, and then the final output of the LDRU is obtained by passing the ReLU activation function. The processing procedure of the Channel-Spatial Attention Joint Enhancement Module (CSA-JEM) includes: For the input feature map F, global average pooling and global max pooling are performed respectively. The two pooled features are then fed into a shared two-layer mapping network to generate channel attention weights Mc(F). The channel attention weight Mc(F) is multiplied element-wise with the input feature map F to obtain the channel enhancement feature F'; For the channel enhancement feature F', average pooling and max pooling are performed along the channel dimension respectively, and the two spatial feature maps obtained are concatenated along the channel dimension; The concatenated feature maps are passed through a 7×7 convolutional layer and then activated by a Sigmoid activation function to generate spatial attention weights Ms(F'). The spatial attention weight Ms(F') is multiplied element-wise with the channel enhancement feature F' to obtain the final output feature F''; The processing procedure of the residual compensation fusion module RCFM includes: Receive shallow features Fs from the encoder and deep features Fd from the decoder; The shallow feature Fs and the deep feature Fd are concatenated along the channel dimension to obtain the concatenated feature Fcat; The spliced feature Fcat is linearly compressed through a 1×1 convolutional layer to obtain the fused feature Ffuse; The fused feature Ffuse is added element-wise to the deep feature Fd to obtain the final output Fout of the module; The processing procedure of the Detail Enhancement Multi-Scale Residual Module (DA-MSRM) includes: For the input feature map X, four convolutional branches with kernel sizes of 1×1, 3×3, 5×5 and 7×7 are passed in parallel. The number of output channels of each branch is one-quarter of the number of input channels, resulting in four multi-scale features F1, F3, F5 and F7. The four multi-scale features are concatenated along the channel dimension to obtain the concatenated feature Fcat; The spliced feature Fcat is passed through a 1×1 convolutional layer and batch normalized to obtain the fused feature Ffuse; The fusion feature F fuse is element-wise added with the input feature map X and activated by a ReLU function to obtain the final output Y of the module.
2. The method according to claim 1, characterized in that, The structure of the LD-GIRNet network is as follows: The input layer is followed by the first LDRU; The encoder includes a first downsampling layer, a second LDRU, a second downsampling layer, and a third LDRU connected in sequence. The decoder includes a third LDRU, a first upsampling layer, a fourth LDRU, a second upsampling layer, and a fifth LDRU connected in sequence. The RCFM is provided between the first downsampling layer and the second upsampling layer, and between the second downsampling layer and the first upsampling layer, respectively, for performing feature fusion; The end of the encoder is connected to the beginning of the decoder via a bottleneck layer; The output layer is connected after the fifth LDRU.
3. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the ghost imaging image reconstruction method as described in any one of claims 1 to 2.
4. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the ghost imaging image reconstruction method as described in any one of claims 1 to 2.