Computational ghost imaging reconstruction method based on global attention network

Abstract

The disclosed whole attention network-based computational ghost imaging reconstruction method irradiates a phase mask generated based on a Hadamard matrix on an MNIST dataset to obtain total light intensity values corresponding to the whole dataset; the total light intensity values are divided into a test set and a training set, data preprocessing is performed, a HAN network model is constructed and trained through the preprocessed training set; the trained HAN network model is verified through the preprocessed test set and the result is output, and the whole recovery process of the original image is realized. The whole attention network-based computational ghost imaging reconstruction method can capture more information features of the image by combining the layer attention module and the multi-head convolution attention module; the generalization effect of the HAN model is verified in the test set, and a high-quality target can be recovered under the premise of extremely low sampling rate.

Description

Technical Field

[0001] This invention belongs to the fields of quantum imaging and artificial intelligence, specifically relating to a computational ghost imaging reconstruction method based on a holistic attention network. Background Technology

[0002] Traditional image acquisition methods typically require sampling a large number of pixels to obtain complete image information. Computational ghost imaging, however, requires only a small number of random measurements to obtain a sparse representation of the image, significantly improving data acquisition efficiency. Furthermore, by measuring the sparse representation of the image, computational ghost imaging can effectively resist noise and distortion interference, thereby improving the information integrity of the image. Computational ghost imaging has applications in medical imaging, materials science, optical engineering, and security monitoring.

[0003] Computational ghost imaging is a technique that recovers object information through correlation measurements of light field intensity. Because it has unique properties that distinguish it from traditional imaging, it has attracted much attention. In this technique, the light emitted by the light source is split into two paths: one as a reference light and the other as a signal light. The reference light does not pass through the object and, after traveling a certain distance, its intensity distribution information is detected by a detector with spatial resolution. The signal light illuminates the object being imaged, and the transmitted or reflected light is collected by a barrel detector without spatial resolution, which records the total light intensity value. By correlating the intensity values ​​of the two light fields, the information of the object can be recovered.

[0004] In recent years, Global Attention Networks (HAN) have emerged. Initially applied to image super-resolution reconstruction, HAN has shown great promise in computational ghost imaging. Traditional channel attention methods lose the connections between different convolutional layers, while HAN's layer attention modules and multi-head convolutional attention modules can further extract information features from both shallow and deep layers of the image. Layer attention modules can adaptively emphasize hierarchical features between different layers, while multi-head convolutional attention modules can selectively capture more information features from each channel. Combining these two techniques improves the model's performance and efficiency. This combination allows each input to allow the network layers to pay attention to the parts of interest to them. Summary of the Invention

[0005] The purpose of this invention is to provide a computational ghost imaging reconstruction method based on a global attention network, which can greatly improve the imaging quality of the reconstructed image and enhance the reconstruction efficiency.

[0006] The technical solution adopted in this invention is a computational ghost imaging reconstruction method based on a global attention network, which specifically includes the following steps:

[0007] Step 1: First, obtain the phase mask generated based on the Hadamard matrix. Then, sequentially illuminate the entire MNIST dataset with the obtained phase mask to obtain the total light intensity value dataset corresponding to the entire MNIST dataset. Divide the total light intensity value dataset into training set and test set.

[0008] Step 2: Preprocess the training set and test set separately, construct the HAN network model, input the preprocessed training set into the HAN network model to train the network, and obtain the trained HAN network model by convergence through the loss function.

[0009] Step 3: Input the preprocessed test set into the trained HAN network model. After the loss converges, the HAN network model outputs the reconstructed image, realizing the entire reconstruction process of the original image.

[0010] The invention is further characterized in that:

[0011] The specific process of step 1 is as follows:

[0012] H1=[1] (1)

[0013]

[0014]

[0015]

[0016] In equations (1), (2), (3), and (4), For Kronecker product; H N The result is an N-order Hadamard matrix; obtain the phase mask sequence I generated based on the Hadamard matrix. m The obtained phase masks are sequentially projected onto the entire MNIST dataset, resulting in a dataset of total light intensity values ​​corresponding to the entire MNIST dataset, denoted as S. m The expression is as follows:

[0017] S m =∫I m (x,y)T(x,y)dxdy (5)

[0018] Where x represents the abscissa of the original image plane, y represents the ordinate of the original image plane, and T(x,y) represents the original image in the MNIST dataset;

[0019] Let the total light intensity value dataset be S. m It is divided into training set and test set.

[0020] Step 2 is as follows:

[0021] Step 2.1: Preprocess and output the training and test sets respectively;

[0022] Step 2.2: Construct the HAN network model and initialize the network parameters. Input the preprocessed training set results from Step 2.1 into the constructed HAN network model to train the parameters of each network layer.

[0023] Step 2.3: Use the mean squared error between the training result of step 2.2 and the corresponding original image as the loss to train the network. Backpropagate the network according to the loss function. At the same time, keep the hyperparameters in the network unchanged and train all parameters in the network until the value of the loss function remains dynamically stable. Otherwise, repeat steps 2.2 to 2.3 until the loss function converges to obtain the trained HAN network model.

[0024] In step 2.1, the preprocessing specifically involves taking the training set data and the test set data as inputs and passing them through two fully connected layers and two convolutional layers in sequence, then performing batch normalization, and finally outputting the preprocessed training set results and test set results through activation function layers respectively.

[0025] Each fully connected layer contains a batch normalization layer, a ReLU activation function layer, and a dropout layer. Each convolutional layer contains a batch normalization layer, a ReLU activation function layer, and a dropout layer. The kernel size of both convolutional layers is 5*5, and the stride is 1. The first convolutional layer has 16 feature layers, and the second convolutional layer has 32 feature layers.

[0026] The formula for batch normalization is as follows:

[0027] BN(K)=(K-μ _B) / sqrt(σ_B^2 + ε) (6)

[0028] Where ε is an infinitesimal quantity close to zero, μ_B represents the mean of image K, and σ_B^2 represents the variance of image K.

[0029] In step 2.2, the HAN network model includes a shallow feature extraction module, four residual group modules, a layer attention module, a multi-head convolutional attention module, and a reconstruction module.

[0030] The shallow feature extraction module includes a convolutional layer with a kernel size of 5*5, a stride of 2, and a feature layer size of 32.

[0031] Each of the four residual group modules includes five residual channel attention modules and one convolutional module. The convolutional result is residually concatenated with the data previously input to the residual group module, and the result is output. The convolutional module has a kernel size of 5*5, a stride of 1, and 32 feature layers. Each residual channel attention module consists of a convolutional layer and a channel attention layer. The convolutional layer has a kernel size of 5*5, a stride of 1, and 32 feature layers. The residual channel attention module performs residual concatenation between the input data and the data passed through the channel attention layer, and outputs and saves the result.

[0032] The layer attention module includes a matrix transpose layer, a soft maximization layer, and a convolutional layer. The convolutional layer is 5*5 in size with a stride of 1, and the feature layer has 32. The layer attention module first obtains the values ​​of the data in the four dimensions of batch, height, width, and channel, transforms them into a two-dimensional matrix, transposes them, multiplies the transposed matrix with the previously transformed two-dimensional matrix, performs a softmax operation on the result and outputs it. The data is then multiplied with the saved matrix, and the data is transformed into the same size and shape as the input data. This data is then added to the data input to the layer attention module and a convolution operation is performed before returning the result.

[0033] The multi-head convolutional attention module consists of three head convolutional attention modules, each including one convolutional layer and one convolutional transpose layer. The first head convolutional attention module has a 1x1 convolutional layer and a stride of 1 for both its convolutional and transpose layers, and a feature layer size of 32. The second head convolutional attention module has a 3x3 convolutional layer and a stride of 2 for both its convolutional and transpose layers, and a feature layer size of 32. The third head convolutional attention module has a 5x5 convolutional layer and a stride of 4, and a feature layer size of 32.

[0034] The reconstruction module includes two convolutional pooling modules and one convolutional transpose layer. In the first convolutional pooling module, the size of the convolutional layer is 5*5, the stride is 1, and the feature layer is 64. In the second convolutional pooling module, the size of the convolutional layer is 5*5, the stride is 1, and the feature layer is 64. Both pooling methods are max pooling. The size of the convolutional transpose layer is 3*3, the stride is 2, and the feature layer is 64.

[0035] The parameter training process for each network layer is as follows:

[0036] Step 2.2.1: Input the preprocessed training set results obtained in Step 2.1 into the shallow feature extraction module, perform convolution operation, save the data and output it;

[0037] Step 2.2.2: The data obtained in step 2.2.1 is passed through four residual group modules in sequence. The data passed through each residual group module is saved and concat is performed in the channel dimension. The concat merged data is then passed through a convolution transpose module and three convolution modules for convolution operation. Finally, the result is saved and input into the layer attention module and the data is saved.

[0038] The convolutional transpose module has a kernel size of 5*5, a stride of 2, and 128 feature layers; all three convolutional modules have a kernel size of 5*5, the first two convolutional modules have a stride of 1, and feature layers of 64 and 32 respectively, and the last convolutional module has a stride of 2 and a feature layer of 32.

[0039] Step 2.2.3: The data from the fourth residual group module in step 2.2.2 is then processed through one convolution module for convolution operation, the result is saved and output; the convolution kernel size of one convolution module is 5*5, the stride is 1, and the feature layer is 32;

[0040] Step 2.2.4: Input the data output in step 2.2.3 into the multi-head convolutional attention module, and perform spatial and channel attention simultaneously. Add the outputs of each head and input them into the sigmoid activation function. Multiply the result with the data saved in step 2.2.3 to obtain the attention and save the output.

[0041] Step 2.2.5: Add the data output from Steps 2.2.1, 2.2.2, and 2.2.4 together, and save and output the result.

[0042] Step 2.2.6: Pass the data output in step 2.2.5 through one convolutional module with a kernel size of 5*5, a stride of 1, and 32 feature layers, and save and output the result;

[0043] Step 2.2.7: Pass the data output in step 2.2.6 through one convolutional transpose layer with a kernel size of 5*5, a stride of 2, and 32 feature layers, and save and output the result;

[0044] Step 2.2.8: Input the data output in Step 2.2.7 into the reconstruction module, and output the final result to obtain the intermediate image.

[0045] intermediate image Represented as:

[0046]

[0047] Where H represents the constructed HAN network model, S ′ mH{S} represents the preprocessed training set. ′ m} indicates the process of substituting the preprocessed training set data into the constructed HAN network model for parameter training.

[0048] In step 2.3, the loss function expression is as follows:

[0049]

[0050] Where n is the number of pixels in the original image, j is the pixel position in the image, and T(x,y) is the original image. Let T(x,y) be the intermediate image. j This represents the pixel value at position j in the original image. Let j be the pixel value at position j in the intermediate image.

[0051] The beneficial effects of this invention are:

[0052] The computational ghost imaging reconstruction method based on the global attention network of this invention effectively improves the reconstruction effect and efficiency of computational ghost imaging. It uses a self-attention mechanism to adaptively learn the global correlation between different depths, channels and positions. The layer attention module can capture the long-distance dependencies between hierarchical layers, and the multi-head convolutional attention module can merge the channel and context information contained in each layer. The combination of the two modules can capture more information features. The generalization effect of the HAN model is verified on the test set. In terms of accuracy and visualization effect, it can recover high-quality targets under the premise of extremely low sampling rate. Attached Figure Description

[0053] Figure 1 This is a flowchart of the computational ghost imaging reconstruction method based on a global attention network according to the present invention;

[0054] Figure 2 This is a flowchart of the HAN network training process in the computational ghost imaging reconstruction method based on the global attention network of the present invention.

[0055] Figure 3 This is a flowchart of the residual group module in the computational ghost imaging reconstruction method of the present invention;

[0056] Figure 4 This is a framework diagram of the residual channel attention module in the computational ghost imaging reconstruction method of the present invention;

[0057] Figure 5 This is a schematic diagram of the structure of the layer attention module in the computational ghost imaging reconstruction method of the present invention;

[0058] Figure 6 This is a framework diagram of the multi-head convolutional attention module in the ghost imaging reconstruction method of this invention;

[0059] Figure 7 The image shows a comparison of the reconstruction results of the character "5" using existing differential ghost imaging reconstruction methods, the TVAL3 algorithm, and the computational ghost imaging reconstruction method based on the global attention network of this invention, at sampling rates of 1%, 3%, 5%, 10%, 20%, and 25%, respectively.

[0060] Figure 8 The images show a comparison of the reconstruction results of "0-9" using existing differential ghost imaging reconstruction methods, the TVAL3 algorithm, and the computational ghost imaging reconstruction method based on the global attention network of this invention, respectively, at a 5% sampling rate. Detailed Implementation

[0061] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0062] The computational ghost imaging reconstruction method based on global attention network of the present invention is as follows: Figure 1 As shown, please follow these steps:

[0063] Step 1: First, obtain the phase mask generated based on the Hadamard matrix. Then, sequentially illuminate the entire MNIST dataset with the obtained phase mask to obtain the total light intensity value dataset corresponding to the entire MNIST dataset. Divide the total light intensity value dataset into training set and test set.

[0064] The specific process of step 1 is as follows:

[0065] H1=[1] (1)

[0066]

[0067]

[0068]

[0069] In equations (1), (2), (3), and (4), For Kronecker product; H N The result is the Nth-order Hadamard matrix; obtain the phase mask sequence I generated based on the Hadamard matrix. m The obtained phase mask is sequentially projected onto the entire MNIST dataset, thus obtaining the total light intensity dataset corresponding to the entire MNIST dataset, denoted as S. m The expression is as follows:

[0070] S m =∫I m (x,y)T(x,y)dxdy (5)

[0071] Where x represents the x-coordinate of the original image plane, y represents the y-coordinate of the original image plane, and I m T(x,y) represents the phase mask sequence generated based on the Hadamard matrix, and T(x,y) represents the original image in the MNIST dataset.

[0072] Let the total light intensity value dataset be S. m Divide into training set and test set;

[0073] Step 2: Preprocess the training set and test set separately to construct the HAN network model. Input the preprocessed training set into the HAN network model to start the training phase of the entire neural network parameters. The trained HAN network model is obtained by converging through the loss function.

[0074] like Figure 1 and Figure 2 As shown, the specific process of step 2 is as follows:

[0075] Step 2.1: Take the training set data and test set data as inputs and pass them through the fully connected layer and convolutional layer with given parameters, then perform batch normalization, and finally output the preprocessed training set results and test set results through the activation function layer respectively.

[0076] In step 2.1, the system sequentially passes through two fully connected layers and two convolutional layers. Each fully connected layer contains a batch normalization layer, a ReLU activation function layer, and a dropout layer. Each convolutional layer contains a batch normalization layer, a ReLU activation function layer, and a dropout layer. The kernel size of both convolutional layers is 5*5, and the stride is 1. The first convolutional layer has 16 feature layers, and the second convolutional layer has 32 feature layers.

[0077] The formula for batch normalization in step 2.1 is as follows:

[0078] BN(K)=(K-μ _B) / sqrt(σ_B^2 + ε) (6)

[0079] Where ε is an infinitesimal quantity close to zero, μ_B represents the mean of image K, and σ_B^2 represents the variance of image K.

[0080] Step 2.2: Construct the HAN network model and initialize the network parameters. The process is as follows: Figure 2 As shown, the preprocessed training set results from step 2.1 are input into the constructed HAN network model to train the parameters of each network layer.

[0081] In step 2.2, the HAN network includes a shallow feature extraction module, four residual group modules, a layer attention module, a multi-head convolutional attention module, and a reconstruction module;

[0082] The shallow feature extraction module includes one convolutional layer with a kernel size of 5*5, a stride of 2, and 32 feature layers.

[0083] like Figure 3 As shown, each of the four residual group modules includes five residual channel attention modules and one convolution module (with a kernel size of 5*5, a stride of 1, and 32 feature layers). The result after convolution is residually concatenated with the data previously input to the residual group module, and the result is output.

[0084] like Figure 4 As shown, each residual channel attention module consists of a convolutional layer (with a kernel size of 5*5, a stride of 1, and a feature layer of 32) and a channel attention layer. The residual channel attention module performs residual concatenation between the input data and the data passed through the channel attention layer, and outputs and saves the result.

[0085] The layer attention module includes a matrix transpose layer, a soft maximization layer, and a convolutional layer. The convolutional kernel size of the convolutional layer is 5*5, the stride is 1, and the feature layer has 32. The layer attention module first obtains the values ​​of the data in the four dimensions of batch, height, width, and channel, transforms them into a two-dimensional matrix, transposes them, multiplies the transposed matrix with the previously transformed two-dimensional matrix, performs a softmax operation on the result and outputs it, multiplies the data with the saved matrix, transforms the data into data of the same size and shape as the input data, adds it to the data input to the layer attention module, performs a convolution operation, and returns the result.

[0086] Figure 5 This is a schematic diagram of the Layer Attention Module (LAM) in the Global Attention Network method. It learns the weights of hierarchical features by considering the correlation between multi-scale layers. The feature map of each layer is regarded as a response to a specific class, and the responses of different layers are interrelated. By obtaining the dependencies between features of different depths, this module can assign different attention weights to features of different depths and automatically improve the representation ability of extracted features.

[0087] like Figure 6As shown, the multi-head convolutional attention module includes three heads, each integrating spatial and channel attention. Each head convolutional attention module includes one convolutional layer and one convolutional transpose layer. The channel descent rate is controlled by adjusting the number of convolutional kernels. The first head convolutional attention module has a 1*1 size convolutional layer and a stride of 1, with 32 feature layers. The second head convolutional attention module has a 3*3 size convolutional layer and a stride of 2, with 32 feature layers. The third head convolutional attention module has a 5*5 size convolutional layer and a stride of 4, with 32 feature layers. The size of the convolutional kernels in each head is different, and the spatial descent rate is controlled by adjusting the size of the convolutional kernels. As an attention module, this module forces the model to retain attentionable information by relying on the bottleneck principle.

[0088] The reconstruction module includes two convolutional pooling modules and one convolutional transpose layer. In the first convolutional pooling module, the size of the convolutional layer is 5*5, the stride is 1, and the feature layer is 64. In the second convolutional pooling module, the size of the convolutional layer is 5*5, the stride is 1, and the feature layer is 64. Both pooling methods are max pooling. The size of the convolutional transpose layer is 3*3, the stride is 2, and the feature layer is 64.

[0089] like Figure 1 and Figure 2 As shown, the specific process of training the parameters of each network layer is as follows:

[0090] Step 2.2.1: Input the preprocessed training set results obtained in Step 2.1 into the convolutional layer of the shallow feature extraction module, perform convolution operation, save the data and output it;

[0091] Step 2.2.2, as follows Figure 2 As shown, the data obtained in step 2.2.1 is sequentially passed through four residual group modules. The data passed through each residual group module is saved and concat is performed in the channel dimension. The concat merged data is then passed through a convolution transpose module (convolution kernel size is 5*5, stride is 2, feature layer is 128) and three convolution modules (three convolution kernel sizes are all 5*5, the first two strides are 1, the feature layers are 64 and 32 respectively, the last stride is 2, the feature layer is 32) for convolution operation. The final result is saved and input into the layer attention module to save the data. It can improve the representation ability of the extracted features by obtaining the dependency relationship between features of different depths and assigning different attention weights to features of different layers.

[0092] Step 2.2.3: The data from the fourth residual group module in step 2.2.2 is then subjected to a convolution operation by another convolution module, and the result is saved and output.

[0093] The convolutional module has a kernel size of 5*5, a stride of 1, and 32 feature layers.

[0094] Step 2.2.4: Input the data output in step 2.2.3 into the multi-head convolutional attention module, and perform spatial and channel attention simultaneously. Add the outputs of each head and input them into the sigmoid activation function. Multiply the result with the data saved in step 2.2.3 to obtain the attention and save the output.

[0095] Step 2.2.5: Add the data output from Steps 2.2.1, 2.2.2, and 2.2.4 together, and save and output the result.

[0096] Step 2.2.6: Pass the data output in step 2.2.5 through one convolutional module with a kernel size of 5*5, a stride of 1, and 32 feature layers, and save and output the result;

[0097] Step 2.2.7: Pass the data output in step 2.2.6 through one convolutional transpose layer with a kernel size of 5*5, a stride of 2, and 32 feature layers, and save and output the result;

[0098] Step 2.2.8: Input the data output in Step 2.2.7 into the reconstruction module, and output the final result to obtain the intermediate image.

[0099] intermediate image Represented as:

[0100]

[0101] Where H represents the constructed HAN network model, S ′ m H{S} represents the preprocessed training set. ′ m} indicates the process of substituting the preprocessed training set data into the constructed HAN network model for parameter training;

[0102] Step 2.3: Use the mean squared error between the intermediate image and the corresponding original image from the training results in Step 2.2 as the loss to train the network. Backpropagate the network according to the loss function. At the same time, keep the hyperparameters in the network unchanged and train all parameters in the network until the value of the loss function remains dynamically stable. Otherwise, repeat Step 2.2 to Step 2.3 until the loss function converges to obtain the trained HAN network model.

[0103] The loss function expression in step 2.3 is as follows:

[0104]

[0105] Where n is the number of pixels in the original image, j is the pixel position in the image, and T(x,y) is the original image. Let T(x,y) be the intermediate image. j This represents the pixel value at position j in the original image. Let j be the pixel value at position j in the intermediate image.

[0106] Step 3: Validate the HAN network model trained in Step 2 using the preprocessed test set;

[0107] Specifically, the preprocessed test set is input into the trained HAN network model. At this point, the parameters of each network layer have been determined. After the loss converges, the output of the model is the final reconstructed image, thus realizing the entire reconstruction process of the original image.

[0108] The computational ghost imaging reconstruction method based on the global attention network of the present invention has the following advantages: it effectively improves the reconstruction effect and efficiency of computational ghost imaging; it uses a self-attention mechanism to adaptively learn the global correlation between different depths, channels and positions; the layer attention module can capture the long-distance dependencies between hierarchical layers; the multi-head convolutional attention module can merge the channel and context information contained in each layer; the combination of the two modules can capture more information features; the generalization effect of the HAN model is verified on the test set, and it can recover high-quality targets under the premise of extremely low sampling rate in terms of accuracy and visualization effect.

[0109] The following examples demonstrate the versatility of the computational ghost imaging reconstruction method based on the global attention network of this invention:

[0110] Example 1 reconstructs images using existing differential ghost imaging reconstruction methods at sampling rates of 1%, 3%, 5%, 10%, 20%, and 25%.

[0111] Example 2 uses the TVAL3 algorithm to reconstruct images at sampling rates of 1%, 3%, 5%, 10%, 20%, and 25%.

[0112] Example 3 uses the computational ghost imaging reconstruction method based on the global attention network of the present invention to reconstruct images at sampling rates of 1%, 3%, 5%, 10%, 20%, and 25%.

[0113] By dividing the collected data images into a training set of 10,000 images and a test set of 992 images, the number of random phase masks was reduced to 5% after the above operation, and good results were obtained.

[0114] The results of comparing the average numerical values ​​of the reconstructed images of the test set under different sampling rates in Examples 1-3 are shown in Table 1 below.

[0115] Table 1

[0116]

[0117] Table 1 shows the experimental data, illustrating the values ​​of various evaluation metrics for the three methods at different sampling rates. It can be observed that as the sampling rate increases, the values ​​of all metrics for computational ghost imaging based on the overall attention network improve. Compared to Example 1 (GI) and Example 2 (CSGI), at sampling rates of 20% and below, the computational ghost imaging reconstruction method of this invention outperforms the differential ghost imaging reconstruction method of Example 1 and the TVAL3 algorithm of Example 2. This indicates that at lower sampling rates, HAN has a significant advantage over the other two methods. However, at a sampling rate of 25%, the metrics of the computational ghost imaging reconstruction method of this invention are slightly lower than CSGI, indicating that at a sampling rate of 20%, the channel's attention information has reached saturation.

[0118] Figure 7 The images show a comparison of the reconstruction results of the character "5" using the methods of Examples 1, 2, and 3 at sampling rates of 1%, 3%, 5%, 10%, 20%, and 25%, respectively. It can be seen that Example 1 (GI) fails to reconstruct the original image well at almost all sampling rates, and the reconstructed images contain a large amount of noise. Only at sampling rates of 20% and above can the image reconstructed by Example 2 (CSGI) show a rough outline, but the reconstructed image lacks outline information. In contrast, the computational ghost imaging reconstruction method of this invention can effectively reconstruct the original image at all sampling rates. Therefore, the computational ghost imaging reconstruction method of this invention is effective and correct even at extremely low sampling rates.

[0119] Figure 8 The images show a comparison of the results of reconstructing "0-9" using the methods of Examples 1, 2, and 3 at a sampling rate of 5%. It was observed that at a sampling rate of 5%, the methods of Examples 1 (GI) and 2 (CSGI) could not effectively recover the original image, and both images contained a large amount of image noise and interference, making them difficult to distinguish. In contrast, the computational ghost imaging reconstruction method of this invention could recover the original image at a sampling rate of 5%, and the recovered image was extremely similar to the original image, demonstrating relatively high accuracy and effectiveness.

Claims

1. A computational ghost imaging reconstruction method based on a global attention network, characterized in that: Specifically, the steps include the following: Step 1: First, obtain the phase mask generated based on the Hadamard matrix. Then, sequentially illuminate the entire MNIST dataset with the obtained phase mask to obtain the total light intensity value dataset corresponding to the entire MNIST dataset. Divide the total light intensity value dataset into training set and test set. Step 2: Preprocess the training set and test set separately, construct the HAN network model, input the preprocessed training set into the HAN network model to train the network, and obtain the trained HAN network model by convergence through the loss function. Step 3: Input the preprocessed test set into the trained HAN network model. After the loss converges, the HAN network model outputs the reconstructed image, realizing the entire reconstruction process of the original image. The specific process of step 2 is as follows: Step 2.1: Preprocess the training set and test set separately and output them; Step 2.2: Construct the HAN network model and initialize the network parameters. Input the preprocessed training set results from Step 2.1 into the constructed HAN network model to train the parameters of each network layer. Step 2.3: Use the mean squared error between the training result of step 2.2 and the corresponding original image as the loss to train the network. Backpropagate the network according to the loss function. At the same time, keep the hyperparameters in the network unchanged and train all parameters in the network until the value of the loss function remains dynamically stable. Otherwise, repeat steps 2.2 to 2.3 until the loss function converges to obtain the trained HAN network model. In step 2.2, the HAN network model includes a shallow feature extraction module, four residual group modules, a layer attention module, a multi-head convolutional attention module, and a reconstruction module; The shallow feature extraction module includes a convolutional layer with a kernel size of 5. 5, step size is 2, feature layer is 32; Each of the four residual group modules includes five residual channel attention modules and one convolution module. The convolutional result is residually concatenated with the data previously input to the residual group module, and the result is output. The convolution kernel size of the convolution module is 5.

5. The stride is 1, and the feature layers are 32. Each residual channel attention module consists of one convolutional layer and one channel attention layer. The kernel size of the convolutional layer is 5.

5. The step size is 1, the feature layer is 32, and the residual channel attention module performs residual connection between the input data and the data that has passed through the channel attention layer, and outputs and saves the result; The layer attention module includes a matrix transpose layer, a soft maximization layer, and a convolutional layer with a size of 5.

5. The stride is 1, and the feature layer is 32. The attention module first obtains the data values ​​in the four dimensions of batch, height, width, and channel, transforms them into a two-dimensional matrix, transposes them, multiplies the transposed matrix with the previously transformed two-dimensional matrix, performs a softmax operation on the result and outputs it. The data is then multiplied with the saved matrix, and the data is transformed into the same size as the input data. This data is then added to the data input to the attention module and a convolution operation is performed before returning the result. The multi-head convolutional attention module includes three head convolutional attention modules, each of which includes a convolutional layer and a convolutional transpose layer; wherein the size of the convolutional layer and the convolutional transpose layer of the first head convolutional attention module is 1.

1. The stride is 1, and the feature layer size is 32; the size of the convolutional layer and the transpose layer of the second head convolutional attention module are both 3.

3. The stride is 2, and the feature layer size is 32; the size of the convolutional layer and the transpose layer of the third head convolutional attention module are both 5.

5. Step size is 4, and the number of feature layers is 32; The reconstruction module includes two convolutional pooling modules and one convolutional transpose layer. The size of the convolutional layer in the first convolutional pooling module is 5.

5. With a stride of 1 and a feature layer size of 64, the size of the convolutional layers in the second convolutional pooling module is 5.

5. Stride size is 1, feature layers are 64, all pooling is max pooling, and the size of the transposed convolutional layer is 3.

3. Step size is 2, and the feature layer is 64.

2. The computational ghost imaging reconstruction method based on a global attention network according to claim 1, characterized in that, The specific process of step 1 is as follows: =[1](1) = (2) = (3) = (4) In equations (1), (2), (3), and (4), For Kronecker product; The result is an N-order Hadamard matrix; obtain the phase mask sequence generated based on the Hadamard matrix. The obtained phase masks are sequentially projected onto the entire MNIST dataset, resulting in a dataset of total light intensity values ​​corresponding to the entire MNIST dataset, denoted as . The expression is as follows: = (5) Where x represents the x-coordinate of the original image plane, and y represents the y-coordinate of the original image plane. Represents the original images in the MNIST dataset; The total light intensity value dataset is denoted as... It is divided into training set and test set.

3. The computational ghost imaging reconstruction method based on a global attention network according to claim 1, characterized in that, In step 2.1, the preprocessing specifically involves taking the training set data and the test set data as inputs and passing them through two fully connected layers and two convolutional layers in sequence, then performing batch normalization, and finally outputting the preprocessed training set results and test set results through activation function layers respectively. Each fully connected layer contains a batch normalization layer, a ReLU activation function layer, and a dropout layer; each convolutional layer contains a batch normalization layer, a ReLU activation function layer, and a dropout layer. The kernel size of both convolutional layers is 5.

5. The stride is 1 for all convolutional layers. The first convolutional layer has 16 feature layers, and the second convolutional layer has 32 feature layers.

4. The computational ghost imaging reconstruction method based on a global attention network according to claim 3, characterized in that, The formula for batch normalization is as follows: BN(K)=(K-μ _B) / sqrt(σ_B^2 + )(6) in, It is an infinitesimal quantity, μ_B represents the mean of image K, and σ_B^2 represents the variance of image K.

5. The computational ghost imaging reconstruction method based on a global attention network according to claim 1, characterized in that, The parameter training process for each network layer is as follows: Step 2.2.1: Input the preprocessed training set results obtained in Step 2.1 into the shallow feature extraction module, perform convolution operation, save the data and output it; Step 2.2.2: The data obtained in step 2.2.1 is passed through four residual group modules in sequence. The data passed through each residual group module is saved and concat is performed in the channel dimension. The concat merged data is then passed through a convolution transpose module and three convolution modules for convolution operation. Finally, the result is saved and input into the layer attention module and the data is saved. The kernel size of the convolution transpose module is 5.

5. The stride is 2, and the feature layer size is 128; the kernel size of each of the three convolutional modules is 5.

5. The stride of the first two convolutional modules is 1, and the feature layers are 64 and 32 respectively. The stride of the last convolutional module is 2, and the feature layer is 32. Step 2.2.3: The data from the fourth residual group module in Step 2.2.2 is then subjected to a convolution operation by another convolution module, the result is saved, and the result is output; the kernel size of the convolution module is 5. 5, step size is 1, feature layer is 32; Step 2.2.4: Input the data output in step 2.2.3 into the multi-head convolutional attention module, and perform spatial and channel attention simultaneously. Add the outputs of each head and input them into the sigmoid activation function. Multiply the result with the data saved in step 2.2.3 to obtain the attention and save the output. Step 2.2.5: Add the data output from Steps 2.2.1, 2.2.2, and 2.2.4 together, and save and output the result. Step 2.2.6: Pass the data output from Step 2.2.5 through one convolutional module with a kernel size of 5.

5. Set the step size to 1, the feature layer to 32, and save and output the results; Step 2.2.7: Pass the data output from Step 2.2.6 through one convolutional transpose layer with a kernel size of 5.

5. Set the step size to 2, the feature layer size to 32, and save and output the results; Step 2.2.8: Input the data output in Step 2.2.7 into the reconstruction module, and output the final result to obtain the intermediate image. ; intermediate image Represented as: = (7) Where H represents the constructed HAN network model, This represents the preprocessed training set. This refers to the process of substituting the preprocessed training set data into the constructed HAN network model for parameter training.

6. The computational ghost imaging reconstruction method based on a global attention network according to claim 5, characterized in that, In step 2.3, the loss function expression is as follows: MSE(T(x,y), )= (8) Where n is the number of pixels in the original image, j is the pixel position in the image, and T(x,y) is the original image. This is an intermediate image. This represents the pixel value at position j in the original image. This represents the pixel value at position j in the intermediate image.

Images