Coding illumination lens-free imaging system based on feature fusion

Abstract

The invention discloses a coded illumination lensless imaging system based on feature fusion, which is characterized in that coded illumination optimization and deep learning feature fusion are combined to construct a complete imaging architecture of physical coding-fast reconstruction-intelligent enhancement. Firstly, a series of separable illumination patterns with complementary information coding characteristics are projected to improve the morbidity of a system matrix in traditional lensless imaging. The method comprises the following steps of: firstly, carrying out primary image reconstruction by adopting a fast closed-form solution algorithm based on least squares to obtain a primary reconstruction result containing complementary information, and finally, carrying out deep information integration on the primary reconstruction result by adopting a feature extraction fusion network, and finishing self-adaptive weighting and fusion of information in different illumination modes on a feature level. Compared with the prior art, the method has the advantages that the imaging detail reduction degree is high, the system robustness is good, and the lens-free imaging quality is effectively improved.

Description

Technical Field

[0001] This invention belongs to the field of lensless imaging technology, and studies feature extraction and fusion networks under coded illumination conditions to address imaging quality issues. Background Technology

[0002] Traditional lens imaging systems are limited by the inherent characteristics of physical lenses and are affected by various factors during the imaging process, such as optical distortion, chromatic aberration, and light path refraction. These problems often affect image quality, especially in applications requiring high resolution and high contrast. Furthermore, lens systems are often large and heavy, creating bottlenecks in portability, cost, and application scope. To overcome these issues, lensless imaging technology is gaining increasing attention.

[0003] Lensless imaging, as an emerging imaging technique, completely eliminates physical lenses and uses computational methods to reconstruct the target scene from the captured image, circumventing aberration and focal length limitations. This offers significant advantages in achieving convenient and lightweight imaging systems. This shift not only enables a more compact and low-cost system architecture but also broadens the imaging band adaptability, especially in the field of microscopy, achieving both large field of view and high resolution, opening new avenues for portable diagnostics, on-site monitoring, and observation under extreme conditions. However, due to the inherent underdeterminism of this system, especially in complex scenes, image reconstruction often faces the problem of non-unique solutions; that is, given the observational data, multiple different reconstruction results may exist. This non-uniqueness complicates the process of restoring the original image, making it difficult for the reconstruction algorithm to determine the optimal solution, thus posing numerous challenges to improving reconstruction quality.

[0004] To address this issue, coded lighting technology has emerged as a controllable, structured lighting method. This technology, by introducing multiple different coding modes, transmits information more efficiently, significantly suppressing system ill-conditioning and improving the stability and accuracy of reconstruction. However, current methods generally lack effective mechanisms for fusing multi-mode lighting information, easily leading to information redundancy or loss, which further restricts the improvement of reconstruction quality.

[0005] To address the aforementioned challenges, this invention proposes a lensless imaging system based on coded illumination using feature fusion. This system performs preliminary reconstruction under multiple illumination modes and efficiently fuses the different initial reconstruction results in the frequency domain. First, the system uses a shifted illumination pattern to perform preliminary image processing, creating complementary coded illumination images from four different illumination modes. Second, the system performs preliminary reconstruction on the images under different illumination modes, resulting in four preliminary reconstructed images from each initial image. Then, a feature extraction and fusion network extracts features from the preliminary reconstruction results, first extracting shallow features, then extracting global features and detailed features from two complementary dimensions. Global features capture consistency information under different illumination conditions, achieving overall scene reconstruction; while detailed features focus on the unique details of each illumination mode, achieving supplementary reconstruction of details. Finally, the network fuses the extracted global and detailed features, decoding to obtain the final result, thus achieving efficient fusion of preliminary reconstruction results under multiple illumination modes. Experimental results show that the system has high reconstruction accuracy and performs well even in complex scenes, demonstrating strong robustness and enabling high-quality lensless imaging. Summary of the Invention

[0006] This invention discloses a lensless imaging system based on coded illumination using feature fusion. This system combines coded illumination optimization with deep learning feature fusion to construct a complete imaging architecture of "physical encoding - rapid reconstruction - intelligent enhancement." First, coded illumination technology is used to actively modulate the scene, projecting a series of separable illumination patterns with complementary information encoding characteristics to improve the ill-conditioned nature of the system matrix in traditional lensless imaging. Second, a fast closed-form algorithm based on least squares is used for preliminary image reconstruction, obtaining an initial reconstruction result containing complementary information. Finally, a feature extraction and fusion network is used to deeply integrate the information from the initial reconstruction result. The feature encoder first extracts shared shallow information, then extracts it separately from two complementary dimensions: global information and detail information. Adaptive weighting and fusion of information under different illumination modes are completed at the feature level. Experimental results show that the present invention achieves high imaging detail restoration, good system robustness, and effectively improves the quality of lensless imaging.

[0007] To facilitate the description of the present invention, the following terms are defined first:

[0008] Definition 1: Classical convolution operation

[0009] Classical convolution is a fundamental mathematical operation in signal and image processing, used to describe a weighted moving average of one function (or signal) on another function (or signal). In image processing and deep learning, convolution is primarily used for feature extraction, filtering, and signal transformation. For a detailed explanation of classical convolution, see "https: / / blog.csdn.net / raoxiaoya / article / details / 131655348".

[0010] Definition 2: The classic CatvsDog dataset

[0011] The CatvsDog dataset is a widely used dataset for image classification and computer vision research, containing images of both dogs and cats. This dataset aims to allow researchers and developers to evaluate the performance of image classification algorithms. The complete dataset can be obtained from "https: / / www.kaggle.com / c / dogs-vs-cats".

[0012] Definition 3: Standard method for forming a point light source

[0013] A point light source is an idealized light source model with an infinitesimally small size that emits light uniformly in all directions. In optical systems, point light sources are often used to analyze the system's point spread function (PSF), which is the system's response to a point light source. For a standard method for forming a point light source, please refer to the literature "Chen K, Li Y, Li Z, et al. Enhancing object recognition for lensless cameras through PSF correction and feature loss[J]. Optics and Laser Technology, 2025, 190113077-113077."

[0014] Definition 4: Standard Binary Amplitude Masking Method

[0015] Binary amplitude masking is a commonly used modulation method in optical imaging and signal processing. It controls the amplitude transmittance of light waves through a two-dimensional matrix containing only 0s and 1s. In lensless imaging, binary amplitude masks are often used to replace lenses to encode the incident light field and form a modulation pattern on the sensor. For a detailed description of the standard binary amplitude masking method, please refer to the literature "Salman MA, Ali A, Aswin S, et al. FlatCam: Thin, Lensless Cameras Using Coded Aperture and Computation[J].IEEE Transactions on Computational Imaging,2017,3(3):384-397."

[0016] Definition 5: Standard method for forming shift point patterns

[0017] A shifted-point pattern is a coded illumination pattern generated by translating a periodically arranged lattice. Specifically, a base pattern is cyclically shifted in both the horizontal and vertical directions to produce different illumination patterns. These patterns are orthogonal and can be used in multi-frame imaging to improve the system condition number. For a detailed description of standard shifted-point pattern formation methods, see “Y. Zheng and M. S. Asif, 'Coded Illumination for Improved Lensless Imaging,' in IEEE Transactions on Computational Imaging, vol. 9, pp. 172-184, 2023.”

[0018] Definition 6: Classic transpose operation

[0019] The classic transpose operation is a fundamental matrix operation that interchanges the rows and columns of a matrix. In linear algebra, signal processing, and machine learning, the transpose operation is a basic operation for constructing model equations, calculating gradients, and solving optimization problems. For a detailed explanation of the classic transpose operation, see "https: / / blog.csdn.net / lee985_ / article / details / 152941838".

[0020] Definition 7: Classical Least Squares Algorithm

[0021] Least squares is a mathematical optimization technique that finds the best function match for data by minimizing the sum of squared errors. In image reconstruction, the least squares algorithm is used to solve the linear inverse problem, that is, to find the solution that minimizes the difference between the observed data and the model prediction. For a detailed explanation of the classic least squares algorithm, see the reference "Gonzalez RC, Woods RE, Masters BR. Digital Image Processing, Third Edition[J]. Journal of BiomedicalOptics, 2009, 14(2):029901."

[0022] Definition 8: Classic method for minimizing parameters

[0023] The classic method of minimizing parameters is a core concept in optimization theory, representing the search for the values ​​of independent variables that minimize the objective function. In image reconstruction and machine learning, this method is used to determine the optimal model parameters or reconstruction results. For a detailed explanation of the classic method of minimizing parameters, please see "https: / / blog.csdn.net / qq_18310041 / article / details / 94440930".

[0024] Definition 9: Classical L2 norm squaring operation

[0025] The classic L2 norm squared operation is a fundamental mathematical tool for measuring the "size" of a vector or matrix, calculated as the sum of the squares of its elements. In optimization problems, it is often used as a data fidelity term or regularization term in the loss function to measure the smoothness of the error or constraint solution. For a detailed explanation of the classic L2 norm squared operation, see the reference "Thang P. Group action and L2-normestimates of geometric problems[J]. Journal of Mathematical Analysis and Applications, 2024, 531(1P2)."

[0026] Definition 10: Standard Restormer block construction method

[0027] The Restormer block is a Transformer block for image restoration that achieves efficient processing on high-resolution images through multi-scale local-global representation learning. The Restormer block incorporates gated depthwise separable convolutions and a multi-head transposed attention mechanism, enabling it to capture long-range dependencies while maintaining computational efficiency. For details on the standard Restormer block construction method, please refer to the paper "Vaswani, Ashish et al. 'Attention is All you Need.' Neural Information Processing Systems (2017)."

[0028] Definition 11: Standard Lite Transformer Block Construction Method

[0029] The Lite Transformer block is a lightweight Transformer block designed specifically for mobile devices. It reduces parameters and computational cost through long-short-range attention mechanisms and a flattened feedforward network while maintaining model performance. For details on the standard Lite Transformer block construction method, please refer to the literature “Devlin, Jacob et al. “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding.” North American Chapter of the Association for Computational Linguistics (2019).”.

[0030] Definition 12: Standard method for constructing reversible neural network blocks

[0031] A reversible neural network block is a network block with reversible properties, allowing sufficient information to be retained during forward propagation to reconstruct the input, thereby reducing memory consumption. A typical reversible block uses an affine coupling layer to split the input into two parts and applies a reversible transformation to ensure the accuracy of information transmission. For details on standard reversible neural network block construction methods, please refer to "https: / / blog.csdn.net / All_to_be_nice_ / article / details / 146960429".

[0032] Definition 13: Classical method for calculating mean squared error

[0033] Mean squared error (MSE) is a commonly used metric to measure the difference between an estimated value and the true value. It is calculated as the average of the squared errors. For images, MSE is calculated as the mean of the squared differences between corresponding pixel values ​​in two images. For a detailed explanation of the classic method for calculating MSE, please refer to "https: / / blog.csdn.net / u013172930 / article / details / 142748899".

[0034] Definition 14: Classical methods for calculating structural similarity

[0035] Structural similarity (SSIM) is a metric for measuring the similarity between two images, taking into account brightness, contrast, and structural information. The SSIM value ranges from -1 to 1, with higher values ​​indicating greater similarity between the two images. For a detailed explanation of classic structural similarity calculation methods, please refer to "https: / / blog.csdn.net / ergevv / article / details / 144727979".

[0036] Definition 15: Classical gradient loss calculation method

[0037] Gradient loss measures the difference between two images in the gradient domain, typically achieved by calculating the difference in the L1 or L2 norm of the image gradients. Gradient loss can emphasize the preservation of image edges and textures. For a detailed explanation of classic gradient loss calculation methods, please refer to "https: / / blog.csdn.net / weixin_45594172 / article / details / 136429289".

[0038] Definition 16: Classical learning-aware image patch similarity calculation method

[0039] The classic Learning Perceptual Image Patch Similarity (LPIPS) is a deep learning-based image quality assessment metric that evaluates quality by calculating the similarity between feature patches in an input image and perceptual features in a reference image. LPIPS extracts features using a pre-trained convolutional neural network, accurately capturing differences in human visual perception and better reflecting human subjective feelings about image quality. For details on the classic learning perceptual image patch similarity calculation method, please refer to the literature “Zhang, Richard et al. “The Unreasonable Effectiveness of Deep Features as a Perceptual Metric.” 2018 IEEE / CVF Conference on Computer Vision and Pattern Recognition (2018): 586-595.”

[0040] Definition 17: Standard method for calculating weighted loss summation

[0041] Weighted loss summation combines the contributions of multiple loss functions according to predetermined weights, with the loss function having the largest weight playing a more significant role in the total loss function. For a detailed explanation of the standard weighted loss summation method, please refer to "https: / / blog.csdn.net / d8dongdong / article / details / 134364194".

[0042] Definition 18: Standard Feature Extraction Method

[0043] Standard feature extraction methods refer to the process of automatically learning or manually designing a set of representative features from raw data. In the field of deep learning, multi-level feature representations are typically learned automatically through the hierarchical structure of convolutional neural networks, ranging from shallow edges and textures to deep semantic features. For details on standard feature extraction methods, please refer to the literature "Wenli Z, Tingsong Z, Jianyi Z, et al. LST-EMG-Net: Long short-term transformer feature fusion network for sEMG gesture recognition [J]. Frontiers in Neurorobotics, 2023, 171127338-1127338."

[0044] Definition 19: Standard Feature Fusion Method

[0045] Standard feature fusion methods refer to the effective combination of features from different sources, levels, or modalities to obtain more comprehensive and robust feature representations. Common feature fusion methods include early fusion (fusion at the input layer), mid-stage fusion (fusion at the intermediate layer), and late-stage fusion (fusion at the decision layer). Standard feature fusion methods aim to fully utilize complementary information to improve model performance. For details on standard feature fusion methods, please refer to the literature "Hu R, Tang X, Hao K. A Geometric-enhanced Feature Fusion Method for Adaptive Position Embedding in Transformer-based Models[C]2025:207-214."

[0046] Definition 20: Standard Feature Concatenation Method

[0047] The standard feature concatenation method involves joining multiple feature tensors along a certain dimension (usually the channel dimension) to form a new feature tensor. This method is often used to fuse features from different branches or layers. For details on the standard feature concatenation method, please see "https: / / blog.csdn.net / qq_23865133 / article / details / 136482657".

[0048] Definition 21: Standard Feature Decoding Method

[0049] Standard feature decoding methods refer to the process of mapping extracted and fused high-level features back to the original data space or the output space required by the task. In tasks such as image reconstruction and segmentation, decoders typically consist of deconvolutional layers, upsampling layers, and convolutional layers, progressively restoring spatial details and generating an output that matches the input size. Standard feature decoding methods need to correspond to the encoder structure to ensure that information can be effectively reconstructed. For details on standard feature decoding methods, please refer to the literature "Yifan W, Weigang C, Tao Y, et al. Dynamic Multi-Graph Convolution based Channel-Weighted Transformer Feature Fusion Network for Epileptic Seizure Prediction.[J]. IEEE transactions on neural systems and rehabilitationengineering : a publication of the IEEE Engineering in Medicine and Biology Society, 2023.".

[0050] Definition 22: The classic Adam algorithm

[0051] Adam (Adaptive Moment Estimation) is an adaptive learning rate optimization algorithm that combines the advantages of momentum methods and RMSProp. It adjusts the learning rate of each parameter by calculating the first and second moment estimates of the gradient. For a detailed explanation of the classic structural similarity calculation method, please refer to the literature "Kingma PD, Ba J. Adam: A Method for Stochastic Optimization.[J]. CoRR, 2014, abs / 1412.6980".

[0052] Definition 23: Standard Codified Lighting Operation

[0053] Standard coded illumination is an active imaging technique that illuminates a scene by controlling a light source to emit a light field with a specific spatial coding pattern, thereby obtaining modulated measurement data on a sensor. In computational imaging, coded illumination can improve the condition number of the system matrix and enhance the ability to acquire specific spatial frequency information. For details on standard coded illumination, please refer to the literature "Angela C, Xu M, RGA. Compressive spectral X-ray tomography based on spatial and spectral coded illumination.[J]. Optics express,2019,27(8):10745-10764."

[0054] Definition 24: Standard Lensless Imaging Method

[0055] Lensless imaging is a computational imaging technique that modulates the light field by using a coded mask instead of a traditional lens to form a coded image that cannot be directly recognized by the human eye. Typical lensless imaging systems include FlatCam and DiffuserCam. For details on standard lensless imaging methods, please refer to the literature “Nick A, Grace K, Reinhard H, et al. DiffuserCam: lensless single-exposure 3D imaging[J]. Optica, 2018, 5(1): 1-1.”.

[0056] Definition 25: Standard Lens-less Image Reconstruction Method

[0057] Standard lensless image reconstruction methods refer to the complete process of recovering the original scene image from coded sensor measurements captured by a lensless camera using computational algorithms. This method first establishes a physical model describing the imaging process of the optical system, and then solves the inverse problem to reconstruct the image. For details on standard lensless image reconstruction methods, please refer to the literature “Cha Zhang and Tsuhan Chen, "Light field capturing with lensless cameras," IEEE International Conference on Image Processing 2005, Genova, Italy, 2005.”

[0058] Definition 26: Standard forward propagation method

[0059] Forward propagation is the process by which data is transferred from the input layer through the hidden layers to the output layer in a neural network. Each layer performs linear transformations and non-linear activations on the input until the final output is obtained. See "https: / / blog.csdn.net / tang7mj / article / details / 147602809" for details.

[0060] Definition 27: Classical method for calculating peak signal-to-noise ratio

[0061] Peak Signal-to-Noise Ratio (PSNR) is a standard metric for evaluating the quality of image compression or transmission. It is calculated by comparing the maximum possible signal strength to the peak noise level. PSNR is typically expressed on a logarithmic scale (decibels); a higher value indicates better image quality. See "https: / / blog.csdn.net / qq_44111805 / article / details / 127676377" for details.

[0062] A lensless imaging system based on feature fusion and coded illumination, comprising the following steps:

[0063] Step 1: Initialize the parameters of the coded illumination lensless imaging system

[0064] The initialization parameters for the coded illumination module include: the (i, j)th complete coded illumination pattern matrix of the coded illumination module, denoted as... The i-th basis illumination vector in the horizontal direction of the encoded illumination is denoted as... The j-th basis illumination vector in the vertical direction of the encoded illumination is denoted as... ;

[0065] Initialize the operating parameters of the lensless imaging system, including: the input light rays of the lensless imaging system, denoted as... The input image of a lensless imaging system is denoted as... The measurement results of the lensless imaging system are denoted as... The classic convolution operation in Definition 1 is denoted as... The regularization coefficient in the process of reconstructing a lensless image is denoted as . The gradient operator in the process of reconstructing a lensless image is denoted as... .

[0066] Given a fixed system, the initial values ​​of all the above parameters are known.

[0067] Step 2: Prepare the dataset

[0068] Using the classic CatvsDog dataset from Definition 2 as the data base, 10,000 images are randomly selected from it to obtain the dataset of this invention, denoted as CvD;

[0069] Using the standard point light source formation method in Definition 3, the input light obtained in step 1 is processed. After processing, a standard point light source is obtained, denoted as POT;

[0070] Using the standard binary amplitude masking method in Definition 4, the standard point light source (POT) is processed to obtain the standard image after amplitude masking, denoted as PSF.

[0071] Step 3: Construct the coded lighting module

[0072] Using the standard shift point pattern formation method defined in Definition 5, the input light obtained in step 1 is processed. After processing, the coded lighting module is obtained, as shown below:

[0073]

[0074] in, This represents the (i, j)th complete coded lighting pattern matrix of the coded lighting module obtained in step 1. Let i be the i-th base illumination vector in the horizontal direction of the encoded illumination obtained in step 1. Let j be the j-th base illumination vector in the vertical direction of the encoded illumination obtained in step 1. This defines six classic transpose operations.

[0075] Step 4: Construct the initial reconstruction module

[0076] The classic least squares algorithm in Definition 7 is used to analyze the measurement results of the lensless imaging system obtained in step 1. The reconstruction process yields a preliminary reconstruction module, as shown below:

[0077]

[0078] in, These are the preliminary imaging results of the lensless imaging system. This is the classic method for minimizing parameters as defined in 8; PSF is the standard image obtained in step 2 after amplitude masking. The convolution operation obtained in step 1, The measurement results of the lensless imaging system obtained in step 1 are as follows. The input image of the lensless imaging system obtained in step 1. It is the classic L2 norm squaring operation in Definition 9. These are the regularization coefficients obtained in step 1 during the process of reconstructing the lensless image. This refers to the gradient operator used in the process of reconstructing the lensless image obtained in step 1.

[0079] Step 5: Construct a feature extraction and fusion network

[0080] Step 5.1: Initialize the feature extraction and fusion network parameters

[0081] Initialize the feature extraction and fusion network parameters, including: the input image of the feature extraction and fusion network is the preliminary imaging result of the lensless imaging system obtained in step 4. The output image of the feature extraction fusion network is denoted as... The optimal solution of the output image of the feature extraction fusion network is denoted as... ;

[0082] Eight Restormer blocks are constructed using the standard Restormer block construction method in Definition 10, denoted as restormer_block1, restormer_block2, restormer_block3, restormer_block4, restormer_block5, restormer_block6, restormer_block7, and restormer_block8.

[0083] Four Lite Transformer blocks, denoted as lt_block1, lt_block2, lt_block3, and lt_block4, are constructed using the standard Lite Transformer block construction method in Definition 11.

[0084] Four INN blocks are constructed using the standard reversible neural network block construction method in Definition 12, denoted as inn_block1, inn_block2, inn_block3, and inn_block4;

[0085] Step 5.2: Construct the feature extraction module

[0086] Step 5.2.1: Construct a shallow feature extraction encoder

[0087] The first four Restormer blocks obtained in step 5.1, restormer_block1, restormer_block2, restormer_block3, and restormer_block4, are sequentially connected to obtain the shallow feature extraction encoder, denoted as LFE_Encoder.

[0088] Step 5.2.2: Construct a complementary feature extraction encoder

[0089] The first two Lite Transformer blocks lt_block1 and lt_block2 obtained in step 5.1 are sequentially connected to obtain the global feature extraction encoder, denoted as GFE_Encoder;

[0090] The first two INN blocks inn_block1 and inn_block2 obtained in step 5.1 are concatenated sequentially to obtain the detail feature extraction encoder, denoted as DFE_Encoder;

[0091] Step 5.3: Construct the feature fusion module

[0092] The last two Lite Transformer blocks lt_block3 and lt_block4 obtained in step 5.1 are connected sequentially to obtain the global feature fusion module, denoted as GF_Fusion;

[0093] The last two INN blocks, inn_block3 and inn_block4, obtained in step 5.1 are sequentially connected to obtain the detail feature fusion module, denoted as DF_Fusion.

[0094] Step 5.4: Construct the feature decoder module

[0095] The last four Restormer blocks obtained in step 5.1, restormer_block5, restormer_block6, restormer_block7, and restormer_block8, are sequentially connected to obtain the feature decoder, denoted as Decoder.

[0096] Step 5.5: Construct the loss function module

[0097] Using the classic mean square error calculation method in Definition 13, the input image of the lensless imaging system obtained in step 1 is processed. The output image of the feature extraction fusion network in step 5.1 After comparison, the mean squared error loss was calculated and denoted as... ;

[0098] Using the classic structural similarity calculation method in Definition 14, the input image of the lensless imaging system obtained in step 1 is processed. The output image of the feature extraction fusion network in step 5.1 The structural similarity loss is calculated by performing a comparison process, and denoted as . ;

[0099] Using the classic gradient loss calculation method in Definition 15, the input image of the lensless imaging system obtained in step 1 is processed. The output image of the feature extraction fusion network in step 5.1 By performing a comparison, the gradient loss is calculated and denoted as... ;

[0100] Using the classic learning-based perceptual image patch similarity calculation method in Definition 16, the input image of the lensless imaging system obtained in step 1 is processed. The output image of the feature extraction fusion network in step 5.1 The perceptual loss was calculated by performing a comparison process and denoted as . ;

[0101] The weighted loss summation method in Definition 17 is used to calculate the mean square error loss. Structural similarity loss gradient loss Perceived loss By performing a weighted summation, we obtain the loss function module, as shown below:

[0102]

[0103] in, Mean squared error loss The weighting coefficients, It is structural similarity loss The weighting coefficients, It is gradient loss The weighting coefficients, It is perceived loss The weighting coefficients.

[0104] Step 5.6: Construct a feature extraction and fusion network

[0105] Using the standard feature extraction method defined in Definition 18, and employing the shallow feature extraction encoder LFE_Encoder obtained in step 5.2.1, the preliminary imaging results of the lensless imaging system obtained in step 4 are processed. Feature extraction is performed to obtain shallow features of the image. ;

[0106] The standard feature extraction method in Definition 18 is adopted, and the global feature extraction encoder GFE_Encoder obtained in step 5.2.2 is used to extract shallow features of the image. Feature extraction is performed to obtain the global features of the image. ;

[0107] The standard feature extraction method in Definition 18 is adopted, and the detail feature extraction encoder DFE_Encoder obtained in step 5.2.2 is used to extract shallow features of the image. Feature extraction is performed to obtain the detailed features of the image. ;

[0108] The standard feature fusion method in Definition 19 is adopted, and the global feature fusion module GE_Fusion obtained in step 5.2 is used to process the global features of the image. Perform feature fusion to obtain fused global features. ;

[0109] The standard feature fusion method in Definition 19 is adopted, and the detail feature fusion module DE_Fusion obtained in step 5.2 is used to process the detail features of the image. Perform feature fusion to obtain fused detailed features. ;

[0110] Using the 20 standard feature concatenation methods defined, global features are fused. and blending details The fused features Fu are obtained by concatenating the data along the channel dimension.

[0111] Using the standard feature decoding method in Definition 21, and employing the feature decoder obtained in step 5.3, the fused features are processed. Decode the image to generate the output image of the feature extraction and fusion network in step 5.1. ;

[0112] We employ the classic Adam algorithm from Definition 22, using the loss function module obtained in step 5.4. The output image of the feature extraction fusion network Iterative updates yield the optimal solution for the output image of the feature extraction fusion network in step 5.1. .

[0113] Step 6: Construct a lensless imaging system with coded illumination

[0114] Using the standard coded illumination operation in Definition 23, the coded illumination module obtained in step 3 is used to perform coded illumination processing on the dataset CvD obtained in step 2 to form a coded illumination image, denoted as Coded_CvD;

[0115] Using the standard lensless imaging method in Definition 24, the coded illumination image Coded_CvD is simulated and encoded to obtain the lensless image dataset of this invention, denoted as Lensless_CvD.

[0116] The Lensless_CvD image dataset obtained in this invention is randomly divided into two parts: a training set and a test set, with a ratio of 9:1. The training set is denoted as Train, and the test set is denoted as Test.

[0117] Using the standard lensless image reconstruction method defined in Definition 25, and employing the preliminary reconstruction module obtained in step 3, preliminary reconstruction is performed on the obtained training set Train to obtain the preliminary imaging result of the training set, denoted as... ;

[0118] The classic Adam algorithm from Definition 22 is used to train the feature extraction and fusion network obtained in step 4, and the preliminary imaging results are used on the training set. As input, the coded illumination lensless imaging system, denoted as CL_Lensless, is obtained after training.

[0119] Step 7: Test the coded illumination lensless imaging system

[0120] Using the standard lensless image reconstruction method defined in Definition 25, and employing the preliminary reconstruction module obtained in step 3, preliminary reconstruction is performed on the obtained test set Test to obtain the preliminary imaging result of the test set, denoted as . ;

[0121] The forward propagation method according to Definition 26 is used to test the coded illumination lensless imaging system CL_Lensless obtained in step 6, and the preliminary imaging results are used with the test set. As input, the test results of the test set Test on the coded illumination lensless imaging system CL_Lensless are obtained and denoted as Result.

[0122] Step 8: Evaluate the coded illumination lensless imaging system

[0123] Using the classic mean square error calculation method in Definition 13, the test results Result of the test set Test obtained in step 7 on the coded illumination lensless imaging system CL_Lensless are processed to calculate the mean square error index of the reconstructed fused image, denoted as MSE.

[0124] Using the classic structural similarity calculation method in Definition 14, the test results Result of the test set Test obtained in step 7 on the coded illumination lensless imaging system CL_Lensless are processed to calculate the structural similarity index of the reconstructed fused image, denoted as SSIM;

[0125] Using the classic peak signal-to-noise ratio (PSNR) calculation method in Definition 27, the test results Result of the test set Test obtained in step 7 on the coded illumination lensless imaging system CL_Lensless are processed to calculate the peak signal-to-noise ratio index of the reconstructed fused image, denoted as PSNR.

[0126] The classic learning-aware image patch similarity calculation method in Definition 15 is used to process the test result Result of the test set Test obtained in step 7 on the coded illumination lensless imaging system CL_Lensless, and calculate the learning-aware image patch similarity index of the reconstructed fused image, denoted as LPIPS;

[0127] This concludes the entire method.

[0128] The innovation and advantages of this invention are as follows: The proposed coded illumination lensless imaging system based on feature fusion combines coded illumination preprocessing with a feature extraction and fusion network, proposing an efficient fusion method for image information applicable to different illumination patterns, and providing a new processing mode for coded illumination lensless imaging images. At the physical level, the system actively modulates scene information through coded illumination to improve the system matrix condition number; at the algorithmic level, it adaptively extracts and fuses multi-mode information through a deep learning network to suppress noise and restore high-frequency details; at the system level, it enhances the rationality and efficiency of image information decoding through feature extraction and fusion of two complementary dimensions: global and detailed. Attached Figure Description

[0129] Figure 1 This is a flowchart illustrating the implementation steps of the present invention.

[0130] Figure 2 The mean square error, structural similarity, peak signal-to-noise ratio, and learned perceptual image patch similarity of the coded illumination lensless imaging system based on feature fusion in this invention are given. Detailed Implementation

[0131] Next, we will combine the appendix Figure 1 The present invention will be described in further detail below.

[0132] This invention provides a lensless imaging system based on feature fusion with coded illumination, the system comprising the following steps:

[0133] Step 1: Initialize the parameters of the coded illumination lensless imaging system

[0134] The initialization parameters for the coded illumination module include: the (i, j)th complete coded illumination pattern matrix of the coded illumination module, denoted as... The i-th basis illumination vector in the horizontal direction of the encoded illumination is denoted as... The j-th basis illumination vector in the vertical direction of the encoded illumination is denoted as... ;

[0135] Initialize the operating parameters of the lensless imaging system, including: the input light rays of the lensless imaging system, denoted as... The input image of a lensless imaging system is denoted as... The measurement results of the lensless imaging system are denoted as... The classic convolution operation in Definition 1 is denoted as... The regularization coefficient in the process of reconstructing a lensless image is denoted as . The gradient operator in the process of reconstructing a lensless image is denoted as... .

[0136] Given a fixed system, the initial values ​​of all the above parameters are known.

[0137] Step 2: Prepare the dataset

[0138] Using the classic CatvsDog dataset from Definition 2 as the data base, 10,000 images are randomly selected from it to obtain the dataset of this invention, denoted as CvD;

[0139] Using the standard point light source formation method in Definition 3, the input light obtained in step 1 is processed. After processing, a standard point light source is obtained, denoted as POT;

[0140] Using the standard binary amplitude masking method in Definition 4, the standard point light source (POT) is processed to obtain the standard image after amplitude masking, denoted as PSF.

[0141] Step 3: Construct the coded lighting module

[0142] Using the standard shift point pattern formation method defined in Definition 5, the input light obtained in step 1 is processed. After processing, the coded lighting module is obtained, as shown below:

[0143]

[0144] in, This represents the (i, j)th complete coded lighting pattern matrix of the coded lighting module obtained in step 1. Let i be the i-th base illumination vector in the horizontal direction of the encoded illumination obtained in step 1. Let j be the j-th base illumination vector in the vertical direction of the encoded illumination obtained in step 1. This defines six classic transpose operations.

[0145] Step 4: Construct the initial reconstruction module

[0146] The classic least squares algorithm in Definition 7 is used to analyze the measurement results of the lensless imaging system obtained in step 1. The reconstruction process yields a preliminary reconstruction module, as shown below:

[0147]

[0148] in, These are the preliminary imaging results of the lensless imaging system. This is the classic method for minimizing parameters as defined in 8; PSF is the standard image obtained in step 2 after amplitude masking. The convolution operation obtained in step 1, The measurement results of the lensless imaging system obtained in step 1 are as follows. The input image of the lensless imaging system obtained in step 1. It is the classic L2 norm squaring operation in Definition 9. These are the regularization coefficients obtained in step 1 during the process of reconstructing the lensless image. This refers to the gradient operator used in the process of reconstructing the lensless image obtained in step 1.

[0149] Step 5: Construct a feature extraction and fusion network

[0150] Step 5.1: Initialize the feature extraction and fusion network parameters

[0151] Initialize the feature extraction and fusion network parameters, including: the input image of the feature extraction and fusion network is the preliminary imaging result of the lensless imaging system obtained in step 4. The output image of the feature extraction fusion network is denoted as... The optimal solution of the output image of the feature extraction fusion network is denoted as... Eight Restormer blocks are constructed using the standard Restormer block construction method in Definition 10, denoted as restormer_block1, restormer_block2, restormer_block3, restormer_block4, restormer_block5, restormer_block6, restormer_block7, and restormer_block8; four Lite Transformer blocks are constructed using the standard Lite Transformer block construction method in Definition 11, denoted as lt_block1, lt_block2, lt_block3, and lt_block4; and four INN blocks are constructed using the standard reversible neural network block construction method in Definition 12, denoted as inn_block1, inn_block2, inn_block3, and inn_block4.

[0152] Step 5.2: Construct the feature extraction module

[0153] Step 5.2.1: Construct a shallow feature extraction encoder

[0154] The first four Restormer blocks obtained in step 5.1, restormer_block1, restormer_block2, restormer_block3, and restormer_block4, are sequentially connected to obtain the shallow feature extraction encoder, denoted as LFE_Encoder.

[0155] Step 5.2.2: Construct a complementary feature extraction encoder

[0156] The first two Lite Transformer blocks lt_block1 and lt_block2 obtained in step 5.1 are sequentially connected to obtain the global feature extraction encoder, denoted as GFE_Encoder;

[0157] The first two INN blocks inn_block1 and inn_block2 obtained in step 5.1 are concatenated sequentially to obtain the detail feature extraction encoder, denoted as DFE_Encoder;

[0158] Step 5.3: Construct the feature fusion module

[0159] The last two Lite Transformer blocks lt_block3 and lt_block4 obtained in step 5.1 are connected sequentially to obtain the global feature fusion module, denoted as GF_Fusion;

[0160] The last two INN blocks, inn_block3 and inn_block4, obtained in step 5.1 are sequentially connected to obtain the detail feature fusion module, denoted as DF_Fusion.

[0161] Step 5.4: Construct the feature decoder module

[0162] The last four Restormer blocks obtained in step 5.1, restormer_block5, restormer_block6, restormer_block7, and restormer_block8, are sequentially connected to obtain the feature decoder, denoted as Decoder.

[0163] Step 5.5: Construct the loss function module

[0164] Using the classic mean square error calculation method in Definition 13, the input image of the lensless imaging system obtained in step 1 is processed. The output image of the feature extraction fusion network in step 5.1 After comparison, the mean squared error loss was calculated and denoted as... ;

[0165] Using the classic structural similarity calculation method in Definition 14, the input image of the lensless imaging system obtained in step 1 is processed. The output image of the feature extraction fusion network in step 5.1 The structural similarity loss is calculated by performing a comparison process, and denoted as . ;

[0166] Using the classic gradient loss calculation method in Definition 15, the input image of the lensless imaging system obtained in step 1 is processed. The output image of the feature extraction fusion network in step 5.1 By performing a comparison, the gradient loss is calculated and denoted as... ;

[0167] Using the classic learning-based perceptual image patch similarity calculation method in Definition 16, the input image of the lensless imaging system obtained in step 1 is processed. The output image of the feature extraction fusion network in step 5.1 The perceptual loss was calculated by performing a comparison process and denoted as . ;

[0168] The weighted loss summation method in Definition 17 is used to calculate the mean square error loss. Structural similarity loss gradient loss Perceived loss By performing a weighted summation, we obtain the loss function module, as shown below:

[0169]

[0170] in, Mean squared error loss The weighting coefficients, It is structural similarity loss The weighting coefficients, It is gradient loss The weighting coefficients, It is perceived loss The weighting coefficients.

[0171] Step 5.6: Construct a feature extraction and fusion network

[0172] Using the standard feature extraction method defined in Definition 18, and employing the shallow feature extraction encoder LFE_Encoder obtained in step 5.2.1, the preliminary imaging results of the lensless imaging system obtained in step 4 are processed. Feature extraction is performed to obtain shallow features of the image. ;

[0173] The standard feature extraction method in Definition 18 is adopted, and the global feature extraction encoder GFE_Encoder obtained in step 5.2.2 is used to extract shallow features of the image. Feature extraction is performed to obtain the global features of the image. ;

[0174] The standard feature extraction method in Definition 18 is adopted, and the detail feature extraction encoder DFE_Encoder obtained in step 5.2.2 is used to extract shallow features of the image. Feature extraction is performed to obtain the detailed features of the image. ;

[0175] The standard feature fusion method in Definition 19 is adopted, and the global feature fusion module GE_Fusion obtained in step 5.2 is used to process the global features of the image. Perform feature fusion to obtain fused global features. ;

[0176] The standard feature fusion method in Definition 19 is adopted, and the detail feature fusion module DE_Fusion obtained in step 5.2 is used to process the detail features of the image. Perform feature fusion to obtain fused detailed features. ;

[0177] Using the 20 standard feature concatenation methods defined, global features are fused. and blending details The fused features Fu are obtained by concatenating the data along the channel dimension.

[0178] Using the standard feature decoding method in Definition 21, and employing the feature decoder obtained in step 5.3, the fused features are processed. Decode the image to generate the output image of the feature extraction and fusion network in step 5.1. ;

[0179] We employ the classic Adam algorithm from Definition 22, using the loss function module obtained in step 5.4. The output image of the feature extraction fusion network Iterative updates yield the optimal solution for the output image of the feature extraction fusion network in step 5.1. .

[0180] Step 6: Construct a lensless imaging system with coded illumination

[0181] Using the standard coded illumination operation in Definition 23, and with the coded illumination module obtained in step 3, the number of shift point patterns is set to 4. The dataset CvD obtained in step 2 is processed with coded illumination to form 40,000 coded illumination images, denoted as Coded_CvD.

[0182] Using the standard lensless imaging method in Definition 24, 40,000 coded illumination images Coded_CvD were simulated and encoded to obtain the lensless image dataset of this invention, denoted as Lensless_CvD;

[0183] The Lensless_CvD image dataset obtained in this invention is randomly divided into two parts: a training set and a test set, with a ratio of 9:1. The first 36,000 images are used as the training set, denoted as Train, and the last 4,000 images are used as the test set, denoted as Test.

[0184] Using the standard lensless image reconstruction method defined in Definition 25, and employing the preliminary reconstruction module obtained in step 3, preliminary reconstruction is performed on the obtained training set Train to obtain the preliminary imaging result of the training set, denoted as... ;

[0185] The classic Adam algorithm from Definition 22 is used to train the feature extraction and fusion network obtained in step 4, and the preliminary imaging results are used on the training set. PyCharm was used as the simulation platform, Python was used as the simulation language, and the initial learning rate was set to 10. −4 After training, the coded illumination lensless imaging system is obtained, denoted as CL_Lensless.

[0186] Step 7: Test the coded illumination lensless imaging system

[0187] Using the standard lensless image reconstruction method defined in Definition 25, and employing the preliminary reconstruction module obtained in step 3, preliminary reconstruction is performed on the obtained test set Test to obtain the preliminary imaging result of the test set, denoted as . ;

[0188] The forward propagation method according to Definition 26 is used to test the coded illumination lensless imaging system CL_Lensless obtained in step 6, and the preliminary imaging results are used with the test set. As input, the test results of the test set Test on the coded illumination lensless imaging system CL_Lensless are obtained and denoted as Result.

[0189] Step 8: Evaluate the coded illumination lensless imaging system

[0190] Using the classic mean square error calculation method in Definition 13, the test results Result of the test set Test obtained in step 7 on the coded illumination lensless imaging system CL_Lensless are processed to calculate the mean square error index of the reconstructed fused image, denoted as MSE.

[0191] Using the classic structural similarity calculation method in Definition 14, the test results Result of the test set Test obtained in step 7 on the coded illumination lensless imaging system CL_Lensless are processed to calculate the structural similarity index of the reconstructed fused image, denoted as SSIM;

[0192] Using the classic peak signal-to-noise ratio (PSNR) calculation method in Definition 27, the test results Result of the test set Test obtained in step 7 on the coded illumination lensless imaging system CL_Lensless are processed to calculate the peak signal-to-noise ratio index of the reconstructed fused image, denoted as PSNR.

[0193] The classic learning-aware image patch similarity calculation method in Definition 15 is used to process the test result Result of the test set Test obtained in step 7 on the coded illumination lensless imaging system CL_Lensless, and calculate the learning-aware image patch similarity index of the reconstructed fused image, denoted as LPIPS;

[0194] The above indicators and comparison results are attached. Figure 2 .

[0195] This concludes the entire method.

[0196] Computer simulation experiments show that, compared to the classic Unet method, the reconstructed image obtained by this method has better performance across various metrics. Specifically, the mean squared error (MSE) and learned perceptual patch similarity (LPIPS) are both lower, indicating that the reconstructed and fused image is closer to the true value. Furthermore, the structural similarity (SSIM) and peak signal-to-noise ratio (PSNR) are higher, indicating that the reconstructed image is of higher quality and possesses clearer detail information. The results are as follows: Figure 2 As shown.

Claims

1. A feature fusion based coded illumination lensless imaging system characterized in that it comprises Comprising the following steps: Step 1, initialize the coded illumination lensless imaging system parameters Initializing working parameters of the coded lighting module, comprising: a (i, j)th complete coded lighting pattern matrix of the coded lighting module, denoted as ; an i th base lighting vector in the horizontal direction of the coded lighting, denoted as ; and a j th base lighting vector in the vertical direction of the coded lighting, denoted as ; Initialization of working parameters of the lensless imaging system, including: input light rays of the lensless imaging system, denoted as ; input image of the lensless imaging system, denoted as ; measurement result of the lensless imaging system, denoted as ; classical convolution operation, denoted as ; regularization coefficient in the process of reconstructing the lensless image, denoted as ; gradient operator in the process of reconstructing the lensless image, denoted as ; All the initial values of the above parameters are known in the case of system determination; Step 2, prepare the data set The classic CatvsDog data set is used as the data basis, and 10,000 pictures are randomly selected from it to obtain the data set of the application, denoted as CvD; The input light rays obtained in step 1 are processed using a standard point light source formation method to obtain a standard point light source, denoted as POT. The input light rays obtained in step 1 are processed using a standard point light source formation method to obtain a standard point light source, denoted as POT. The standard point light source POT is processed by using the standard binary amplitude mask method to obtain the standard image after amplitude mask processing, denoted as PSF; Step 3, build a coded illumination module Using a standard shift point pattern formation method, the input light obtained in step 1 is processed. After processing, the coded lighting module is obtained, as shown below: wherein is the (i,j)th complete coded illumination pattern matrix of the coded illumination module obtained in step 1, is the ith base illumination vector in horizontal direction of the coded illumination obtained in step 1, is the jth base illumination vector in vertical direction of the coded illumination obtained in step 1, is the classical transpose operation; Step 4, build a preliminary reconstruction module Using the classical least square algorithm, the measured results of the lensless imaging system obtained in step 1 are processed The reconstruction is performed to obtain a preliminary reconstruction module, as follows: wherein, is the preliminary imaging result of the lensless imaging system, is the classical minimization parameter solving method; PSF is the standard image after amplitude mask processing obtained in step 2, is the convolution operation obtained in step 1, is the measurement result of the lensless imaging system obtained in step 1, is the input image of the lensless imaging system obtained in step 1, is the classical L2 norm square operation is the regularization coefficient in the process of reconstructing the lensless image obtained in step 1, is the gradient operator in the process of reconstructing the lensless image obtained in step 1; Step 5, build a feature extraction and fusion network Step 5.1, initialize the feature extraction and fusion network parameters The initialization of the feature extraction fusion network parameters comprises: the input image of the feature extraction fusion network is the preliminary imaging result of the lensless imaging system obtained in step 4 ; the output image of the feature extraction fusion network is denoted as ; and the optimal solution of the output image of the feature extraction fusion network is denoted as ; Eight Restormer blocks are constructed by using the standard Restormer block construction method, denoted as restormer_block1, restormer_block2, restormer_block3, restormer_block4, restormer_block5, restormer_block6, restormer_block7, and restormer_block8; Four Lite Transformer blocks are constructed by using the standard Lite Transformer block construction method, denoted as lt_block1, lt_block2, lt_block3, and lt_block4; Four INN blocks are constructed by using the standard reversible neural network block construction method, denoted as inn_block1, inn_block2, inn_block3, and inn_block4; Step 5.2, build a feature extraction module Step 5.2.1, construct a shallow feature extraction encoder The first four Restormer blocks restormer_block1, restormer_block2, restormer_block3, and restormer_block4 obtained in step 5.1 are sequentially connected to obtain a shallow feature extraction encoder, denoted as LFE_Encoder; Step 5.2.2, construct a complementary feature extraction encoder The first two Lite Transformer blocks lt_block1 and lt_block2 obtained in step 5.1 are sequentially connected to obtain a global feature extraction encoder, denoted as GFE_Encoder; The first two INN blocks inn_block1 and inn_block2 obtained in step 5.1 are sequentially connected to obtain a detail feature extraction encoder, denoted as DFE_Encoder; Step 5.3, build a feature fusion module The last two Lite Transformer blocks lt_block3 and lt_block4 obtained in step 5.1 are sequentially connected to obtain a global feature fusion module, denoted as GF_Fusion; The last two INN blocks inn_block3 and inn_block4 obtained in step 5.1 are sequentially connected to obtain a detail feature fusion module, denoted as DF_Fusion; Step 5.4, constructing a feature decoder module The last four Restormer blocks restormer_block5, restormer_block6, restormer_block7, and restormer_block8 obtained in step 5.1 are sequentially connected to obtain a feature decoder, denoted as Decoder; Step 5.5, constructing a loss function module The input image of the lensless imaging system obtained in step 1 is calculated by using a classical mean square error calculation method The output image of the feature extraction fusion network in step 5.1 is compared and processed to calculate the mean square error loss, denoted as ;​ The input image of the lensless imaging system obtained in step 1 is processed by using a classical structural similarity calculation method The output image of the feature extraction fusion network in step 5.1 is processed by using a classical structural similarity calculation method The structural similarity loss is calculated by using a classical structural similarity calculation method, and is denoted as ; Using the classical gradient loss calculation method, the input image of the lensless imaging system obtained in step 1 is calculated The output image of the feature extraction fusion network in step 5.1 is calculated The contrast processing is performed, and the gradient loss is calculated, denoted as ; The classic learning-based perceptual image patch similarity calculation method is used to process the input image of the lensless imaging system obtained in step 1. The output image of the feature extraction fusion network in step 5.1 The perceptual loss is calculated by performing a comparison process and is denoted as . ; The mean square error loss , structural similarity loss , gradient loss , perceptual loss are weighted and summed to obtain a loss function module, as follows: wherein, is a weighting coefficient for the mean squared error loss is a weighting coefficient for the structural similarity loss is a weighting coefficient for the gradient loss is a weighting coefficient for the perceptual loss ​​​​ Step 5.6, constructing a feature extraction fusion network Using the standard feature extraction method, the shallow feature extraction encoder LFE_Encoder obtained in step 5.2.1 is used to extract features from the preliminary imaging results of the lensless imaging system obtained in step 4 to extract features from the preliminary imaging results of the lensless imaging system obtained in step 4 ; Using the standard feature extraction method, the global feature extraction encoder GFE_Encoder obtained in step 5.2.2 is used to extract the shallow features of the image Feature extraction is performed to obtain the global features of the image ; Using the standard feature extraction method, the shallow features of the image are extracted using the detail feature extraction encoder DFE_Encoder obtained in step 5.2.2 Feature extraction is performed to obtain the detail features of the image ; The global feature fusion module GE_Fusion obtained in step 5.2 is used to fuse the global features of the image by using the standard feature fusion method The global feature fusion module GE_Fusion obtained in step 5.2 is used to fuse the global features of the image by using the standard feature fusion method ; Using the standard feature fusion method, the detail feature fusion module DE_Fusion obtained in step 5.2 is used to fuse the detail features of the image The feature fusion is performed to obtain the fused detail features ; The standard feature splicing method is adopted to fuse the global feature and the detail feature The splicing is performed in the channel dimension to obtain the fused feature Fu; Using the standard feature decoding method, the feature decoder obtained in step 5.3 is used to process the fused features. Decode the image to generate the output image of the feature extraction and fusion network in step 5.

1. ; Using the classical Adam algorithm, the loss function module obtained in step 5.4 is used , the output image of the feature extraction fusion network , the output image of the feature extraction fusion network in step 5.1 is obtained by iterative updating ; Step 6, constructing an encoding and illumination lensless imaging system Using the encoding and illumination module obtained in step 3, the dataset CvD obtained in step 2 is processed by encoding and illumination to form an encoding and illumination image, denoted as Coded_CvD, using a standard encoding and illumination operation. Using a standard lensless imaging method, the encoding and illumination image Coded_CvD is simulated to obtain a lensless image dataset of the present application, denoted as Lensless_CvD. The lensless image dataset Lensless_CvD is randomly divided into a training set and a test set in a ratio of 9:1, and the training set is denoted as Train and the test set is denoted as Test. Using the standard lensless image reconstruction method, the preliminary reconstruction module obtained in step 3 is used to preliminarily reconstruct the obtained training set Train, to obtain a preliminary imaging result of the training set, denoted as ; The feature extraction and fusion network obtained in step 4 is trained using the classical Adam algorithm, and the preliminary imaging results of the training set are used As input, the encoding illumination lensless imaging system obtained after training is denoted as CL_Lensless. Step 7, testing the encoding and illumination lensless imaging system Using the standard lensless image reconstruction method, the preliminary reconstructed module obtained in step 3 is used to preliminarily reconstruct the obtained test set Test, to obtain a preliminary imaging result of the test set, denoted as ; The coded illumination lensless imaging system CL_Lensless obtained in step 6 is tested using the standard forward propagation method, using the test set to obtain preliminary imaging results As input, the test set Test is obtained on the coded illumination lensless imaging system CL_Lensless, denoted as Result. Step 8, evaluating the encoding and illumination lensless imaging system Using a classic mean square error calculation method, the test results Result of the test set Test on the encoding and illumination lensless imaging system CL_Lensless obtained in step 7 are processed to calculate the mean square error index of the reconstructed fusion image, denoted as MSE. Using a classic structural similarity calculation method, the test results Result of the test set Test on the encoding and illumination lensless imaging system CL_Lensless obtained in step 7 are processed to calculate the structural similarity index of the reconstructed fusion image, denoted as SSIM. Using a classic peak signal-to-noise ratio calculation method, the test results Result of the test set Test on the encoding and illumination lensless imaging system CL_Lensless obtained in step 7 are processed to calculate the peak signal-to-noise ratio index of the reconstructed fusion image, denoted as PSNR. Using a classic learning perceptual image block similarity calculation method, the test results Result of the test set Test on the encoding and illumination lensless imaging system CL_Lensless obtained in step 7 are processed to calculate the learning perceptual image block similarity index of the reconstructed fusion image, denoted as LPIPS. At this point, the entire method ends.

Images