Fresnel aperture encoding imaging method and device based on two-stage reconstruction and noise matching

Abstract

The application discloses a kind of based on two-stage reconstruction and noise matching's Fresnel aperture encoding imaging method and device, comprising: by the real image after pre-processing is with Fresnel aperture, obtains simulation image by convolution operation;The simulation image is carried out luminance conversion and noise fitting operation, so that its noise distribution is close to real data, obtains coding chart;The coded chart is carried out inverse propagation reconstruction, obtains initial reconstruction image;According to fast fourier transform module and convolutional neural network, construct target FFTConv-UNet model;The initial reconstruction image is input into the target FFTConv-UNet model, and target reconstruction image is obtained.The application can improve the quality of picture reconstruction, and can be widely applied in image reconstruction technical field.

Description

Technical Field

[0001] This invention relates to the field of image reconstruction technology, and in particular to a Fresnel aperture coded imaging method and apparatus based on two-stage reconstruction and noise matching. Background Technology

[0002] Fresnel aperture-coded imaging is a novel optical computational imaging method. Traditional optical imaging techniques rely on lens systems to focus light rays to form images; for example, cameras or optical microscopes typically contain multiple sets of lens elements. While this imaging method can provide high-quality images, its equipment is expensive and structurally complex. In Fresnel aperture-coded imaging, the imaging system relies on the encoding and image reconstruction of optical modulation elements (i.e., Fresnel aperture masks), eliminating the dependence on complex lenses. Therefore, it has advantages such as simple structure, low cost, and ease of integration. Currently, image reconstruction methods in Fresnel aperture-coded imaging technology include traditional iterative optimization methods and data-driven deep learning methods.

[0003] Traditional iterative optimization methods perform iterative reconstruction operations on the acquired, encoded images. In iterative reconstruction, the image estimate is continuously updated, gradually approximating the true image to obtain a relatively accurate reconstruction result. However, iterative reconstruction methods are computationally intensive, especially for high-resolution, large-data images. The reconstruction process requires significant time and computational resources, limiting its use in applications with high real-time requirements. Furthermore, the algorithm may get trapped in local optima and fail to find the globally optimal reconstruction result; the choice of parameters also has a significant impact on the reconstruction outcome.

[0004] Existing deep learning methods for binarized Fresnel aperture imaging are primarily based on end-to-end neural network architectures, meaning that the input is an encoded image, and the output is a clear, reconstructed image. Currently, due to the lack of real-world data for network training, training mainly relies on simulated data. However, the noise distribution in real-world data is more complex than in simulated data, resulting in poor image reconstruction quality from models trained on simulated data. Furthermore, existing methods fail to fully utilize optical models, leading to low image reconstruction quality. Summary of the Invention

[0005] In view of this, the main objective of the embodiments of the present invention is to provide a Fresnel aperture coding imaging method and apparatus based on two-stage reconstruction and noise matching, in order to solve at least one of the problems of the prior art. The present invention can improve the image reconstruction quality.

[0006] To achieve the above objectives, one aspect of the present invention provides a Fresnel aperture coded imaging method based on two-stage reconstruction and noise matching, comprising:

[0007] The preprocessed real image is convolved with the Fresnel aperture to obtain the simulated image;

[0008] The simulated image is subjected to brightness conversion and noise fitting operations to obtain an encoded image;

[0009] The encoded image is reconstructed by backpropagation to obtain an initial reconstructed image;

[0010] Construct the target FFTConv-UNet model based on the Fast Fourier Transform module and the convolutional neural network;

[0011] The initial reconstructed image is input into the target FFTCoV-UNet model to obtain the target reconstructed image.

[0012] In some embodiments, before convolving the preprocessed real image with the Fresnel aperture to obtain the simulated image, the following steps are included:

[0013] The real image is preprocessed by magnification and black filling to obtain the preprocessed real image.

[0014] In some embodiments, performing brightness conversion and noise fitting operations on the simulated image to obtain an encoded image includes the following steps:

[0015] The original n-level grayscale image of the real image is acquired through a sensor;

[0016] Construct an exponential function based on the original grayscale image and the real image;

[0017] Obtain the first mean and first variance of the original grayscale image;

[0018] Gaussian noise is obtained by linearly fitting the first mean and the first variance.

[0019] Based on the exponential function and the Gaussian noise, the simulated image is subjected to brightness conversion and noise fitting operations to obtain the encoded image.

[0020] In some embodiments, the formula used to perform backpropagation reconstruction on the coded map to obtain an initial reconstructed image includes:

[0021]

[0022] In the formula, O R O represents the initial reconstructed image; I represents the ground truth image; H represents the encoded image. -1 This represents the inverse of the Fresnel aperture-coded transfer function in the frequency domain; F represents the Hadamard product; F{·} represents the Fourier transform operation; F-1 {·} represents the inverse Fourier transform operation.

[0023] In some embodiments, constructing the target FFTCoV-UNet model based on the Fast Fourier Transform module and the convolutional neural network includes the following steps:

[0024] An encoder-decoder structure is used to fuse the Fast Fourier Transform module with the convolutional neural network to obtain the initial FFTCOnov-UNet model.

[0025] Construct the target loss function based on pixel loss, Laplace loss, and fast Fourier transform loss;

[0026] The initial FFTCOnov-UNet model is trained based on the encoding graph and the target loss function to obtain the target FFTCOnov-UNet model.

[0027] In some embodiments, the formula used to construct the target loss function based on pixel loss, Laplacian loss, and Fast Fourier Transform loss includes:

[0028] L = L piexl +λ F L FFT +λ L L laplacian

[0029] In the formula,

[0030]

[0031] Where L represents the target loss function; L piexl Indicates pixel loss; L FFT Indicates Laplace loss; L laplacian λ represents the loss of the Fast Fourier Transform; F The weights of the Laplace loss are represented by λ. L The weights represent the Fast Fourier Transform loss; O represents the real image; O R The initial reconstructed image is represented by ; N represents the total number of pixels in the image; F(·) represents the Fourier transform operation; ||·|2 represents the L2 norm operation; ||·|1 represents the L1 norm operation; This represents the Laplace operator.

[0032] To achieve the above objectives, another aspect of the present invention proposes a Fresnel aperture-coded imaging device based on two-stage reconstruction and noise matching, the device comprising:

[0033] The first module is used to perform a convolution operation between the preprocessed real image and the Fresnel aperture to obtain a simulated image;

[0034] The second module is used to perform brightness conversion and noise fitting operations on the simulated image to obtain an encoded image;

[0035] The third module is used to perform backpropagation reconstruction on the encoded image to obtain an initial reconstructed image;

[0036] The fourth module is used to construct the target FFTConv-UNet model based on the Fast Fourier Transform module and the convolutional neural network.

[0037] The fifth module is used to input the initial reconstructed image into the target FFTCoV-UNet model to obtain the target reconstructed image.

[0038] To achieve the above objectives, another aspect of the present invention provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the Fresnel aperture coded imaging method based on two-stage reconstruction and noise matching described above.

[0039] To achieve the above objectives, another aspect of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the Fresnel aperture coded imaging method based on two-stage reconstruction and noise matching described above.

[0040] To achieve the above objectives, another aspect of the present invention provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions, causing the computer device to perform the aforementioned Fresnel aperture-coded imaging method based on two-stage reconstruction and noise matching.

[0041] The embodiments of the present invention include at least the following beneficial effects: The present invention provides a Fresnel aperture coded imaging method and apparatus based on two-stage reconstruction and noise matching. This scheme obtains a simulated image by convolving a preprocessed real image with a Fresnel aperture; performs brightness conversion and noise fitting operations on the simulated image to make its noise distribution close to the real data, obtaining a coded image; performs backpropagation reconstruction on the coded image to obtain an initial reconstructed image; constructs a target FFTCOnov-UNet model based on a fast Fourier transform module and a convolutional neural network; and inputs the initial reconstructed image into the target FFTCOnov-UNet model to obtain the target reconstructed image, which can improve the quality of image reconstruction. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 This is a flowchart of the Fresnel aperture coded imaging method based on two-stage reconstruction and noise matching provided in the embodiments of the present invention;

[0044] Figure 2 This is a 16-level grayscale original image captured by the sensor provided in this embodiment of the invention;

[0045] Figure 3 This is a flowchart of acquiring simulation data and adding noise to the simulation data provided in an embodiment of the present invention;

[0046] Figure 4 This is a schematic diagram of the FFTConv-UNet network architecture provided in an embodiment of the present invention;

[0047] Figure 5 This is a schematic diagram of the FFTConv Block structure provided in an embodiment of the present invention;

[0048] Figure 6 This is a schematic diagram of the Conv Block structure provided in an embodiment of the present invention;

[0049] Figure 7 This is a flowchart of the two-stage algorithm provided in an embodiment of the present invention;

[0050] Figure 8 This is a schematic diagram of the Fresnel aperture coded imaging system provided in an embodiment of the present invention;

[0051] Figure 9 This is a schematic diagram of the experimental apparatus provided in an embodiment of the present invention;

[0052] Figure 10 This is a schematic diagram of the brightness calibration fitting results provided in an embodiment of the present invention;

[0053] Figure 11 This is a schematic diagram of the noise distribution fitting results provided in an embodiment of the present invention;

[0054] Figure 12 This is a comparative schematic diagram of the target test image and the one-stage BP reconstruction image provided in an embodiment of the present invention;

[0055] Figure 13 This is a comparative schematic diagram of the first-stage reconstructed image and the second-stage reconstructed image provided in an embodiment of the present invention;

[0056] Figure 14 This is a schematic diagram comparing the image reconstruction effects of the method of this invention with those of existing technologies;

[0057] Figure 15 This is a schematic diagram of the hardware structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this invention; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this invention as detailed in the appended claims.

[0059] It should be noted that although functional modules are divided in the system diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the system or the order in the flowchart. The terms "first / S100" and "second / S200" in the specification, claims, and the foregoing drawings may be used herein to describe various concepts, but unless specifically stated otherwise, these concepts are not limited by these terms. These terms are used only to distinguish one concept from another. For example, first information may also be referred to as second information without departing from the scope of the embodiments of the invention, and similarly, second information may also be referred to as first information. Depending on the context, the words "if" or "when" as used herein may be interpreted as "when," "in response to a determination," or "in the event of a determination."

[0060] The terms “at least one,” “multiple,” “each,” “any,” etc., used in this invention, “at least one” includes one, two, or more than two; “multiple” includes two or more than two; “each” refers to each of the corresponding multiple; and “any” refers to any one of the multiple.

[0061] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein is for the purpose of describing embodiments of the invention only and is not intended to limit the invention.

[0062] Before providing a detailed description of the embodiments of the present invention, some of the nouns and terms involved in the embodiments of the present invention will be explained first. The nouns and terms involved in the embodiments of the present invention are subject to the following interpretations.

[0063] Fresnel aperture coded imaging: Fresnel aperture coded imaging is a lensless imaging technique that follows the Fresnel diffraction principle. When light waves pass through a specific coded aperture, Fresnel diffraction occurs, altering their propagation path and intensity. The coded light signal contains more information, which is then used by a decoding algorithm to reconstruct the target image.

[0064] Binarization: Binarization refers to the transmission rate containing only two values, 1 and -1. Theoretically, Fresnel aperture encoding uses zone plates with continuously sinusoidal transmission rates, but zone plates with continuously varying transmission rates are difficult to manufacture, so in practice they are processed using a binarization method.

[0065] Noise matching: Noise matching refers to matching the noise distribution of simulated data with that of actual captured data.

[0066] like Figure 1 As shown, this embodiment of the invention provides a Fresnel aperture coded imaging method based on two-stage reconstruction and noise matching, which may include, but is not limited to, steps S100 to S500:

[0067] Step S100: Convolve the preprocessed real image with the Fresnel aperture to obtain the simulation image;

[0068] Step S200: Perform brightness conversion and noise fitting operations on the simulated image to obtain an encoded image;

[0069] Step S300: Perform backpropagation reconstruction on the encoded image to obtain an initial reconstructed image;

[0070] Step S400: Construct the target FFTConv-UNet model based on the Fast Fourier Transform module and the convolutional neural network;

[0071] Step S500: Input the initial reconstructed image into the target FFTConv-UNet model to obtain the target reconstructed image.

[0072] In some embodiments, prior to step S100, a preprocessing operation is performed on the real image, including magnification and black filling, to obtain a preprocessed real image. Then, a simulation image is obtained by convolving the preprocessed real image with a Fresnel aperture.

[0073] In some embodiments, step S200 may include, but is not limited to, steps S210 to S250:

[0074] Step S210: Acquire the n-level grayscale original image of the real image using a sensor;

[0075] Step S220: Construct an exponential function based on the original grayscale image and the real image;

[0076] Step S230: Obtain the first mean and first variance of the original grayscale image;

[0077] Step S240: Perform linear fitting on the first mean and the first variance to obtain Gaussian noise;

[0078] Step S250: Based on the exponential function and the Gaussian noise, perform brightness conversion and noise fitting operations on the simulated image to obtain the encoded image.

[0079] In step S210 of some embodiments, the real image is replaced with uniform gray squares whose pixel values ​​increase incrementally every certain number of pixels, generating an n-level grayscale original image displayed on the LCD screen, and the n-level grayscale original image is captured by a sensor. For example, as... Figure 2 As shown, the real image is replaced with uniform gray squares with incrementing values ​​of 16 pixels each, with pixel values ​​of 0, 15, 31, ..., 239, 255, for a total of 16 grayscale images. This generates a 16-level grayscale original image, which is then displayed on the LCD screen. The 16-level grayscale original image is then captured by a sensor.

[0080] In step S220 of some embodiments, the brightness conversion is fitted using an exponential function, resulting in the following formula:

[0081] O0=aO b +c

[0082] In the formula, O0 represents the sensor-acquired data; O represents the real image; a, b, and c are the parameters to be fitted. Specifically, a is the coefficient of the fitting exponential term, which physically represents the mapping from the normalized grayscale value of the LCD display to the normalized grayscale value of the sensor-acquired data; b is the exponential coefficient, reflecting the nonlinear relationship of brightness calibration; and c represents the sensor's response to weak ambient light.

[0083] In steps S230 to S240 of some embodiments, the first mean and first variance of the n-level grayscale original image data acquired by the sensor are linearly fitted to obtain the gain coefficient and Gaussian noise. The relationship between noise and the mean of the grayscale original image can be obtained based on the noise distribution, as shown in the following formula:

[0084]

[0085] In the formula, σ(O0) 2 The first variance of the sensor-acquired data represents the noise level; E(O0) represents the first mean of the sensor-acquired data; g represents the gain coefficient. This represents Gaussian noise.

[0086] In step S250 of some embodiments, an exponential function is used to perform brightness conversion on the simulation image, and corresponding Gaussian noise is added to fit the simulation image to obtain noisy simulation data, i.e., coded image.

[0087] In some embodiments, steps S100 to S200, such as Figure 3 As shown, a real target image with a pixel size of 256×256 is magnified and filled with black to enlarge the pixel size to 1500×1500. Then, a simulated image is obtained by performing Fresnel diffraction (i.e., convolution with Fresnel aperture (FZA)). The simulated image is then subjected to brightness conversion and noise fitting to obtain noisy simulated data that is close to the real data, i.e., the coded image.

[0088] In step S300 of some embodiments, in the first stage of image reconstruction, backpropagation reconstruction is performed on the obtained coded map to obtain an initial reconstructed image, and the formula used includes:

[0089]

[0090] In the formula, O R O represents the initial reconstructed image; I represents the ground truth image; H represents the encoded image. -1 This represents the inverse of the Fresnel aperture-coded transfer function in the frequency domain; F represents the Hadamard product; F{·} represents the Fourier transform operation; F -1 {·} denotes the inverse Fourier transform operation. Substituting the coded image I into the formula yields the initial reconstructed image. In this step, noise in I is allowed to enter the solution. Although the BP algorithm cannot handle noise, it can quickly recover the basic structure and some details of the image. That is, the first stage introduces optical prior information to complete the initial reconstruction of the image.

[0091] In some embodiments, step S400 may include, but is not limited to, steps S410 to S430:

[0092] Step S410: Using an encoder-decoder structure, the fast Fourier transform module is fused with the convolutional neural network to obtain the initial FFTCOnov-UNet model.

[0093] Step S420: Construct the target loss function based on pixel loss, Laplace loss, and Fast Fourier Transform loss;

[0094] Step S430: Train the initial FFTCOnov-UNet model according to the encoding graph and the target loss function to obtain the target FFTCOnov-UNet model.

[0095] In steps S410 to S430 of some embodiments, the aforementioned noise-matched encoded graph is divided into a training set and a test set, and the initial FFTConv-UNet model is trained for inference. Figure 4 As shown, the initial FFT Conv-UNet model deeply integrates the Fast Fourier Transform (FFT) with a convolutional neural network, adopts an encoder-decoder structure, and includes 5 levels of downsampling (halving the input image size M through a Maxpool Block) and upsampling (doubling the input size M through an UpConv Block). The number of channels C expands from 32 to 64 to 128 to 256 to 512. The encoder and decoder are connected by a Skip Connection.

[0096] In some embodiments, such as Figure 5 The FFTConv Block shown is the core component of the FFTConv-UNet model. It combines spatial domain convolution and FFT-ReLU flow (a module formed by combining FFT and ReLU) to achieve spatial-frequency domain co-learning through a dual-path architecture. The spatial flow branch adopts the original Conv Block structure from UNet, such as... Figure 6 As shown, it contains two 3×3 convolutional layers, each followed by BatchNorm (batch normalization) and ReLU activation functions. The main function of spatial flow is to extract local spatial features of the image, capturing detailed information and local structure.

[0097] In some embodiments, the model's loss function is a function used in deep learning to measure the difference between the model's predictions and the actual data. In the FFTConv-UNet model, the target loss function consists of pixel loss, Laplacian loss, and Fast Fourier Transform loss. The expression for the target loss function is as follows:

[0098] L = L piexl +λ F L FFT +λ L L laplacian

[0099] In the formula, pixel loss L piexl The most basic loss function, used to measure the pixel-level difference between the reconstructed image and the real image, is defined as:

[0100]

[0101] The Fast Fourier Transform loss function reflects the difference in amplitude spectrum between the reconstructed image and the real image after Fourier transform, and is defined as:

[0102]

[0103] The Laplacian loss function uses a 3x3 Laplacian kernel to convolve the image, and then calculates the absolute difference between the reconstructed image and the real image after the Laplacian transform. The definition of Laplacian loss is as follows:

[0104]

[0105] Where L represents the target loss function; L piexl Indicates pixel loss; L FFT Indicates Laplace loss; L laplacian λ represents the loss of the Fast Fourier Transform; F The weights of the Laplace loss are represented by λ. F =0.1; λ L The weights representing the Fast Fourier Transform loss, optionally, λ L =0.08; O represents the real image; O R This represents the initial reconstructed image; N represents the total number of pixels in the image, for example, if the image has 200×200 pixels, then N is 40000; F(·) represents the Fourier transform operation; ||·|2 represents the L2 norm operation; ||·|1 represents the L1 norm operation; This represents the Laplace operator.

[0106] In step S500 of some embodiments, in the second stage of image reconstruction, the initial reconstructed image is input into the target FFTCoV-UNet model to obtain the target reconstructed image.

[0107] In some embodiments, steps S300 to S500, such as Figure 7 As shown, a coded image with a pixel size of 1500×1500 after noise matching is reconstructed through the first stage of backpropagation to restore the general outline and details of the image, resulting in the initial reconstructed image with a pixel size of 512×512. Then, the image is reconstructed through the second stage of the FFTConv-UNet network to obtain a clear image, and the target reconstructed image with a pixel size of 256×256 is output.

[0108] In some optional embodiments, a Fresnel aperture coded imaging system composed of a CMOS image sensor, an LCD screen, a computer, and other optical components is used to perform Fresnel aperture coded imaging based on two-stage reconstruction and noise matching, but the components used are not limited to this. The CMOS image sensor is used to acquire image data, the LCD screen serves as a light source and displays the image, the computer is used for image reconstruction, and other optical components may include a Fresnel aperture mask (a mask made based on the Fresnel diffraction principle). For example, as... Figure 8 As shown, in the Fresnel aperture coded imaging system, the FZA mask is fixed 3mm in front of the CMOS image sensor via a specially designed 3D-printed shell. The mask and the camera together form the Fresnel aperture coded camera. The target image to be captured is displayed in the center of the LCD screen, while the area outside the target image is covered in black (i.e., the LCD displays a pre-processed real image). The LCD screen is placed 300mm parallel to the plane of the Fresnel aperture coded imaging camera. Image data is captured by the Fresnel aperture coded camera and then transmitted to a computer, where a two-stage reconstruction algorithm is run to reconstruct the object image.

[0109] In some embodiments, a monochrome camera, model QHY163m, was selected as the shooting camera, employing a 16-megapixel Panasonic 4 / 3-inch CMOS sensor. The experimental setup diagram is shown below. Figure 9 As shown. The CMOS sensor has a pixel size of 3.8μm and an output resolution of 4656×3522. Single-exposure shooting was used in the experiment without pixel binning. The dataset used was the Places365-Standard dataset, a large-scale scene recognition dataset developed by the MITCSAIL team (a finely labeled dataset covering 365 scene categories, such as bedroom, street, park, airport, etc.). The target image on the LCD screen was replaced with uniform gray squares whose values ​​increased every 16 pixels, with values ​​of 0, 15, 31, ..., 239, 255. A total of 16 grayscale images were displayed on the LCD screen for shooting, as shown. Figure 2 As shown in the figure. After brightness calibration and noise distribution fitting, the noise matching result is obtained. The brightness calibration result is shown in the figure. Figure 10 As shown, the fitted curve is an exponential function curve; the noise fitting result is as follows. Figure 11 As shown in the figure. Noise is added to the simulated image data to complete the noise matching process, resulting in a noise-matched encoded map. Then, a one-stage fast reconstruction is performed on the encoded map, and the reconstruction result is shown in the figure. Figure 12 As shown, the upper part is the target test image, and the lower part is the image after a first-stage reconstruction, which quickly restores the basic structure and some details of the image. The results of the first-stage reconstruction (such as...) are then displayed. Figure 13Part (a) of the model is trained on the target FFTConv-UNet model, and a two-stage reconstruction is performed to obtain the final reconstructed image as shown in Figure 1. Figure 13 As shown in section (b) of the diagram. During model training, the AdamW optimizer was used with an initial learning rate of 0.0002, betas parameters set to (0.9, 0.999), and weight decay set to 0.01.

[0110] This invention employs a novel noise matching method, requiring only 16 grayscale images to obtain a noise distribution close to that of a real image. Due to a lack of real data, existing technologies generally use simulated data for model training. This invention can match simulated data to data close to real-world images, solving the problem of insufficient real data in Fresnel aperture coded imaging.

[0111] Table 1

[0112]

[0113]

[0114] Secondly, this invention employs a two-stage reconstruction method (a one-stage BP algorithm and a two-stage FFTConv-UNet model) to recover images and proposes a novel network structure. The image reconstruction quality surpasses existing technologies (such as image reconstruction methods using TwIST, ADMM, UNet, LinkNet, SUNet, and ATTUNet techniques). Figure 14 As shown, for each image, the higher the PSNR (Peak Signal-to-Noise Ratio, reflecting the noise in the reconstructed image) and SSIM (Structural Similarity Index, reflecting the detail in the reconstructed image), the higher the reconstruction quality. A summary of the average image reconstruction metrics is shown in Table 1. Therefore, compared with existing technologies, the method of this invention achieves better image reconstruction quality.

[0115] This invention also provides a Fresnel aperture coding imaging device based on two-stage reconstruction and noise matching, which can realize the above-mentioned Fresnel aperture coding imaging method based on two-stage reconstruction and noise matching. The device includes:

[0116] The first module is used to perform a convolution operation between the preprocessed real image and the Fresnel aperture to obtain a simulated image;

[0117] The second module is used to perform brightness conversion and noise fitting operations on the simulated image to obtain an encoded image;

[0118] The third module is used to perform backpropagation reconstruction on the encoded image to obtain an initial reconstructed image;

[0119] The fourth module is used to construct the target FFTConv-UNet model based on the Fast Fourier Transform module and the convolutional neural network.

[0120] The fifth module is used to input the initial reconstructed image into the target FFTCoV-UNet model to obtain the target reconstructed image.

[0121] It is understood that the content of the above method embodiments is applicable to the present device embodiments. The specific functions implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0122] This invention also provides an electronic device, which includes a processor and a memory. The memory stores a computer program, and when the processor executes the computer program, it implements the aforementioned Fresnel aperture-coded imaging method based on two-stage reconstruction and noise matching. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0123] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0124] refer to Figure 15 , Figure 15 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes:

[0125] The processor 601 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention.

[0126] The memory 602 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 602 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 602 and is called and executed by the processor 601 to execute the Fresnel aperture coded imaging method based on two-stage reconstruction and noise matching of the embodiments of this invention.

[0127] The input / output interface 603 is used to implement information input and output;

[0128] The communication interface 604 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0129] Bus 605 transmits information between various components of the device (e.g., processor 601, memory 602, input / output interface 603, and communication interface 604);

[0130] The processor 601, memory 602, input / output interface 603, and communication interface 604 are connected to each other within the device via bus 605.

[0131] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned Fresnel aperture-coded imaging method based on two-stage reconstruction and noise matching.

[0132] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0133] This invention also provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions, causing the computer device to perform the aforementioned Fresnel aperture-coded imaging method based on two-stage reconstruction and noise matching.

[0134] In summary, the Fresnel aperture coded imaging method and apparatus based on two-stage reconstruction and noise matching of the present invention have the following advantages:

[0135] 1. The embodiments of the present invention employ a noise matching method, which requires only 16 grayscale images to obtain a noise distribution close to that of real images. Furthermore, by preprocessing the simulation data, the noise distribution is made to approximate that of real data, thus solving the problem of lack of real data in Fresnel aperture coding imaging during network training.

[0136] 2. This invention employs a two-stage reconstruction method, combining the advantages of iterative reconstruction and deep learning techniques. In the first stage, the backpropagation (BP) algorithm is used to quickly recover the basic structure of the image. In the second stage, the forward convolution of FFT-Block (Fast Fourier Transform module) is innovatively combined with the UNet (named for its U-shaped network structure) encoding and decoding framework to construct a novel network structure, FFTConv-UNet, thereby improving the image reconstruction quality.

[0137] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this invention are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is altered and sub-operations described as part of a larger operation are executed independently.

[0138] Furthermore, although the invention has been described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the described functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the invention. Rather, given the properties, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the module will be understood within the scope of conventional skill of an engineer. Therefore, those skilled in the art can implement the invention as set forth in the claims using ordinary techniques without excessive experimentation. It is also understood that the specific concepts disclosed are merely illustrative and not intended to limit the scope of the invention, which is determined by the full scope of the appended claims and their equivalents.

[0139] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0140] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0141] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0142] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0143] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0144] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

[0145] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of the present invention.

Claims

1. A Fresnel aperture-coded imaging method based on two-stage reconstruction and noise matching, characterized in that, Includes the following steps: The preprocessed real image is convolved with the Fresnel aperture to obtain the simulated image; The simulated image is subjected to brightness conversion and noise fitting operations to obtain an encoded image; The encoded image is reconstructed by backpropagation to obtain an initial reconstructed image; Based on the Fast Fourier Transform module and convolutional neural network, a target FFTCoV-UNet model is constructed, including: An encoder-decoder structure is used to fuse the Fast Fourier Transform module with the convolutional neural network to obtain the initial FFTCOnov-UNet model. A target loss function is constructed based on pixel loss, Laplacian loss, and Fast Fourier Transform loss; the initial FFTCOnov-UNet model is trained based on the encoding map and the target loss function to obtain the target FFTCOnov-UNet model. The initial reconstructed image is input into the target FFTCoV-UNet model to obtain the target reconstructed image.

2. The Fresnel aperture coded imaging method based on two-stage reconstruction and noise matching according to claim 1, characterized in that, Before convolving the preprocessed real image with the Fresnel aperture to obtain the simulated image, the following steps are included: The real image is preprocessed by magnification and black filling to obtain the preprocessed real image.

3. The Fresnel aperture coded imaging method based on two-stage reconstruction and noise matching according to claim 1, characterized in that, The process of performing brightness conversion and noise fitting operations on the simulated image to obtain the encoded image includes the following steps: The original n-level grayscale image of the real image is acquired through a sensor; Construct an exponential function based on the original grayscale image and the real image; Obtain the first mean and first variance of the original grayscale image; Gaussian noise is obtained by linearly fitting the first mean and the first variance. Based on the exponential function and the Gaussian noise, the simulated image is subjected to brightness conversion and noise fitting operations to obtain the encoded image.

4. The Fresnel aperture coded imaging method based on two-stage reconstruction and noise matching according to claim 1, characterized in that, The formula used to perform backpropagation reconstruction on the encoded image to obtain the initial reconstructed image includes: ； In the formula, This represents the initial reconstructed image; Represents a real image; Represents the encoding diagram; This represents the inverse of the Fresnel aperture-coded transfer function in the frequency domain; It represents the Hadamardi (or Hadama) stack; This indicates the Fourier transform operation; This indicates the inverse Fourier transform operation.

5. The Fresnel aperture coded imaging method based on two-stage reconstruction and noise matching according to claim 1, characterized in that, The formula used to construct the target loss function based on pixel loss, Laplacian loss, and Fast Fourier Transform loss includes: ； In the formula, ； ； ； in, Represent the target loss function; Indicates pixel loss; Indicates Laplace loss; This represents the loss of the Fast Fourier Transform; The weights representing the Laplace loss; The weights representing the loss of the Fast Fourier Transform; Represents a real image; This represents the initial reconstructed image; Indicates the total number of pixels in the image; This indicates the Fourier transform operation; Indicates L2 norm operation; Indicates L1 norm operation; This represents the Laplace operator.

6. A Fresnel aperture-coded imaging device based on two-stage reconstruction and noise matching, characterized in that, include: The first module is used to perform a convolution operation between the preprocessed real image and the Fresnel aperture to obtain a simulated image; The second module is used to perform brightness conversion and noise fitting operations on the simulated image to obtain an encoded image; The third module is used to perform backpropagation reconstruction on the encoded image to obtain an initial reconstructed image; The fourth module is used to construct the target FFTCOnov-UNet model based on the Fast Fourier Transform module and the convolutional neural network; specifically, the fourth module is used for: An encoder-decoder structure is adopted to fuse the Fast Fourier Transform module with the convolutional neural network to obtain an initial FFTCOnov-UNet model; a target loss function is constructed based on pixel loss, Laplacian loss, and Fast Fourier Transform loss; the initial FFTCOnov-UNet model is trained based on the encoded map and the target loss function to obtain the target FFTCOnov-UNet model. The fifth module is used to input the initial reconstructed image into the target FFTCoV-UNet model to obtain the target reconstructed image.

7. An electronic device, characterized in that, Including the processor and memory; The memory is used to store programs; The processor executes the program to implement the method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The storage medium stores a program that is executed by a processor to implement the method as described in any one of claims 1 to 5.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 5.

Images