A 3D compressed imaging method and device based on self-supervised deep learning

Abstract

This invention relates to a 3D compressed imaging method and apparatus based on self-supervised deep learning. The method includes: imaging a 3D object using a coded measurement-based compressed imaging method; constructing a self-supervised deep learning network model; and optimizing and reconstructing the imaging results of the 3D object using the self-supervised deep learning network model. Compared with supervised learning, this invention not only eliminates the need for large amounts of labeled data, reducing data requirements and labor costs, but also eliminates the need to obtain the true values ​​of the 3D image, allowing the learning of useful representations from the data itself, thus improving the algorithm's adaptability and generalization; furthermore, it can effectively learn even with insufficient data, exhibiting good scalability. This invention solves the problems of insufficient resolution, slow data processing speed, weak processing capability for complex scenes, and high imaging costs associated with traditional 3D imaging.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional compression imaging technology, specifically to a three-dimensional compression imaging method and apparatus based on self-supervised deep learning. Background Technology

[0002] 3D imaging is a technology used to acquire, process, and present three-dimensional objects, with increasingly widespread applications in fields such as medicine, aviation, and geological exploration. 3D imaging can improve human understanding of three-dimensional objects and provide these fields with more accurate data and more vivid image representations. Currently, commonly used 3D imaging methods include compressed sensing-based 3D imaging and supervised deep learning-based 3D compressed imaging. Existing 3D imaging technologies often have some shortcomings, such as: insufficient resolution, inability to present finer structures; slow data processing speed, unable to meet real-time requirements; weak ability to handle complex scenes, prone to occlusion and artifacts; and high imaging costs.

[0003] The basic idea of ​​compressed sensing-based 3D imaging methods is to use compressed sensing technology to compress and sample 3D data, reducing the dimensionality and storage requirements. Then, reconstruction algorithms are used to restore the original data, reducing imaging costs and improving imaging speed and quality. However, this method has the following drawbacks: 1. It is not applicable to all imaging tasks; for certain specific tasks, a higher data sampling rate may be required, causing the compressed sensing method to fail. 2. When the target scene is complex, it is easily affected by noise, artifacts, and other factors, impacting imaging accuracy. 3. The algorithm has high complexity, requiring long computation times when dealing with large datasets. To address these issues, a 3D compressed imaging method based on supervised deep learning can be used for improvement.

[0004] The basic idea of ​​the supervised deep learning-based 3D compressed imaging method is to use a deep learning model to perform supervised learning on the input data, thereby achieving high-precision and high-efficiency 3D imaging. Compared with compressed sensing-based 3D imaging methods, this method can adapt to different imaging tasks, achieve high-precision and high-efficiency imaging, and handle large-scale, high-dimensional data. However, this method also has some shortcomings: First, it requires a large amount of labeled data, which has high requirements for the data and is costly; second, it cannot directly obtain the ground truth (GT) of the 3D image; third, adjusting and optimizing the algorithm requires a lot of manpower and time. Summary of the Invention

[0005] The purpose of this invention is to provide a three-dimensional compressed imaging method and apparatus based on self-supervised deep learning. This three-dimensional compressed imaging method and apparatus based on self-supervised deep learning can overcome the shortcomings of the prior art and combine the advantages of coded measurement and self-supervised deep learning. It can not only effectively compress three-dimensional data and reduce the cost of data storage and transmission, but also improve the quality and accuracy of three-dimensional imaging.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] In a first aspect of the present invention, a three-dimensional compressed imaging method based on self-supervised deep learning is disclosed.

[0008] Specifically, the method includes:

[0009] A coded measurement-based compressed imaging method is used to image three-dimensional objects;

[0010] A self-supervised deep learning network model is constructed, and the imaging results of 3D objects are optimized and reconstructed using the self-supervised deep learning network model.

[0011] Furthermore, the method of using coded measurement-based compressed imaging to image a three-dimensional object includes:

[0012] Construct a three-dimensional compressed imaging system based on code-pair measurements;

[0013] The three-dimensional compression imaging system is used to image three-dimensional objects and obtain sparse signals;

[0014] The sparse signal is sampled using a pairwise coding measurement method, and the pairwise coding measurement values ​​are obtained by simultaneous exposure using two different optical paths.

[0015] Furthermore, the construction of a self-supervised deep learning network model, and the optimization and reconstruction of the imaging results of 3D objects using the self-supervised deep learning network model, includes:

[0016] Construct a self-supervised deep learning network model and use the acquired pairwise encoded measurement values ​​to build a training dataset for self-supervised training;

[0017] The self-supervised deep learning network model is trained using the training dataset;

[0018] By utilizing a pre-trained self-supervised deep learning network model, a real-time reconstruction framework from two-dimensional coded measurements to three-dimensional images is established, enabling the optimization and reconstruction of 3D scenes from pairwise coded measurements.

[0019] Furthermore, the self-supervised deep learning network model is a convolutional neural network.

[0020] Furthermore, the loss function of the self-supervised deep learning network model is:

[0021]

[0022] Where γ is a weighting parameter used to balance the two loss functions. and The proportion of;

[0023] The characterization exchange measurement is used to evaluate the error between the encoded measurement obtained by predicting one of the paired encoded measurements through a neural network and simulating it with another forward imaging operator, and the other input encoded measurement. The formula is as follows:

[0024]

[0025] Self-measurement is used to evaluate the consistency between a coded measurement predicted from a two-dimensional coded measurement using a neural network and simulated by its own forward imaging operator and its corresponding input coded measurement. The formula is as follows:

[0026]

[0027] Paired coded measurements are obtained by simultaneous exposure using two different optical paths. One of the optical paths is randomly modulated by a dynamic diffractive optical element to obtain the measurement value. The forward propagation operator for this optical path is Φ1. The other optical path is unmodulated, and the measured value is obtained. The forward propagation operator at this point is denoted as Φ2. This represents the measurement values ​​obtained through self-supervised deep learning training. Inverse mapping to 3D images Indicates from the measured value Inverse mapping to 3D images

[0028] In a second aspect of the invention, a three-dimensional compression imaging device based on self-supervised deep learning is disclosed.

[0029] Specifically, the device includes a 3D object imaging module and a 3D object imaging result optimization and reconstruction module. The 3D object imaging module is used to image a 3D object using a coded measurement-based compressed imaging method; the 3D object imaging result optimization and reconstruction module is used to construct a self-supervised deep learning network model and use the self-supervised deep learning network model to optimize and reconstruct the imaging result of the 3D object.

[0030] Compared with the prior art, the advantages of the present invention are:

[0031] (1) This invention utilizes unsupervised learning to learn useful representations from the inherent characteristics of the data, thereby enabling tasks such as 3D imaging. Compared to supervised learning, this invention not only eliminates the need for large amounts of labeled data, reducing data requirements and labor costs, but also eliminates the need to obtain the true values ​​of 3D images. It can learn useful representations from the data itself, improving the algorithm's adaptability and generalization. Furthermore, it can effectively learn even when data is insufficient, exhibiting good scalability. This invention solves the problems of insufficient resolution, slow data processing speed, weak processing capability for complex scenes, and high imaging costs in traditional 3D imaging.

[0032] (2) The self-supervised learning-based 3D compressed imaging method proposed in this invention consists of two processes: encoding compression and decoding reconstruction. The process involves compressing the image from a high-dimensional space to a low-dimensional space using a sampling matrix encoding measurement, and then reconstructing the image from the low-dimensional space to a high-dimensional space using a reconstruction algorithm. This invention combines the advantages of encoding measurement and self-supervised deep learning, which can not only effectively compress 3D data and reduce the cost of data storage and transmission, but also improve the quality and accuracy of 3D imaging. Attached Figure Description

[0033] Figure 1 This is a flowchart of the three-dimensional imaging compression method in this invention;

[0034] Figure 2 This is a schematic diagram illustrating the sampling and imaging principle of a three-dimensional object;

[0035] Figure 3 This is the optical path diagram of a three-dimensional compressed imaging system;

[0036] Figure 4 This is a schematic diagram of the deep learning architecture design for 3D compressed imaging reconstruction.

[0037] Figure 5 This is a schematic diagram of the design principle of self-supervised deep learning based on pairwise coding measurement, where (a) is traditional supervised deep learning and (b) is self-supervised deep learning in this invention.

[0038] Figure 6 This is a design scheme diagram for building a deep convolutional neural network architecture based on modified U-net;

[0039] Figure 7 This is the overall flowchart of the three-dimensional imaging compression method in this invention. Detailed Implementation

[0040] The present invention will be further described below with reference to the accompanying drawings:

[0041] like Figure 1 This paper presents a three-dimensional compressed imaging method based on self-supervised deep learning.

[0042] Specifically, the method includes:

[0043] A coded measurement-based compressed imaging method is used to image three-dimensional objects;

[0044] A self-supervised deep learning network model is constructed, and the imaging results of 3D objects are optimized and reconstructed using the self-supervised deep learning network model.

[0045] Furthermore, the method of using coded measurement-based compressed imaging to image a three-dimensional object includes:

[0046] Construct a three-dimensional compressed imaging system based on code-pair measurements;

[0047] The three-dimensional compression imaging system is used to image three-dimensional objects and obtain sparse signals;

[0048] The sparse signal is sampled using a pairwise coding measurement method, and the pairwise coding measurement values ​​are obtained by simultaneous exposure using two different optical paths.

[0049] Furthermore, the construction of a self-supervised deep learning network model, and the optimization and reconstruction of the imaging results of 3D objects using the self-supervised deep learning network model, includes:

[0050] Construct a self-supervised deep learning network model and use the acquired pairwise encoded measurement values ​​to build a training dataset for self-supervised training;

[0051] The self-supervised deep learning network model is trained using the training dataset;

[0052] By utilizing a pre-trained self-supervised deep learning network model, a real-time reconstruction framework from two-dimensional coded measurements to three-dimensional images is established, enabling the optimization and reconstruction of 3D scenes from pairwise coded measurements.

[0053] The sampling imaging principle of three-dimensional objects is as follows Figure 2 As shown, Δ z Indicates the sampling interval. The scattering field of a three-dimensional object. After Born approximation, and then discretization, we obtain... Therefore, the discrete model of this scattering field can be represented using the angular spectrum method (ASM), and the equation can be simply written as:

[0054]

[0055] Where η is the scattering density function of the three-dimensional object, represented by η(x′,y′,z′). Represents the Fourier transform of η. lΔ represents the inverse Fourier transform operator. z Represents distance, exp[iklΔ z ] indicates at a distance lΔ z Phase delay at the point, Indicates the distance lΔ z The propagation transfer function.

[0056] The final linear measurement values ​​are as follows:

[0057]

[0058] in, Represents the scattering expression, and They represent in direction and The number of pixels in the orientation detector.

[0059] Define the forward imaging operator Φ, which is an M×N matrix representing the dimensionality reduction sampling operation. Its specific description is related to the physical processes and parameters involved in the imaging process. The specific formula is:

[0060] Φ=G 2D QB

[0061] in, Represents a block diagonal matrix. It is a matrix representation of 2DDFT, and its size is “bldiag” represents a block diagonal matrix. G 2D This represents the 2D inverse DFT matrix.

[0062] Q = [P] 1 ,P 2 ,...,P L ],LP i Let P be a matrix with size i = 1, ..., L. in Representation matrix P l The element value in row m1 and column m2, Δ k Indicates the sampling interval.

[0063] The self-supervised learning-based three-dimensional compressed imaging method described in this invention consists of two processes: encoding compression and decoding reconstruction. First, the image is compressed from a high-dimensional space to a low-dimensional space through sampling matrix encoding measurement. Then, the image is reconstructed from the low-dimensional space to the high-dimensional space through a reconstruction algorithm.

[0064] During the encoding process, coded measurement pairs (CMPs) are used to sample the sparse signal. CMPs refer to measuring the sparse signal in two directions. For example... Figure 3 As shown, the leftmost side is a three-dimensional object, and the right side samples and images this three-dimensional object. The principle of sampling and imaging is as follows: Figure 2 As shown. During the sampling process, a bandpass filter and polarizer are first used to standardize the optical path, selecting polarized light signals within a specific frequency range. Then, a beam splitter divides the optical path into two parts, and two different optical paths are simultaneously exposed to obtain paired coded measurement values. One part of the optical path is reflected by the beam splitter, randomly modulated by an LCoS element, and then the measurement value is obtained through CMOS 1. (The forward propagation operator is denoted as Φ1). The distance from the rightmost plane of the object to the beam splitter is z1, the distance from the beam splitter to the LCoS plane is z2, the distance from the rightmost plane of the object to the LCoS plane is d1, d1 = z1 + z2, and the distance from the LCoS plane to the CMOS1 plane is d3. The other optical path directly uses CMOS2 to obtain the measurement value without modulation. (The forward propagation operator is denoted as Φ2). The distance from the rightmost plane of the object to the CMOS2 plane is d2.

[0065] Measured values and measured values The two can be represented as:

[0066]

[0067]

[0068] For the optical path containing Φ2, the optical path E propagating to the CMOS2 element CMOS1 It can be represented as:

[0069]

[0070] Therefore, Φ2 can be expressed as:

[0071]

[0072] in, P represents the l-th matrix l The element values ​​in the m1-th row and m2-th column, l = 1, 2, ..., L,

[0073] The scattering field of the optical path containing Φ1 changes after phase modulation by the LCoS element, from the scattering field before reaching the LCoS from the three-dimensional object. It can be represented as:

[0074]

[0075] in,

[0076]

[0077] After reflection by the LCoS, the phase modulation on the LCoS is blkdiag(R). 11 ,R 22 ,...,R NN ), It can be represented as:

[0078]

[0079] Where R = blkdiag(R 11 ,R 22 ,...,R NN ), which represents the phase modulation of the optical field by LCoS.

[0080] From the reflected light to the CMOS1 element, optical path E CMOS1 It can be represented as:

[0081]

[0082] Therefore, Φ1 can be represented as:

[0083]

[0084] in, This represents the phase factor corresponding to a distance of d3. Therefore, it can be seen that the forward imaging operators Φ1 and Φ2 corresponding to different coding measurements are known.

[0085] A self-supervised deep learning network model was established and trained using paired encoded measurements to create a real-time reconstruction framework from 2D encoded measurements to 3D images. The self-supervised deep learning network model is a convolutional neural network.

[0086] In a typical decoding process, the sampled measurements need to be reconstructed into the original 3D image using a reconstruction algorithm. A typical compressed sensing reconstruction process can be represented as:

[0087]

[0088] The first term is the fidelity term, and the second term is the regularization term. λ represents the measured value, and λ represents the parameter that balances sparsity and reconstruction error.

[0089] In addition to the reconstruction methods mentioned above, a large number of data-driven reconstruction methods based on deep learning theory have emerged in recent years. The core idea is to replace the optimization-based compressed sensing reconstruction process with deep learning methods, achieving high-resolution reconstruction of signals from low-dimensional measurements. The basic idea of ​​applying deep learning in compressed imaging is to first train neural network parameters with a large amount of data, and then use a pre-trained deep convolutional neural network to reconstruct high-resolution images from low-dimensional measurements. A schematic diagram of the design and optimization of the neural network architecture in a classic deep learning optical imaging process is shown below. Figure 4 As shown, the design of this deep learning framework mainly includes the following five research contents: A) constructing training dataset, B) designing network structure, C) selecting loss function, D) regularization and E) optimization.

[0090] The principle of classic supervised deep learning methods is as follows: Figure 5 As shown in (a). On a large training dataset. The deep convolutional neural network obtained through training Once the training process is complete, it can be used to measure directly from the two-dimensional code. This method reconstructs 3D images. However, its application in 3D compressed imaging reconstruction faces several major challenges: 1) Current mainstream deep learning methods are mainly applicable to 2D imaging, with limited applications in 3D imaging; 2) In 3D imaging, it is difficult to know the ground truth of the target; 3) The deep neural networks corresponding to 3D imaging have too many unknown parameters, which can easily lead to overfitting; 4) The automated construction of deep neural network architectures and the optimization of training mechanisms remain significant challenges in deep learning.

[0091] In practical imaging systems, it is impossible to directly obtain the true three-dimensional light field distribution, meaning it is impossible to label it. A training dataset is constructed for supervised training. To avoid this limitation, this invention still uses pairwise encoded measurements to construct the training dataset and designs a special loss function that fits physical constraints for training the deep convolutional neural network using a self-supervised deep learning method. The preliminary design scheme to be adopted is as follows: Figure 5 As shown in (b). First, an imaging system is used to obtain paired encoded measurements as a training set, and then... Figure 6The planned neural network learns from two sets of measurements to obtain two trained neural networks. These two networks are then used to reconstruct and generate 3D image estimates. The network loss function parameters are then optimized using self-measurement and cross-measurement to obtain a better 3D image estimate. This scheme allows for the training of deep convolutional neural networks from pairwise encoded measurement pairs captured by multiple co-exposures under label-free, real 3D lighting conditions using a self-supervised deep learning method. The designed loss function is required to ensure consistency between the coded measurements of the estimated 3D light field under different imaging conditions.

[0092] To obtain self-supervised deep learning Paired-code measurements are obtained by simultaneously exposing two different optical paths. One of the optical paths is randomly modulated by a dynamic diffractive optical element (LCoS) to acquire the measurement value. (The forward propagation operator is denoted as Φ1), and the other optical path is unmodulated, obtaining the measurement value. (The forward propagation operator is denoted as Φ2). This involves multi-group pairwise coding measurements obtained through multiple collaborative measurements. A training dataset is constructed for self-supervised deep learning training. It is important to note that the final reconstruction stage utilizes a pre-trained DCNN architecture to achieve non-iterative real-time reconstruction of 3D images from a single 2D coded measurement.

[0093] The loss function in classic supervised learning is defined as follows:

[0094]

[0095] However, for three-dimensional light field imaging, the true value Typically unknown. Under unlabeled ground truth conditions, a self-supervised deep learning method is proposed to utilize pairwise encoded measurements captured by a real imaging system. Construct a training dataset, and define the loss function for self-supervised deep learning as follows:

[0096]

[0097]

[0098] Notice, This represents the measurement values ​​obtained through self-supervised deep learning training. Inverse mapping to 3D images Indicates from the measured value Inverse mapping to 3D images

[0099] The loss function for self-supervised learning is defined as:

[0100]

[0101] Where γ is a weighting parameter that balances the two loss functions. and The proportion. First item The term "swap-measurement" is used to evaluate the error between the encoded measurement obtained by predicting one of the paired encoded measurements through a neural network and simulating it with another forward imaging operator, and the other input encoded measurement. Its formula is as follows:

[0102]

[0103] Another item The term "self-measurement" is used to evaluate the consistency between the encoded measurement predicted by a neural network and simulated by its own forward imaging operator from a two-dimensional encoded measurement and its corresponding input encoded measurement. Its formula is as follows:

[0104]

[0105] The proposed design scheme for establishing a deep convolutional neural network architecture based on modified U-net in this invention is as follows: Figure 6 As shown, its general framework is based on the existing U-net network. The difference lies in the operations such as convolution and pooling adopted. The specific operations performed in each process are labeled with different types of arrows in the diagram.

[0106] The overall flowchart is as follows Figure 7 As shown, the algorithm is divided into two parts. The upper part focuses on training and optimizing the neural network. Specifically, for 3D scenes, a forward imaging operator is first constructed, and an imaging system is built. Paired encoded measurements are obtained through collaborative exposure as the training dataset. Then, the proposed self-supervised deep learning framework is used to train and optimize the network. In the lower part, for real 3D scenes, the same imaging system is used to obtain measurements. Then, the trained and optimized network is used to reconstruct the 3D scene estimation.

[0107] The present invention also includes a three-dimensional compressed imaging device based on self-supervised deep learning. The device includes a three-dimensional object imaging module and a three-dimensional object imaging result optimization and reconstruction module. The three-dimensional object imaging module is used to image a three-dimensional object using a compressed imaging method based on coded measurement. The three-dimensional object imaging result optimization and reconstruction module is used to construct a self-supervised deep learning network model and use the self-supervised deep learning network model to optimize and reconstruct the imaging result of the three-dimensional object.

[0108] In summary, this invention utilizes unsupervised learning to learn useful representations from the inherent characteristics of data, thereby achieving tasks such as 3D imaging. The proposed 3D compressed imaging method based on self-supervised learning consists of two processes: encoding compression and decoding reconstruction. The process involves compressing the image from a high-dimensional space to a low-dimensional space using a sampling matrix encoding measurement, and then reconstructing the image from the low-dimensional space back to the high-dimensional space using a reconstruction algorithm. This invention combines the advantages of encoding measurement and self-supervised deep learning, effectively compressing 3D data, reducing data storage and transmission costs, and improving the quality and accuracy of 3D imaging. Compared to supervised learning, this invention not only eliminates the need for large amounts of labeled data, reducing data requirements and labor costs, but also eliminates the need to obtain the true values ​​of 3D images, allowing the learning of useful representations from the data itself, thus improving the algorithm's adaptability and generalization. Furthermore, it can effectively learn even with insufficient data, exhibiting good scalability. This invention solves the problems of insufficient resolution, slow data processing speed, weak handling of complex scenes, and high imaging costs associated with traditional 3D imaging.

[0109] The above-described embodiments are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made by those skilled in the art to the technical solutions of the present invention without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A three-dimensional compressed imaging method based on self-supervised deep learning, characterized in that, The method includes: A coded measurement-based compressed imaging method is used to image three-dimensional objects; Construct a self-supervised deep learning network model and use it to optimize and reconstruct the imaging results of 3D objects; The construction of a self-supervised deep learning network model, and the optimization and reconstruction of the imaging results of 3D objects using the self-supervised deep learning network model, includes: Construct a self-supervised deep learning network model and use the acquired pairwise encoded measurement values ​​to build a training dataset for self-supervised training; The self-supervised deep learning network model is trained using the training dataset; By utilizing a pre-trained self-supervised deep learning network model, a real-time reconstruction framework from two-dimensional coded measurements to three-dimensional images is established, enabling the optimization and reconstruction of 3D scenes from pairwise coded measurements. The loss function of the self-supervised deep learning network model is: in, These are weight parameters used to balance the two loss functions. and The proportion of gravity; The characterization exchange measurement is used to evaluate the error between the encoded measurement obtained by predicting one of the paired encoded measurements through a neural network and simulating it with another forward imaging operator, and the other input encoded measurement. The formula is as follows: Self-measurement is used to evaluate the consistency between a coded measurement predicted from a two-dimensional coded measurement using a neural network and simulated by its own forward imaging operator and its corresponding input coded measurement. The formula is as follows: Paired coded measurements are obtained by simultaneous exposure using two different optical paths. One of the optical paths is randomly modulated by a dynamic diffractive optical element to obtain the measurement value. The forward propagation operator for this optical path is The other optical path was unmodulated, and the measurement value was obtained. The forward propagation operator at this time is denoted as , This represents the measurement values ​​obtained through self-supervised deep learning training. Inverse mapping to 3D images Indicates from the measured value Inverse mapping to 3D images .

2. The method according to claim 1, characterized in that, The method of imaging a three-dimensional object using a coded measurement-based compressed imaging method includes: Construct a three-dimensional compressed imaging system based on code-pair measurements; The three-dimensional compression imaging system is used to image three-dimensional objects and obtain sparse signals; The sparse signal is sampled using a pairwise coding measurement method, and the pairwise coding measurement values ​​are obtained by simultaneous exposure using two different optical paths.

3. The method according to claim 1, characterized in that, The self-supervised deep learning network model is a convolutional neural network.

4. A three-dimensional compression imaging device based on self-supervised deep learning, characterized in that, The device includes a three-dimensional object imaging module and a three-dimensional object imaging result optimization and reconstruction module; The three-dimensional object imaging module is used to image a three-dimensional object using a coded measurement-based compressed imaging method. The 3D object imaging result optimization and reconstruction module is used to construct a self-supervised deep learning network model and use the self-supervised deep learning network model to optimize and reconstruct the imaging results of 3D objects. The construction of a self-supervised deep learning network model, and the optimization and reconstruction of the imaging results of 3D objects using the self-supervised deep learning network model, includes: Construct a self-supervised deep learning network model and use the acquired pairwise encoded measurement values ​​to build a training dataset for self-supervised training; The self-supervised deep learning network model is trained using the training dataset; By utilizing a pre-trained self-supervised deep learning network model, a real-time reconstruction framework from two-dimensional coded measurements to three-dimensional images is established, enabling the optimization and reconstruction of 3D scenes from pairwise coded measurements. The loss function of the self-supervised deep learning network model is: in, These are weight parameters used to balance the two loss functions. and The proportion of gravity; The characterization exchange measurement is used to evaluate the error between the encoded measurement obtained by predicting one of the paired encoded measurements through a neural network and simulating it with another forward imaging operator, and the other input encoded measurement. The formula is as follows: Self-measurement is used to evaluate the consistency between a coded measurement predicted from a two-dimensional coded measurement using a neural network and simulated by its own forward imaging operator and its corresponding input coded measurement. The formula is as follows: Paired coded measurements are obtained by simultaneous exposure using two different optical paths. One of the optical paths is randomly modulated by a dynamic diffractive optical element to obtain the measurement value. The forward propagation operator for this optical path is The other optical path was unmodulated, and the measurement value was obtained. The forward propagation operator at this time is denoted as , This represents the measurement values ​​obtained through self-supervised deep learning training. Inverse mapping to 3D images Indicates from the measured value Inverse mapping to 3D images .

Images