Snapshot polarization imager information reconstruction method based on combination of physical model and neural network
By combining physical models and neural networks in snapshot polarization imager for information reconstruction, the problems of long design time of two-dimensional low-pass filters and the impact of interference signals in the prior art are solved, and high-precision Stokes parameter reconstruction is achieved. And the method has the advantages of time-saving and convenient.
Patent Information
- Application Number
- CN202411322362.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-23
- Publication Date
- 2024-12-03
AI Technical Summary
The existing snapshot polarization imager needs to design a two-dimensional low-pass filter with appropriate time during the information reconstruction process, and the reconstruction effect is affected by the interference signal, resulting in poor reconstruction accuracy of Stokes parameter.
Using a method based on physical models combined with neural networks, by acquiring interference images, using physical models and neural networks for processing, building loss functions and automatic stop rules, using gradient descent algorithm to train neural networks, and optimizing parameters to achieve high-precision replication of Stokes parameters structure.
It saves time in designing low-pass filters, reduces the impact of interfering signals, improves the reconstruction accuracy of Stokes parameters, and can quickly optimize neural network parameters under the guidance of physical models, realizing time-saving and convenient information reconstruction.
Smart Images

Figure CN119068074A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the field of snapshot polarization imagers and information reconstruction, and in particular to a method for reconstructing information of snapshot polarization imagers based on a physical model combined with a neural network. Background Art
[0002] With the broad application prospects of polarization imaging technology in various fields, snapshot polarization imagers have attracted attention. They obtain the polarization information of the target light through a single imaging, fundamentally solving various problems existing in the measurement of time-sharing polarization imagers.
[0003] A spatially modulated snapshot polarimeter (MSPSIP) based on an improved Savart polarizer was proposed in 2012. When the target light is incident, it will be sheared into four beams of linear polarization, and the four beams have a fixed lateral offset. After passing through the polarizer and the imaging lens, an interference fringe image is formed on the CCD image plane. The four Stokes parameter images of the target light can be obtained by processing the interference image through a demodulation algorithm.
[0004] The information reconstruction method based on coherent demodulation theory introduces a signal with the same frequency as the two-dimensional carrier to multiply the signal around the two-dimensional carrier information back to low frequency, and then extracts the low-frequency signal through a two-dimensional low-pass filter to complete the extraction of a single Stokes parameter. By repeating this process, the reconstruction of all Stokes parameters can be completed. Although the reconstruction effect of this method is better than the traditional method of using reference light calibration, this method requires time to design a suitable two-dimensional low-pass filter, and the low-frequency signal extracted by the two-dimensional low-pass filter will contain interference signals from other channels, resulting in poor reconstruction effect. Summary of the invention
[0005] In order to solve the above problems, the present invention proposes a snapshot polarization imager information reconstruction method based on a physical model combined with a neural network, so as to save time and quickly reconstruct all the information of each Stokes parameter, thereby improving the reconstruction accuracy of the four Stokes parameters.
[0006] To achieve the above object, the present invention provides the following technical solutions:
[0007] The present invention provides a snapshot polarization imager information reconstruction method based on a physical model combined with a neural network, which comprises the following steps:
[0008] Step 1: Obtain interference image using snapshot polarization imager , the interference image is obtained through the physical model shown in formula (1) On Light intensity at position :
[0009] (1)
[0010] In formula (1), They represent the four Stokes parameter images. The light intensity at the location, represents the spatial carrier frequency, and has:
[0011] (2)
[0012] In formula (2), is the central wavelength of the incident light of the snapshot polarization imager, is the back focal length of the imaging lens in the snapshot polarization imager; is the transverse shear of the Savart veneer, and:
[0013] (3)
[0014] In formula (3), is the thickness of the Savart veneer, and They are the ordinary light refractive index and the extraordinary light refractive index in the Savart veneer respectively;
[0015] Step 2: Build a neural network And the interference image After processing, the four reconstructed Stokes parameter images are obtained. ,in, represents the i-th Stokes parameter image of the initial reconstruction;
[0016] Step 3: Substitute it into the physical model shown in equation (1) to obtain the preliminary reconstructed interference image superior Light intensity at position ;
[0017] Step 4: Construct the loss function according to formula (4) :
[0018] (4)
[0019] In formula (4), is the total variation regularization; is the square of the L2 norm; For neural networks Parameters;
[0020] Step 5: Use formula (5) to construct the automatic stop rule:
[0021] (5)
[0022] In formula (5), Indicates the neural network from the pth iteration to the p+qth iteration The mean of the structural similarity of the output reconstructed interference images; It means that the neural network from the pqth iteration to the pth iteration The mean of the structural similarity of the output reconstructed interference images; is the threshold for automatically stopping iteration;
[0023] Step 6: Use the gradient descent algorithm to derive the neural network Perform training and calculate the loss function To update the network parameters until the equation (5) is satisfied, stop the neural network Iterative training, so as to finally obtain the optimal Stokes parameter reconstruction model , used to reconstruct the information of the four Stokes parameters of the incident light.
[0024] The method for reconstructing information of a snapshot polarization imager based on a physical model combined with a neural network according to the present invention is also characterized in that the neural network in step 2 It consists of an encoder and a decoder, wherein the encoder includes: w encoding modules; the decoder includes: w symmetrical decoding modules;
[0025] The encoding module contains: two convolutional layers, a batch normalization layer, and a downsampling layer based on the maximum pooling operation;
[0026] The decoding module includes: two convolutional layers, a batch normalization layer, and an upsampling layer based on deconvolution operations; each convolutional layer is preceded by a "same" padding operation; the activation function connected after each convolutional layer uses the ReLU function;
[0027] The feature map obtained by each encoding module is concatenated with the feature map of the same size output by the symmetrical decoding module and then input into the next decoding module for processing, thereby forming a skip connection between each encoding module and the symmetrical decoding module;
[0028] The interference image Enter the neural network The four Stokes parameter images are initially reconstructed. .
[0029] An electronic device of the present invention includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the method, and the processor is configured to execute the program stored in the memory.
[0030] The present invention provides a computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and the computer program executes the steps of the method when executed by a processor.
[0031] Compared with the existing reconstruction method, the beneficial effects of the present invention are as follows:
[0032] 1. Compared with the reconstruction method based on coherent demodulation theory, the information reconstruction method of the present invention saves the time for designing and testing the best low-pass filter, inputs the interference image into the optimized Stokes parameter reconstruction model, extracts feature information by the encoder, and then processes it by the decoder, which fundamentally solves the problem of mutual interference between different Stokes parameter information in the reconstruction process based on coherent demodulation theory, and finally better realizes the information reconstruction of the four Stokes parameters of the incident light, thereby improving the reconstruction accuracy.
[0033] 2. Compared with the traditional neural network training method, the neural network training method of the present invention does not need to spend time preparing a large amount of real training data sets. It only needs to input an interference image obtained by a snapshot polarization imager to complete the optimization of the neural network parameters under the guidance of the physical model. Therefore, the information reconstruction method of the present invention has the advantages of saving time and being convenient. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] Figure 1 is a flow chart of the snapshot polarization imager information reconstruction method of the present invention;
[0035] Figure 2 The system structure diagram of the present invention using a snapshot polarization imager to obtain an interference image;
[0036] Figure 3 It is a neural network structure diagram based on the physical model of the present invention;
[0037] Figure 4 is a comparison diagram between Stokes parameter images obtained by using the reconstruction method of the present invention and the reconstruction method based on coherent demodulation theory;
[0038] In the figure, Filter is the filter, MSP 1 , MSP 2 Both are improved Savart polarizers consisting of two Savart plates with a half-wave plate in the middle. HWP is the half-wave plate, A is the polarizer, and L is the imaging lens. 1The optical axis of the left plate is 45° to the positive direction of the y and z axes, the optical axis of the right plate is 45° to the positive direction of the y axis and the negative direction of the z axis, and the fast axis of the half-wave plate is 45° to the positive direction of the x and y axes; MSP 2 The optical axis direction of the left plate is 45° to the positive direction of the x and z axes, the optical axis direction of the right plate is 45° to the positive direction of the x axis and the negative direction of the z axis, the fast axis of the half-wave plate is 45° to the positive direction of the x and y axes; the fast axis direction of the HWP is 22.5° to the y axis; the fast axis direction of L is 45° to the x axis; SSIM is the structural similarity between the reconstructed Stokes parameter image and the real Stokes parameter image. DETAILED DESCRIPTION
[0040] In this embodiment, Figure 1 As shown, a snapshot polarization imager information reconstruction method based on a physical model combined with a neural network includes the following steps:
[0041] Step 1, by Figure 2 Snapshot of the polarization imager acquiring the interference image shown In this embodiment, the interference image is a grayscale image of size 256×256. Jones matrix of snapshot polarization imaging system The expression of is shown in formula (1):
[0042] (1)
[0043] In formula (1), , , and Respectively represent polarizer A, half-wave plate HWP, MSP 1 and MSP 2 The Jones matrix of . So we have:
[0044] (2)
[0045] In formula (2), The Jones matrix representing the incident light of the system, , They represent the components of the electric vector of the incident light on the x-axis and y-axis respectively. The Jones matrix represents the output light after system modulation, , Respectively represent the components of the electric vector of the output light on the x-axis and y-axis, thereby obtaining the interference image On Light intensity at position The expression is shown in formula (3):
[0046] (3)
[0047] In formula (3), They represent the four Stokes parameter images. The light intensity at the location, represents the spatial carrier frequency, and has:
[0048] (4)
[0049] In formula (4), is the central wavelength of the incident light of the snapshot polarization imager; is the back focal length of the imaging lens in the snapshot polarization imager; in this embodiment, , . is the transverse shear of the Savart veneer, and:
[0050] (5)
[0051] In formula (5), is the thickness of the Savart veneer; and They are the ordinary light refractive index and the extraordinary light refractive index in the Savart single plate respectively; in this embodiment, , , .
[0052] Step 2: Build a neural network And the interference image After processing, the four reconstructed Stokes parameter images are obtained:
[0053] like Figure 3 As shown, the neural network It consists of an encoder and a decoder, wherein the encoder includes: w encoding modules, and the decoder includes: w symmetrical decoding modules; in this embodiment, w=4; the encoding module includes two convolutional layers with a dimension of 3×3 convolution, a batch normalization layer, and a downsampling layer implemented by a maximum pooling operation with a dimension of 2×2; the maximum pooling operation can reduce the size of the output feature map, reduce the operation time, and improve the robustness of the extracted features. The decoding module includes two convolutional layers with a dimension of 3×3 convolution, a batch normalization layer, and an upsampling layer implemented by a deconvolution operation with a dimension of 2×2; the batch normalization layer makes the output data distribution of each module conform to the standard normal distribution, avoiding gradient explosion or disappearance during the training process, and reducing the influence of neural network parameter initialization on the final parameter optimization effect.
[0054] A "same" padding operation is set before each 3×3 convolution in the neural network; this operation fills a circle of 0s around the input image, which ensures that the size of the input image and the output image are consistent and prevents the loss of image edge information;
[0055] The activation function of each 3×3 convolution connection in the neural network uses the ReLU function;
[0056] The feature map obtained by each encoding module is spliced together with the feature map of the same size output by the symmetrical decoding module, and input into the next decoding module for processing, thereby forming a jump connection between each encoding module and the symmetrical decoding module; the jump connection fully integrates the feature maps extracted at different stages, so that the neural network can better capture and understand the feature information of the input image, thereby performing better in Stokes parameter reconstruction.
[0057] The interference image Enter the neural network In the figure, we get the four initially reconstructed Stokes parameter images. ,in, represents the i-th Stokes parameter image of the initial reconstruction;
[0058] Step 3: Substitute it into the physical model shown in equation (3) to obtain the preliminary reconstructed interference image superior Light intensity at position ;
[0059] Step 4: Construct the loss function according to formula (6) :
[0060] (6)
[0061] In formula (6), is the total variation regularization; is the square of the L2 norm; For neural networks Parameters;
[0062] Step 5: Set an automatic stop rule for the iteration process. When the change of the structural similarity (SSIM) is less than the threshold, the iteration process stops automatically. The expression of the automatic stop rule is shown in formula (7):
[0063] (7)
[0064] In formula (7), Indicates the neural network from the pth iteration to the p+qth iteration The mean of the structural similarity of the output reconstructed interference images; It means that the neural network from the pqth iteration to the pth iteration The mean of the structural similarity of the output reconstructed interference images; is the threshold for automatically stopping iterations. Compared with the traditional method of setting a fixed number of iterations, setting an automatic stop rule can prevent the fixed number of iterations from being too few, resulting in poor reconstruction results, and can also avoid too many meaningless iterations. Moreover, since this rule considers the average SSIM of the adjacent q iterations, it can also avoid the premature termination of the iteration process due to the occasional small optimization effect of adjacent single iterations. Among them, the mean value of structural similarity Calculated by formula (7):
[0065] (8)
[0066] In formula (8), , Represent the real interference image and neural network respectively The average value of the light intensity at all positions of the output reconstructed interference image; , Represent the real interference image and neural network respectively The variance of the light intensity at all positions of the output reconstructed interference image; , There are 2 parameters respectively, and they are equal to 6.5025 and 58.5225; Representing real interference images and neural networks The covariance of the light intensity at all positions of the output reconstructed interference image.
[0067] Step 6: Use the gradient descent algorithm to train the neural network Perform training and calculate the loss function The network parameters are updated until equation (7) is satisfied, and the iterative training of the neural network is stopped. Finally, the optimal Stokes parameter reconstruction model is obtained. , used to reconstruct the information of the four Stokes parameters of the incident light.
[0068] Different from the traditional neural network training method, the neural network training of the present invention only needs one interference image to complete the neural network parameter training under the guidance of the physical model. Neural network training parameters The adjustment formula is shown in formula (9):
[0069] (9)
[0070] In formula (9), is the optimized parameter of the nth iteration; Parameters optimized for the previous iteration; Represents the loss function after the last iteration; is the learning rate; express Respectively Find the partial derivative of each parameter in; in this embodiment, .
[0071] Figure 4 The figure shows the comparison between four Stokes parameter images obtained by the reconstruction method of the present invention and the reconstruction method based on the coherent demodulation theory (method 1). SSIM is the structural similarity between the reconstructed Stokes parameter image and the real parameter image. It can be clearly seen from the figure that the SSIM of the reconstruction result of the present invention is higher, indicating that the reconstructed Stokes parameter image is more similar to the real Stokes parameter image, which proves that the reconstruction effect of the reconstruction method of the present invention is better.
[0072] In this embodiment, an electronic device includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the above method, and the processor is configured to execute the program stored in the memory.
[0073] In this embodiment, a computer-readable storage medium stores a computer program on the computer-readable storage medium, and the computer program executes the steps of the above method when executed by a processor.
Claims
1. A snapshot polarization imager information reconstruction method based on physical model combined with neural network, It is characterized in that The following steps are involved: Step 1: Obtain interference image using snapshot polarization imager , the interference image is obtained through the physical model shown in formula (1) On Light intensity at position : (1) In formula (1), They represent the four Stokes parameter images. The light intensity at the location, represents the spatial carrier frequency, and has: (2) In formula (2), is the central wavelength of the incident light of the snapshot polarization imager, is the back focal length of the imaging lens in the snapshot polarization imager; is the transverse shear of the Savart veneer, and: (3) In formula (3), is the thickness of the Savart veneer, and They are the ordinary light refractive index and the extraordinary light refractive index in the Savart veneer respectively; Step 2: Build a neural network And the interference image After processing, the four reconstructed Stokes parameter images are obtained. ,in, represents the i-th Stokes parameter image of the initial reconstruction; Step 3: Substitute it into the physical model shown in equation (1) to obtain the preliminary reconstructed interference image superior Light intensity at position ; Step 4: Construct the loss function according to formula (4) : (4) In formula (4), is the total variation regularization; is the square of the L2 norm; For neural networks Parameters; Step 5: Use formula (5) to construct the automatic stop rule: (5) In formula (5), Indicates the neural network from the pth iteration to the p+qth iteration The mean of the structural similarity of the output reconstructed interference images; It means that the neural network from the pqth iteration to the pth iteration The mean of the structural similarity of the output reconstructed interference images; is the threshold for automatically stopping iteration; Step 6: Use the gradient descent algorithm to derive the neural network Perform training and calculate the loss function To update the network parameters until the equation (5) is satisfied, stop the neural network Iterative training, so as to finally obtain the optimal Stokes parameter reconstruction model , used to reconstruct the information of the four Stokes parameters of the incident light.
2. According to claim 1, a snapshot polarization imager information reconstruction method based on a physical model combined with a neural network, It is characterized in that The neural network in step 2 It consists of an encoder and a decoder, wherein the encoder includes: w encoding modules; the decoder includes: w symmetrical decoding modules; The encoding module contains: two convolutional layers, a batch normalization layer, and a downsampling layer based on the maximum pooling operation; The decoding module includes: two convolutional layers, a batch normalization layer, and an upsampling layer based on deconvolution operations; each convolutional layer is preceded by a "same" padding operation; the activation function connected after each convolutional layer uses the ReLU function; The feature map obtained by each encoding module is concatenated with the feature map of the same size output by the symmetrical decoding module and then input into the next decoding module for processing, thereby forming a skip connection between each encoding module and the symmetrical decoding module; The interference image Enter the neural network The four Stokes parameter images are initially reconstructed. .
3. An electronic device comprising a memory and a processor, It is characterized in that The memory is used to store a program that supports a processor to execute the method according to claim 1 or 2, and the processor is configured to execute the program stored in the memory.
4. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the steps of the method according to claim 1 or 2 are performed.
