A method, system and training method for de-mosaicking reconstruction of a focal plane polarized image

Abstract

The application discloses a kind of focal plane polarization image demosaicing reconstruction method, system and training method, belong to image processing technical field, including: to single-channel mosaic image is direction separation and initial reconstruction, obtains four-channel initial polarization intensity image;Original image and initial reconstruction result input shallow feature extraction network modeling different polarization direction between physical correlation and output rough reconstruction priori, according to the rough reconstruction result corresponding stokes component constructs continuous angle periodic structure feature, and combines material response probability graph to deformable sampling offset and applies boundary constraint;Adaptive fusion polarization structure feature and texture details.Based on fusion feature prediction correction residual, output full-resolution four-channel polarization intensity reconstruction image.The application can keep polarization parameter reliability while effectively improving the reconstruction quality of complex edge and high-frequency texture area.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically to a method, system, and training method for de-mosaic reconstruction of focal plane polarized images. Background Technology

[0002] Polarization imaging can acquire physical information such as surface roughness, surface normal, reflection characteristics and material properties that are difficult to characterize with traditional intensity images. It has important application value in scenarios such as target detection, reflection separation, 3D reconstruction, road scene perception, underwater imaging and industrial inspection.

[0003] A focal plane polarization camera integrates a periodically arranged micro-polarization array on the sensor's focal plane to acquire light intensity responses in multiple polarization directions, such as 0°, 45°, 90°, and 135°, in a single exposure. It boasts advantages such as compact structure, fast acquisition speed, and suitability for dynamic scenes. However, since each pixel only records light intensity information in one polarization direction, a mosaic reconstruction method is needed to recover the full-resolution four-channel polarization intensity image.

[0004] Unlike ordinary image de-mosaicing, focal plane polarization image reconstruction not only requires restoring a clear intensity image, but also maintaining the consistency of polarization physical parameters such as Stokes parameters, degree of linear polarization (DoLP), and angle of linear polarization (AoLP). If the reconstruction errors in the four polarization directions are inconsistent, the related errors will be further amplified during the polarization parameter calculation process, leading to fake textures, angle jumps, or distortion of polarization parameters.

[0005] Existing interpolation methods typically rely on manual priors such as local smoothing, directional interpolation, or channel correlation, which can easily produce jagged edges, blurring, or polarization parameter distortion in complex edges, high-reflectivity regions, and high-frequency texture areas. While optimization methods can improve reconstruction quality through sparse representation, nonlocal similarity, or regularization constraints, they usually require iterative solutions, resulting in high computational costs and making them difficult to meet the needs of real-time or embedded applications.

[0006] Existing deep learning methods can learn nonlinear mappings from focal plane polarized mosaic images to full-resolution polarized images. However, many methods primarily rely on intensity images or simple channel stacking features for reconstruction, failing to fully utilize the physical consistency between polarization channels, the periodicity of AoLP angles, and the differences in polarization responses across different material regions. Consequently, problems such as oversmoothing, texture aliasing, or angular discontinuities still easily arise at material boundaries, in areas with strong reflection, weak polarization, and high-frequency textures.

[0007] Therefore, there is still a need for a focal plane polarization image demosaic reconstruction system and training method that can simultaneously take into account intensity image reconstruction, polarization parameter consistency, and texture restoration of complex material regions. Summary of the Invention

[0008] In view of this, the present invention provides a method, system and training method for de-mosaic reconstruction of focal plane polarization images. The method is designed for single-channel polarization mosaic images acquired by focal plane polarization sensors. It constructs a collaborative reconstruction network composed of implicit material response branches, polarization structure flow, AoLP periodic texture flow and material response gated fusion module to achieve reconstruction of four-channel full-resolution polarization intensity images.

[0009] To achieve the above objectives, the present invention adopts the following technical solution: On one hand, the present invention provides a method for de-mosaic reconstruction of a focal plane polarization image, comprising: S1, acquire a single-channel polarized mosaic image collected by a split-focus plane polarization sensor, and separate the single-channel polarized mosaic image according to the sampling direction of the micro-polarization column to obtain multiple sparse sampling channels; S2, perform initial reconstruction on the sparse sampling channel to obtain a four-channel initial polarization intensity image, and perform shallow feature extraction on the single-channel polarization mosaic image and the four-channel initial polarization intensity image to obtain shared shallow features; S3, perform material response analysis on the shared shallow features to generate a multi-channel material response probability map; S4, perform polarization structure modeling on the shared shallow features, model the physical correlation between different polarization directions, output polarization structure features, predict coarse residuals from the polarization structure features, and combine the coarse residuals with the four-channel initial polarization intensity images to obtain a coarse reconstructed polarization image. S5. Calculate the corresponding Stokes component based on the coarsely reconstructed polarization image, and normalize the Stokes component to construct a continuous angular periodic structure feature. S6, perform texture extraction on the angular periodic structure features to obtain local texture features, and combine them with the material response probability map to obtain texture detail features; S7, adaptively fuse the polarization structure features, the texture detail features, and the material response probability map to obtain the fused features; S8. Based on the fusion features, predict the final correction residual, and combine the final correction residual with the four-channel initial polarization intensity image to output a full-resolution four-channel polarization intensity reconstructed image.

[0010] Preferably, the sparse sampling channel includes four polarization directions: 0 degrees, 45 degrees, 90 degrees and 135 degrees.

[0011] Preferably, a multi-channel material response probability map is generated from shared shallow features, wherein at least one channel is used to characterize the stable polarization response or strong polarization response region obtained by network adaptive learning, and the remaining channels are used to characterize the potential material response differences in different image regions.

[0012] Preferably, the angular periodic structural features are derived from the Stokes component corresponding to the coarsely reconstructed polarization image. and Normalization yields the result, which is used to characterize and The corresponding bi-angular periodic information.

[0013] Preferably, S6 specifically includes: Texture extraction is performed on the angular periodic structure features to obtain local texture features; The material boundary strength is identified based on the material response probability map. The material boundary strength is used to suppress the deformable sampling offset predicted by the angular periodic structure features. Then, feature extraction is performed to obtain texture detail features. Preferably, the deformable sampling offset is predicted based on the angular periodic structure features and the texture channel statistical features extracted from the initial reconstructed image, and the material boundary strength is calculated based on the gradients of the strongly polarized material response channels in the horizontal and vertical directions in the material response probability map. The material boundary strength is then used to suppress the deformable sampling offset across the material boundary.

[0014] Preferably, S7 specifically includes: Pixel-level fusion weights are generated based on polarization structure features, texture detail features, and material response probability maps. The contributions of polarization structure flow and angular periodic texture flow are adaptively adjusted in different image regions based on these pixel-level fusion weights to obtain fusion features.

[0015] On the other hand, the present invention also provides a system for implementing the above-described method for de-mosaic reconstruction of a focal plane polarization image, comprising: A split-focus plane polarization sensor is used to acquire single-channel polarized mosaic images collected by the split-focus plane polarization sensor. The channel separation module is used to separate the single-channel polarized mosaic image according to the sampling direction of the micro-polarization column to obtain multiple sparse sampling channels; An initial reconstruction module is used to perform initial reconstruction on the sparse sampling channels to obtain a four-channel initial polarization intensity image. The shallow feature extraction module is used to perform shallow feature extraction on the single-channel polarized mosaic image and the four-channel initial polarization intensity image to obtain shared shallow features. An implicit material response branch is used to perform material response analysis on the shared shallow features and generate a multi-channel material response probability map. The polarization structure flow is used to model the polarization structure of the shared shallow features to model the physical correlation between different polarization directions, output polarization structure features, predict coarse residuals from the polarization structure features, and combine the coarse residuals with the four-channel initial polarization intensity images to obtain a coarse reconstructed polarization image. An angular periodic structure construction unit is used to calculate the corresponding Stokes component based on the coarsely reconstructed polarization image, and to normalize the Stokes component to construct a continuous angular periodic structure feature. An angular periodic texture stream is used to extract texture from the angular periodic structural features, obtain local texture features, and combine them with the material response probability map to obtain texture detail features. The material response gated fusion module is used to adaptively fuse the polarization structure features, the texture detail features, and the material response probability map to obtain fused features; The output module is used to predict the final correction residual based on the fusion features, and combine the final correction residual with the four-channel initial polarization intensity image to output a full-resolution four-channel polarization intensity reconstructed image.

[0016] Thirdly, the present invention also provides a training method for the above-mentioned focal plane polarization image de-mosaic reconstruction method, characterized in that a composite loss function is used to train the reconstruction method, wherein the composite loss function includes at least four-channel polarization intensity reconstruction loss, Stokes parameter loss, AoLP period consistency loss, and physical consistency loss based on material response probability map weighting.

[0017] As can be seen from the above technical solutions, compared with the prior art, this invention discloses a method, system, and training method for de-mosaic reconstruction of focal plane polarized images. First, it enhances the physical correlation between the four polarization directions through polarization structure flow, reducing error accumulation in the calculation of Stokes parameters, DoLP, and AoLP. Second, it guides texture recovery by using AoLP periodic structure features constructed from coarsely reconstructed Stokes components, alleviating angle jumps and structural discontinuities at AoLP periodic boundaries. Third, it reduces cross-regional texture crosstalk at material boundaries by constraining deformable sampling offsets through material response probability maps. Fourth, it improves the reconstruction stability of complex edges, high-reflection, and high-frequency texture regions by adaptively allocating the contributions of polarization structure flow and texture detail flow in different regions through a material response-gated fusion mechanism. Fifth, it enhances the reliability of reconstructed polarization parameters through material response-weighted physical consistency constraints. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0019] Figure 1 This is an overall flowchart of the focal plane polarization image de-mosaic reconstruction method described in this invention.

[0020] Figure 2 This is a schematic diagram of the overall network model of the network that synergistically fuses polarization structure features and AoLP periodic texture features as described in this invention.

[0021] Figure 3 This is a schematic diagram illustrating the polarization image reconstruction effect of the present invention. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] This invention discloses a method for de-mosaic reconstruction of a focal plane polarization image, such as... Figure 1 As shown, it includes: S1, acquire a single-channel polarized mosaic image collected by a split-focus plane polarization sensor, and separate the single-channel polarized mosaic image according to the sampling direction of the micro-polarization column to obtain multiple sparse sampling channels.

[0024] Specifically, the 2x2 superpixels of the focal plane polarization sensor are equipped with micro-polarizers at four different angles: 0°, 45°, 90°, and 135°. The input single-channel polarization mosaic image is denoted as X, and the output target is a four-channel full-resolution polarization intensity image. ,in , , and These represent full-resolution polarization intensity images at 0°, 45°, 90°, and 135°, respectively.

[0025] The single-channel polarization mosaic observation model can be expressed as equation (1): (1) Where θ represents the polarization direction. This represents the binary sampling mask corresponding to the polarization direction θ. The image represents the potential full-resolution polarization intensity image with polarization direction θ, and ⊙ represents element-wise multiplication. Equation (1) shows that at each spatial location, the focal plane polarization sensor records the light intensity response in only one polarization direction.

[0026] S2, perform initial reconstruction on the sparse sampling channels to obtain a four-channel initial polarization intensity image, and perform shallow feature extraction on the single-channel polarization mosaic image and the four-channel initial polarization intensity image to obtain shared shallow features.

[0027] Based on the micro-polarization array arrangement, X is separated into sparsely sampled images with four polarization directions. Bilinear interpolation is used as a deterministic initial reconstruction operator. The initial polarization intensity image B of the four channels is obtained: (2) in, This represents the initially reconstructed four-channel polarization intensity image. In other embodiments, It can also be achieved using Newton interpolation, bicubic interpolation, directional interpolation, or edge-preserving interpolation.

[0028] The single-channel polarization mosaic image X is stitched together with the four-channel initial polarization intensity image B along the channel dimension, and then processed by a shallow feature extraction module. Obtain shared shallow features : (3) Where [X,B] represents the channel dimension concatenation operation. This represents a shallow feature extraction module consisting of one or more convolutional layers, normalization layers, and nonlinear activation layers.

[0029] S3, perform material response analysis on the shared shallow features to generate a multi-channel material response probability map M: (4) in, This represents the material response prediction module, and Softmax represents the channel normalization function. In a preferred embodiment of the invention, M includes three channels, wherein the first channel... The first channel represents the region of strong or stable polarization response, while the remaining channels represent the latent material response components learned adaptively by the network. To avoid overly strong prior material responses in the early stages of training, M can be compared with a uniformly distributed... Linear mixing was performed to obtain the response probability map of the weakened material. : (5) in, This represents a uniform probability distribution with the same number of channels as M. The material response injection coefficient is represented by . The linear mixing described above is mainly used in the training phase; in the deployment and inference phase, subsequent material boundary estimation and gating fusion can directly use the unmixed material response probability map M. For ease of description, the following examples use . This example illustrates the computational flow during the training phase; the inference phase can then utilize the computational resources at the corresponding locations. Replace with M.

[0030] S4. Perform polarization structure modeling on the shared shallow features, model the physical correlation between different polarization directions, output polarization structure features, predict coarse residuals from the polarization structure features, and combine the coarse residuals with the four-channel initial polarization intensity images to obtain a coarse reconstructed polarization image.

[0031] Polarization structure flow P based on shared shallow features Feature extraction is performed to model the physical correlations between the four polarization directions. The polarization structure flow P includes multiple residual blocks and depthwise separable smooth convolutional layers. The residual blocks enhance the correlation features between polarization channels, while the depthwise separable smooth convolutional layers suppress local high-frequency perturbations and maintain the consistency of the low-frequency polarization structure. The polarization structure flow outputs polarization structure features. And reconstructed from rough head Predicting bounded gross residuals: (6) (7) in, Indicates the coarse reconstruction residual. This represents the coarse residual scaling factor, and tanh represents the hyperbolic tangent function. Indicates a rough reconstruction head. This represents a coarsely reconstructed polarization image, and clip(·,0,1) restricts the pixel values ​​to the range of 0 to 1.

[0032] S5. Calculate the corresponding Stokes component based on the coarsely reconstructed polarization image, and normalize the Stokes component to construct a continuous angular periodic structure feature.

[0033] Based on the coarsely reconstructed polarization image Calculate Stokes parameters , and : (8) Furthermore, the degree of linear polarization DoLP and the angle of linear polarization AoLP are calculated: (9) (10) in, ε To avoid the stability constant of division by zero, atan2 is used according to and Determine the angular quadrant. For images reconstructed from coarse images... The calculated Stokes components are denoted as and .

[0034] The AoLP periodic structure features are used to provide a continuous directional structure representation at the angular periodic boundaries. Specifically, based on the Stokes components corresponding to the coarse reconstruction results... and Constructing normalized biangular periodic structure features : (11) (12) (13) in, and Corresponding to and It is used to continuously characterize the local angular orientation structure and provide directional priors for texture recovery.

[0035] S6, perform texture extraction on the angular periodic structure features to obtain local texture features, and combine them with the material response probability map to obtain texture detail features.

[0036] AoLP periodic texture flow T includes multi-scale convolutional branches and deformable convolutional modules. The multi-scale convolutional branches can include 3x3, 5x5, and 1x1 convolutions, whose outputs are concatenated along the channel dimension to enhance edge and texture responses at different scales. This incorporates AoLP periodic structural features. Texture channel statistical features After concatenation, the input offset prediction convolutional layer is used to obtain the deformable convolution sampling offset O: (14) in, This represents the statistical features of the texture channels extracted from the initial reconstructed image. Indicates the offset prediction convolutional layer. This represents the offset scaling factor.

[0037] Furthermore, based on the weakened material response probability diagram Strongly polarized material response channels Estimate material boundary strength : (15) in, x and y represents the gradient operators in the horizontal and vertical directions, respectively. This is achieved using material boundary strength. Suppressing the sampling offset O yields an offset constrained by the material boundary. : (16) Where β represents the boundary suppression coefficient. The deformable convolution module constrained by material boundaries outputs texture detail features. : (17) in, This represents the initial texture features output by the multi-scale convolution branch, and DeformConv represents the deformable convolution operation. This represents the texture refinement module.

[0038] S7, adaptively fuse the polarization structure features, the texture detail features, and the material response probability map to obtain the fused features.

[0039] The material response gating fusion module integrates polarization structure features Texture details And weakened material response probability diagram The initial gating weights are obtained by concatenating the components, performing a 1x1 convolution, and then applying the sigmoid function. : (18) in, This indicates a gated weight prediction convolutional layer. Further based on... The strongly polarized material response channel and other material response channels in the middle construct the material bias term. : (19) And the final fusion weight G is obtained: (20) in, This represents the material bias injection coefficient.

[0040] Fusion features Represented as: (twenty one) In this context, a larger G indicates a higher contribution from the polarization structure flow, while a smaller G indicates a higher contribution from the texture detail flow. Those skilled in the art will understand that the number of channels in the material response probability map, the kernel size and combination of the multi-scale convolutions, and the kernel size of the gated weight prediction layer can all be equivalently replaced or adjusted based on the actual sensor array, computing resources, or deployment platform, and still fall within the scope of the present invention.

[0041] S8. Based on the fusion features, predict the final correction residual, and combine the final correction residual with the four-channel initial polarization intensity image to output a full-resolution four-channel polarization intensity reconstructed image.

[0042] The final reconstructed head R is based on fusion features Predict the final bounded residual and add it to the initial reconstructed image B to output a full-resolution four-channel polarization intensity reconstructed image Y: (twenty two) Where R represents the final reconstructed head, represents the final residual scaling factor, and Y represents the final output four-channel full-resolution polarization intensity reconstructed image.

[0043] On the other hand, the present invention also provides a system for implementing the above-described method for de-mosaic reconstruction of a focal plane polarization image, such as... Figure 2 As shown, it includes: A split-focus plane polarization sensor is used to acquire single-channel polarized mosaic images collected by the split-focus plane polarization sensor. The channel separation module is used to separate the single-channel polarized mosaic image according to the sampling direction of the micro-polarization column to obtain multiple sparse sampling channels; An initial reconstruction module is used to perform initial reconstruction on the sparse sampling channels to obtain a four-channel initial polarization intensity image. The shallow feature extraction module is used to perform shallow feature extraction on the single-channel polarized mosaic image and the four-channel initial polarization intensity image to obtain shared shallow features. An implicit material response branch is used to perform material response analysis on the shared shallow features and generate a multi-channel material response probability map. The polarization structure flow is used to model the polarization structure of the shared shallow features to model the physical correlation between different polarization directions, output polarization structure features, predict coarse residuals from the polarization structure features, and combine the coarse residuals with the four-channel initial polarization intensity images to obtain a coarse reconstructed polarization image. An angular periodic structure construction unit is used to calculate the corresponding Stokes component based on the coarsely reconstructed polarization image, and to normalize the Stokes component to construct a continuous angular periodic structure feature. An angular periodic texture stream is used to extract texture from the angular periodic structural features, obtain local texture features, and combine them with the material response probability map to obtain texture detail features. The material response gated fusion module is used to adaptively fuse the polarization structure features, the texture detail features, and the material response probability map to obtain fused features. The output module is used to predict the final correction residual based on the fusion features, and combine the final correction residual with the four-channel initial polarization intensity image to output a full-resolution four-channel polarization intensity reconstructed image.

[0044] In a preferred training embodiment of the present invention, a polarization-sensing composite loss function can be used for end-to-end optimization. The total loss function can be expressed as: (twenty three) in, This represents the image reconstruction loss of the four-channel polarization intensity. Indicates average strength loss. This represents the Stokes parameter loss. Indicates the loss of linear polarization degree. This represents the range constraint loss of DoLP. This represents the periodic consistency loss of AoLP. This represents the gradient loss in AoLP. Indicates the loss of physical consistency. This represents the total variation loss in the material response probability plot. This represents the material response probability equilibrium loss; , , , , , , , and These represent the weights of the corresponding loss terms.

[0045] The periodic consistency loss of AoLP can be expressed as: (twenty four) in, Indicates the predicted linear polarization angle. This represents the true linear polarization angle. The physical consistency loss is used to constrain the approximate conservation relationship between the intensities in the four polarization directions: (25) In one embodiment, the physical consistency loss can be adaptively weighted by the material response probability map to improve the physical constraint strength in strongly polarized or stable polarized regions.

[0046] In an optional embodiment of the present invention, the network can be trained using publicly available polarization image datasets or self-acquired focal plane polarization image data. Training samples can be enhanced through methods such as cropping, rotation, flipping, and brightness normalization. Network training can employ the Adam optimizer or other gradient optimization algorithms. After training, the method can be deployed on a server, embedded computing platform, or image processing device connected to a focal plane polarization camera.

[0047] In practical applications, this invention can be used for industrial defect detection, reflection separation, robot vision, road scene perception, underwater imaging, and other scenarios that require the recovery of high-quality polarization intensity images and polarization parameters from a focal plane polarization sensor.

[0048] In one verification method of the present invention, the network is trained under full supervision using a full-resolution reference polarization image and its corresponding focal plane degradation simulation data, or trained and verified using a publicly available polarization dataset. Figure 3 As shown, (3A) is the input RAW mosaic image; (3B) is the image obtained from deterministic initial reconstruction. Figure (3C) shows the result obtained using the method of the present invention. Figures; (3D), (3E), and (3F) are the initial reconstructed DoLP diagram, the reconstructed DoLP diagram of the present invention, and the reference DoLP diagram, respectively; (3G), (3H), and (3I) are the initial reconstructed AoLP diagram, the reconstructed AoLP diagram of the present invention, and the reference AoLP diagram, respectively.

[0049] Depend on Figure 3 It can be seen that, compared with the initial reconstruction results, the method of the present invention achieves better results. The image can better preserve the outline of objects, text edges and local textures; in the DoLP image, it can reduce pseudo textures and local fluctuations caused by sparse sampling and interpolation errors; in the AoLP image, it can improve the structural continuity of the angular field and reduce instability near the periodic boundary. Figure 3 The evaluation results shown demonstrate that the present invention has good performance in maintaining the spatial continuity of polarization parameters and suppressing edge pseudo-textures.

[0050] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0051] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for de-mosaic reconstruction of a focal plane polarization image, characterized in that, include: S1, acquire a single-channel polarized mosaic image collected by a split-focus plane polarization sensor, and separate the single-channel polarized mosaic image according to the sampling direction of the micro-polarization column to obtain multiple sparse sampling channels; S2, perform initial reconstruction on the sparse sampling channel to obtain a four-channel initial polarization intensity image, and perform shallow feature extraction on the single-channel polarization mosaic image and the four-channel initial polarization intensity image to obtain shared shallow features; S3, perform material response analysis on the shared shallow features to generate a multi-channel material response probability map; S4, perform polarization structure modeling on the shared shallow features, model the physical correlation between different polarization directions, output polarization structure features, predict coarse residuals from the polarization structure features, and combine the coarse residuals with the four-channel initial polarization intensity images to obtain a coarse reconstructed polarization image. S5. Calculate the corresponding Stokes component based on the coarsely reconstructed polarization image, and normalize the Stokes component to construct a continuous angular periodic structure feature. S6, perform texture extraction on the angular periodic structure features to obtain local texture features, and combine them with the material response probability map to obtain texture detail features; S7, adaptively fuse the polarization structure features, the texture detail features, and the material response probability map to obtain the fused features; S8. Based on the fusion features, predict the final correction residual, and combine the final correction residual with the four-channel initial polarization intensity image to output a full-resolution four-channel polarization intensity reconstructed image.

2. The method for de-mosaic reconstruction of a focal plane polarized image according to claim 1, characterized in that, The sparse sampling channel includes four polarization directions: 0 degrees, 45 degrees, 90 degrees, and 135 degrees.

3. The method for de-mosaic reconstruction of a focal plane polarized image according to claim 1, characterized in that, A multi-channel material response probability map is generated from shared shallow features, where at least one channel is used to characterize a stable polarization response or a strong polarization response region obtained by network adaptive learning, and the remaining channels are used to characterize the potential material response differences in different image regions.

4. The method for de-mosaic reconstruction of a focal plane polarization image according to claim 1, characterized in that, The angular periodic structure features are derived from the Stokes component corresponding to the coarsely reconstructed polarization image. and Normalization yields the result, which is used to characterize and The corresponding bi-angular periodic information.

5. The method for de-mosaic reconstruction of a focal plane polarized image according to claim 1, characterized in that, S6 specifically includes: Texture extraction is performed on the angular periodic structure features to obtain local texture features; The material boundary strength is identified based on the material response probability map. The material boundary strength is then used to suppress the deformable sampling offset predicted by the angular periodic structure features. Finally, feature extraction is performed to obtain texture detail features.

6. The method for de-mosaic reconstruction of a focal plane polarized image according to claim 5, characterized in that, The deformable sampling offset is predicted based on the angular periodic structure features and the texture channel statistical features extracted from the initial reconstructed image. The material boundary strength is calculated based on the gradients of the strongly polarized material response channels in the horizontal and vertical directions in the material response probability map. The material boundary strength is used to suppress the deformable sampling offset across the material boundary.

7. The method for de-mosaic reconstruction of a focal plane polarized image according to claim 1, characterized in that, S7 specifically includes: Pixel-level fusion weights are generated based on polarization structure features, texture detail features, and material response probability maps. The contributions of polarization structure flow and angular periodic texture flow are adaptively adjusted in different image regions based on these pixel-level fusion weights to obtain fusion features.

8. A system for implementing a focal plane polarization image de-mosaic reconstruction method as described in any one of claims 1 to 7, characterized in that, include: A split-focus plane polarization sensor is used to acquire single-channel polarized mosaic images collected by the split-focus plane polarization sensor. The channel separation module is used to separate the single-channel polarized mosaic image according to the sampling direction of the micro-polarization column to obtain multiple sparse sampling channels; An initial reconstruction module is used to perform initial reconstruction on the sparse sampling channels to obtain a four-channel initial polarization intensity image. The shallow feature extraction module is used to perform shallow feature extraction on the single-channel polarized mosaic image and the four-channel initial polarization intensity image to obtain shared shallow features. An implicit material response branch is used to perform material response analysis on the shared shallow features and generate a multi-channel material response probability map. The polarization structure flow is used to model the polarization structure of the shared shallow features to model the physical correlation between different polarization directions, output polarization structure features, predict coarse residuals from the polarization structure features, and combine the coarse residuals with the four-channel initial polarization intensity images to obtain a coarse reconstructed polarization image. An angular periodic structure construction unit is used to calculate the corresponding Stokes component based on the coarsely reconstructed polarization image, and to normalize the Stokes component to construct a continuous angular periodic structure feature. An angular periodic texture stream is used to extract texture from the angular periodic structural features, obtain local texture features, and combine them with the material response probability map to obtain texture detail features. The material response gated fusion module is used to adaptively fuse the polarization structure features, the texture detail features, and the material response probability map to obtain fused features; The output module is used to predict the final correction residual based on the fusion features, and combine the final correction residual with the four-channel initial polarization intensity image to output a full-resolution four-channel polarization intensity reconstructed image.

9. A training method for a focal plane polarization image de-mosaic reconstruction method as described in any one of claims 1 to 7, characterized in that, The reconstruction method is trained using a composite loss function, which includes at least four-channel polarization intensity reconstruction loss, Stokes parameter loss, AoLP period consistency loss, and physical consistency loss weighted by material response probability map.

Images