Polarization spectrum imaging device, calculation reconstruction system and method thereof

Abstract

According to the polarization spectral imaging device, incident light of a target object is divided into a dispersion imaging light path and a polarization imaging light path, and the dispersion imaging light path sequentially passes through a dispersion element and a first rear lens group and then forms a dispersion spectral image containing spectral information of the target object on a monochromatic detector array; the polarization imaging light path forms a polarization image containing polarization information of a target object on the DoFP detector array after passing through the second rear lens group; the polarization spectrum calculation and reconstruction system performs mathematical modeling on a transmission process of polarization spectrum information of a target object by using a forward projection mathematical model, and a neural network model performs calculation and reconstruction on dispersion spectrum projection data and polarization projection data output by the forward projection mathematical model and generates a polarization spectrum image; the imaging device has the advantages of high data acquisition real-time performance and high light utilization efficiency, and the polarization spectrum image obtained through calculation and reconstruction of the calculation and reconstruction system is high in quality.

Description

Technical Field

[0001] This invention relates to the field of hyperspectral imaging technology, and in particular to a polarization spectral imaging device, a computational reconstruction system, and a method thereof. Background Technology

[0002] Polarimetric Spectral Imaging (PSI) is a novel technique for acquiring multidimensional optical properties of targets, emerging in the late 20th century based on spectral imaging and polarization imaging. It integrates polarization measurement, spectral detection, and imaging, enabling the acquisition and spectral imaging of the same target across the entire spectral band, thus obtaining high-resolution spectral and full polarization information for each pixel while simultaneously detecting the target's spatial morphology. Through this integrated image-spectrum-polarization imaging, targets can be detected and identified comprehensively in terms of physical shape, characteristic spectra, and polarization degree, making it promising for applications in target detection, camouflage identification, atmospheric monitoring, and image enhancement. Currently, typical polarimetric spectral imaging systems primarily enhance spectral imaging by adding the ability to acquire target polarization information.

[0003] Conventional interferometric polarization spectral imaging techniques are all scanning-based. Whether push-broom or Fourier transform, they require long-term point-by-point or line-by-line scanning to obtain complete Fourier spectrum or hyperspectral full polarization information. This results in low light utilization efficiency, long imaging time, large size, complex system, and high cost. Multidimensional coded aperture compression snapshot imaging methods can achieve real-time imaging, but they have low light utilization efficiency and poor reconstructed image quality. Lens array-based snapshot imaging methods can also achieve real-time imaging, but the image resolution is low.

[0004] In the prior art, the invention with publication number "CN 117686089 A" and titled "Dual-channel Integral Field-of-View Snapshot Hyperspectral Imaging System and Image Fusion Method" involves a system where target radiation is converged by an imaging objective lens and then split into two channels by a beam-splitting prism. The transmitted light is focused onto the focal plane of a panchromatic imaging detector to acquire a high spatial resolution panchromatic image of the target. The reflected light is focused onto a microlens array, integrated by the microlens array, and then sequentially passed through a plane folding mirror, a collimating mirror, a compound dispersion prism, and a focusing mirror. Finally, the focusing mirror focuses the light onto the focal plane of a spectral imaging detector to obtain a low spatial resolution spectral image of the target. While this invention can acquire high spatial resolution spectral images, it suffers from complex hardware, resulting in low light utilization efficiency and consequently poor reconstructed image quality.

[0005] In the prior art, the invention with publication number "CN 118190162 A" and titled "A Compressed Sensing Hyperspectral Coding Structured Light Imaging System and Method" includes an illumination light generation module, a spectral modulation module, a spatial coding module, and an image acquisition and reconstruction module. The spectral modulation module modulates the spectrum of the illumination beam using a spatial light modulator A, and uses two identical dispersive elements to achieve beam splitting and combining, outputting a spectrally coded parallel illumination beam. The spatial coding module generates striped structured light to illuminate the target using a spatial light modulator B. The image acquisition and reconstruction module uses a monochrome camera to acquire images of different colors of structured light illumination, and a computer solves for the target's spectral reflectance and depth information. In this system, the spatial modulation of the beam by the spatial light modulator B results in low quality of the subsequently constructed spectral image. Summary of the Invention

[0006] To overcome the shortcomings of the prior art, the present invention aims to propose a polarization spectral imaging device, a computational reconstruction system, and a method thereof. The imaging device uses a depolarizing beam splitter to divide the incident light from the target object into a dispersive imaging path and a polarization imaging path. The dispersive imaging path sequentially passes through a dispersive element and a first rear mirror group, forming a dispersive spectral image containing the spectral information of the target object on a monochromatic detector array. The polarization imaging path passes through a second rear mirror group, forming a polarization image containing the polarization information of the target object on a focal plane polarimeter (DoFP) detector array. The polarization spectral computational reconstruction system uses a forward projection mathematical model to mathematically model the transmission process of polarization spectral information formation from the target object, and outputs dispersive spectral projection data and polarization projection data to a neural network model. The neural network model performs computational reconstruction on the dispersive spectral projection data and polarization projection data to generate a polarization spectral image. The polarization spectral imaging device has the advantages of high real-time data acquisition and high light utilization efficiency. The polarization spectral image obtained by the computational reconstruction system is also of high quality, solving the problems of low light utilization efficiency and poor reconstructed image quality in conventional polarization spectral imaging techniques.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A polarization spectral imaging device includes a front lens group 2, a polarization-reducing beam splitter 3, a dispersive element 4, a first rear mirror group 5, a second rear mirror group 7, a monochromatic detector array 6, and a focal-plane polarimeter (DoFP) detector array 8. Light emitted from a target object 1 is converted into a parallel beam after passing through the front lens group 2. This parallel beam is then split into a dispersive imaging path and a polarization imaging path after passing through the polarization-reducing beam splitter 3. The dispersive imaging path passes sequentially through the dispersive element 4 and the first rear mirror group 5, forming a dispersive spectral image containing the spectral information of the target object 1 on the plane of the monochromatic detector array 6. The polarization imaging path passes through the second rear mirror group 7, forming a polarization image containing the polarization information of the target object 1 on the plane of the DoFP detector array 8.

[0008] Furthermore, each pixel of the DoFP detector array 8 integrates four miniature polarizing filters, which are arranged at different angles of 0°, 45°, 90°, and 135°. The DoFP detector array 8 can acquire polarization images containing polarization information of the target object 1.

[0009] A polarization spectrum calculation and reconstruction system includes a forward projection mathematical model and a neural network model. The forward projection mathematical model mathematically models the transmission process of polarization spectrum information formed by a beam emitted from a target object 1 passing through an imaging device, and then outputs dispersive spectral projection data and polarization projection data to the neural network model. The neural network model calculates and reconstructs the dispersive spectral projection data and polarization projection data to generate a polarization spectrum image.

[0010] Furthermore, the forward projection mathematical model is the following least squares optimization problem: in, Y and Y represent the optimal solutions to the least squares optimization problem. 0 Y 45 Y 90 , Y 135 These represent dispersive spectral projection data, 0° polarization projection data, 45° polarization projection data, 90° polarization projection data, and 135° polarization projection data, respectively. Represents the sum of the total intensity of polarized and unpolarized light. Represents linearly polarized light at 0° in the horizontal direction. and linearly polarized light at 90° vertical direction The difference in light intensity, Represents linearly polarized light at a 45° angle. and linearly polarized light in the 135° direction The difference in light intensity, Represents the dispersion process, where w represents the degree of discretization of the spectral response sensitivity of an 8-pixel monochromatic detector or DoFP detector array. , These represent the transmittance and reflectance of the depolarizing beam splitter 3, respectively.

[0011] Furthermore, the neural network model includes a spectral encoder, a polarization encoder, and a joint decoder. The spectral encoder and polarization encoder are used to extract different levels of encoded features from dispersive spectral projection data and polarization projection data, respectively. The encoded features interact with each other through a cross-activation layer connecting the spectral encoder and the polarization encoder. The joint decoder decodes the extracted encoded features layer by layer to reconstruct the polarization spectral image.

[0012] Furthermore, both the spectral encoder and the polarization encoder include convolutional layers, lightweight residual convolutional modules, and alternating 2x downsampling layers. ), where the convolutional layer consists of a regular convolution with a kernel size of 3×3.

[0013] Furthermore, the joint decoder includes alternating 2x upsampling layers ( ), lightweight residual convolution modules and convolutional layers.

[0014] Furthermore, the lightweight residual convolution module includes layer normalization, depthwise separable convolution, two 1×1 convolutions, and Gaussian error linear units.

[0015] Furthermore, the cross-activation layer is defined as follows: for the features extracted by the same level of lightweight residual convolution module in the spectral encoder and polarization encoder described above... and ,use right The gate function for cross-activation is shown below: in, Represents the gate function. Represents nonlinear activation functions such as ReLU, GELU, and Sigmoid. Indicates by Linear transformation layers composed of convolutions, This represents element-wise multiplication between high-dimensional features; Similarly, using right The gate function for cross-activation is shown below: A polarization spectrum calculation and reconstruction method, based on a polarization spectrum imaging device and a polarization spectrum calculation and reconstruction system, specifically includes the following steps: S1: The monochromatic detector array 6 and the DoFP detector array 8 in the polarization spectral imaging device receive the dispersive imaging optical path and the polarization imaging optical path of the target object 1, respectively, and convert the dispersive imaging optical path and the polarization imaging optical path into dispersive light signals and polarization light signals, respectively, and output them to the polarization spectral calculation and reconstruction system. S2: The polarization spectral calculation and reconstruction system uses a forward projection mathematical model to convert the dispersive light signal in step S1 into dispersive spectral projection data, and converts the polarization light signal in step S1 into 0° polarization projection data, 45° polarization projection data, 90° polarization projection data, and 135° polarization projection data. S3: The deep neural network model in the polarization spectrum calculation and reconstruction system uses the dispersive spectral projection data and 0° polarization projection data, 45° polarization projection data, 90° polarization projection data and 135° polarization projection data described in step S2 to complete the calculation and reconstruction to obtain the polarization spectrum image of the target object 1.

[0016] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. The polarization image, i.e., the projection data, containing the polarization spectral information of the target acquired by the polarization spectral imaging device in this invention does not have the time-division scanning process of common interferometric polarization spectral imaging systems, thus having high real-time performance; the polarization spectral imaging device does not have the coding aperture of common snapshot polarization spectral imaging systems, and all incident beams reach the detector for projection data acquisition, thus having high light utilization efficiency.

[0017] 2. The polarization images acquired by the polarization spectral imaging device in this invention have recorded complete spatial information since they have not undergone any light intensity or spectral coding modulation. They can be used as priors to guide spectral image reconstruction, and therefore the polarization spectral images obtained by calculation and reconstruction are of high quality.

[0018] 3. In the polarization spectral calculation and reconstruction system of this invention, the architecture consisting of a spectral encoder, a polarization encoder, and a joint decoder in a neural network model achieves efficient polarization spectral image reconstruction through multimodal feature collaborative extraction and deep fusion. The spectral encoder and the polarization encoder operate in parallel, respectively extracting data from dispersive spectral projection data. and polarization projection data Y, Y 0 Y 45 Y 90 Y 135Multi-scale features are extracted, and the two are used to achieve hierarchical feature interaction through cross-modal cross-activation layers to construct a physically related dual-stream feature representation. The joint decoder integrates the deep semantic features output by the spectral encoder and polarization encoder with the shallow detail information in the original projection data (transferred through cross-level residual connections) through upsampling operations, and finally reconstructs a high-fidelity polarization spectral image. This neural network model significantly enhances the representation ability of complex physical features while ensuring real-time computing efficiency through lightweight module design and cross-modal information interaction mechanism.

[0019] 4. The polarization spectral calculation and reconstruction method of the present invention converts the dispersive light signal and polarized light signal of the target object output by the imaging device into a polarization spectral image through a polarization spectral calculation and reconstruction system. This calculation and reconstruction method utilizes the complete spatial information of the target object and the low-loss light rays in the polarization spectral calculation and reconstruction system. After multimodal feature collaborative extraction and deep fusion by the deep neural network model in the reconstruction system, the high-fidelity polarization spectral image of the target object is finally generated rapidly.

[0020] In summary, compared with the prior art, the polarization spectral imaging device of the present invention has the advantages of high real-time data acquisition and high light utilization efficiency; the polarization spectral image obtained by the polarization spectral calculation and reconstruction system is also of high quality; and the polarization spectral calculation and reconstruction method can quickly generate high-fidelity polarization spectral images of the target object. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the polarization spectral imaging device in this invention.

[0022] Figure 2 This is a schematic diagram of the neural network model in this invention.

[0023] Figure 3 This is a schematic diagram of the lightweight residual convolution module in this invention.

[0024] Figure 4 This is a graph showing the total intensity components of polarized and unpolarized light in different spectral bands of hyperspectral images obtained by calculation and reconstruction in this invention.

[0025] Figure 5 This is a polarization degree component map of the hyperspectral images of different spectral bands obtained by calculation and reconstruction in this invention.

[0026] Figure 6 This is a polarization angle component map of the hyperspectral images of different spectral bands obtained by calculation and reconstruction in this invention.

[0027] In the diagram, 1. Target object; 2. Front lens group; 3. Polarization-depolarizing beam splitter; 4. Dispersion element; 5. First rear mirror group; 6. Monochromatic detector array; 7. Second rear mirror group; 8. DoFP detector array. Detailed Implementation

[0028] The following is combined Figures 1 to 6 The present invention will be further described in detail below: like Figure 1 As shown, a polarization spectral imaging device includes a front lens group 2, a depolarization beam splitter 3, a dispersive element 4, a first rear mirror group 5, a second rear mirror group 7, a monochromatic detector array 6, and a DoFP detector array 8; wherein, the depolarization beam splitter 3 can be an optical element that splits the imaging optical path into two paths, such as a depolarization beam splitter prism or a plane mirror, and the dispersive element 4 can be a component that achieves the same dispersive purpose, such as a dispersive prism, a grating, or a diffractive optical element.

[0029] The target object 1 is located on the front focal plane of the front imaging optical path. The light emitted by the target object 1 becomes a parallel beam after passing through the front lens group 2. The parallel beam is split into a dispersive imaging optical path and a polarization imaging optical path after passing through the depolarization beam splitter 3. The dispersive imaging optical path passes through the dispersive element 4 (which creates a misalignment of different wavelengths) and the first rear mirror group 5 in sequence, forming a dispersive spectral image containing the spectral information of the target object 1 on the plane of the monochromatic detector array 6. The polarization imaging optical path passes through the second rear mirror group 7, forming a polarization image containing the polarization information of the target object 1 on the plane of the DoFP detector array 8. The polarization image has not undergone any intensity or spectral encoding modulation. The polarization spectral image of the target object 1 can be reconstructed by calculating and inverting the dispersive spectral image and the polarization image.

[0030] The front lens group 2 includes one or more lenses for converting light emitted from a point on the target object 1 located on its front focal plane into a parallel beam.

[0031] The depolarizing beam splitter 3 used in this embodiment of the invention is specifically a non-polarizing beam splitter (NPBS), which consists of a pair of precision-machined and calibrated right-angle prisms. These prisms are tightly joined by optical bonding or mechanical assembly to ensure that wavefront distortion is controlled to a minimum. This device can accurately split S-polarized light (vertically polarized light) and P-polarized light (parallel polarized light) according to a preset reflection-transmission ratio within a specific wavelength range. Its core characteristic is that it can maintain the polarization state of the incident beam: after beam splitting, whether it is the reflected beam or the transmitted beam, its S-polarization state and P-polarization state are consistent with the incident beam, without introducing additional polarization changes.

[0032] The dispersive element 4 disperses the dispersive imaging optical path, creating misalignments of different wavelengths, which facilitates subsequent calculation, inversion, and reconstruction of the spectral information of the target object 1.

[0033] The first rear mirror group 5 includes one or more lenses for converging the dispersed dispersive imaging light path onto the focal plane of the monochromatic detector array 6 to form a dispersive spectral image.

[0034] The second rear mirror group 7 includes one or more lenses for converging the polarization imaging optical path onto the focal plane of the DoFP detector array 8 to form a polarization image.

[0035] The monochromatic detector array 6 is used to convert the optical signal of the dispersive imaging optical path into an electrical signal to obtain a dispersive spectral image containing the spectral information of the target object 1 and with wavelength-related spatial misalignment.

[0036] Furthermore, each pixel of the DoFP detector array 8 integrates four miniature polarizing filters, which are arranged at different angles of 0°, 45°, 90° and 135°. The DoFP detector array 8 can acquire polarized images containing polarization information of the target object 1, thereby allowing the simultaneous acquisition of light intensity information from different polarization angles. Real-time acquisition of polarization images can be achieved without conventional mechanical rotation of polarizers or multiple exposures.

[0037] A polarization spectral calculation and reconstruction system includes a forward projection mathematical model and a feature-cross-activation deep neural network model for polarization spectral image calculation and reconstruction. The forward projection mathematical model mathematically models the transmission process of polarization spectral information formed by a beam emitted from a target object 1 passing through an imaging device, and then outputs dispersive spectral projection data and polarization projection data to the neural network model. The dispersive spectral projection data and polarization projection data provide accurate prior information and training data for the neural network model. The neural network model calculates and reconstructs the dispersive spectral projection data and polarization projection data to generate a polarization spectral image.

[0038] Let the four Stokes components of the polarization spectrum image of target object 1 be as follows: , and ,in This represents the sum of the intensities of polarized and unpolarized light. This represents linearly polarized light with a horizontal angle of 0° and a vertical angle of 90°. and The difference in light intensity, This represents linearly polarized light at 45° and 135° directions. and The difference in light intensity, This represents the intensity difference between left-handed and right-handed circularly polarized light. In natural light sources and most imaging scenarios, light mainly exists in a linearly polarized form, with a small circularly polarized component that can be ignored. Represents its two-dimensional spatial coordinates, The coordinates represent its spectral dimensions.

[0039] For the polarization imaging optical path, the DoFP detector array 8 records the polarization light intensity in four directions: 0°, 45°, 90°, and 135°, respectively. , , , The relationship between these polarized light intensities and the four Stokes components can be expressed by formula (1): (1) In formula (1), the spectral response range of the pixels in the monochromatic or DoFP detector array 8 is denoted as... The spectral response sensitivity is For a dispersive imaging optical path, assuming the transmittance of the depolarizing beam splitter 3 is... The reflectivity is No absorption .

[0040] The dispersion equation for dispersive element 4 is expressed as follows: Then, in the monochromatic detector array 6, the position... The intensity of polarized light, i.e., the intensity value g(x,y) of the polarized light signal, can be expressed by formula (2): (2) Position in monochromatic detector array 6 Corresponding pixel The polarization intensity g(m,n) can be expressed as formula (3): (3) In formula (3), This indicates the pixel size of the detector.

[0041] Corresponding pixels The polarized light intensity at a certain point can be expressed as a weighted sum of the spectra, as shown in formula (4): (4) In formula (4), I 0(m,n) I 45(m,n) I 90(m,n) I 135(m,n) Representing pixels The polarization intensity at 0°, 45°, 90° and 135° respectively.

[0042] To facilitate subsequent calculation and reconstruction, Given a unit length in space, the wavelength misalignment of a pixel is given by a unit length in the spectral dimension. Formulas (2) and (4) are expressed in discrete form. The polarization spectral quantity cube is denoted as a 4-dimensional tensor. ,in and These represent the number of pixels in the height and width directions of the spatial dimension after discretization, respectively. This represents the number of channels in the discrete spectral dimension. , representing the four polarization directions. The spectral response sensitivity of an 8-pixel monochromatic or DoFP detector array can be discretized and expressed as: At this point, the polarized light intensity in formula (2) can be transformed into formula (5): (5) In formula (4) I 0(m,n) I 45(m,n) I 90(m,n) I 135(m,n) This can be converted into formula (6): (6) Construct the following operator The dispersion process of dispersive element 4 in the spectral dimension is described in detail as shown in formula (7): (7) At this point, the forward projection mathematical models of the dispersive imaging optical path and the polarization imaging optical path described by formulas (5) and (6) can be expressed in a unified tensor form as shown in formula (8): (8) In formula (8), Y, Y 0 Y 45 Y 90 Y 135 These represent dispersive spectral projection data, 0° polarization projection data, 45° polarization projection data, 90° polarization projection data, and 135° polarization projection data, respectively.

[0043] Furthermore, the forward projection mathematical model described in equation (8) can be expressed as a least squares optimization problem as shown in equation (9): (9) In formula (9), This represents the optimal solution to a least squares optimization problem based on maximum a posteriori theory, where Y, Y 0 Y 45 Y 90 Y 135These represent dispersive spectral projection data, 0° polarization projection data, 45° polarization projection data, 90° polarization projection data, and 135° polarization projection data, respectively. X represents a certain dimension of the 4-dimensional tensor. Represents the sum of the total intensity of polarized and unpolarized light. Represents linearly polarized light at 0° in the horizontal direction. and linearly polarized light at 90° vertical direction The difference in light intensity, Represents linearly polarized light at a 45° angle. and linearly polarized light in the 135° direction The difference in light intensity, Representing the dispersion process, w represents the degree of discretization of the spectral response sensitivity of a 6-pixel monochromatic detector array or an 8-pixel DoFP detector array. , These represent the transmittance and reflectance of the depolarizing beam splitter 3, respectively.

[0044] Because the high-dimensional polarization spectral data cube is compressed and sampled, the dimension of the projected data is much smaller than the dimension of the polarization spectral data cube to be reconstructed. Therefore, equation (9) is an ill-posed optimization problem. Although the ill-posedness of the reconstruction problem can be improved by introducing a regularization term and then using an iterative solution with alternating descent, the reconstruction results are poor, and the iterative process is time-consuming, making it unsuitable for real-time imaging of dynamic scenes. To address this, this invention, based on deep learning theory, constructs... Figure 2 The feature cross-activation deep neural network model shown achieves efficient real-time computation and reconstruction of polarization spectral images.

[0045] Furthermore, the neural network model includes a spectral encoder, a polarization encoder, and a joint decoder. The spectral encoder and polarization encoder are used to extract different levels of encoded features from dispersive spectral projection data and polarization projection data, respectively. The encoded features interact with each other through a cross-activation layer connecting the spectral encoder and the polarization encoder. The joint decoder decodes the extracted encoded features layer by layer to reconstruct the polarization spectral image. There is a simple residual connection between the encoder and the decoder to fully extract the original projection information without increasing the computational load.

[0046] Furthermore, both the spectral encoder and the polarization encoder include convolutional layers, lightweight residual convolutional modules, and alternating 2x downsampling layers. The convolutional layers consist of regular convolutions with a kernel size of 3×3. These convolutional layers are used to extract shallow features. Without loss of generality, for a spectral encoder, the number of input channels is... The number of output channels is For a polarization encoder, the number of its input channels is: The output channel count is also 64, facilitating subsequent feature cross-activation. The five alternating lightweight residual convolutional layers and the 2x downsampling layer used here gradually reduce the input feature map size while increasing channel information, thereby extracting high-level abstract features without loss of generality. The output channel counts of the five lightweight residual convolutional layers are 64, 128, 256, 512, and 1024, respectively. The number of input channels is the same as the output channel count of the previous module. The downsampling layer reduces the feature map size, allowing the network to focus on more abstract high-level features rather than low-level features.

[0047] Furthermore, the joint decoder includes alternating 2x upsampling layers ( The network employs a lightweight residual convolutional module and convolutional layers. The 2x upsampling layer gradually increases the spatial dimension of the input feature map, enhancing feature representation while restoring the original image size. The four alternating 2x upsampling layers and lightweight residual convolutional modules progressively restore the spatial resolution of the input while further enhancing detailed information and high-level abstract features in the feature map. Each convolutional module effectively integrates the expanded spatial information during upsampling and reduces computation and parameter count through lightweight design, improving network efficiency and robustness. Generally, the four lightweight residual convolutional modules have 1024, 512, 256, and 128 input channels and 512, 256, 128, and 64 output channels, respectively. This progressive reduction in channel count preserves rich feature information, effectively capturing features at different scales and semantic levels. Finally, the convolutional layers are standard 3×3 convolutions with 64 input and 64 output channels. Its output is the final reconstructed polarization spectrum image. .

[0048] Furthermore, such as Figure 3As shown, the lightweight residual convolution module includes Layer Normalization (Layer Normalization), Depthwise Convolution (DW Conv), two 1×1 convolutions (Conv 1×1), and a Gaussian Error Linear Unit (GeLU). Layer Normalization accelerates the training process and improves model stability. Depthwise Convolution is a highly efficient convolution operation widely used in lightweight neural network design, significantly reducing computational complexity and parameter count without loss of generality; in this invention, the size of the separable convolution kernel is 3×3. The Gaussian Error Linear Unit is a smooth, non-linear activation function that also outputs in the negative region, allowing for better gradient propagation in deep networks and mitigating gradient vanishing. The two 1×1 convolutions, with the first following the Depthwise Convolution, allow adjustment of the number of channels without changing the feature map space size, while the second, following the Gaussian Error Linear Unit, enhances the network's non-linear expressive power.

[0049] Furthermore, the cross-activation layer is defined as follows: for the features extracted by the same level of lightweight residual convolution module in the spectral encoder and polarization encoder described above... and ,use right The gate function for cross-activation is shown in equation (10): (10) In formula (10), Represents the gate function. Represents nonlinear activation functions such as ReLU, GELU, and Sigmoid. Indicates by Linear transformation layers composed of convolutions, This represents element-wise multiplication between high-dimensional features; Similarly, using right The gate function for cross-activation is shown in equation (11): (11) The above gate function is used to process the output of each residual convolution module of the spectral encoder and polarization encoder. Cross-activation can promote information exchange during the coding process and improve the information extraction capability of the module.

[0050] A polarization spectrum calculation and reconstruction method, based on a polarization spectrum imaging device and a polarization spectrum calculation and reconstruction system, specifically includes the following steps: S1: The monochromatic detector array 6 and the DoFP detector array 8 in the polarization spectral imaging device receive the dispersive imaging optical path and the polarization imaging optical path of the target object 1, respectively, and convert the dispersive imaging optical path and the polarization imaging optical path into dispersive light signals and polarization light signals, respectively, and output them to the polarization spectral calculation and reconstruction system. S2: The polarization spectral calculation and reconstruction system uses a forward projection mathematical model to convert the dispersive light signal in step S1 into dispersive spectral projection data, and converts the polarization light signal in step S1 into 0° polarization projection data, 45° polarization projection data, 90° polarization projection data, and 135° polarization projection data. S3: The deep neural network model in the polarization spectrum calculation and reconstruction system uses the dispersive spectral projection data and 0° polarization projection data, 45° polarization projection data, 90° polarization projection data and 135° polarization projection data described in step S2 to complete the calculation and reconstruction to obtain the polarization spectrum image of the target object 1.

[0051] Example In this embodiment of the invention, the depolarization beam splitter 3 of the polarization spectral imaging device is specifically a depolarization beam splitter prism, and the dispersive element 4 is specifically a dispersive prism. The polarization spectrum of the target object is measured in the 520-690 nm band using the polarization spectral imaging device, and then a polarization spectral image is reconstructed using a polarization spectral calculation and reconstruction system. Figure 4 It can be seen that the hyperspectral images of the target object in the 520-690nm spectral band have high resolution and clear details. Figure 5 It can be seen that the polarization degree of the target object varies in the 520-690nm spectral range. Figure 6 As can be seen, the polarization angle images of the target object in the 520-690nm spectral band are rich in detail and of high quality. Compared with existing snapshot imaging methods based on multi-dimensional coded aperture compression or lens arrays, the polarization spectral images of the target object obtained by this invention have the advantages of fast generation speed and high image quality.

[0052] The working principle of this invention is as follows: In this invention, the light emitted from the target object becomes a parallel beam after passing through the front lens group. This parallel beam is then split into a dispersive imaging path and a polarization imaging path after passing through the depolarization beam splitter. The dispersive imaging path passes sequentially through a dispersive element and a first rear mirror group, forming a dispersive spectral image containing the target object's spectral information on the monochromatic detector array plane. The polarization imaging path passes through a second rear mirror group, forming a polarization image containing the target object's polarization information on the DoFP detector array plane. The forward projection mathematical model in the polarization spectral calculation and reconstruction system mathematically models the received polarization spectral information and outputs dispersive spectral projection data and polarization projection data to the neural network model. These data are then processed by deep convolution of the spectral encoder and polarization encoder in the neural network model to ultimately reconstruct a high-fidelity polarization spectral image. .

Claims

1. A polarization spectral imaging device, comprising a front lens group (2), a polarization-reducing beam splitter (3), a dispersive element (4), a first rear lens group (5), a second rear lens group (7), a monochromatic detector array (6), and a focal plane polarization detector array (8), characterized in that, The light emitted by the target object (1) becomes a parallel beam after passing through the front lens group (2). The parallel beam is divided into a dispersive imaging path and a polarization imaging path after passing through the depolarization beam splitter (3). The dispersive imaging path passes through the dispersive element (4) and the first rear mirror group (5) in sequence, and forms a dispersive spectral image containing the spectral information of the target object (1) on the plane of the monochromatic detector array (6). The polarization imaging path passes through the second rear mirror group (7) and forms a polarization image containing the polarization information of the target object (1) on the plane of the focal plane polarization detector array (8).

2. The polarization spectral imaging device as described in claim 1, characterized in that, Each pixel of the focal plane polarization detector array (8) integrates four micro polarization filters, which are arranged at different angles of 0°, 45°, 90° and 135° respectively. The focal plane polarization detector array (8) can acquire polarization images containing polarization information of the target object (1).

3. A polarization spectral calculation and reconstruction system, based on the polarization spectral imaging device of claim 1, characterized in that, It includes a forward projection mathematical model and a neural network model. The forward projection mathematical model mathematically models the transmission process of the beam emitted by the target object (1) through the imaging device to form polarization spectral information, and then outputs dispersive spectral projection data and polarization projection data to the neural network model. The neural network model calculates and reconstructs the dispersive spectral projection data and polarization projection data to generate a polarization spectral image.

4. The polarization spectrum calculation and reconstruction system as described in claim 3, characterized in that, The forward projection mathematical model is the following least squares optimization problem: in, Y and Y represent the optimal solutions to the least squares optimization problem. 0 Y 45 Y 90 Y 135 These represent dispersive spectral projection data, 0° polarization projection data, 45° polarization projection data, 90° polarization projection data, and 135° polarization projection data, respectively. Represents the sum of the total intensity of polarized and unpolarized light. Represents linearly polarized light at 0° in the horizontal direction. and linearly polarized light at 90° vertical direction The difference in light intensity, Represents linearly polarized light at a 45° angle. and linearly polarized light in the 135° direction The difference in light intensity, Representing the dispersion process, w represents the degree of discretization of the spectral response sensitivity of the pixel of the monochromatic detector (6) or the focal plane polarization detector array (8). , These represent the transmittance and reflectance of the depolarized beam splitter (3), respectively.

5. The polarization spectrum calculation and reconstruction system as described in claim 3, characterized in that, The neural network model includes a spectral encoder, a polarization encoder, and a joint decoder. The spectral encoder and polarization encoder are used to extract different levels of encoded features from dispersive spectral projection data and polarization projection data, respectively. The encoded features interact with each other through a cross-activation layer connecting the spectral encoder and the polarization encoder. The joint decoder decodes the extracted encoded features layer by layer to reconstruct the polarization spectral image.

6. The polarization spectrum calculation and reconstruction system as described in claim 5, characterized in that, Both the spectral encoder and the polarization encoder include convolutional layers, lightweight residual convolutional modules, and alternating 2x downsampling layers, wherein the convolutional layers consist of conventional convolutions with a kernel size of 3×3.

7. The polarization spectrum calculation and reconstruction system as described in claim 5, characterized in that, The joint decoder comprises alternating 2x upsampling layers, lightweight residual convolutional modules, and convolutional layers.

8. The polarization spectrum calculation and reconstruction system as described in claim 6, characterized in that, The lightweight residual convolution module includes layer normalization, depthwise separable convolution, two 1×1 convolutions, and Gaussian error linear units.

9. The polarization spectrum calculation and reconstruction system as described in claim 5, characterized in that, The cross-activation layer is defined as follows: for the features extracted by the same level of lightweight residual convolution module in the spectral encoder and polarization encoder described above. and ,use right The gate function for cross-activation is shown below: in, Represents the gate function. Represents nonlinear activation functions such as ReLU, GELU, and Sigmoid. Indicates by Linear transformation layers composed of convolutions, This represents element-wise multiplication between high-dimensional features; use right The gate function for cross-activation is shown below: 。 10. A polarization spectrum calculation and reconstruction method, based on the polarization spectrum imaging device of claim 1 and the polarization spectrum calculation and reconstruction system of claim 3, specifically comprising the following steps: S1: The monochromatic detector array (6) and the focal plane polarization detector array (8) in the polarization spectral imaging device receive the dispersive imaging optical path and the polarization imaging optical path of the target object (1) respectively, and convert the dispersive imaging optical path and the polarization imaging optical path into dispersive light signals and polarization light signals respectively and output them to the polarization spectral calculation and reconstruction system. S2: The polarization spectral calculation and reconstruction system uses a forward projection mathematical model to convert the dispersive light signal in step S1 into dispersive spectral projection data, and converts the polarization light signal in step S1 into 0° polarization projection data, 45° polarization projection data, 90° polarization projection data, and 135° polarization projection data. S3: The deep neural network model in the polarization spectrum calculation and reconstruction system uses the dispersive spectral projection data and 0° polarization projection data, 45° polarization projection data, 90° polarization projection data and 135° polarization projection data described in step S2 to complete the calculation and reconstruction to obtain the polarization spectrum image of the target object (1).

Images