Polarization spectrum modulation type multispectral stereo imaging method

Abstract

The invention discloses a polarization spectrum modulation type multispectral stereo imaging method which solves the problem that in the prior art, an existing polarization spectrum imaging scheme does not achieve three-dimensional reconstruction of a target and achieves the purpose that the three-dimensional reconstruction of the target is not achieved under the condition that active illumination or special marks are not needed. Synchronous high-precision reconstruction of physical attributes and three-dimensional geometry of the surface of the target is realized; the method comprises the following steps: uniformly modulating polarization information and spectral information of incident light of a target through a three-way optical filter and a polaroid to obtain modulated incident light information; performing aliasing acquisition of a plurality of polarization spectrum components on the modulated incident light information through a multi-color detector combination to obtain aliasing light intensity; performing linear solution on the aliasing light intensity to obtain each polarization spectrum component vector, calculating the polarization degree and the polarization angle of each channel, and further deriving a surface zenith angle and an azimuth angle; reconstructing the three-dimensional morphology of each spectrum channel through normal gradient field frequency domain integration; and finally, fusing the multi-spectral three-dimensional data to obtain a multi-spectral three-dimensional imaging result.

Description

Technical Field

[0001] This invention relates to the field of polarization imaging technology, and in particular to a polarization spectral modulation type multi-band stereo imaging method. Background Technology

[0002] In 1979, Japanese scientist Koshikawa pioneered the theory of three-dimensional reconstruction of target surface morphology based on polarization information. In 1988, American scholar Wolff systematically analyzed Fresnel reflection theory, revealing the relationship between the polarization characteristics of light waves on the object surface and the macroscopic geometric morphology. By introducing azimuth angle and degree of polarization, he achieved three-dimensional reconstruction of specular reflection targets.

[0003] A team from the Institute of Space Optics, School of Physics, Xi'an Jiaotong University, proposed a method for modulating polarization spectral intensity using only a single fixed multi-stage delayer, and a DIP-SP reconstruction method that combines a sparse prior-based physical model with a deep image prior based on an untrained neural network. This method compresses and encodes the Stokes parameters using a modulation module, then uses a reconstruction algorithm to demodulate the spectral image cube containing the Stokes parameters. It offers advantages in system simplification and accurate reconstruction of polarization spectral information, and also exhibits strong noise resistance. However, this scheme relies on a series of complex algorithms, which cannot guarantee its real-time performance and versatility.

[0004] Existing technology uses a quarter-wave plate to convert the linearly polarized light of the target beam into circularly polarized light. After processing by a multi-stage phase delayer, a polarization beam splitter generates two beams with a certain discrete angle. The beams contain polarization information. The grating of the spectral imaging module performs dispersion processing on the two beams entering the slit, and forms corresponding target images in two regions of the area array detector assembly. The pixels of the target images received by the area array detector assembly that have the same incident angle and the same spectral channel in the two regions are calculated together to obtain the spectral, polarization, and spatial information of the target.

[0005] Existing technologies use polarization modulators and snapshot imaging spectrometers to spectrally modulate the full Stokes parameters. By constructing a complete imaging physics model or an imaging physics model based on compressed sensing, and then using untrained artificial neural networks, the full Stokes parameter images under each narrow band can be reconstructed.

[0006] Existing technology uses two polarization cameras to acquire polarization channel information and polarization spectral channel information. The polarization spectral channel uses a blazed grating to generate dispersion and produce spectral separation. After being modulated by a reflective micromirror array, the dispersed light carrying spatial and spectral compression information is reflected and imaged on a focal plane polarization camera in the polarization spectral channel. Depending on the different DMD loading encoding modes during acquisition, a restoration algorithm based on Hadamard transform, compressed sensing theory, and deep learning theory is used to reconstruct the polarization spectral image. Finally, the polarization spectral imaging subsystem obtains image data of the target in two-dimensional space, one-dimensional spectrum, and one-dimensional polarization.

[0007] Traditional polarization 3D reconstruction only reconstructs the 3D shape of the target without considering its material composition. Spectral imaging can use data cubes of the target in different spectral bands to identify the material composition of the target, but it lacks depth clues and cannot simultaneously acquire material composition and 3D shape.

[0008] Existing polarization spectral imaging systems mostly employ spectral splitting modules and polarization modulators / filters. These modules utilize dispersive or filtering devices, and the use of grating-based splitting and filtering equipment complicates the optical structure. The addition of spectrometers and other equipment increases system costs and makes maintenance difficult. Furthermore, as the number of spectral channels increases, the energy acquired through the splitting module decreases, leading to a decline in the accuracy of the acquired polarization spectral information. Moreover, most polarization spectral devices only acquire two-dimensional image information of the target and do not achieve three-dimensional reconstruction. Summary of the Invention

[0009] This invention provides a polarization spectral modulation type multi-band stereo imaging method, which solves the problem that existing polarization spectral imaging schemes in the prior art cannot achieve three-dimensional reconstruction of the target. It realizes the synchronous high-precision reconstruction of the physical properties and three-dimensional geometry of the target surface without the need for active illumination or special marking.

[0010] This invention provides a polarization spectral modulation type multi-band stereo imaging method, the method comprising: The incident light to the target is uniformly modulated using a three-way filter and a polarizer to obtain modulated incident light information; the modulated incident light information is then mixed with multiple polarization spectral components using a combination of multi-color detectors to obtain mixed light intensity; and the mixed light intensity is linearly solved to obtain vectors of each polarization spectral component. The polarization spectral component vectors corresponding to different spectral channels in each imaging sub-path are extracted to obtain the polarization spectral data matrix in each imaging sub-path; Using the polarization spectral data matrix and Stokes vector formula in each imaging sub-path, the Stokes vector corresponding to different spectral channels in each imaging sub-path is calculated; The degree of polarization and the polarization angle are calculated based on the Stokes vector to obtain the degree of polarization and the polarization angle corresponding to different spectral channels in each imaging sub-optical path. Based on the degree of polarization and the polarization angle, the zenith angle and azimuth angle corresponding to different spectral channels in each imaging sub-path are calculated using Fresnel's law of reflection and motion recovery structure, respectively. The surface normal gradient field is calculated based on the zenith angle and azimuth angle corresponding to different spectral channels in each imaging sub-path, and the three-dimensional surface morphology corresponding to the center wavelength of each channel is reconstructed by frequency domain integration. By fusing the three-dimensional surface topography of all spectral channels, multi-band three-dimensional reconstruction is achieved through spatial alignment and spectral weighting to obtain the final imaging result.

[0011] In one possible implementation, the uniform modulation of polarization and spectral information of the incident light to the target using a three-way filter and a polarizer yields modulated incident light information; the acquisition of mixed light intensity by aliasing multiple polarization spectral components of the modulated incident light information using a multi-color detector combination includes: The target light is spectrally selectively filtered using a three-channel filter to obtain light of a selected wavelength; The polarization state of the selected wavelength light is selected using a polarizer to obtain selectively polarized light. The selected polarized light is focused using a lens to obtain an initial image; The initial image is then processed by a color detector to obtain the aliased light intensity after all spectral polarization modulations are received by the color detector.

[0012] In one possible implementation, the color detector receives the aliased light intensity after all spectral polarization modulations, expressed as: ; Among them, among them, Indicates the first The received light intensity of each imaging sub-optical path spectral channel; Indicates the color detector at the Each imaging sub-optical path Spectral transmittance of the channel; This indicates the center wavelength of the first channel of the three-way filter in the first sub-optical path; This indicates that the center wavelength of the filter is Spectral transmittance when the color detector is an R-channel; Indicates the first The angle of the polarizer in the individual optical path; This indicates that the wavelength emitted by the target is The light component passes through the polarizer at an angle of 0°. The intensity of light after; This represents the center wavelength of the second channel of the three-way filter in the first sub-optical path and the center wavelength of the first channel of the three-way filter in the second sub-optical path; This indicates that the center wavelength of the filter is Spectral transmittance when the color detector is an R-channel; This indicates that the wavelength emitted by the target is The light component passes through the polarizer at an angle of 0°. The intensity of light after; Indicates the first The center wavelength of the third channel of the three-way filter in the individual optical path; This indicates that the center wavelength of the filter is... Spectral transmittance when the color detector is an R-channel; This indicates that the wavelength emitted by the target is The light component passes through the polarizer at an angle of 0°. The intensity of light after; This indicates that the center wavelength of the filter is Spectral transmittance when the color detector is a G-channel; Indicates the first The center wavelength of the second channel of the three-way filter in the individual optical path; This indicates that the center wavelength of the filter is Spectral transmittance when the color detector is a G-channel; This indicates that the wavelength emitted by the target is The light component passes through the polarizer at an angle of 0°. The intensity of light after; This indicates that the center wavelength of the filter is Spectral transmittance when the color detector is a G-channel; This indicates that the center wavelength of the filter is Spectral transmittance when the color detector is in channel B; This indicates that the wavelength emitted by the target is The light component passes through the polarizer at an angle of 0°. The intensity of light after; Indicates the first The center wavelength of the first channel of the three-way filter in the individual optical path; This indicates that the center wavelength of the filter is Spectral transmittance when the color detector is a G-channel; This indicates that the center wavelength of the filter is Spectral transmittance when the color detector is a G-channel; Indicates the first Each imaging sub-optical path The received light intensity of the spectral channel; Indicates the first Each imaging sub-optical path The received light intensity of the spectral channel.

[0013] In one possible implementation, the linear calculation of the received light intensity to obtain the polarization spectral component vectors corresponding to different spectral channels of each imaging sub-path includes: The intensity of the aliased light after receiving all spectral polarization modulations in each imaging sub-path of the color detector is expressed as a system of linear equations. Determine the constraints on the coefficient matrix; wherein the constraint is that the coefficient matrix is ​​of full rank; Under the constraints, the coefficient matrix is ​​solved to obtain the polarization spectral component vectors corresponding to different spectral channels of each imaging sub-path.

[0014] In one possible implementation, the step of calculating the Stokes vector corresponding to different spectral channels in each imaging sub-path using the polarization spectral data matrix and the Stokes vector formula in each imaging sub-path includes: For the polarization spectral data matrix in each imaging sub-optical path, extract the polarization spectral component components corresponding to different spectral channels respectively; Using the Stokes vector formula, the Stokes vectors corresponding to different spectral channels in each imaging sub-path are calculated based on the polarization spectral component components corresponding to the different spectral channels.

[0015] In one possible implementation, the polarization spectral component vectors corresponding to different spectral channels of each imaging sub-optical path are represented as: ; in, This indicates that the wavelength emitted by the target is The light component passes through the polarizer at an angle of 0°. The intensity of light after; This indicates the center wavelength of the first channel of the three-way filter in the first sub-optical path; Indicates the first The angle of the polarizer in the individual optical path; This indicates that the wavelength emitted by the target is The light component passes through the polarizer at an angle of 0°. The intensity of light after; This represents the center wavelength of the second channel of the three-way filter in the first sub-optical path and the center wavelength of the first channel of the three-way filter in the second sub-optical path; This indicates that the wavelength emitted by the target is The light component passes through the polarizer at an angle of 0°. The intensity of light after; Indicates the first The center wavelength of the third channel of the three-way filter in the individual optical path; This indicates that the wavelength emitted by the target is The light component passes through the polarizer at an angle of 0°. The intensity of light after; express; This indicates that the wavelength emitted by the target is The light component passes through the polarizer at an angle of 0°. The intensity of light after; Indicates the first The center wavelength of the first channel of the three-way filter in the individual optical path.

[0016] In one possible implementation, the step of calculating the zenith angle and azimuth angle corresponding to different spectral channels in each imaging sub-path based on the degree of polarization and the polarization angle, using Fresnel's law of reflection and motion-restoring structure respectively, includes: Based on the polarization degree, the zenith angle corresponding to different spectral channels in each imaging sub-path is calculated; Based on the coarse depth information obtained from the motion recovery structure, the polarization angle is corrected according to the coarse depth information to obtain the azimuth angles corresponding to different spectral channels in each imaging sub-path.

[0017] In one possible implementation, the degree of polarization is expressed as: ; The polarization angle is expressed as: ; The zenith angle is represented as: ; The azimuth angle is expressed as:

[0018] in, Indicates the center wavelength of the stereoscopic image; Indicates the target pixel in The refractive index below; Indicates the target pixel in The zenith angle below; Indicates wavelength as The first component of the Stokes vector of the target pixel; Indicates wavelength as The second component of the Stokes vector of the target pixel; Indicates wavelength as The third component of the Stokes vector of the target pixel; Indicates wavelength as The polarization degree of the target pixel at that time; This represents the azimuth angle obtained by the SFM (Structure from Motion) method; This indicates that the true azimuth angle is obtained by correcting the polarization angle (AOP) using the SFM method.

[0019] In one possible implementation, the reconstructing of the three-dimensional surface topography corresponding to the center wavelength of each channel through frequency domain integration includes: Construct a cost function that aims to minimize the gradient difference; The cost function is solved in the frequency domain to obtain the reconstructed surface normal gradient field; Using an integral algorithm, the three-dimensional surface morphology corresponding to the center wavelength of each channel is calculated based on the reconstructed gradient field of the cotton normal.

[0020] In one possible implementation, the cost function is expressed as: ; in, The x-coordinate represents the surface normal gradient field; The ordinate represents the surface normal gradient field; The x-coordinate represents the measured normal gradient field; The ordinate represents the measured normal gradient field; Indicates the target is The target surface function.

[0021] One or more technical solutions provided in this invention have at least the following technical effects or advantages: This invention obtains the polarization spectral components of each channel through linear calculation, effectively separating the light intensity information of different polarization states, providing a high-fidelity data foundation for subsequent polarization analysis, and reducing cross-interference. It integrates the polarization vectors of multiple sub-paths and multispectral channels to form a matrix, achieving structured data storage and efficient management, facilitating large-scale matrix operations and improving processing efficiency. Based on a physical optics model, it accurately quantifies the polarization state of light, converting the original light intensity into standard polarization parameters, providing theoretical support for polarization characteristic analysis. It derives key polarization attributes from Stokes vectors, directly reflecting the polarization modulation characteristics of the target surface, enhancing the perception of physical properties such as material and roughness. It integrates Fresnel's law (and motion recovery structures) to achieve dual-angle calculation from polarization information to surface orientation, breaking through the texture dependence limitation of traditional 3D reconstruction. By integrating the normal gradient field in the frequency domain, it effectively suppresses noise accumulation errors, achieving high-precision, seamless 3D topography reconstruction, and supports independent processing of subspectral channels. Spatial alignment eliminates pose differences between channels, and spectral weighted fusion takes into account the advantages of reconstruction across different bands, ultimately outputting a 3D model with both high-resolution geometric details and multispectral characteristics, significantly improving the completeness of imaging information. Attached Figure Description

[0022] Figure 1 A flowchart of the steps of the polarization spectral modulation type multi-band stereo imaging method provided in the embodiments of the present invention; Figure 2 This is a schematic diagram of the optical path of each imaging sub-projection provided in the embodiments of the present invention; Figure 3 Wavelengths provided for embodiments of the present invention The target 3D reconstruction image below; Figure 4 This is a schematic diagram of the final imaging result provided in an embodiment of the present invention; Figure 5 Provided for embodiments of the present invention Schematic diagram of the imaging sub-optical path; Figure 6 The target three-dimensional reconstruction image provided for the embodiments of the present invention. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0024] This invention provides a polarization spectral modulation type multi-band stereo imaging method, see [link to relevant documentation]. Figure 1 The method includes the following steps S101 to S107.

[0025] S101, the polarization and spectral information of the incident light of the target are uniformly modulated by a three-way filter and a polarizer to obtain modulated incident light information; multiple polarization spectral components of the modulated incident light information are mixed and obtained by a combination of multi-color detectors to obtain mixed light intensity; the mixed light intensity is linearly solved to obtain the vector of each polarization spectral component. Specifically, in step S101, the polarization and spectral information of the incident light of the target are uniformly modulated by a three-way filter and a polarizer to obtain modulated incident light information; the modulated incident light information is obtained by aliasing multiple polarization spectral components by a combination of multi-color detectors to obtain aliased light intensity. This includes the following steps S1011 to S1014. The optical paths of each imaging sub-path are described below. Figure 2 .

[0026] S1011 uses a three-channel filter to perform spectrally selective filtering of the target light to obtain selective wavelength light; S1012, using a polarizer to select the polarization state of light of a selected wavelength to obtain selectively polarized light; S1013 uses the lens to focus selectively polarized light to obtain an initial image; S1014, based on the color detector, perform color imaging on the initial image to obtain the aliased light intensity after all spectral polarization modulations received by the color detector.

[0027] Here, the color detector receives the aliased light intensity after all spectral polarization modulations, expressed as: (1.1) in, Indicates the first Each imaging sub-optical path The received light intensity of the spectral channel; Indicates the color detector at the Each imaging sub-optical path Spectral transmittance of the channel; This indicates the center wavelength of the first channel of the three-way filter in the first sub-optical path; This indicates that the center wavelength of the filter is Spectral transmittance when the color detector is an R-channel; Indicates the first The angle of the polarizer in the individual optical path; This indicates that the wavelength emitted by the target is The light component passes through the polarizer at an angle of 0°. The intensity of light after; This represents the center wavelength of the second channel of the three-way filter in the first sub-optical path and the center wavelength of the first channel of the three-way filter in the second sub-optical path; This indicates that the center wavelength of the filter is Spectral transmittance when the color detector is an R-channel; This indicates that the wavelength emitted by the target is The light component passes through the polarizer at an angle of 0°. The intensity of light after; Indicates the first The center wavelength of the third channel of the three-way filter in the individual optical path; This indicates that the center wavelength of the filter is... Spectral transmittance when the color detector is an R-channel; This indicates that the wavelength emitted by the target is The light component passes through the polarizer at an angle of 0°. The intensity of light after; This indicates that the center wavelength of the filter is Spectral transmittance when the color detector is a G-channel; Indicates the first The center wavelength of the second channel of the three-way filter in the individual optical path; This indicates that the center wavelength of the filter is Spectral transmittance when the color detector is a G-channel; This indicates that the wavelength emitted by the target is The light component passes through the polarizer at an angle of 0°. The intensity of light after; This indicates that the center wavelength of the filter is Spectral transmittance when the color detector is a G-channel; This indicates that the center wavelength of the filter is Spectral transmittance when the color detector is in channel B; This indicates that the wavelength emitted by the target is The light component passes through the polarizer at an angle of 0°. The intensity of light after; Indicates the first The center wavelength of the first channel of the three-way filter in the individual optical path; This indicates that the center wavelength of the filter is Spectral transmittance when the color detector is a G-channel; This indicates that the center wavelength of the filter is Spectral transmittance when the color detector is a G-channel; Indicates the first Each imaging sub-optical path The received light intensity of the spectral channel; Indicates the first Each imaging sub-optical path The received light intensity of the spectral channel.

[0028] Specifically, in step S101, the received light intensity is linearly calculated to obtain the polarization spectral component vectors corresponding to different spectral channels of each imaging sub-path, including: (1) The intensity of the mixed light after all spectral polarization modulation received by the color detector in each imaging sub-optical path is expressed as a system of linear equations. (2) Determine the constraints of the coefficient matrix; where the constraint is: the coefficient matrix is ​​of full rank; (3) Solve the coefficient matrix under the constraints to obtain the polarization spectral component vectors corresponding to different spectral channels of each imaging sub-path.

[0029] For example, see Figure 1 The light emitted (or reflected) by the target enters There are several different imaging sub-optical paths. In the first... In the imaging sub-optical path, the target light sequentially passes through a three-channel filter, a polarizer, and a lens. The three-channel filter performs spectral selective filtering of the target light, allowing light of a specific center wavelength to pass through; the polarizer selects the polarization state of the passing target light, filtering out light with a specific polarization direction; subsequently, the lens focuses the filtered and polarized target light to form a clear image, which is ultimately imaged onto the color sensor. The target light passes through the three-channel filter and has a center wavelength of... The light reached the first Taiwan color detector The spectral transmittance of the channel receiving light can be characterized as... The intensity of light received This is represented by formula (1.1).

[0030] S102, extract the polarization spectral component vectors corresponding to different spectral channels in each imaging sub-optical path to obtain the polarization spectral data matrix in each imaging sub-optical path; Here, the polarization spectral component vectors corresponding to different spectral channels of each imaging sub-path are expressed as: (1.2) in, This indicates that the wavelength emitted by the target is The light component passes through the polarizer at an angle of 0°. The intensity of light after; This indicates the center wavelength of the first channel of the three-way filter in the first sub-optical path; Indicates the first The angle of the polarizer in the individual optical path; This indicates that the wavelength emitted by the target is The light component passes through the polarizer at an angle of 0°. The intensity of light after; This represents the center wavelength of the second channel of the three-way filter in the first sub-optical path and the center wavelength of the first channel of the three-way filter in the second sub-optical path; This indicates that the wavelength emitted by the target is The light component passes through the polarizer at an angle of 0°. The intensity of light after; Indicates the first The center wavelength of the third channel of the three-way filter in the individual optical path; This indicates that the wavelength emitted by the target is The light component passes through the polarizer at an angle of 0°. The intensity of light after; express; This indicates that the wavelength emitted by the target is The light component passes through the polarizer at an angle of 0°. The intensity of light after; Indicates the first The center wavelength of the first channel of the three-way filter in the individual optical path.

[0031] For example, according to formula (1.1), the first The color detector has the following system of linear equations: when hour: (1.3) when hour: (1.4) when hour: (1.5) To ensure that the system of equations (1.3) to (1.5) has a unique solution, the coefficient matrix must be of full rank to obtain the unique solution of the system, which is equation (1.2). Specifically, the coefficient matrix is ​​represented as follows: (1.6).

[0032] S103, using the polarization spectral data matrix and Stokes vector formula in each imaging sub-path, calculate the Stokes vector corresponding to different spectral channels in each imaging sub-path; Specifically, in step S103, the Stokes vector corresponding to different spectral channels in each imaging sub-path is calculated using the polarization spectral data matrix and Stokes vector formula in each imaging sub-path, including the following steps S1031 to S1032.

[0033] S1031, for the polarization spectral data matrix in each imaging sub-optical path, extract the polarization spectral component components corresponding to different spectral channels respectively; S1032, using the Stokes vector formula, calculates the Stokes vector corresponding to different spectral channels in each imaging sub-path based on the polarization spectral component components corresponding to different spectral channels.

[0034] For example, the center wavelength is The three polarization angles below are polarization spectral components Substituting into the formula below, we obtain the Stokes vector. .

[0035] (1.7) in, This indicates that the wavelength emitted by the target is The light component passes through the polarizer at an angle of 0°. The intensity of light after; This indicates that the wavelength emitted by the target is The light component passes through the polarizer at an angle of 0°. The intensity of light after; This indicates that the wavelength emitted by the target is The light component passes through the polarizer at an angle of 0°. The intensity of light afterward.

[0036] S104, calculate the degree of polarization and polarization angle based on the Stokes vector to obtain the degree of polarization and polarization angle corresponding to different spectral channels in each imaging sub-path; Here, the degree of polarization is expressed as: (1.8) The polarization angle is expressed as: (1.9) in, Indicates wavelength as The first component of the Stokes vector of the target pixel; Indicates wavelength as The second component of the Stokes vector of the target pixel; Indicates wavelength as The third component of the Stokes vector of the target pixel; Indicates degree of polarization; Indicates the center wavelength of the stereoscopic image; Indicates the target pixel in The refractive index below.

[0037] For example, Stokes vector Substituting these formulas into the following equations yields the polarization degree formula (1.8) and the polarization angle formula (1.9). Combining Fresnel's law, the reflectivity of the reflected light in each vibration direction is related to the incident angle, and can be determined based on the polarization degree. Find the zenith angle. According to formula (1.10), the coarse depth of each pixel of the target is obtained using the SFM method, and the polarization angle of each pixel is fused with the coarse depth. Correction is performed to obtain the target's true azimuth angle. .

[0038] S105, based on the degree of polarization and the polarization angle, the zenith angle and azimuth angle corresponding to different spectral channels in each imaging sub-optical path are calculated using Fresnel's law of reflection and motion recovery structure, respectively. Specifically, in step S105, based on the degree of polarization and the polarization angle, the zenith angle and azimuth angle corresponding to different spectral channels in each imaging sub-path are calculated using Fresnel's law of reflection and the motion recovery structure, respectively, including the following steps S1051 to S1052.

[0039] S1051, based on the degree of polarization, calculate the zenith angle corresponding to different spectral channels in each imaging sub-optical path; S1052, based on the coarse depth information obtained from the motion recovery structure, the polarization angle is corrected according to the coarse depth information to obtain the azimuth angle corresponding to different spectral channels in each imaging sub-optical path.

[0040] For example, zenith angle Represented as: (1.10) Azimuth is expressed as: (1.11) in, Indicates the center wavelength of the stereoscopic image; Indicates the target pixel in The refractive index below; Indicates the target pixel in The zenith angle below; Indicates wavelength as The first component of the Stokes vector of the target pixel; Indicates wavelength as The second component of the Stokes vector of the target pixel; Indicates wavelength as The third component of the Stokes vector of the target pixel; Indicates wavelength as The polarization degree of the target pixel at that time; This represents the azimuth angle obtained by the SFM (Structure from Motion) method; This indicates that the true azimuth angle is obtained by correcting the polarization angle (AOP) using the SFM method.

[0041] S106. The surface normal gradient field is calculated based on the zenith angle and azimuth angle corresponding to different spectral channels in each imaging sub-optical path, and the three-dimensional surface morphology corresponding to the center wavelength of each channel is reconstructed by frequency domain integration. Specifically, in step S106, the three-dimensional surface morphology corresponding to the center wavelength of each channel is reconstructed by frequency domain integration, including the following steps S1061 to S1063.

[0042] S1061, construct a cost function with the objective of minimizing gradient difference; Here, the cost function is expressed as: (1.12) in, The x-coordinate represents the surface normal gradient field; The ordinate represents the surface normal gradient field; The x-coordinate represents the measured normal gradient field; The ordinate represents the measured normal gradient field; Indicates the target is The target surface function.

[0043] S1062, solve the cost function in the frequency domain to obtain the reconstructed surface normal gradient field; S1063 uses an integral algorithm to calculate the three-dimensional surface morphology corresponding to the center wavelength of each channel based on the reconstructed gradient field of the bicotyl cotton normal.

[0044] For example, based on the obtained zenith angle and azimuth The target's normal and normal gradient can be obtained. Using the minimum target surface proposed by Frankot Chellappa gradient field and measuring gradient field The target surface function is solved by the difference method, and the cost function is shown in formula (1.12).

[0045] By using a discrete Fourier basis function system to expand the surface function in the Frankot-Chellappa algorithm, the surface function of the object can be obtained. Regarding the gradient field The expression: (1.13) in, Represents the Discrete Fourier Transform; The number of columns representing the two-dimensional image of the target; The number of rows in the two-dimensional image representing the target; Indicates the inverse Fourier transform; Indicates the center wavelength of the stereoscopic image; Indicates the target pixel in The refractive index below; Represents the Fourier coefficients of the discrete microoperator in the x-direction; This represents the Fourier coefficients of the discrete microoperator in the y-direction.

[0046] The wavelength can be obtained by using an integration algorithm. The three-dimensional surface morphology of the target, such as Figure 3 As shown.

[0047] S107, a 3D surface topography fusion of all spectral channels, achieves multi-spectral 3D reconstruction through spatial alignment and spectral weighting. See [link / reference]. Figure 4 This yields the final imaging results.

[0048] In a specific embodiment provided by the present invention, see [link to specific embodiment]. Figure 5 Number of color detectors This system was used to interpret the full polarization information across four spectral channels and reconstruct the three-dimensional information of the target. The angles of the polarizers were respectively... The center wavelengths of the four spectral channels are respectively The center wavelength selectively transmitted by the three-channel filter 1 is... The center wavelength selectively transmitted by the three-channel filter 2 is... The center wavelength of selective transmission of the three-channel filter 3 is... The center wavelength of selective transmission of the three-channel filter 4 is... .

[0049] For color detector 1, the received light intensity ( The characteristics are as follows: (1.14) Step 2: According to the above equation, the first detector has the following system of linear equations: (1.15) Substituting the specific values, we obtain the coefficient matrix of the above linear equation system. The coefficient matrix is ​​full rank, meaning there exists a unique solution. , , .

[0050] Step 3: For color detectors 2, 3, and 4, repeat steps 1 and 2. The final 12 polarization spectral components of the target can be solved as follows: (1.16) Step 4: The polarization spectral components below Substitute into the following formula (1.8): (1.17) It can be obtained linearly. Stokes vector below .

[0051] Step 5: Substituting Stokes equations and the angle of polarization and polarization degree The relational formulas (1.8) and (1.9) can be used to obtain... , .

[0052] Step 6: For wavelength According to the degree of polarization With zenith angle and wavelength Lower refractive index The relational formula (1.10) is used to solve for the zenith angle. .

[0053] Step 7: Use the SFM method to obtain the coarse depth of each pixel of the target, and fuse the coarse depth pairs. Correction is performed to obtain the true azimuth angle. .

[0054] Step 8: Based on the azimuth angle and zenith The normals of the target micro-surface elements are obtained, and the Frankot-Chellappa surface integral algorithm is used to apply the surface function to the target surface. Reconstruction will be carried out.

[0055] Step 9: For other wavelengths , , The reconstruction method can be obtained by repeating steps 4, 5, 6, 7, and 8. , , Achieve 3D reconstruction of targets in different spectral bands; reconstruction results are as follows. Figure 6 As shown.

[0056] This invention splits the light emitted by the target object into n independent optical paths through a beam splitting system. The spectral-polarization modulation system is composed of multiple three-way filters and polarizers. Different filters allow light of specific wavelengths to pass through and cooperate with polarizers at different angles, so that the spectral and polarization characteristics of each light path are precisely modulated.

[0057] This invention is highly innovative in the design and application of three-way filters and polarizers. First, the combination of the three-way filter and polarizer is not a simple stacking, but an optimized design based on a deep understanding of spectral and polarization characteristics. The accompanying polarizer modulates the polarization of light at a specific transmission angle, ensuring that the transmitted light not only has a specific spectral channel but also a specific polarization direction, ultimately achieving the simultaneous acquisition of multiple polarization spectral components of the target.

[0058] This invention designs a polarization spectral modulation scheme with a simple structure and high light energy utilization. It obtains accurate polarization spectral information through linear calculation, and then uses the obtained polarization spectral information to solve for the degree of polarization and the polarization angle. Based on the degree of polarization and the polarization angle, it calculates the azimuth angle and zenith angle of the micro-surface element normal. Finally, it uses a surface integration algorithm to reconstruct the surface morphology of different targets.

[0059] The key lies in the designed polarization spectral imaging system and the method for linearly solving accurate polarization multispectral information. This ensures that the polarization multispectral information is linearly and accurately solved from the original signal, thereby realizing the three-dimensional reconstruction of the target under multiple spectral bands.

[0060] The various embodiments described in this specification are presented in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. All or part of this invention can be used in numerous general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, mobile communication terminals, multiprocessor systems, microprocessor-based systems, programmable electronic devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices, etc.

[0061] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the present invention.

Claims

1. A polarization spectrum modulation type multi-spectral stereoscopic imaging method, characterized by, include: The polarization and spectral information of the incident light to the target are uniformly modulated by a three-way filter and a polarizer to obtain modulated incident light information. The modulated incident light information is obtained by aliasing multiple polarization spectral components through a combination of multi-color detectors to obtain the aliased light intensity; the aliased light intensity is then linearly solved to obtain the vector of each polarization spectral component. The polarization spectral component vectors corresponding to different spectral channels in each imaging sub-path are extracted to obtain the polarization spectral data matrix in each imaging sub-path; Using the polarization spectral data matrix and Stokes vector formula in each imaging sub-path, the Stokes vector corresponding to different spectral channels in each imaging sub-path is calculated; The degree of polarization and the polarization angle are calculated based on the Stokes vector to obtain the degree of polarization and the polarization angle corresponding to different spectral channels in each imaging sub-optical path. Based on the degree of polarization and the polarization angle, the zenith angle and azimuth angle corresponding to different spectral channels in each imaging sub-path are calculated using Fresnel's law of reflection and motion recovery structure, respectively. The surface normal gradient field is calculated based on the zenith angle and azimuth angle corresponding to different spectral channels in each imaging sub-path, and the three-dimensional surface morphology corresponding to the center wavelength of each channel is reconstructed by frequency domain integration. By fusing the three-dimensional surface topography of all spectral channels, multi-band three-dimensional reconstruction is achieved through spatial alignment and spectral weighting to obtain the final imaging result.

2. The polarization spectrum modulation type multi-spectral stereoscopic imaging method according to claim 1, characterized in that, The polarization and spectral information of the incident light to the target are uniformly modulated using a three-way filter and a polarizer to obtain modulated incident light information. The modulated incident light information is obtained by aliasing multiple polarization spectral components using a combination of multi-color detectors to obtain the aliased light intensity, including: The target light is spectrally selectively filtered using a three-channel filter to obtain light of a selected wavelength; The polarization state of the selected wavelength light is selected using a polarizer to obtain selectively polarized light. The selected polarized light is focused using a lens to obtain an initial image; The initial image is then processed by a color detector to obtain the aliased light intensity after all spectral polarization modulations are received by the color detector.

3. The polarization-spectrally modulated multi-spectral stereoscopic imaging method according to claim 1, wherein, The color detector receives the aliased light intensity after all spectral polarization modulations, expressed as: ； wherein, represents the first imaging sub-path of the received light intensity of the spectral channel; represents the spectral transmittance of the color detector in the first imaging sub-path of the channel; represents the spectral transmittance when the filter center wavelength is and the color detector is the R channel; represents the angle of the polarizer in the first sub-path; represents the light intensity after the light component with a wavelength of emitted by the target passes through the polarizer with an angle of ; represents the center wavelength of the second channel of the tri-band filter in the first sub-path and the center wavelength of the first channel of the tri-band filter in the second sub-path; represents the spectral transmittance when the filter center wavelength is and the color detector is the R channel; represents the light intensity after the light component with a wavelength of emitted by the target passes through the polarizer with an angle of ; represents the center wavelength of the third channel of the tri-band filter in the first sub-path; represents the spectral transmittance when the filter center wavelength is and the color detector is the R channel; represents the light intensity after the light component with a wavelength of emitted by the target passes through the polarizer with an angle of ; represents the spectral transmittance when the filter center wavelength is and the color detector is the G channel; represents the center wavelength of the second channel of the tri-band filter in the first sub-path; represents the center wavelength of the second channel of the tri-band filter in the first sub-path; represents the spectral transmittance when the filter center wavelength is and the color detector is the G channel; represents the light intensity after the light component with a wavelength of emitted by the target passes through the polarizer with an angle of ; represents the spectral transmittance when the filter center wavelength is and the color detector is the G channel; represents the spectral transmittance when the filter center wavelength is Spectral transmittance when the color detector is in channel B; This indicates that the wavelength emitted by the target is The light component passes through the polarizer at an angle of 0°. The intensity of light after; Indicates the first The center wavelength of the first channel of the three-way filter in the individual optical path; This indicates that the center wavelength of the filter is Spectral transmittance when the color detector is a G-channel; This indicates that the center wavelength of the filter is Spectral transmittance when the color detector is a G-channel; Indicates the first Each imaging sub-optical path The received light intensity of the spectral channel; Indicates the first Each imaging sub-optical path The received light intensity of the spectral channel.

4. The polarization-spectrally modulated multi-spectral stereoscopic imaging method of claim 1, wherein, The linear calculation of the received light intensity to obtain the polarization spectral component vectors corresponding to different spectral channels of each imaging sub-path includes: The intensity of the aliased light after receiving all spectral polarization modulations in each imaging sub-path of the color detector is expressed as a system of linear equations. Determine the constraints on the coefficient matrix; wherein the constraint is that the coefficient matrix is ​​of full rank; Under the constraints, the coefficient matrix is ​​solved to obtain the polarization spectral component vectors corresponding to different spectral channels of each imaging sub-path.

5. The polarization-spectrally modulated multi-spectral stereoscopic imaging method of claim 1, wherein, The calculation of the Stokes vector corresponding to different spectral channels in each imaging sub-path using the polarization spectral data matrix and the Stokes vector formula includes: For the polarization spectral data matrix in each imaging sub-optical path, extract the polarization spectral component components corresponding to different spectral channels respectively; The Stokes vector formula is used to calculate the Stokes vector corresponding to different spectral channels in each imaging sub-optical path according to the polarization spectral component corresponding to the different spectral channels.

6. The polarization-spectrally modulated multi-spectral stereoscopic imaging method according to claim 5, wherein, The polarization spectral component vector corresponding to different spectral channels in each imaging sub-optical path is represented as: ； in, This indicates that the wavelength emitted by the target is The light component passes through the polarizer at an angle of 0°. The intensity of light after; This indicates the center wavelength of the first channel of the three-way filter in the first sub-optical path; Indicates the first The angle of the polarizer in the individual optical path; This indicates that the wavelength emitted by the target is The light component passes through the polarizer at an angle of 0°. The intensity of light after; This represents the center wavelength of the second channel of the three-way filter in the first sub-optical path and the center wavelength of the first channel of the three-way filter in the second sub-optical path; This indicates that the wavelength emitted by the target is The light component passes through the polarizer at an angle of 0°. The intensity of light after; Indicates the first The center wavelength of the third channel of the three-way filter in the individual optical path; This indicates that the wavelength emitted by the target is The light component passes through the polarizer at an angle of 0°. The intensity of light after; express; This indicates that the wavelength emitted by the target is The light component passes through the polarizer at an angle of 0°. The intensity of light after; Indicates the first The center wavelength of the first channel of the three-way filter in the individual optical path.

7. The polarization-spectrally modulated multi-spectral stereoscopic imaging method of claim 1, wherein, The zenith angle and the azimuth angle corresponding to different spectral channels in each imaging sub-optical path are calculated according to the degree of polarization and the polarization angle by using the Fresnel reflection law and the motion recovery structure, respectively, and the calculation includes: The zenith angle corresponding to different spectral channels in each imaging sub-optical path is calculated according to the degree of polarization. The azimuth angle corresponding to different spectral channels in each imaging sub-optical path is obtained by performing angle correction on the polarization angle according to the coarse depth information obtained by the motion recovery structure.

8. The polarization-spectrally modulated multi-spectral stereoscopic imaging method of claim 1, wherein, The degree of polarization is represented as: ； The polarization angle is represented as: ； The zenith angle is represented as: ； The azimuth angle is represented as: ； wherein, represents the center wavelength of the stereoscopic imaging; represents the refractive index of the target pixel at ; represents the zenith angle of the target pixel at ; represents the first component of the Stokes vector of the target pixel at a wavelength of ; represents the second component of the Stokes vector of the target pixel at a wavelength of ; represents the third component of the Stokes vector of the target pixel at a wavelength of ; represents the degree of polarization of the target pixel at a wavelength of ; represents the azimuth angle obtained by the SFM method; represents that the real azimuth angle is obtained by using the azimuth angle obtained by the SFM method to correct the polarization angle AOP.

9. The polarization-spectrally modulated multi-spectral stereoscopic imaging method of claim 1, wherein, The three-dimensional surface topography corresponding to each channel center wavelength is reconstructed by frequency domain integration, and the reconstruction includes: A cost function is constructed with the minimum gradient difference as the target; The cost function is solved in the frequency domain to obtain the reconstructed surface normal gradient field; The three-dimensional surface topography corresponding to each channel center wavelength is calculated by using the integral algorithm.

10. The polarization-spectrally modulated multi-spectral stereoscopic imaging method of claim 9, wherein, The cost function is represented as: ； wherein denotes the x-coordinate of the surface normal gradient field; denotes the y-coordinate of the surface normal gradient field; denotes the x-coordinate of the measured normal gradient field; denotes the y-coordinate of the measured normal gradient field; denotes the target surface function of the target under .

Images