Spectral polarization imaging three-dimensional reconstruction method and device based on pixel array and medium

Abstract

The invention discloses a spectral polarization imaging three-dimensional reconstruction method and device based on a pixel array, and a medium, and relates to the technical field of image processing, and the method mainly comprises the steps: collecting original images, which are fixedly obtained by pixels, of corresponding spectral bands and polarization angles; extracting sub-image information under each spectral band and polarization angle according to pixel positions, and stacking to generate a four-dimensional data volume; constructing a polarization geometric model according to the four-dimensional data body, and constructing an actual expression of the linear polarization degree in combination with a Fresnel reflection theory; constructing a redundant dispersion constraint equation and a theoretical expression of the relation between the diffuse reflection spectrum polarization degree and the incident angle and the imaging target refractive index; substituting the redundant dispersion constraint equation into a combined residual function of the theoretical expression and the actual expression, and performing parameter inversion on the combined residual function; and carrying out three-dimensional shape recovery under imaging target surface normal reconstruction based on an inversion result. According to the method, time delay and space registration errors caused by time-sharing acquisition are eliminated.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly relates to a three-dimensional reconstruction method, device and medium for spectral polarization imaging based on a pixel array. Background Art

[0002] Current target three-dimensional reconstruction technologies mainly rely on single-dimensional information acquisition means, such as spectral imaging, polarization imaging or traditional three-dimensional imaging methods. However, the application of these technologies in complex scenarios has significant limitations. Spectral imaging can effectively identify the chemical composition of materials by analyzing the reflection or transmission characteristics of objects in different bands, but it is insufficient in capturing surface structure details and spatial topography; polarization imaging utilizes the polarization modulation characteristics of light waves to analyze surface roughness, microstructure or stress distribution. However, it has a high dependence on the refractive index of the material and cannot directly provide the depth information of the target; three-dimensional imaging technologies (such as binocular vision, structured light) can reconstruct the macroscopic contour of objects, but in scenes with textureless, highly reflective or transparent surfaces, the reconstruction accuracy may decrease or even fail due to missing feature points or reflection interference. In addition, traditional multimodal imaging systems usually adopt a multi-device parallel architecture with time-sharing or beam splitting, and fuse data in different dimensions through post-processing. This design not only introduces registration errors due to time delay and spatial misalignment, reducing the real-time performance and accuracy in dynamic scenarios, but also makes the system bulky and the optical path debugging complex due to the integration of multiple sets of optical elements (such as filter wheels, polarization rotators), making it difficult to meet the requirements of portable or high-efficiency deployment in industrial production lines. More critically, there are deficiencies in the multi-source information collaborative modeling of existing technologies: for example, polarization three-dimensional reconstruction relies on known material refractive index parameters. In practical applications, complex targets are often composed of multiple materials, and the reconstruction results are prone to ambiguity when the refractive index is unknown or variable; the physical correlation between multispectral data and polarization information has not been fully exploited, resulting in the inversion model being sensitive to noise and having insufficient robustness. These problems together limit the detection ability of existing technologies in complex materials, multi-scale structures and dynamic environments, and it is difficult to achieve high-precision and high-stability three-dimensional topography reconstruction. Summary of the Invention

[0003] Aiming at the problem of difficult information superposition in existing target three-dimensional reconstruction, the present invention proposes a three-dimensional reconstruction method for spectral polarization imaging based on a pixel array. With each pixel of the image sensor as an array, interference filter films and polarization filter films of each band are periodically arranged in an array on the surface of the image sensor. The method specifically includes the following steps: S1: Through a single exposure of the image sensor, an original image of the imaging target in multiple spectral bands is acquired. Each pixel in the image sensor fixedly obtains the original image corresponding to its respective spectral band and polarization angle. S2: Decode the original image, extract the sub-image information under each spectral band and polarization angle according to the pixel position, and stack them to generate a four-dimensional data volume containing intensity information, spatial coordinates, spectral bands, and polarization angles. S3: Construct a polarization geometry model based on the four-dimensional data volume, extract the degree of linear polarization, polarization angle, and azimuth angle under each spectral band, and construct the actual expression of the degree of linear polarization in combination with Fresnel reflection theory. S4: Construct a redundant dispersion constraint equation that expresses the refractive index as a rational function of wavelength, and a theoretical expression for the relationship between the diffuse reflection spectral degree of polarization, the incident angle, and the refractive index of the imaging target. S5: Substitute the redundant dispersion constraint equation into the joint residual function of the theoretical expression and the actual expression, and perform parameter inversion with nonlinear least squares optimization of the joint residual function through the LM optimization algorithm. S6: Perform three-dimensional topography restoration under the reconstruction of the surface normal of the imaging target based on the parameters obtained by inversion.

[0004] In the present invention, the intensity information of multiple spectral bands and polarization angles is synchronously acquired through single exposure, which not only eliminates the time delay and spatial registration error caused by time-sharing acquisition in traditional multi-modal systems, but also greatly improves the imaging efficiency and data consistency in dynamic scenes. And the multi-source information fusion based on the four-dimensional data volume, constructing a complete physical characterization framework including material properties, surface microstructure, and macroscopic topography, breaks through the perception bottleneck of single-dimensional imaging under complex materials and multi-scale structures.

[0005] Further, the polarization filter film includes four polarization angles of 0°, 45°, 90°, and 135°, and the interference filter film covers at least two spectral bands.

[0006] Further, the polarization geometry model extracts the degree of linear polarization, polarization angle, and azimuth angle based on the Stokes vector, and the expression is: In the formula, is the constant label of the spectral band, is the wavelength of the th spectral segment, is the total light intensity of the spectral wavelength is the light intensity difference between the horizontal polarization component and the vertical polarization component of the spectral wavelength , is the light intensity difference between the 45° polarization component and the 135° polarization component of the spectral wavelength , is the degree of linear polarization of the spectral wavelength , is the spectral wavelength The polarization angle, is the azimuth angle.

[0007] Furthermore, in the step S3, the actual expression of the degree of linear polarization is: In the formula, is the constant label of the spectral band, is the wavelength of the th segment of the spectrum, is the actual degree of linear polarization of the spectral wavelength is the reflectivity of the s-polarized light of the spectral wavelength ; is the reflectivity of the p-polarized light of the spectral wavelength .

[0008] Furthermore, in the step S4, the redundant dispersion constraint equation is constructed based on the Cauchy dispersion equation, and the expression is: In the formula, is the refractive index of the spectral wavelength , and A, B, and C are the coefficients of the dispersion equation to be determined.

[0009] Furthermore, in the step S4, the theoretical expression of the relationship between the diffuse reflection spectral polarization degree, the incident angle, and the refractive index of the imaging target is: In the formula, is the theoretical degree of linear polarization of the spectral wavelength , is the incident angle of the th spectral band to be determined.

[0010] Furthermore, in the step S5, the expression of the joint residual function is: In the formula, is the joint residual of the theoretical expression and the actual expression.

[0011] Furthermore, in the step S6, the three-dimensional topography restoration is specifically: reconstructing the surface normal of the imaging target based on the parameters obtained by inversion, constructing a minimization target by solving the two-dimensional Poisson equation, performing gradient integration on the normal gradient field, and obtaining the surface function for three-dimensional topography restoration. The expression is: In the formula, is the normal vector at the spatial coordinate , They are the components of the normal vector in the X, Y, and Z axis directions, respectively. is the surface function, is the azimuth angle, is the component of the normal gradient field in the X axis direction, is the component of the normal gradient field in the Y axis direction, is the Fourier transform, is the inverse Fourier transform, is the imaginary unit, is the frequency component in the X axis direction in the spatial domain, is the frequency component in the Y axis direction in the spatial domain.

[0012] The present invention also includes a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the steps of the above-mentioned three-dimensional reconstruction method for spectral polarization imaging based on a pixel array are implemented.

[0013] The present invention also includes a device for processing data, including: a memory, on which a computer program is stored; a processor, configured to execute the computer program in the memory to implement the steps of the above-mentioned three-dimensional reconstruction method for spectral polarization imaging based on a pixel array.

[0014] Compared with the prior art, the present invention has at least the following beneficial effects: (1) The three-dimensional reconstruction method for spectral polarization imaging based on a pixel array proposed by the present invention synchronously obtains the light intensity information of multiple spectral bands and polarization angles through a single exposure, which not only eliminates the time delay and spatial registration error caused by time-sharing acquisition in traditional multi-modal systems, but also greatly improves the imaging efficiency and data consistency in dynamic scenes; (2) Based on the multi-source information fusion of the four-dimensional data volume, a complete physical characterization framework including material properties, surface microstructures, and macroscopic morphologies is constructed, breaking through the perception bottleneck of single-dimensional imaging under complex materials and multi-scale structures; (3) By introducing the joint constraint of redundant band observation data and the Cauchy dispersion equation for inversion, it effectively overcomes the dependence of traditional polarization three-dimensional reconstruction on the prior knowledge of material refractive index, significantly reduces the reconstruction ambiguity of multi-material mixed targets, and at the same time suppresses noise interference through multi-band redundant information, improving the stability in low-light and high-noise environments; (4) Through the gradient integration method of solving the two-dimensional Poisson equation in the frequency domain, the surface normal distribution is efficiently converted into a three-dimensional height map, taking into account the accurate restoration of both detail features and overall morphology, and avoiding the cumulative error problem in traditional spatial domain integration. Description of the Drawings

[0015] Figure 1It is a step diagram of a three-dimensional reconstruction method for spectral polarization imaging based on a pixel array; Figure 2 It is a schematic diagram of a subsampling block and its decoding process. Specific implementation manner

[0016] The following are specific embodiments of the present invention and in combination with the accompanying drawings, the technical solutions of the present invention are further described, but the present invention is not limited to these embodiments.

[0017] Current target three-dimensional reconstruction technologies mainly rely on a single information dimension of spectroscopy, polarization, or traditional three-dimensional imaging, but face significant limitations in complex scenarios. Spectral imaging analyzes material composition through multi-band reflection characteristics, but it is difficult to capture surface topography details; polarization imaging can characterize microstructures or roughness, but its reconstruction accuracy highly depends on the known refractive index of the material, and it cannot directly provide depth information; traditional three-dimensional imaging (such as structured light, binocular vision) can reconstruct macroscopic contours, but it is prone to reconstruction failure due to feature loss or interference in scenes with textureless, highly reflective, or transparent surfaces. In addition, existing systems that fuse multi-modal information mostly adopt a time-sharing and beam-splitting hardware architecture. For example, spectral and polarization data are collected time-sharing through a rotating filter wheel or polarizer, resulting in time delay and spatial registration error, poor data consistency in dynamic scenes, and a large system volume and complex optical path, making it difficult to meet the high-efficiency deployment requirements of industrial inspection or mobile platforms. More critically, existing methods have deficiencies in multi-source information collaborative modeling: polarization three-dimensional reconstruction requires assuming that the refractive index of the material is known, but actual complex targets are often composed of multiple materials, and reconstruction ambiguity is easily introduced when the refractive index is unknown; the physical correlation between multi-spectral and polarization data has not been fully exploited, resulting in the inversion model being sensitive to noise and having insufficient robustness. These problems jointly restrict the accuracy and adaptability of existing technologies in complex materials, multi-scale structures, and dynamic environments.

[0018] Aiming at the above defects, the present invention proposes a three-dimensional reconstruction method for spectral polarization imaging based on a pixel array. This method uses each pixel of the image sensor as an array, and periodically arranges interference filter films and polarization filter films of each band on the surface of the image sensor. Through single exposure, spectral, polarization, and spatial information are synchronously obtained, and a joint inversion model that fuses multi-dimensional physical constraints is constructed to break through the perception bottleneck of traditional technologies, as Figure 1 shown, mainly including the following steps: S1: Through single exposure of the image sensor, the original images of the imaging target in multiple spectral bands are collected, and each pixel in the image sensor fixedly obtains the original images corresponding to its respective spectral bands and polarization angles; S2: Decode the original images, extract the sub-image information of each spectral band and polarization angle according to the pixel positions, and stack them to generate a four-dimensional data volume containing intensity information, spatial coordinates, spectral bands, and polarization angles; S3: Construct a polarization geometry model based on the four-dimensional data volume, extract the degree of linear polarization, polarization angle, and azimuth angle in each spectral band, and construct the actual expression of the degree of linear polarization in combination with Fresnel reflection theory; S4: Construct a redundant dispersion constraint equation that expresses the refractive index as a rational function of wavelength, and a theoretical expression for the relationship between the degree of diffuse reflection spectral polarization, the incident angle, and the refractive index of the imaging target; S5: Substitute the redundant dispersion constraint equation into the joint residual function of the theoretical expression and the actual expression, and perform parameter inversion under nonlinear least squares optimization of the joint residual function through the LM optimization algorithm; S6: Perform three-dimensional topography restoration under the reconstruction of the surface normal of the imaging target based on the parameters obtained by inversion.

[0019] The following elaborates on the technical implementation and innovative advantages of this solution in detail in combination with specific embodiments.

[0020] In the image acquisition and preprocessing stage, a pixel-level multi-spectral polarization imaging system is used to synchronously capture target information. The core of the imaging system is a customized image sensor, on the surface of which a variety of interference filter films and polarization filter films are integrated in a periodic array manner, and each pixel corresponds to a unique combination of spectral band and polarization angle. Specifically, the interference filter film covers at least two spectral bands (such as the visible to near-infrared range), and the polarization filter films are periodically arranged at four angles of 0°, 45°, 90°, and 135°, ensuring that each pixel only receives the light intensity signal of a specific wavelength and polarization direction under a single exposure. Through this design, the system can synchronously capture the multi-spectral intensity distribution and polarization modulation information of the target in a single-frame image without relying on mechanical rotating components (such as filter wheels or polarization rotators) required for traditional time-sharing acquisition, eliminating spatial misalignment and motion artifacts caused by time-sequence delay or mechanical motion in time-sharing multi-modal imaging from the root. During the acquisition process, the control module triggers the exposure of the image sensor and synchronously receives the original image data stream. The original image consists of multiple sub-sampled blocks, and each sub-sampled block contains m (number of spectral bands) × 4 (number of polarization angles) independent channels. The spectral and polarization attributes of each pixel are uniquely determined by the preset parameters of the filter film on its surface. This pixel-level parallel sampling mechanism not only significantly shortens the data acquisition time but also ensures a high spatio-temporal consistency of multi-dimensional information in a dynamic scene, providing a reliable data basis for subsequent high-precision three-dimensional reconstruction.

[0021] In the image decoding and data volume construction stage, multi-dimensional information analysis and structured reorganization are performed on the original image obtained by a single exposure. As Figure 2As shown, the original image is composed of periodically arranged subsampled blocks, and within each subsampled block, pixel channels with different spectral bands and polarization angles are distributed. During the decoding process, first, according to the preset pixel arrangement rule, each pixel is scanned row by row and column by column. Based on the parameters of the interference filter film and polarization filter film integrated on its surface, the corresponding spectral band (such as ), and the light intensity information at polarization angles (0°, 45°, 90°, 135°) are identified and extracted. Subsequently, through spatial coordinate mapping, the spectral-polarization data of each pixel is reallocated to the corresponding sub-image channels according to the spatial coordinates , forming a four-dimensional data volume that includes spatial coordinates , spectral dimension ( ), and polarization angle ( ). To ensure the spatial alignment of the images between different channels, a linear interpolation algorithm is used to restore the resolution of the sub-images, eliminating the local information loss or pixel interval caused by pixel-level sampling, and finally generating a multi-dimensional data set consistent with the original sensor resolution. This process is completely based on the filter film arrangement rules predefined by the hardware and does not rely on external registration algorithms, fundamentally avoiding the spatio-temporal misalignment problems introduced by time-sharing acquisition or mechanical movement in traditional multi-modal systems. At the same time, through strict spectral-polarization channel binding, the high consistency of multi-dimensional data at the pixel level is ensured.

[0022] Based on the multi-spectral polarization data in the four-dimensional data volume, the present invention further correlates it with the physical properties and geometric morphology of the target surface. For each spectral band, by analyzing the Stokes vector, not only the degree of linear polarization ( ) and polarization angle ( ) are extracted, but also combined with the distribution characteristics of the azimuth angle ( ), the global and local variation laws of the normal direction of the target surface are deduced.

[0023] Specifically, the three components of the Stokes vector respectively characterize the total light intensity, the difference between the orthogonal components of linearly polarized light, and the modulation characteristics of the circularly polarized component. Among them, the square root of the sum of the squares of and divided by the total light intensity: That is, the degree of linear polarization of the spectral wavelength , which directly reflects the modulation ability of the target surface to light waves with different polarization states, while the polarization angle depicts the direction of the polarization principal axis. Then, by introducing the azimuth angle , the polarization angle is mapped to the azimuth distribution of the surface normal in three-dimensional space, and then combined with the zenith angle Solve for the angle of incidence to construct a complete normal vector field.

[0024] Furthermore, to relate the degree of linear polarization to the angle of incidence, the present invention combines Fresnel reflection theory and establishes an actual expression for the degree of linear polarization through the reflectivity difference between s-polarized light and p-polarized light ( and ). During this process, the calculation of the reflectivities and strictly depends on the angle of incidence and the wavelength-dependent refractive index . For example, the expression for the s-polarized reflectivity is: reveals the non-linear coupling relationship between the material refractive index and the geometric parameters ( ). And the expression for the p-polarized reflectivity is: further highlights the significant influence of the wavelength dependence of the refractive index on polarization modulation.

[0025] Meanwhile, considering the key role played by the redundancy of multi-spectral data at this stage: through the polarization observation data obtained at different wavelengths, the Cauchy dispersion equation ( ) is used to perform cross-band smoothing constraints on the refractive index. For example, when the target surface is composed of an unknown composite material, the refractive index inversion in a single band may lead to error accumulation due to noise or local anomalies (such as saturation distortion in the specular reflection region), while the multi-band data can effectively suppress such interference through the global fitting of the dispersion equation. Specifically, if the dispersion equation contains three undetermined coefficients (A, B, C), at least four observation data in different bands (N≥4) are required to construct an overdetermined system of equations, so as to achieve a unique solution of the parameters through least squares optimization. This positive definite inversion strategy based on redundant data not only improves the adaptability of the model to complex materials (such as multi-layer coatings, anisotropic materials), but also significantly reduces the morphological reconstruction distortion caused by refractive index assumption deviation in the traditional single-band model.

[0026] In addition, combining the theoretical model of the degree of polarization of diffuse reflection spectra, the quantitative relationship between the degree of linear polarization, the angle of incidence, and the refractive index is derived. Based on Fresnel reflection theory and the diffuse reflection hypothesis, the expression for the theoretical degree of polarization is: Through strict mathematical derivation, this expression deeply integrates the optical properties (refractive index dispersion) of the material with geometric parameters (angle of incidence), depicting the physical essence of polarization modulation on the target surface at different wavelengths. For example, at low angles of incidence ( ), the theoretical degree of polarization approaches zero, consistent with the specular reflection characteristics of a smooth surface; while at high angles of incidence ( ), the degree of polarization is significantly enhanced, reflecting the modulation effect of surface roughness or microstructures on light waves.

[0027] Relate the theoretical derivation model to the actual observation model and construct a joint residual function: By minimizing the residual function, the angle of incidence and dispersion coefficients such as A, B, and C can be inversely retrieved simultaneously. In this process, the redundancy of multi-wavelength data plays a dual role: firstly, through the wavelength-dependent refractive index continuity constraint (Cauchy equation), it ensures the physical rationality of material parameters; secondly, through the complementarity of multi-angle polarization observations, it enhances the model's analytical ability for complex surfaces (such as anisotropic textures and multi-material mixtures). For example, on the surface of a composite material, the refractive index in different regions may vary differently with wavelength, and multi-wavelength constraints can balance local contradictions through global optimization, avoiding overfitting problems caused by single-wavelength data.

[0028] Then, based on the constructed joint residual function, the present invention efficiently solves this function through the Levenberg-Marquardt (LM) algorithm to achieve simultaneous high-precision inversion of the optical properties of the material and geometric morphology parameters. Among them, the LM algorithm achieves a balance between the second-order convergence of the Gauss-Newton method and the stability of the gradient descent method by dynamically adjusting the damping factor , and is especially suitable for optimization problems with strong nonlinear coupling and high parameter dimensions.

[0029] Specifically, the inversion process starts with an initial parameter estimate. The initial value of the angle of incidence can be set through the approximate relationship (such as ) between the azimuth angle and the angle of incidence in the polarization geometry model; the dispersion coefficients are empirically assigned based on the refractive index dispersion curves of typical materials (such as the known dispersion laws of glass or polymers). In each iteration, the algorithm first calculates the theoretical degree of polarization at the current parameters and its partial derivatives with respect to the parameters (Jacobian matrix J), and then constructs an increment equation: where is the parameter increment vector, is the damping factor, is the identity matrix. By adjusting , the algorithm selects the optimal step size in the parameter space: when the residual decreases significantly, is reduced to accelerate convergence (close to the Gauss-Newton method); when the residual increases or fluctuates, is increased to enhance stability (close to the gradient descent method).

[0030] The redundancy of multi-spectral data further highlights its advantages at this stage. For example, when the observed data in a certain band (such as ) deviates from the theoretical model due to noise or saturation distortion, the multi-band constraint can automatically reduce the contribution of abnormal data by weighting the residuals, avoiding the interference of local errors to the global solution. In addition, the intrinsic smoothness of the Cauchy dispersion equation (the refractive index changes continuously with wavelength) provides physical rationality constraints for parameter optimization. For example, the dispersion coefficient B is usually negative (most materials exhibit normal dispersion in the visible light band), and such prior knowledge can be incorporated into the optimization process through parameter boundary conditions (such as B < 0) to prevent the emergence of non-physical solutions.

[0031] To ensure the convergence and efficiency of the algorithm, two termination conditions are set during the iteration process: firstly, the residual decrease rate is lower than the threshold, indicating that the optimization enters a stable state; secondly, the norm of the parameter increment is less than the preset tolerance (such as ), indicating that the parameters tend to be optimal. For complex scenarios (such as multi-material mixed surfaces), the algorithm also supports a hierarchical optimization strategy: first fix the dispersion coefficient to invert the geometric parameters ( ), and then optimize the material parameters (A, B, C) based on the geometric results, reducing the complexity of the coupling problem through alternating iterations.

[0032] The optimized output parameter set ( ) not only satisfies the minimum residual of the multi-band data and the theoretical model, but also ensures the physical interpretability of the results through physical constraints (such as refractive index continuity, incident angle range). This process significantly improves the robustness and accuracy of 3D shape reconstruction in scenarios with unknown materials, dynamic lighting, and noise interference through the deep integration of a rigorous mathematical framework and physical laws.

[0033] According to the optimized output parameter set, the normal components , , and of each pixel position are extracted to obtain the target surface normal , and the formula is expressed as follows: where , is the surface function at the pixel position . Then, based on the obtained target surface normal , construct the normal gradient field: Among them, is the gradient field in the X-axis direction of the normal, is the gradient field in the Y-axis direction of the normal. Subsequently, by solving the two-dimensional Poisson equation to construct the minimization objective, perform gradient integration on the discovered gradient field of the target surface to obtain the result of 3D reconstruction, that is, given Restore the surface function The process of, the objective cost function is: When the value of the above formula reaches the minimum, it indicates that the normal gradient field is orthogonally projected onto the integrable surface, and the surface function can be represented by a linear combination of the basis functions :

[0034] Among them, the basis function is a set of functions defined in the two-dimensional space. Just like the "basis vectors" in the vector space, more complex functions can be constructed through their linear combinations. What needs to be constructed in the present invention is the surface function , represents a two-dimensional form, is the expansion coefficient of the surface function If there exists an optimal coefficient combination such that the cost function has the minimum value, its expression is (this is a weighted average least squares estimation expression, making a compromise between the gradient information in two directions (X-axis, Y-axis)):

[0035] Among them, is the optimal basis function coefficient reconstructed, is the coefficient estimated according to the gradient in the X-axis direction, is the coefficient estimated according to the gradient in the Y-axis direction, is the weighting factor related to the error or confidence, representing the energy or contribution degree of this basis function in the X-axis and Y-axis directions respectively. Further expressed as:

[0036] Perform Fourier transform on the basis function and the optimal coefficient combination solution and substitute them into the expression of to obtain Related expressions for the normal gradient field:

[0037] Among them, and represent the discrete Fourier transform and the inverse Fourier transform respectively, is the imaginary unit, is the frequency component in the X-axis direction in the spatial domain, is the frequency component in the Y-axis direction in the spatial domain. Through Fourier transform, the discrete integral problem in the time domain is transformed into the frequency domain. The obtained surface function is converted into a point cloud or a mesh model, and the restored target three-dimensional morphology can be intuitively displayed.

[0038] The present invention also includes a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the steps of the three-dimensional reconstruction method based on pixel array spectral polarization imaging are implemented.

[0039] The present invention also includes a device for processing data, including: A memory, on which a computer program is stored; A processor, configured to execute the computer program in the memory to implement the steps of the three-dimensional reconstruction method based on pixel array spectral polarization imaging.

[0040] In summary, the three-dimensional reconstruction method based on pixel array spectral polarization imaging proposed by the present invention synchronously obtains the light intensity information of multiple spectral bands and polarization angles through single exposure, which not only eliminates the time delay and spatial registration error caused by time-sharing acquisition in the traditional multi-modal system, but also greatly improves the imaging efficiency and data consistency in dynamic scenes.

[0041] Based on the multi-source information fusion of the four-dimensional data volume, a complete physical characterization framework including material properties, surface microstructure and macroscopic morphology is constructed, breaking through the perception bottleneck of single-dimensional imaging under complex materials and multi-scale structures.

[0042] By introducing the joint constraint of redundant band observation data and the Cauchy dispersion equation for inversion, the dependence on the prior knowledge of material refractive index in traditional polarization three-dimensional reconstruction is effectively overcome, the reconstruction ambiguity of multi-material mixed targets is significantly reduced, and at the same time, the noise interference is suppressed by multi-band redundant information, improving the stability in low-light and high-noise environments.

[0043] Through the gradient integral method of solving the two-dimensional Poisson equation in the frequency domain, the surface normal distribution is efficiently converted into a three-dimensional height map, taking into account the accurate restoration of both detail features and overall morphology, and avoiding the cumulative error problem in traditional spatial domain integration.

[0044] It should be noted that all directional indications (such as up, down, left, right, front, back...) in the embodiments of the present invention are only used to explain the relative positional relationship, movement conditions, etc. between components in a specific posture (as shown in the drawings). If the specific posture changes, the directional indications will also change accordingly.

[0045] In addition, in the present invention, descriptions such as "first", "second", "one", etc. are only for descriptive purposes and should not be construed as indicating or implying their relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, the meaning of "a plurality" is at least two, such as two, three, etc., unless otherwise specifically defined.

[0046] In the present invention, unless otherwise clearly specified and defined, terms such as "connection" and "fixation" should be understood in a broad sense. For example, "fixation" can be a fixed connection, a detachable connection, or integrated; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and can be the communication inside two components or the interaction relationship between two components, unless otherwise clearly defined. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0047] In addition, the technical solutions between various embodiments of the present invention can be combined with each other, but it must be based on the ability of those of ordinary skill in the art to implement. When the combination of technical solutions results in contradictions or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the protection scope required by the present invention.

Claims

1. A three-dimensional reconstruction method for spectral polarization imaging based on a pixel array, characterized in that, Taking each pixel of the image sensor as an array, interference filter films and polarization filter films of each wavelength band are periodically arranged in an array on the surface of the image sensor, specifically including the steps: S1: Through a single exposure of the image sensor, the original images of the imaging target in multiple spectral bands are acquired. Each pixel in the image sensor fixedly obtains the original images corresponding to its respective spectral bands and polarization angles. S2: Decode the original images, extract the sub-image information of each spectral band and polarization angle according to the pixel positions, and stack them to generate a four-dimensional data volume containing intensity information, spatial coordinates, spectral bands, and polarization angles. S3: Construct a polarization geometric model based on the four-dimensional data volume, extract the degree of linear polarization, polarization angle, and azimuth angle of each spectral band, and construct an actual expression of the degree of linear polarization in combination with Fresnel reflection theory. S4: Construct a redundant dispersion constraint equation expressing the refractive index as a rational function of wavelength, and a theoretical expression for the relationship between the diffuse reflection spectral degree of polarization, the incident angle, and the refractive index of the imaging target. S5: Substitute the redundant dispersion constraint equation into the joint residual function of the theoretical expression and the actual expression, and perform parameter inversion with nonlinear least squares optimization of the joint residual function through the LM optimization algorithm. S6: Perform three-dimensional shape recovery of the imaging target surface normal reconstruction based on the parameters obtained by inversion.

2. The three-dimensional reconstruction method of spectral polarization imaging based on a pixel array according to claim 1, wherein The polarization filter film includes four polarization angles of 0°, 45°, 90°, and 135°, and the interference filter film covers at least two spectral bands.

3. The three-dimensional reconstruction method of spectral polarization imaging based on a pixel array according to claim 1, characterized in that In the step S3, the polarization geometric model extracts the degree of linear polarization, polarization angle, and azimuth angle based on the Stokes vector, and the expression is: In the formula, is the constant label of the spectral band, is the wavelength of the th segment of the spectrum, is the total light intensity at the spectral wavelength , is the difference in light intensity between the horizontal polarization component and the vertical polarization component at the spectral wavelength , is the difference in light intensity between the 45° polarization component and the 135° polarization component at the spectral wavelength , is the degree of linear polarization at the spectral wavelength , is the polarization angle at the spectral wavelength , is the azimuth angle.

4. The three-dimensional reconstruction method of spectral polarization imaging based on a pixel array according to claim 1, wherein, In the step S3, the actual expression of the degree of linear polarization is: Wherein, is the constant label of the spectral band, is the wavelength of the th segment of the spectrum, is the spectral wavelength of the actual degree of linear polarization, is the spectral wavelength of the reflectivity of s-polarized light, is the spectral wavelength of the reflectivity of p-polarized light.

5. The three-dimensional reconstruction method of spectral polarization imaging based on a pixel array according to claim 4, characterized in that, In the step S4, the redundant dispersion constraint equation is constructed based on the Cauchy dispersion equation, and the expression is: In the formula, is the spectral wavelength is the refractive index, and A, B, and C are the coefficients of the dispersion equation to be determined.

6. The three-dimensional reconstruction method of spectral polarization imaging based on a pixel array according to claim 5, wherein In the step S4, the theoretical expression for the relationship between the diffuse reflection spectral degree of polarization, the incident angle, and the refractive index of the imaging target is: In the formula, is the spectral wavelength of the theoretical degree of linear polarization, is the incident angle of the th spectral band to be obtained.

7. A three-dimensional reconstruction method of spectral polarization imaging based on a pixel array according to claim 6, characterized in that In the step S5, the expression of the joint residual function is: In the formula, is the combined residual of the theoretical expression and the actual expression.

8. A three-dimensional reconstruction method for spectral polarization imaging based on a pixel array according to claim 7, characterized in that In the step S6, the three-dimensional shape recovery is specifically: reconstruct the surface normal of the imaging target based on the parameters obtained by inversion, construct a minimization target by solving the two-dimensional Poisson equation, perform gradient integration on the normal gradient field, and obtain a surface function for three-dimensional shape recovery. The expression is: In the formula, is the normal vector at , are the components of the normal vector in the X, Y, and Z axis directions respectively, is the surface function, is the azimuth angle, is the component of the normal gradient field in the X axis direction, is the component of the normal gradient field in the Y axis direction, is the Fourier transform, is the inverse Fourier transform, is the imaginary unit, is the frequency component in the X axis direction in the spatial domain, is the frequency component in the Y axis direction in the spatial domain.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by a processor, it implements the steps of the method for three-dimensional reconstruction of spectral polarization imaging based on a pixel array according to any one of claims 1 to 8.

10. A device for processing data, characterized in that, Including: A memory, on which a computer program is stored; A processor, configured to execute the computer program in the memory to implement the steps of the method for three-dimensional reconstruction of spectral polarization imaging based on a pixel array according to any one of claims 1 to 8.

Images