Disambiguation of surface normals using single-view mueller shape-from-polarization

Abstract

A method for disambiguating surface normals on an object in a space. The method includes illuminating the object in the space by a light source and capturing an image of the illuminated object in the space by an imaging component. A polarimetry measurement of the object is generated to identify light scattering angles. The method further includes calculating a plurality of surface normals of the object, calculating a rough depth estimate of the object in the space, calculating a depth-based surface normal, and selecting a surface normal of the plurality of surface normals and the depth-based surface normal with a dot product greater than or equal to every other dot product of the plurality of surface normals and the depth-based surface normal.

Description

DISAMBIGUATION OF SURFACE NORMALS USING SINGLE-VIEW MUELLER SHAPE-FROM-POLARIZATIONCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of U.S. Provisional Application No. 63 / 637,272 filed April 22, 2024. the specification of which is incorporated herein in its entirety' by reference.FIELD OF THE INVENTION

[0002] The present invention is directed to a shape-from-polarization process capable of imaging a depth map of a scene from passive illumination and polarization imaging.BACKGROUND OF THE INVENTION

[0003] Shape from Polarization (SIP) is an imaging method that tries to reconstruct an object's 3D shape (surface normals and / or depth) from polarization images. Shape from Polarization (SIP) is dominated by non-physics-based approaches such as machine learning. Analytic approaches assume either purely diffuse or purely specular light scattering on the object being reconstructed. Neither of these two extreme cases represents the polarized light scattering of common materials in an indoor environment. Thus, there exists a present need for a SfP method capable of capturing polarized light scattering of common materials in an indoor environment through a physics-based approach.BRIEF SUMMARY OF THE INVENTION

[0004] It is an objective of the present invention to provide systems, devices, and methods that allow for a shape-from-polarization process capable of imaging a depth map of a scene from passive illumination and polarization imaging, as specified in the independent claims. Embodiments of the invention are given in the dependent claims. Embodiments of the present invention can be freely combined with each other if they are not mutually exclusive.

[0005] The present invention features a method for disambiguating surface normals on an object in a space. The method may comprise illuminating the object in the space by a light source at a known light location. The method may further comprise capturing an image of the illuminated object in the space by an imaging component at a known imaging location, generating a polarimetry measurement of the illuminated object based on the captured image, identifying, based on the polarimetry measurement, a plurality of light scattering angles over the object, based on the known light location and the known imaging location. The method may furthercomprise calculating, based on the plurality of light scattering angles, the known light location, and the know n imaging location, a plurality of surface normals of the object, calculating, based on the plurality of light scattering angles, a rough depth estimate of the object in the space, calculating, based on the rough depth estimate, a depth-based surface normal, and selecting a surface normal of the plurality' of surface normals and the depth-based surface normal with a dot product greater than or equal to every' other dot product of the plurality of surface normals and the depth-based surface normal.

[0006] One of the unique and inventive technical features of the present invention is the generation of a depth map from passive illumination and polarization imaging. Without w ishing to limit the invention to any theory or mechanism, it is believed that the technical feature of the present invention advantageously provides for a physics-based approach for depth imaging that accounts for the polarized light scattering of common materials in an indoor environment. None of the presently know n prior references or work has the unique inventive technical feature of the present invention.

[0007] Furthermore, the inventive technical feature of the present invention is counterintuitive. The reason that it is counterintuitive is because it contributed to a surprising result. One of ordinary skill in the art would implement an existing efficient and optimized machine learning algorithm trained on polarization images for determining surface normals of an image. The present invention implements a physics-based algorithm for deriving surface normals from polarization images gathered from light sources and imaging components at known locations. Surprisingly, this algorithm provides for reliable and efficient selection and disambiguation of surface normals without the need for additional imaging modalities. Thus, the inventive technical feature of the present invention contributed to a surprising result.

[0008] Any feature or combination of features described herein are included within the scope of the present invention provided that the features included in any such combination are not mutually inconsistent as will be apparent from the context, this specification, and the knowledge of one of ordinary skill in the art. Additional advantages and aspects of the present invention are apparent in the following detailed description and claims.BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING(S)

[0009] The features and advantages of the present invention will become apparent from a consideration of the following detailed description presented in connection with theaccompanying drawings in which:

[0010] FIG. 1A shows a diagram of surface normal ambiguity as implemented in the present invention.

[0011] FIG. IB shows a diagram of acquisition geometry as implemented in the present invention.

[0012] FIG. 1C shows a diagram of a Rusinkiewicz coordinate system as implemented in the present invention.

[0013] FIG. 2A shows a flow chart of a method for disambiguating surface normals using a rough depth estimate from single-view polarimetric imaging.

[0014] FIG. 2B shows a schematic diagram of a system for disambiguating surface normals using a rough depth estimate from single-view polarimetric imaging.

[0015] FIG. 3 shows a simulated Mueller matrix image of a sphere with an angular velocity of 35° and a wavelength of 662 nm.

[0016] FIG. 4A shows a ground truth Rusinkiewicz angle image of a sphere with an angular velocity of 35°, taken at angle 0h.

[0017] FIG. 4B shows a ground truth Rusinkiewicz angle image of a sphere with an angular velocity of 35°, taken at angle 0d.

[0018] FIG. 4C shows a ground truth Rusinkiewicz angle image of a sphere with an angular velocity of 35°, taken at angle <J)d.

[0019] FIG. 5 A shows an ambiguous surface normal solution of the sphere from a n ground-truth Rusinkiewicz angle.

[0020] FIG. 5B shows an ambiguous surface normal solution of the sphere from a n’ ground-truth Rusinkiewicz angle.

[0021] FIG. 6A shows the surface normal calculated from a depth I estimate using the ground truth 0d.

[0022] FIG. 6B shows the surface normal calculated from andepthdepth estimate using the ground truth 0d.

[0023] FIG. 7A shows ground truth Rusinkiewicz angles images of a sphere taken at angles 0h, 0d, and (])dwith a signal -to-noise ratio (SNR)of 10000.

[0024] FIG. 7B shows ground truth Rusinkiewicz angles images of a sphere taken at angles 0h, 0d, and <J)d with an SNR of 1000.

[0025] FIG. 7C shows ground truth Rusinkiewicz angles images of a sphere taken at angles 0h,0d, and <j)dwith an SNR of 100.

[0026] FIG. 7D shows ground truth Rusinkiewicz angles images of a sphere taken at angles 0h, 0d, and (])dwith an SNR of 10.

[0027] FIG. 8A shows normals calculated from Rusinkiewicz angles at an SNR of 10000.

[0028] FIG. 8B shows normals calculated from Rusinkiewicz angles at an SNR of 1000.

[0029] FIG. 8C shows normals calculated from Rusinkiewicz angles at an S / N ratio of 100.

[0030] FIG. 8D shows normals calculated from Rusinkiewicz angles at an S / N ratio of 10.

[0031] FIG. 9A shows an estimate of depth calculated from angle 0dat various SNRs after smoothing with a 9x9 pixel median fdter.

[0032] FIG. 9B shows a normal estimate from the gradient of the depth at various SNRs.

[0033] FIG. 10A shows a result of disambiguating surface normals with an SNR of 10000.

[0034] FIG. 10B shows a result of disambiguating surface normals with an SNR of 1000.

[0035] FIG. 10C shows a result of disambiguating surface normals with an SNR of 100.

[0036] FIG. 10D shows a result of disambiguating surface normals with an SNR of 10.

[0037] FIG. 11 shows a line graph showing how the SNR affects the mean-absolute-error of the disambiguated surface normal with two different source distances and with and without using a smoothing filter on the depth estimate.

[0038] FIG. 12 shows a diagram of the angle of linear polarization (AoLP) of each side of a spoon. In (a) the concave and in (b) the convex side. The shape from polarization 7t-ambiguity creates AoLP invariance.

[0039] FIG. 13 shows a diagram of a SfP disambiguation pipeline. The inputs are: a monocular Mueller image, prior knowledge of the measurement configuration and materials’ polarization bidirectional reflectance distribution function (pBRDF). Three computational steps are: a non-linear optimization to estimate the scattering geometry, and transform the scattering angles into two ambiguous surface normals and depth. The depth is used to create a third surface normal solution. Finally, the normal solution from depth is used to resolve the ambiguity.

[0040] FIG. 14 shows a diagram of a crude estimate of depth formed using a known source and camera position combined with the estimated Rusinkiewicz angle, 0d.

[0041] FIGs 15A-15C show diagrams of ambiguous surface normal estimates from monocular Mueller images for a planar object tilted horizontally by 45° (FIG. 15 A), a mixed concave and convex object (FIG. 1 B), and the Stanford bunny (FIG. 15C). Local details on the surface, such as the bunny’s hind leg, are still visible at an SNR of 10. Therefore, if the ir-ambiguity is solved, SfP can be robust to noise.

[0042] FIGs 16A-16D show diagrams of surface normal estimates from monocular Mueller images for singular surfaces. The Mean Average Estimation (MAE) is reported in the upper left of each image and the percent correct from the disambiguation in the lower right. Each surface is shown with four levels of SNR: i. 104; ii. 103; iii. 100; and iv. 10. FIG. 16A shows a planar object tilted horizontally by 45°. FIG. 16B shows a convex sphere. FIG. 16C shows a concave sphere. FIG. 16D shows a saddle surface. The disambiguation approach results in MAE below 2° and greater than 99% correct up to an SNR of 100. At an SNR of 10, more pixels are disambiguated incorrectly, resulting in MAE’s of 13.3° and above. Only 48.9% of pixels are disambiguated correctly for the convex sphere and 52.9% for the saddle surface.

[0043] FIGs 17A-17B show diagrams of surface normal estimates from monocular Mueller images for an object with concave and convex regions (FIG. 17A) and the Stanford bunny (FIG. 17B). Each object is shown with four levels of SNR: i. 104; ii. 103; iii. 100; and iv. 10. The number in the upper left comer is the MAE, and in the lower right is the percent of pixels disambiguated correctly. The MAE is below 10° at SNR=100 for the simpler shape in FIG. 17A, but increases to 12.1° for the bunny in FIG. 17B. This demonstrates that when the segments corresponding to similar curvature are smaller, there are fewer pixels for depth estimation and thus more susceptibility to noise.

[0044] FIGs 18A-18B show diagrams of ambiguous surface normal estimates from a simulated COTS polarimeter with unpolarized (FIG. 18 A) and horizontally polarized (FIG. 18B) illumination at SNR = 104.

[0045] FIGs 19A-19B show diagrams of surface normal estimates at SNR = 104from a simulated COTS polarimeter, in unpolarized (FIG. 19A) and horizontally polarized (FIG. 19B) illumination. The number in the upper left comer is the MAE, and in the lower right is the percent of pixels disambiguated correctly. Both the MAE and the percent correct are degraded compared to Mueller imaging in FIGs 17A-17B.

[0046] FIG. 20 shows a flowchart of the surface normal disambiguation method from single-view polarimetric imaging using an imprecise depth estimate. The surface normals are shown by encoding the x, y, z directions of the normals into the RGB color channels. A Mueller image is simulated using the known measurement geometry and the pBRDF model. This image provides the surface normals up to the a-ambiguity in the azimuth angle. From the same Mueller image, a depth estimate is obtained using the known source and camera location. The gradient of the depth gives an imprecise, but unambiguous estimate of the surface normal, which is used to disambiguate the estimates.

[0047] FIGs 21A-21B show normals calculated with the ambiguoussolution from the fit, showing mixing of the solutions over the sphere, calculated from the simulated, noise-free Mueller image.

[0048] FIG. 22 shows acquisition geometry' where the depth 1 is calculated from 0d, and known camera and source positions 's and ^c. The view direction oo^ is known from the camera properties. The depth is the distance of the object from the camera and can be determined using the law of sines on the brown triangle as in Equation 16.

[0049] FIGs 23A-23B show the surface normals calculated from the depth estimate using the ground truth 0d. In FIG. 23A, the ground truth depth of the sphere calculated from the incident angle 0d using Equation 16. The gradient of this depth from Equation 16 gives the surface normal solution in FIG. 23B. This estimate is used to disambiguate the conventional surface normal solutions shown in FIGs 21A-21B.

[0050] FIG. 24 shows Rusinkiewicz estimates with varying SNR simulated in Equation 10. The range of 0dis more than an order of magnitude smaller than the other angles and SNR reduction degrades the correlation structure across the sphere. The last column shows the same 4>dsolution but converted to modulo 180, in order to visualize which variations are due to the noise and which are due to the ambiguity'.

[0051] FIG. 25 shows normals calculated from the fitted Rusinkiewicz angles at each SNR. These solutions contain the 7i-ambiguity in the azimuth, but have a precise zenith angle. This is visible in the mixing of the concave and convex solutions in some areas, and slow' variation in normals in other areas. They are not significantly sensitive to noise, as the solutions do not substantially change as the SNR decreases.

[0052] FIG. 26 shows the disambiguated surface normal estimates at four SNR levels. The choice between ambiguous solutions is pixel-wise based on the smallest angular difference between the imprecise normal estimate from the depth. The percentage of pixels with an angular error below' 10° is reported.

[0053] FIG. 27 shows a graph of the effect of the noise on the MAE of the disambiguation method at two different source distances. The MAE increases as the SNR decreases. This shows that some measurement geometries may decrease the susceptibility to noise of this disambiguation method.

[0054] FIG. 28A shows a flowchart diagram of a method for disambiguating surface normals of the present invention.

[0055] FIG. 28B shows a schematic diagram and flowchart of a non- transitory computing medium for disambiguating surface normals of the present invention.

[0056] FIG. 28C shows a schematic diagram of a system for disambiguating surface normals of the present invention.DETAILED DESCRIPTION OF THE INVENTION

[0057] Following is a list of elements corresponding to a particular element referred to herein:

[0058] 110 light source

[0059] 120 imaging component

[0060] 130 computing system

[0061] 132 processor

[0062] 134 memory component

[0063] 200 obj ect

[0064] 1000 system

[0065] The term “surface normal” is defined herein as the imaginary line perpendicular to a flat surface, or perpendicular to the tangent plane at a point on a non-flat surface.

[0066] The term “azimuth angle” is defined herein as a horizontal angle measured clockwise from any fixed reference plane or easily established base direction line.

[0067] The term “polarimetry ” is defined herein as a technique to measure the polarization of light.

[0068] The term “dot product” is defined herein as an algebraic operation that takes two equal-length coordinate vectors and returns a single number.

[0069] The term “Rusinkiewicz angle” is defined herein as an angle that represents the scattering geometry of a polarimetric measurement.

[0070] The term “least squares operation” is defined herein as a statistical procedure to find the best fit for a set of data points.

[0071] The term “depth map” is defined herein as an image that contains information relating to the distance of the surfaces of scene objects from a viewpoint.

[0072] The term “Mueller matrix” is defined herein as a mathematical description of how light is altered by an optical element or a sample under study.

[0073] The term “binary mask” is defined herein as a method of selecting pixels based onintensity values or position, or by using binary segmentation techniques.

[0074] Referring now to FIG. 28A. the present invention features a computer-implemented method for disambiguating surface normals on an object (200) in a space. In some embodiments, the method may comprise illuminating the object (200) in the space by a light source (110) at a known light location. The method may further comprise capturing an image of the illuminated object (200) in the space by an imaging component (120) at a known imaging location. The method may further comprise generating a polarimetry measurement of the illuminated object (200) based on the captured image. The method may further comprise identifying, based on the polarimetry measurement, a plurality of light scattering angles over the object (200), based on the known light location and the known imaging location. The method may further comprise calculating, based on the plurality of light scattering angles, the known light location, and the known imaging location, a plurality of surface normals of the object (200). The method may further comprise calculating, based on the plurality of light scattering angles, a rough depth estimate of the object (200) in the space. The method may further comprise calculating, based on the rough depth estimate, a depth-based surface normal. The method may further comprise selecting a surface normal of the plurality of surface normals and the depth-based surface normal with a dot product greater than or equal to every other dot product of the plurality of surface normals and the depth-based surface normal. The method may further comprise deriving a depth map of the object (200) from the surface normal of the plurality of surface normals with the dot product greater than or equal to every other dot product of the plurality of surface normals and the depth-based surface normal.

[0075] In some embodiments, the polarimetry measurement may comprise a Mueller matrix image. In some embodiments, the Mueller matrix image may be derived from a mixed diffuse and specular polarization bidirectional reflectance distribution function (pBRDF) model. In some embodiments, the plurality of light scattering angles may comprise a plurality of Rusinkiewicz angles. In some embodiments, the plurality of Rusinkiewicz angles may comprise an angle representing a deviation from specular reflection, an incident angle of light from the light source (110) onto the object (200), an azimuth angle, or a combination thereof. In some embodiments, identify ing, based on the polarimetry measurement, the plurality of light scattering angles over the object (200) may comprise a least squares operation. In some embodiments, selecting the surface normal of the plurality of surface normals and the depth-based surface normal with the dot product greater than or equal to every other dot product of the plurality of surface normalsand the depth-based surface normal may comprise using a binary7mask. In some embodiments, the method may further comprise smoothing, by a smoothing filter, the rough depth estimate.

[0076] Referring now to FIG. 28B, the present invention features a non-transitory computing medium (134) for disambiguating surface normals on an object (200) in a space, comprising computer-readable instructions. When executed by a processor (132), the computer-readable instructions may cause the processor (132) to illuminate the object (200) in the space by a light source (110) at a known light location. The computer-readable instructions may further cause the processor (132) to capture an image of the illuminated object (200) in the space by an imaging component (120) at a known imaging location. The computer-readable instructions may further cause the processor (132) to generate a Mueller matrix image of the illuminated object (200) based on the captured image. The computer-readable instructions may further cause the processor (132) to perform a least squares operation on the Mueller matrix image to identify a plurality of Rusinkiewicz angles over the object (200) fitted to the Mueller matrix image, based on the known light location and the known imaging location. The computer-readable instructions may further cause the processor (132) to calculate, based on the plurality of Rusinkiewicz angles, the known light location, and the known imaging location, a plurality of surface normals of the object (200).

[0077] The computer-readable instructions may further cause the processor (132) to calculate, based on the plurality of Rusinkiewicz angles, a rough depth estimate of the object (200) in the space. The computer-readable instructions may further cause the processor (132) to calculate, based on the rough depth estimate, a depth-based surface normal. The computer-readable instructions may further cause the processor (132) to select a surface normal of the plurality of surface normals and the depth-based surface normal with a dot product greater than or equal to every other dot product of the plurality of surface normals and the depth-based surface normal. The computer-readable instructions may further cause the processor (132) to derive a depth map of the object (200) from the surface normal of the plurality of surface normals with the dot product greater than or equal to every other dot product of the plurality of surface normals.

[0078] In some embodiments, the Mueller matrix image may be derived from a mixed diffuse and specular polarization bidirectional reflectance distribution function (pBRDF) model. In some embodiments, the computer-readable instructions may further cause the processor (132) to smooth, by a smoothing filter, the rough depth estimate. In some embodiments, the plurality ofRusinkiewicz angles comprise an angle representing a deviation from specular reflection, an incident angle of light from the light source (110) onto the object (200), an azimuth angle, or a combination thereof.

[0079] Referring now to FIG. 28C, the present invention features a system (1000) for disambiguating surface normals on an obj ect (200) in a space. In some embodiments, the system (1000) may comprise a light source (110) at a known light location, configured to illuminate the object (200) in the space. The system (1000) may further comprise an imaging component (120) at a known imaging location, configured to capture an image of the object (200) in the space. The system (1000) may further comprise a computing system (130) operatively coupled to the light source (110) and the imaging component (120), comprising a processor (132) configured to execute computer-readable instructions and a memory component (134) operatively coupled to the processor (132). The memory component (134) may comprise computer-readable instructions.

[0080] The computer-readable instructions may comprise generating a polarimetry measurement of the object (200) based on the captured image. The computer-readable instructions may further comprise identifying, based on the polarimetry measurement, a plurality of light scattering angles over the object (200), based on the known light location and the known imaging location. The computer-readable instructions may further comprise calculating, based on the plurality of light scattering angles, the known light location, and the known imaging location, a plurality of surface normals of the object (200). The computer-readable instructions may further comprise calculating, based on the plurality of light scattering angles, a rough depth estimate of the object (200) in the space. The computer-readable instructions may further comprise calculating a dot product for each surface normal of the plurality of surface normals, based on the rough depth estimate. The computer-readable instructions may further comprise selecting a surface normal of the plurality of surface normals with a dot product greater than or equal to every other dot product of the plurality of surface normals. The computer-readable instructions may further comprise deriving a depth map of the object (200) from the surface normal of the plurality of surface normals with the dot product greater than or equal to every other dot product of the plurality of surface normals and the depth-based surface normal.

[0081] In some embodiments, the polarimetry’ measurement may comprise a Mueller matrix image. In some embodiments, the Mueller matrix image may be derived from a mixed diffuseand specular polarization bidirectional reflectance distribution function (pBRDF) model. In some embodiments, the plurality of light scattering angles may comprise a plurality of Rusinkiewicz angles. In some embodiments, the plurality of Rusinkiewicz angles may comprise an angle representing a deviation from specular reflection, an incident angle of tight from the tight source (110) onto the object (200), an azimuth angle, or a combination thereof. In some embodiments, identifying, based on the polarimetry measurement, the plurality7of light scattering angles over the object (200) may comprise a least squares operation. In some embodiments, selecting a surface normal of the plurality of surface normals with a dot product greater than or equal to every7other dot product of the plurality of surface normals may comprise using a binary mask. In some embodiments, the computer-readable instructions may further comprise smoothing, by a smoothing filter, the rough depth estimate.

[0082] Referring now to FIG. 2A, the present invention features a method for disambiguating surface normals on an object (200) in a space. In some embodiments, the method may comprise illuminating the object (200) in the space by a light source (110) at a known light location. The method may further comprise capturing an image of the illuminated object (200) in the space by an imaging component (120) at a known imaging location. The method may further comprise generating a polarimetry measurement of the illuminated object (200) based on the captured image. The method may further comprise identifying, based on the polarimetry measurement, a plurality of light scattering angles over the object (200). based on the known light location and the known imaging location.

[0083] The method may further comprise calculating, based on the plurality of light scattering angles, the known light location, and the known imaging location, a plurality of surface normals of the object (200). The method may further comprise calculating, based on the plurality of light scattering angles, a rough depth estimate of the object (200) in the space. The method may further comprise calculating, based on the rough depth estimate, a depth-based surface normal. The method may further comprise selecting a surface normal of the plurality of surface normals and the depth-based surface normal with a dot product greater than or equal to every other dot product of the plurality of surface normals and the depth-based surface normal.

[0084] In some embodiments, the polarimetry7measurement may comprise a Mueller matrix image. In some embodiments, the plurality of light scattering angles may comprise a plurality of Rusinkiewicz angles. In some embodiments, the plurality' of Rusinkiewicz angles may comprisean angle representing a deviation from specular reflection, an incident angle of light from the light source (110) onto the object (200), an azimuth angle, or a combination thereof. In some embodiments, identifying, based on the polarimetry measurement, the plurality of light scattering angles over the object (200) may comprise a least squares operation. In some embodiments, the method may further comprise deriving a depth map from the surface normal of the plurality of surface normals with the dot product greater than or equal to every other dot product of the plurality of surface normals and the depth-based surface normal. In some embodiments, selecting the surface normal of the plurality of surface normals and the depth-based surface normal with the dot product greater than or equal to every other dot product of the plurality of surface normals and the depth-based surface normal may comprise using a binary7mask. In some embodiments, the method may further comprise smoothing, by a smoothing filter, the rough depth estimate.

[0085] Referring now to FIG. 2B, the present invention features a system (1000) for disambiguating surface normals on an obj ect (200) in a space. In some embodiments, the system (1000) may comprise a light source (110) at a known light location, configured to illuminate the object (200) in the space. The system (1000) may further comprise an imaging component (120) at a known imaging location, configured to capture an image of the object (200) in the space. The system (1000) may further comprise a computing system (130) operatively coupled to the light source (110) and the imaging component (120), comprising a processor (132) configured to execute computer-readable instructions and a memory component (134) operatively coupled to the processor (132), comprising computer-readable instructions.

[0086] The computer-readable instructions may comprise generating a polarimetry' measurement of the object (200) based on the captured image. The computer-readable instructions may further comprise identifying, based on the polarimetry measurement, a plurality of light scattering angles over the object (200), based on the known light location and the known imaging location. The computer-readable instructions may further comprise calculating, based on the plurality of light scattering angles, the known light location, and the known imaging location, a plurality of surface normals of the object (200). The computer-readable instructions may further comprise calculating, based on the plurality of light scattering angles, a rough depth estimate of the object (200) in the space. The computer-readable instructions may further comprise calculating a dot product for each surface normal of the plurality of surface normals, based on the rough depth estimate. The computer-readable instructions may further comprise selecting a surface normal of the plurality of surface normals with a dot product greater than or equal to every- other dotproduct of the plurality of surface normals.

[0087] In some embodiments, the polarimetry measurement may comprise a Mueller matrix image. In some embodiments, the plurality of light scattering angles may comprise a plurality’ of Rusinkiewicz angles. In some embodiments, the plurality of Rusinkiewicz angles may comprise an angle representing a deviation from specular reflection, an incident angle of light from the light source (110) onto the object (200), an azimuth angle, or a combination thereof. In some embodiments, identifying, based on the polarimetry measurement, the plurality of light scattering angles over the object (200) may comprise a least squares operation. In some embodiments, the computer-readable instructions may further comprise deriving a depth map from the surface normal of the plurality of surface normals with the dot product greater than or equal to every other dot product of the plurality’ of surface normals and the depth-based surface normal. In some embodiments, selecting the surface normal of the plurality’ of surface normals and the depth-based surface normal with the dot product greater than or equal to every’ other dot product of the plurality of surface normals and the depth-based surface normal may comprise using a binary mask. In some embodiments, the computer-readable instructions may further comprise smoothing, by a smoothing filter, the rough depth estimate.

[0088] In some embodiments, the known light location may comprise a known distance from the object. In some embodiments, the known imaging location may comprise a known distance from the object.

[0089] EXAMPLE

[0090] The following is a non-limiting example of the present invention. It is to be understood that said example is not intended to limit the present invention in any way. Equivalents or substitutes are within the scope of the present invention.

[0091] The present invention features SIP image processing techniques designed for monocular Mueller imaging to determine the binary selection between two degenerate surface normals. A depth estimate was generated as an intermediate step to resolving the shape ambiguity. This approach focused on a scattering geometry’ representation where noise primarily affected this depth estimate rather than the ambiguous surface normals. Leveraging this understanding, effectively smoothed the depth estimates while preserving crucial surface normal information. These findings highlighted the inherent SfP information in monocular Mueller imaging and are significant for the design of polari- metric image processing techniques.

[0092] When light reflects off an object, electrons at the surface oscillate in a preferential orientation and re-emit polarized light. The orientation of the reflected light’s polarization depends on many factors: the material of the surface, the wavelength of the light, the type of reflection, the illuminating polarization, the incident direction of the illumination, and the orientation of the surface. Consequently, polarization is used in numerous image processing tasks. For example, polarization is harnessed to remove reflections from images captured through glass. Similarly, differences in the polarization signal between specular and diffuse reflection enable their separation, allowing for isolation of the intrinsic surface color. The dependence of polarization on index of refraction and material properties allows for material classification. SfP is a field of study dedicated to relating polarization and the orientation of surfaces to enhance shape detection. With the recent increase in availability of commercial off-the-shelf (COTS) polarimeters, polarization measurements are becoming more accessible. A COTS measurement of the angle of linear polarization (AoLP) reveals detailed and robust shape information in polarization, as shown in FIG. 12. Notably, measurement noise in a polarization image perturbs the polarization reconstruction. Despite the robustness, this noise often leads to errors in the image processing task being performed.

[0093] SfP has the potential to enable passive and monocular shape detection. However, existing SfP methods use linear polarization to recover the surface normals, which are subject to inherent ambiguities. To resolve these ambiguities, SfP is often combined with other imaging modalities. For example, time-of-flight sensors scan many points of an object to measure depth, however, they typically have low spatial resolution. Similarly, multi-view systems rely on feature matching between different views, but they are ineffective at reconstructing smooth surfaces. In these cases, the high-resolution nature of SfP compensates for these limitations, while the complementary imaging modality solves the ambiguity in SfP.

[0094] The present invention features a method for resolving the SfP ambiguity using single-view-, single-modality polarization imaging. This method implemented a novel physics-based image processing algorithm that leverages full polarimetric information to achieve disambiguation. The pipeline is illustrated in FIG. 13. The method was evaluated to partial polarimetry, involving linear Stokes measurements, and compared to the full polarimetric case.

[0095] The Fresnel equations describe how the polarization of light changes upon interaction with a surface, depending on the surface orientation, material refractive index, and initialpolarization state. Light can be polarized parallel (p-polarized) and perpendicular (s-polarized) to the plane of incidence. Many SfP methods use the Fresnel equations to relate the surface normal direction to the AoLP and the degree of linear polarization (DoLP). However, these relationships introduce several ambiguities in the surface normal.

[0096] The DoLP provides the zenith angle of the normal, which is the angle between the normal and the view direction. The dependence of DoLP on the angle of incidence, and therefore the zenith angle, differs for specular and diffuse reflection. Specular reflection is a single reflection from a macroscopic surface, following the law of reflection. Diffuse reflection involves transmission into the material, followed by scattering back out. For specular reflection, the DoLP as a function of the angle of incidence peaks at Brewster's angle, resulting in an ambiguity where two possible zenith angles exist on either side of the peak. In contrast, diffuse reflection exhibits a monotonic increase in DoLP with the angle of incidence, so there is one unique solution for the zenith angle. The zenith ambiguity for specular reflection in SfP can be avoided by assuming an object only exhibits diffuse reflection. However, this assumption is weak for shiny objects. Alternative methods to solve the zenith ambiguity have been devised. For example, taking multi-spectral measurements and observing which way the DoLP shifts with wavelength resolves the ambiguity in specular reflection. The tested method bypassed the zenith ambiguity by modeling the reflection as a combination of specular and diffuse reflection, and using the full polarimetric information of the object.

[0097] Simultaneously, the AoLP provides the azimuth angle of the surface normal, which is the angle of the normal projected onto the image plane. Given a polarimetric measurement at a single surface point, the normal can be determined up to four possible azimuth angles. These four solutions occur because of two distinct ambiguities. The first ambiguity arises from another difference betw een diffuse and specular reflection. The second stems from the inherent ambiguity in the dependence of linear polarization on surface orientation.

[0098] The first azimuth ambiguity results from the fact that the polarization at the surface is a combination of s- and p-polarized light. Specular reflection is typically dominated by s-polarization. Meanwhile, since diffuse reflection involves transmission, it tends to be more p-polarized. This causes a JT / 2 radian ambiguity in the surface normal. If the dominant type of reflection is known, this n / 2-ambiguity can be resolved. Thus, many approaches assume either a purely specular or a purely diffuse model for the entire object, or by separating the reflectiontypes. Some works adopt a mixed model by selecting or determining the dominant reflection type at each pixel. Other works combine specular and diffuse into a single scattering model. The tested method addressed the n / 2 azimuth ambiguity by using a reflectance model that incorporated the relative amount of diffuse and specular reflection.

[0099] The second azimuth angle ambiguity, which is the focus of this study, arises from the inherent n-invariance of the AoLP about the scattered ray direction. This results in a n-ambiguity in the surface normal. In other words, the surface normal is specified in the image plane as an orientation rather than a direction. The azimuth 7t-ambiguity causes similarly shaped convex and concave surfaces to be indistinguishable given an AoLP measurement. This effect is demonstrated in FIG. 12 using a kitchen tablespoon, where the AoLP images of the concave and convex sides of the spoon are visually indistinguishable. The radial pattern observed on the spoon’s surface indicates the strong dependence of AoLP on the surface orientation. However, on opposite sides of the spherical surface, the AoLP shifts by 180°, cycling back to the same measured AoLP.

[0100] Correspondingly, the surface normals on opposite sides of the spherical surface have the same projected orientation in the image plane. Therefore, it is clear how the opposing convex and concave surfaces appear to have the same AoLP pattern. This work aims to solve the a-azimuth ambiguity in simulated monocular Mueller polarimetric measurements by integrating physics-based modeling and image processing.

[0101] The objective of the method of the present invention was to inform SfP algorithm development by investigating fundamental information content in polarization imaging. It was demonstrated that the n-ambiguity could be resolved from a monocular view using image processing if there was a full polarimetric characterization of the material and prior knowledge of the source and camera locations. The polarized light-matter interaction was modeled using mixed specular and diffuse pBRDF. It has previously been shown that depth and shape retrieval is possible in an idealized noise-free case. Here, an inverse problem was solved to recover the scattering geometry from a simulated monocular Mueller matrix image, while introducing measurement noise. The recovered scattering geometry, expressed by the Rusinkiewicz angles, provided the surface normals corresponding to the ambiguous concave and convex surfaces, as well as a crude depth map. It has previously been shown that directly using this depth estimate to resolve the ambiguity is highly sensitive to noise, leading to rapid degradation in the normalestimation.

[0102] A smoothing assumption was introduced to the depth map, and the smoothed surface was used to resolve the ambiguity. The SfP method was extended to different surface types and more complex objects. Representing the problem using the Rusinkiewicz parametrization enabled image processing to refine the highly noise-sensitive depth map, while preserving essential high-frequency shape details, without imposing restrictive geometric priors. Finally, the method was evaluated for a simulated measurement configuration corresponding to a COTS polarimeter to assess the feasibility of passive or single-shot capture. The results indicated that the ambiguous normals remained robust even in high-noise Mueller images and in partial polarimetric measurements. While the unambiguous depth information was initially present, it deteriorated with increasing measurement noise. The present invention presents a means for monocular, single-modality shape acquisition using full polarimetric information. Additionally, prior studies have shown that image processing and imposing non-orthographic projection can benefit learning and multi-modal SfP algorithms.

[0103] The tested method for resolving the 7t-azimuth ambiguity in SfP using monocular Mueller polarimetry is shown in FIG. 13. The method took as input: 1) a Mueller matrix image, 2) the relative camera and light source positions, ’c and ^s, and 3) a pBRDF model for the material. The scattered ray direction was also known for each pixel from the camera properties, incorporating the non-orthographic projection. The computation consisted of 1) a nonlinear optimization to estimate the scattering geometry from the Mueller matrix image and prior knowledge. In step 2) ambiguous surface normals were calculated from the recovered scattering geometry. Step 3) estimated a depth map from the scattering geometry, followed by image processing techniques to smooth the depth estimate and mitigate measurement noise. This depth map provided an imprecise estimate of the surface normals. Finally, the ambiguous solutions were compared to the imprecise normals to resolve the ambiguity.

[0104] The polarized light-matter interaction at a point on the object was described by a Mueller matrix. A Mueller matrix fully describes how polarization is linearly transformed between an incoming and outgoing ray. These ray directions depend on the object shape, depth, and positions of the source and camera. A Mueller image includes all polarimetric information about an object, including AoLP, DoLP. depolarization, retardance, diattenuation, and circular polarization. In comparison, linear Stokes imaging only adds an AoLP and DoLP to theradiometric image. When a Mueller matrix is specified as a function of the scattering geometry, it becomes a pBRDF. A pBRDF quantifies how a material changes the polarization of incident light from any angle. The outgoing polarization is a function of the incoming and outgoing ray directions, as well as the polarization of the incident light. The pBRDF model used specified the relative amount of first surface and diffuse reflection depending on the scattering geometry, material properties, and the illumination spectrum. The model formulated a non-depolarizing Mueller-Jones Matrix (MJM), which was the dominant process in the triple-degenerate (TD) model, where the depolarization was decoupled from the MJM.

[0105] Rusinkiewicz angles: The scattering geometry7that parametrized the pBRDF model was in the form of the Rusinkiewicz coordinates. The Rusinkiewicz angles described the incident and scattered ray directions relative to the local surface normal. The scattering geometry for a camera and source separation of angle Q. and the Rusinkiewicz angles at a single point of a surface are shown in FIGs 1B-1C. The camera and source were defined to be in the x-z plane, w here the camera axis defined the z-axis. The direction of the incident and outgoing rays, co and co0pointed towards the source and camera, respectively, and defined the scattering plane. As stated, the scattered ray oo0was prior knowledge and corresponded to each pixel’s view direction in the non-orthographic projection. The halfway vector h bisected these ray directions. The Rusinkiewicz angles were especialty useful because they had physical interpretations. For an isotropic surface, the incident and outgoing rays were specified independently from global coordinates using three Rusinkiewicz angles: 0h, 0d, and 4>d. The angle from the surface normal to the halfway vector 0hdescribes deviations from specular scattering and ranges from 0 to jr / 2. When 0h= 0, the reflection is specular. The angle 0dalso ranged from 0 to n / 2 and was the angle between ay and h. Thus, 0dis the angle of incidence if h is regarded as the surface normal to a sub-resolution surface, also called a microfacet, where Fresnel reflection occurs. Finally, <|)dis the angle between two planes: the scattering plane and the plane spanned by n and h. The angle <|)dranges from ~T to n, but due to the azimuth ambiguity, polarization observations are invariant to 7i shifts in <|)d.

[0106] Shape from Mixed Polarization: The relative amount of diffuse and specular reflection in the mixed pBRDF model depended on the scattering geometry, as well as the material properties. The specular component, more accurately called the first surface component, of the pBRDF model w as based on Fresnel reflection from a microfacet and did not dependdirectly on the surface normal of the object. The first-surface component depended only on the separation of the incoming and outgoing rays, described by 0d. The diffuse term was added with a weight based on the direction of the surface normal relative to the microfacet given by 0h, as well as material properties. The diffuse term itself was a function of <|)d, except it was invariant to <f)d+71. The diffuse component of the pBRDF model followed a pattern centered at 0h= 0°, unlike other models centered on the camera axis. This caused the diffuse pattern to center around the point on the object where perfect specular reflection occurs, and the ambiguous normal solutions were equivalent at this location because <|)dwas undefined.

[0107] Optimizing the Inverse Problem: The first step of the SfP method was to solve an inverse problem, where the scattering geometry was in the form of the Rusinkiewicz angles and is estimated from a monocular Mueller matrix image. All images were simulated. A Mueller matrix image was reconstructed from a series of flux images, denoted ^p. The flux image was related to the Mueller matrix ’m by p = W ’m , where W was the measurement matrix representing the polarization states measured by the polarimeter. Gaussian noise was introduced to the simulated measurement by the addition of a noise flux vector ’nSNR. The Mueller matrix image of an object was described by the pBRDF model as a function of the Rusinkiewicz angles, so ’p = W m (0h, 0d, >d). Therefore, the inverse problem was solved in simulation using a nonlinear least squares optimization with the merit function given by, argwhere (0h, 0d, >d) are the ground truth Rusinkiewicz angles before measurement noise is added and (0 0 4>d) are the Rusinkiewicz angles estimated from the simulated noisy measurement. The noise in each flux image,nsNR i, is simulated by sampling from a zero-mean Gaussian distribution with the standard deviation given by o, =A W matrix was implemented, representing a polarimeter consisting of a polarization camera (analyzer) and a polarized illumination source (generator), each consisting of a linear polarizer and a rotating retarder. The illumination wavelength was assumed to be 662 nm. There were 40 images acquired at different analyzer and generator states. A minimum of at least 16 independent images was required to reconstruct the full polarimetric information in a Mueller matrix, so using 40 measurements introduced redundancy, which minimized the influence of noise. Later, the W matrix wassimplified to simulate the partial polarimetric image taken by a COTS polarimeter with unpolarized and then linearly polarized illumination.

[0108] The ground truth Rusinkiewicz angles for a concave and a convex sphere with Q = 35° and a source distance of 20 cm were determined. Both surfaces had the same range of depths within the image and they have equivalent radii of curvature. The surfaces had different specular locations, where 0h= 0 and cpdwas undefined, since their depths at each point varied due to opposite curvature. Here, the azimuthal ambiguity of the surface normal was observed in the images of the cpdangle. The concave and convex surfaces differed by q>d± n. When cpdwas represented as modulo 180° (as would be determined from a polarization measurement) then the cpdpattern for the concave and the convex surface appeared similar about the specular location. Similarly, 0hwas symmetric about the specular point. On the other hand. 0dof the concave and the convex surface was visibly different. This implied that there was a correlation between the depth of a point on a surface and 0d. When solving the optimization, 0dwas constrained to correspond to objects within a predetermined reasonable range of depths. These limits were determined using the prior knowledge of the camera and source locations, as well co^. Additionally, the optimization was performed with four initial values of cpdand combined pixels from the result with the low est residual.

[0109] Surface Normal Calculation: Once the inverse problem was solved, the surface normals were calculated pixelwise using the estimated Rusinkiewicz angles. The incident ray co . was determined by rotating the scattered ray by 20dsuch that co = R(20d, v )co Here R was a three-dimensional rotation matrix where the first argument was the amount of rotation and the second argument was the axis of rotation. The vector v was perpendicular to the plane spanned by co 0 and the source and camera locations, ’s - ^c. The halfway vector was the bisector of the ray directions,

[0110] The surface normal w as calculated by rotating the halfway vector through 0habout the vector v which was perpendicular to h and co., then rotating this vector through q>dabout h. leading toDue to the azimuthal ir-ambiguity in polarization, there is another equally valid solution where the vector is rotated through cf>d+ n, giving

[0111] These ambiguous surface normals contained a combination of normals that made up a concave surface and others that made up a convex surface. For a planar surface, the ambiguity of mixed tilt angles within the image. To disambiguate the normals, the direction of surface curvature or tilt was estimated.

[0112] Depth and Surface Normal Estimation: By solving the inverse problem to estimate the scattering geometry, the depth of the object was estimated at each point through 0d. To understand the relationship between 0dand the shape of the object, it was considered that the location where an incident and outgoing ray intersect depended on the object depth. For example, a closer object position resulted in a larger angle of incidence. As illustrated in FIG. 14, the distance from the object surface to the camera position, / , was calculated from the known camera and source positions, and the camera properties given by am Specifically,where w = / / ’s - c / / was the distance between the source and the camera and a = arccos(-co o •(^s - ^c) / w).

[0113] The distance I was the depth in the non-orthographic projection since it depended on the direction ofwhich increased toward the edges of the field of view To not mistake a slightly concave surface for a convex surface, the depth was calculated in the orthographic projection d by.where proj c(J > y ) was the pro *jection of / co onto the plane perpendicular to c. The depth provided an unambiguous estimate of the surface normals. The surface normals calculated from the depth estimate were given by the gradient of the depth map such thatand p is a camera-specific parameter that represents an estimate of the ratio between a physical length in the scene and the number of pixels it spans in the image.

[0114] When noise was present, erroneous high frequency variations of nbA were induced. Therefore, a smoothing constraint was imposed by approximating the depth with polynomials. A least-quares fit was performed up to the second order coefficients in the x and y direction, C0-4,

[0115] Then d replaced d in Eq. 6. The first three orders of polynomials, excluding cross terms, incorporated just enough information to estimate if the depth represented a concave or a convex surface in the x and y directions, and at what angle the surface was tilted.

[0116] To account for objects with varying curvature, the depth was fit in segments. The areas of constant curvature were determined by calculating the divergence of the ambiguous normal solutions. Either solution, V • n0or V •could be used, and the divergence image was segmented using the multi-Otsu thresholding algorithm. The corresponding depth of each segment was fit as a single surface using Eq. 7. Segmenting based on the ambiguous normal estimates allowed us to leverage the high-frequency, noise-robust shape information in SfP. Meanwhile, the crude depth estimate was used only for the binary choice between the ambiguous solutions.

[0117] Disambiguation: Once the imprecise estimate of the normals nAwas obtained, the robust normals from SfP were disambiguated. The normals were disambiguated pixel-wise by comparing the dot product of n0and nxwith nAand choosing the normal solution with the largestdot product. The final unambiguous normal solution is ii —

[0118] Since the surface normals were disambiguated by calculating a gradient of the data per Eq. 6, the ambiguity in SfP was not resolvable by observing a single location on an object. According to this model the ambiguity was resolved by observing the second order characteristics of the polarization data over a surface. The surface normals calculated from polarization were detailed and highly robust to noise, so the correct solution was left to a binary choice between two surfaces. Therefore n . did not need to be highly accurate, it just needed to be closer to the true solution than the ambiguous one. The only assumption about the shape being imposed was that the object was composed of smaller continuous, smooth surfaces.

[0119] Optimization Results: The Rusinkiewicz angles were estimated from a simulated Mueller matrix image of a convex sphere and the non-linear optimization expressed by Eq. 1. These estimates are shown in FIGs 7A-7D for four different values of SNR. Here, it was evident that 0ftand 4>dwere more robust to noise, compared to 0rfwhich degraded to the point where, at SNR= 10, the image has minimal spatial structure. In other words, the depth retrievals using 0rfand Eqs. 4 and 5 were more susceptible to noise compared to the ambiguous surface normals. This increased susceptibility arose because 0dvaries more gradually across the object compared to the other two angles.

[0120] Ambiguous Normal Solutions: The Rusinkiewicz angle estimates provided the robust, but ambiguous solutions for nQand n but led to the noisy estimate of depth from 0d. FIGs 15A-15C show nQandcalculated using Equations 2 and 3 for three objects, each with high SNR and low SNR. Notice it is impossible to tell which features on the object in FIG. 15B are concave and which are convex based solely on the surface normals. Much of the high-frequency information, such as the bunny’s fur, is still visible at an SNR of 10. The robustness of the surface normal solutions from SfP means the largest source of error was due to the ambiguous solutions rather than to noise.

[0121] Resolving the 7t-Ambiguity on a Single Surface: From here, the binary choicebetween these detailed normal solutions was made. The depth was calculated using Eqs. 4 and 5 with the 0rffrom Eq. 1. The high susceptibility to noise of 0^ meant the depth must then be approximated by a polynomial given by Eq. 7 before it is used in Eq. 6 to obtain n Finally, the surface normal solution n was chosen using Eq. 8. The result of disambiguated surface normals are shown for four surface types in FIGs 16A-16D: a plane tilted horizontally at 45°, a convex sphere, a concave sphere, and a saddle surface that is convex in the y-direction and concave in the x-direction. The surface normal calculations were performed at four values of SNR. Each of the surfaces was treated as a single continuous surface, approximated by a single polynomial. At low levels of noise, nearly all surface normals were disambiguated correctly. The disambiguation started to break down between SNR values of 100 and 10. The disambiguation was reliable, resulting in MAE below 2° and greater than 99% of pixels disambiguated correctly for each surface up to an SNR of 100. At an SNR of 10, more pixels were disambiguated incorrectly, resulting in MAE’s of 13.3° and above. At the specular point, there was no ambiguity, and the ambiguous surfaces were similar near specularity'. As a result, the pixels near the specular point were the first to disambiguate incorrectly but they resulted in little error because rotating 4>dthrough 7i made a small change in the orientation of the surface normal when 0hwas near zero. The disambiguation for the sphere and the saddle surface resulted in a larger error at an SNR of 10 than the concave sphere. This is likely due to the fit for the depth aligning more with the true surface for some noise realization than others since most spatial information in 9d is lost at SNR = 10, as shown in FIGs 7A-7D.

[0122] Resolving the ir-Ambiguity on a Complex Object: The disambiguation of more complex objects with varying curvature are showoi in FIGs 17A-17B. Here, the depth estimate was fit in segments in order to smooth areas with continuous curvature. The segments were determined based on the values of the divergence of nQor n In these cases, the final binary mask representing the choice of nfrom Equation 8 was smoothed to minimize the error due to the gradient in Eq. 6 transitioning betw een segments of the depth. For the convex and concave objects, the MAE was below 10° through SNR = 100. However, SNR = 103was required for the normal estimates of the bunny to be below 10° MAE. This demonstrated that when the depth was fit to smaller segments for a more complex object, ambiguity errors occurred sooner, resulting in the loss of high-frequency information.

[0123] Resolving the n-ambiguity from partial polarimetry: FIGs 18A-19B illustrate the disambiguation algorithm applied to partial polarimetry for SNR = 104The non-depolarizing MJM could not be decoupled from the ideal depolarizer with partial polarimetry. Therefore, the Mueller matrix in Eq. 1 included depolarization for the following results. The W matrix corresponded to a micro-polarizer array, as in a COTS polarimeter, configured with linear polarizers at 0°, 45°, 90°, and 135°. FIG. 18A shows the ambiguous normal estimates for simulating unpolarized illumination, mimicking a passive Stokes polarization measurement, while FIG. 18B shows the ambiguous normal estimates with simulated horizontally polarized illumination. These normal estimates had a similar accuracy to the ambiguous normal estimates from Mueller polarimetry in FIG. 15C. This observation verified that detailed shape information remained in the normal estimates even when partial polarimetry was used. The problem was left to whether 0dstill had enough shape information to resolve the ambiguity in these estimates.

[0124] FIG. 19A shows the disambiguation result for unpolarized illumination and FIG. 19B shows the result for horizontally polarized illumination. For unpolarized illumination, the percentage of pixels that were correct in the ambiguous solutions in FIG. 18A were 57.8% and 42.2%, so the percentage of pixels chosen correctly upon applying the disambiguation algorithm was between those, thus providing no benefit. Therefore, the algorithm was not resolving the ambiguity' with passive polarimetry because the rate of correct pixels was comparable to guessing each pixel at random. If 100% of the pixels were chosen correctly, the MAE for unpolarized illumination would be 9.6°. This demonstrated that there was shape information retrieved from partial polarimetry with a micro-polarizer array, as shown in FIG. 12, but the unambiguous information in 0dretrieved from the algorithm was lost. For horizontally polarized illumination, the percentage of correct pixels in the ambiguous solutions in FIG. 18B were 58.3% and 41.7%, so the disambiguation provided a small but not significant improvement. The improvement between unpolarized and polarized illumination demonstrated the potential of the algorithm to work with partial polarimetry if the polarimetric measurement system was designed to prioritize the estimation of a crude depth map to disambiguate. Notably, the result for horizontally polarized illumination at SNR = 104was an improvement from the Mueller polarimetry result at SNR = 10. Therefore, the performance of the system should be considered along with the task requirements to decide if using partial polarimetry , with a simpler system, is beneficial.

[0125] The present invention addresses the longstanding azimuth n-ambiguity in polarimetric imaging by demonstrating fundamental information content in a completepolarimetric characterization. The approach implemented monocular Mueller imaging and posed an inverse problem to estimate the scattering geometry of an object, given the known locations of the camera, source, and pBRDF of the material. The representation of the scattering geometry in the form of the Rusinkiewicz angles allowed the ambiguity problem to be solved in a single modality with image processing. An explicit relationship between the object’s depth and a single Rusinkiewicz angle was used to select between the ambiguous pair of surface normals. The ambiguous surface normals were estimated with high accuracy for high-noise monocular Mueller measurements. The depth estimate was neither as accurate nor robust to noise compared to the surface normals. However, the feasibility of estimating the binary choice between the ambiguous surface normals was demonstrated. The SfP performance was improved by smoothing constraints that were applied to depth only. Isolating this smoothing operation was a key advantage of applying image processing in the scattering geometry representation.

[0126] The present method of disambiguation on Mueller images resulted in an MAE consistently below 10° for a single surface with an SNR as low as 100. For an SNR of 10, the effectiveness of the disambiguation varied with the noise realization. For increasingly complex objects, the depth was smoothed in smaller patches and the smaller the area the depth was fit over, the more susceptible to noise the disambiguation was. The MAE stayed below 10° through an SNR of 100 for simple objects, but required an SNR of 1,000 for more complex objects.

[0127] While the ambiguous surface normals were estimated with high accuracy, the disambiguation algorithm was not effective for passive linear Stokes polarimetry. The percentage of pixels chosen correctly was nearly 50%. The resulting MAE at SNR = 104was as high as 41.9° for a polarization analyzer corresponding to a micro-polarizer array and unpolarized illumination. The MAE was 33.7° for horizontally polarized illumination, which offered a small improvement in disambiguation. This improvement indicated that a simplified measurement setup could be optimized for recovering unambiguous surface normals.

[0128] For comparison, other physics-based methods applied to real measurements achieved MAE of 41.4° - 49.1° from partial polarimetric images by employing shading and convexity constraints. Learning-based methods have achieved MAE of 18.5°. By using multi-view measurements, learning methods have achieved MAE as low as 1.41°. Therefore, the present partial polarimetric simulated results are comparable to real measurements with physics-based methods. The simulated Mueller polarimetric results at SNR = 100 are comparableto some learning based methods applied to real measurements. The application of image processing with full polarimetric information and the assumption of non-orthographic projection in this work could further improve machine learning SfP approaches.

[0129] The following is another non-limiting example of the present invention. It is to be understood that said example is not intended to limit the present invention in any way. Equivalents or substitutes are within the scope of the present invention.

[0130] The present example investigated, through simulation, a method of disambiguating the surface normals using an imprecise depth estimate calculated from a single-view Mueller image. Both the conventional surface normals and the depth estimates were enabled by a mixed specular and diffuse pBRDF, so no additional imaging modality is required. The method was investigated under the following assumptions: 1) the camera and source locations were known precisely, and 2) the pBRDF model of the material exactly matched the simulated light-matter interactions. The robustness of the method was tested under several levels of simulated measurement noise and two distances from the source to the object.

[0131] A chart for disambiguating surface normals is shown in FIG. 20. The surface normals in this figure are represented by encoding the x, y, z directions of the normals into RGB color channels. Therefore, the varying colors represent different orientations of surface normals. The present method was tested on a simulated Mueller measurement of a sphere. The Mueller image of the sphere was simulated using the pBRDF model and the measurement geometry. A per-pixel least-squares optimization allowed for the Rusinkiewicz angles to be determined from this image, which were used to calculate the two ambiguous surface normal estimates. The normals calculated from the Mueller image had an ambiguous azimuth but an accurate zenith, thus producing two solutions nA and n U . These solutions were disambiguated using the imprecise depth estimate, which was obtained from the same Mueller image using the angle of incidence. The gradient of the depth provided an imprecise surface normal estimate nAover the object. The normal estimate from the depth had an unambiguous azimuth angle, but an imprecise zenith angle. The final unambiguous SfP normals were determined pixel-wise by choosing which of the ambiguous normals were closest to the imprecise estimate from the depth. Therefore, the same Mueller measurement provided the surface normals up to the ambiguity, as well as the additional imprecise estimate of the surface normals used for the disambiguation.

[0132] Computing the Rusinkiewicz angles required that the camera and source positions were known, as well as the shape of the object. Rusinkiewicz angles for a spherical object, = 35°, Lcam~ 16.5 cm and Lsrc= 90cm are shown in FIGs 4A-4C. The Mueller image was generated by evaluating a selected pBRDF at the Rusinkiewicz angles for each pixel. The pBRDF function also required the known material properties of the object, which depended on the wavelength of the source. The model assumed a source waveband centered at / . = 662nm. The simulated ground truth Mueller image of a sphere given this information is shown in FIG. 3. The left edge of the sphere is slightly in shadow.

[0133] The inverse problem was estimating the Rusinkiewicz angles from the single-view simulated Mueller image. The measurements were described by the flux vector ~"p, which was created by multiplying the Mueller image M by the measurement matrix W representing a polarimeter, in this case, the RGB950. The flux vector measurement was therefore given by ’p = WR B950M. Noise is introduced into this measurement by adding the noise flux vector nSNR. Since the simulated measurement was created by the pBRDF model, which depended on the Rusinkiewicz angles, the scattering geometry was estimated by performing a least squares optimization on the simulated flux measurement. The per-pixel merit function was given by, argininwhere (0h, 9d, c[>d) are the ground truth Rusinkiewicz angles in FIGs 4A-4C, and (0ft, 0 c[>d) were the estimated Rusinkiewicz angles. The noise flux vector ’nSNR was created by sampling from a Gaussian distribution with zero mean, and standard deviation given byIn Equation 9, the noise was multiplied by WRGB950I50 is the pseudoinverse of the measurement matrix. This projected the noise vector in consistency space to aid with smoothing the merit function. The SfP method was tested at a range of noise levels by varying the signal-to-noise ratio (SNR).

[0134] Two ambiguous surface normal solutions can be computed from the estimatedRusinkiewicz angles using the following calculation. Firstis rotated through 20dabout a vector v perpendicular toand ’s - ^c, to arrive atThen h was calculated asNext, h was rotated through 0habout a vector v perpendicular to h and oo . . Then this vector was rotated through 4>dabout h to find the surface normal nA,Due to the a-ambiguity inthere was another solution for the surface normal vector n U :

[0135] A plot of nA and n U from the ground truth Rusinkiewicz angles are shown in FIGs 5A-5B. Here, nA corresponds to a convex surface, while n U is concave. When these normals are calculated with the ambiguous 4>dFrom the fit, mixing of the solutions was observed over the sphere, since each pixel was calculated independently. These ambiguous mixed solutions for nA and n U calculated from the simulated, noise-free Mueller image are shown in FIGs 21A-21B. This estimate had an ambiguous azimuth angle, but a precise zenith angle.

[0136] The AOI of the microfacet 0dwas used to calculate an imprecise depth estimate from the Mueller image and knowledge of the acquisition geometry. The distance between the camera and source positions was given byThe angle between this normalized line segment and the observed ray direction is a — arccos(The depth I is one side of the brown triangle shown in FIG. 22. Depth w as related to these known quantities using the law of sines on the middle triangle:To disambiguate nA and n U , another estimate of the surface normals is obtained using Equation1 to calculate the depth I of the object from 0d. The surface normal nA at each pixel (x, y) in the image was determined by calculating the gradient of the depth then normalizing the vector,where p is the ratio of the length of the object to the number of pixels in the image the object subtends. This estimate had an unambiguous azimuth angle, but an imprecise zenith angle.

[0137] The final step is the selection between surface normals nA or n U to disambiguate the surface normal solution. This choice is made by maximizing the dot product with nA. This computation is done using a binary' mask to choose the disambiguated normal independently at each pixel.

[0138] The depth estimate of the sphere using the ground truth 0d and the surface normals from the depth is shown in FIGs 23A-23B. Without noise perturbations, the surface normal w as estimated without ambiguity, and it closely matched the convex ground truth surface normal nA show i in FIG. 5A.

[0139] SNR values ranged from 10,000 to 10 in descending powers of 10. The Rusinkiewicz angles were estimated using Equation 9 at varying SNRs and shown in Figure 9.This least-squares optimization method often converged to local minima which were highly dependent on the initial solution. Therefore the fit was performed with four starting solutions of 4> and combines the results with the lowest residuals. The starting solutions were 0h= 45°, 0d= 18°, and f>d= [0°, 45°, 90°, 135°]. Note, the range of 0din FIG. 24 was more than an order of magnitude smaller than the other two Rusinkiewicz angles. The 0destimation was the most susceptible to noise because the gradient of the merit function with respect to 0dwas relatively small. A given noise perturbation had a larger effect on the global minima compared to the effect of noise on the other two Rusinkiewicz angles. Therefore, the SNR reduction degraded the correlation structure across the sphere more for 0dthan the other two angles. The high frequency variation that was visible inat all SNR values was due to the azimuth re-ambiguity. The optimization converged to different ambiguoussolutions independently for each pixel. The last column showed the same cb d , solution but converted to modulo 180, in order to visualize which variations were due to the noise and which were due to the ambiguity.

[0140] From Equations 13 and 14, the ambiguous normal solutions nA and U were obtained from the Rusinkiewicz estimates and shown in FIG. 25. These solutions show the azimuth re-ambiguity, but had a precise zenith angle. By visually comparing to the ground truth normals, note that some areas obtained the convex solution, while other areas obtained the concave solution. In other areas, the solutions varied pixel-wise, following the pattern in thesolution from the fit in FIG. 24. Note that these solutions exhibited low susceptibility to noise. Even at low values of SNR, slow variation of the normals was still visible in some areas of the sphere.

[0141] Depth was estimated from the same simulated Mueller image that gave the conventional ambiguous surface normal solutions in FIG. 25. The depth estimates from 0dand the normals from the gradient of the depth nA using Equation 17 are shown in FIGs 9A-9B. The normals from depth exhibited more degradation due to noise than the ambiguous normals nA and n U . However, the depth was potentially useful if the related surface normal had sufficient quality to resolve the re-ambiguity of the higher quality surface normal estimates in FIG. 25. Since 0dwas more susceptible to noise compared to the other two Rusinkiew icz angles, the depth estimate and the subsequent estimate of the normals w ere also degraded. The lower bound on 0dwas constrained in the least-squares optimization based on the approximate distance of the object, within an order of magnitude. The upper bound was constrained based on approximately what 0dcorresponded to zero depth. It was assumed that the depth varied slowly with neighboring pixels so a 9 x 9 pixel median filter was applied to the depth estimate. The normal estimate from depth had an unambiguous azimuth angle, but an imprecise zenith angle.Therefore, nA can be used to disambiguate nA and n U .

[0142] Finally, the normal solutions nA and U in FIG. 25 were disambiguated by determining which solution had the smallest angular difference from the imprecise normal estimate nA in FIGs 9A-9B. This calculation was performed at each pixel independently. These results are shown in FIG. 26 along with the percent of pixels with an angular error less than 10° from the ground truth normals. At the highest SNR, most of the sphere was accurately disambiguated, except in the center where the two solutions were the most similar. This disambiguation routine was performed on two acquisition setups with Lsrc= 90cm and Lsrc= 20cm. The effect of the noise on the MAE of each of these geometries are show n in FIG. 27. This figure demonstrates that the acquisition geometry has an effect on the SfP performance.

[0143] The SfP single-view Mueller simulation demonstrated here indicated that the azimuth ra-ambiguity of the surface normals could be disambiguated from an imprecise depth estimate with dependency on both the acquisition geometry and noise statistics. This Mueller image was described by a mixed diffuse and specular pBRDF model, and assumed prior knowledge of the camera and source positions. The simulated results, even at low SNR were comparable to the MAE from physics based methods which use an unmixed reflectance model, performed on Stokes measurements. The results of the method at high SNR were also comparable to learning based methods performed on measurements.

[0144] The computer system can include a desktop computer, a workstation computer, a laptop computer, a netbook computer, a tablet, a handheld computer (including a smartphone), a server, a supercomputer, a w earable computer (including a SmartWatchTM), or the like and can include digital electronic circuitry, firmware, hardware, memory, a computer storage medium, a computer program, a processor (including a programmed processor), an imaging apparatus, wired / wdreless communication components, or the like. The computing system may include a desktop computer with a screen, a tower, and components to connect the two. The tow er can store digital images, numerical data, text data, or any other kind of data in binary form,hexadecimal form, octal form, or any other data format in the memory' component. The data / images can also be stored in a server communicatively coupled to the computer system. The images can also be divided into a matrix of pixels, known as a bitmap that indicates a color for each pixel along the horizontal axis and the vertical axis. The pixels can include a digital value of one or more bits, defined by the bit depth. Each pixel may comprise three values, each value corresponding to a major color component (red, green, and blue). A size of each pixel in data can range from 8 bits to 24 bits. The network or a direct connection interconnects the imaging apparatus and the computer system.

[0145] The term "processor" encompasses all kinds of apparatus, devices, and machines for processing data, including by way of example a programmable microprocessor, a microcontroller comprising a microprocessor and a memory component, an embedded processor, a digital signal processor, a media processor, a computer, a system on a chip, or multiple ones, or combinations, of the foregoing. The apparatus can include special-purpose logic circuitry', e.g., an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit). Logic circuitry may comprise multiplexers, registers, arithmetic logic units (ALUs), computer memory, look-up tables, flip-flops (FF), wires, input blocks, output blocks, read-only memory, randomly accessible memory, electronically-erasable programmable read-only memory', flash memory, discrete gate or transistor logic, discrete hardware components, or any combination thereof. The apparatus also can include, in addition to hardware, code that creates an execution environment for the computer program in question, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, a cross-platform runtime environment, a virtual machine, or a combination of one or more of them. The apparatus and execution environment can realize various different computing model infrastructures, such as web services, distributed computing and grid computing infrastructures. The processor may include one or more processors of any type, such as central processing units (CPUs), graphics processing units (GPUs), special-purpose signal or image processors, field-programmable gate arrays (FPGAs), tensor processing units (TPUs), and so forth.

[0146] A computer program (also known as a program, software, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, declarative or procedural languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, object, or other unit suitable for use in a computing environment. A computer program may, but need not, correspondto a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, subprograms, or portions of code). A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communication network.

[0147] Embodiments of the subject matter and the operations described herein can be implemented in digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions, encoded on computer storage medium for execution by, or to control the operation of, a data processing apparatus.

[0148] A computer storage medium can be, or can be included in, a computer-readable storage device, a computer-readable storage substrate, a random or serial access memory array or device, or a combination of one or more of them. Moreover, while a computer storage medium is not a propagated signal, a computer storage medium can be a source or destination of computer program instructions encoded in an artificially generated propagated signal. The computer storage medium can also be, or can be included in, one or more separate physical components or media (e.g., multiple CDs, drives, or other storage devices). The operations described in this specification can be implemented as operations performed by a data processing apparatus on data stored on one or more computer-readable storage devices or received from other sources.

[0149] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, R.F, Bluetooth, storage media, computer buses, etc., or any suitable combination of the foregoing. Computer program code for carrying out operations for aspects of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C#, Ruby, or the like, conventional procedural programming languages, such as Pascal, FORTRAN, BASIC, or similar programming languages, programming languages that have both object-oriented and procedural aspects, such as the "C" programming language, C++, Python, or the like, conventionalfunctional programming languages such as Scheme, Common Lisp, Elixir, or the like, conventional scripting programming languages such as PHP, Perl, Javascript, or the like, or conventional logic programming languages such as PROLOG, ASAP, Datalog. or the like.

[0150] The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0151] The processes and logic flows described in this specification can be performed by one or more programmable processors executing one or more computer programs to perform actions by operating on input data and generating output. The processes and logic flows can also be performed by, and apparatus can also be implemented as, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit).

[0152] Processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processors of any kind of digital computer. Generally, a processor will receive instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a processor for performing actions in accordance with instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto-optical disks, or optical disks.

[0153] However, a computer need not have such devices. Moreover, a computer can be embedded in another device, e.g., a mobile telephone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a Global Positioning System (GPS) receiver, or a portable storage device (e.g., a universal serial bus (USB) flash drive), to name just a few. Devices suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor and the memory' can be supplemented by, or incorporated in, specialpurpose logic circuitry.

[0154] Computers typically include known components, such as a processor, an operating system, system memory, memory storage devices, input-output controllers, input-output devices, and display devices. It will also be understood by those of ordinary skill in the relevant art that there are many possible configurations and components of a computer and may also include cache memory, a data backup unit, and many other devices. To provide for interaction with a user, embodiments of the subject matter described in this specification can be implemented on a computer having a display device, e.g., an LCD (liquid crystal display), LED (light emitting diode) display, or OLED (organic light emitting diode) display, for displaying information to the user.

[0155] Examples of input devices include a keyboard, cursor control devices (e.g., a mouse or a trackball), a microphone, a scanner, and so forth, wherein the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be in any form of sensory feedback, e.g., visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, speech, or tactile input. Examples of output devices include a display device (e.g., a monitor or projector), speakers, a printer, a network card, and so forth. Displaydevices may include display devices that provide visual information, this information typically may be logically and / or physically organized as an array of pixels. In addition, a computer can interact with a user by sending documents to and receiving documents from a device that is used by the user; for example, by sending web pages to a web browser on a user's client device in response to requests received from the web browser.

[0156] An interface controller may also be included that may comprise any of a variety of known or future software programs for providing input and output interfaces. For example, interfaces may include what are generally referred to as “Graphical User Interfaces’" (often referred to as GUI’s) that provide one or more graphical representations to a user. Interfaces are typically enabled to accept user inputs using means of selection or input known to those of ordinary skill in the related art. In some implementations, the interface may be a touch screen that can be used to display information and receive input from a user. In the same or alternative embodiments, applications on a computer may employ an interface that includes what are referred to as “command line interfaces” (often referred to as CLI’s). CLI’s typically provide atext based interaction between an application and a user. Typically, command line interfaces present output and receive input as lines of text through display devices. For example, some implementations may include what are referred to as a “shell” such as Unix Shells known to those of ordinary skill in the related art, or Microsoft® Windows Powershell that employs object-oriented type programming architectures such as the Microsoft® .NET framework.

[0157] Those of ordinary skill in the related art will appreciate that interfaces may include one or more GUTs, CLI’s or a combination thereof. A processor may include a commercially available processor such as a Celeron, Core, or Pentium processor made by Intel Corporation®, a SPARC processor made by Sun Microsystems®, an Athlon, Sempron, Phenom, or Opteron processor made by AMD Corporation®, or it may be one of other processors that are or will become available. Some embodiments of a processor may include what is referred to as multi-core processor and / or be enabled to employ parallel processing technology in a single or multi-core configuration. For example, a multi-core architecture typically comprises two or more processor “execution cores”. In the present example, each execution core may perform as an independent processor that enables parallel execution of multiple threads. In addition, those of ordinary skill in the related field will appreciate that a processor may be configured in what is generally referred to as 32 or 64 bit architectures, or other architectural configurations now known or that may be developed in the future.

[0158] A processor typically executes an operating system, which may be, for example, a Windows type operating system from the Microsoft Corporation®; the Mac OS X operating system from Apple Computer Corp.®; a Unix® or Linux®-type operating system available from many vendors or what is referred to as an open source; another or a future operating system; or some combination thereof. An operating system interfaces with firmware and hardware in a well-known manner, and facilitates the processor in coordinating and executing the functions of various computer programs that may be written in a variety7of programming languages. An operating system, typically in cooperation with a processor, coordinates and executes functions of the other components of a computer. An operating system also provides scheduling, input-output control, file and data management, memory management, and communication control and related sendees, all in accordance with known techniques.

[0159] Connecting components may be properly termed as computer-readable media. For example, if code or data is transmitted from a website, server, or other remote source using acoaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technology such as infrared, radio, or microwave signals, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technology are included in the definition of medium. Combinations of media are also included within the scope of computer-readable media.

[0160] Although there has been shown and described the preferred embodiment of the present invention, it will be readily apparent to those skilled in the art that modifications may be made thereto which do not exceed the scope of the appended claims. Therefore, the scope of the invention is only to be limited by the following claims. In some embodiments, the figures presented in this patent application are draw n to scale, including the angles, ratios of dimensions, etc. In some embodiments, the figures are representative only and the claims are not limited by the dimensions of the figures. In some embodiments, descriptions of the inventions described herein using the phrase ‘'comprising” includes embodiments that could be described as “consisting essentially of’ or “consisting of’, and as such the written description requirement for claiming one or more embodiments of the present invention using the phrase “consisting essentially of’ or “consisting of’ is met.

[0161] The reference numbers recited in the below' claims are solely for ease of examination of this patent application, and are exemplary, and are not intended in any way to limit the scope of the claims to the particular features having the corresponding reference numbers in the drawings.

Claims

WHAT IS CLAIMED IS:

1. A computer-implemented method for disambiguating surface normals on an object (200) in a space, the method comprising: a. illuminating the object (200) in the space by a light source (110) at a known light location; b. capturing an image of the illuminated object (200) in the space by an imaging component (120) at a known imaging location; c. generating a polarimetry measurement of the illuminated object (200) based on the captured image; d. identifying, based on the polarimetry measurement, a plurality of light scattering angles over the object (200), based on the known light location and the known imaging location; e. calculating, based on the plurality of light scattering angles, the known light location, and the known imaging location, a plurality of surface normals of the object (200); f. calculating, based on the plurality of light scattering angles, a rough depth estimate of the object (200) in the space; g. calculating, based on the rough depth estimate, a depth-based surface normal; h. selecting a surface normal of the plurality of surface normals and the depth-based surface normal with a dot product greater than or equal to every other dot product of the plurality of surface normals and the depth-based surface normal; and i. deriving a depth map of the object (200) from the surface normal of the plurality of surface normals with the dot product greater than or equal to every other dot product of the plurality of surface normals and the depth-based surface normal.

2. The method of claim 1 , wherein the polarimetry measurement comprises a Mueller matrix image.

3. The method of claim 2, wherein the Mueller matrix image is derived from a mixed diffuse and specular polarization bidirectional reflectance distribution function (pBRDF) model.

4. The method of claim 1, wherein the plurality of light scattering angles comprise a plurality of Rusinkiewicz angles.

5. The method of claim 4, wherein the plurality' of Rusinkiewicz angles comprise an angle representing a deviation from specular reflection, an incident angle of light from the light source (110) onto the object (200). an azimuth angle, or a combination thereof.

6. The method of claim 1, wherein identifying, based on the polarimetry' measurement, the plurality' of light scattering angles over the object (200) comprises a least squares operation.

7. The method of claim 1, wherein selecting the surface normal of the plurality of surface normals and the depth-based surface normal with the dot product greater than or equal to every other dot product of the plurality of surface normals and the depth-based surface normal comprises using a binary mask.

8. The method of claim 1 further comprising smoothing, by a smoothing fdter. the rough depth estimate.

9. A non-transitory computing medium (134) for disambiguating surface normals on an object (200) in a space, comprising computer-readable instructions that, when executed by a processor (132). causes the processor (132) to: a. illuminate the object (200) in the space by a light source (110) at a known light location; b. capture an image of the illuminated object (200) in the space by an imaging component (120) at a known imaging location; c. generate a Mueller matrix image of the illuminated object (200) based on the captured image; d. perform a least squares operation on the Mueller matrix image to identify a plurality’ of Rusinkiewicz angles over the object (200) fitted to the Mueller matrix image, based on the known light location and the known imaging location; e. calculate, based on the plurality of Rusinkiewicz angles, the known light location, and the known imaging location, a plurality’ of surface normals of the object (200); f. calculate, based on the plurality' of Rusinkiewicz angles, a rough depth estimate of the object (200) in the space; g. calculate, based on the rough depth estimate, a depth-based surface normal; h. select a surface normal of the plurality7of surface normals and the depth-based surface normal with a dot product greater than or equal to every7other dot product of the plurality of surface normals and the depth-based surface normal; and i. derive a depth map of the object (200) from the surface normal of the plurality7of surface normals with the dot product greater than or equal to every other dot product of the plurality7of surface normals.

10. The non-transitory computing medium (134) of claim 9, wherein the Mueller matriximage is derived from a mixed diffuse and specular polarization bidirectional reflectance distribution function (pBRDF) model.

11. The non-transitory computing medium (134) of claim 9. wherein the computer-readable instructions further cause the processor (132) to smooth, by a smoothing filter, the rough depth estimate.

12. The non-transitory computing medium (134) of claim 9, wherein the plurality of Rusinkiewicz angles comprise an angle representing a deviation from specular reflection, an incident angle of light from the light source (110) onto the object (200), an azimuth angle, or a combination thereof.

13. A system (1000) for disambiguating surface normals on an object (200) in a space, the system (1000) comprising: a. a light source (110) at a known light location, configured to illuminate the object (200) in the space; b. an imaging component (120) at a known imaging location, configured to capture an image of the obj ect (200) in the space; and c. a computing system (130) operatively coupled to the light source (110) and the imaging component (120), comprising a processor (132) configured to execute computer-readable instructions and a memory component (134) operatively coupled to the processor (132), comprising computer-readable instructions for: i. generating a polarimetry measurement of the object (200) based on the captured image; ii. identifying, based on the polarimetry measurement, a plurality of light scattering angles over the object (200), based on the known light location and the known imaging location; iii. calculating, based on the plurality of light scattering angles, the known light location, and the known imaging location, a plurality of surface normals of the object (200); iv. calculating, based on the plurality7of light scattering angles, a rough depth estimate of the object (200) in the space; v. calculating a dot product for each surface normal of the plurality of surface normals, based on the rough depth estimate; vi. selecting a surface normal of the plurality of surface normals with a dot product greater than or equal to every other dot product of the plurality7ofsurface normals; and vii. deriving a depth map of the object (200) from the surface normal of the plurality- of surface normals with the dot product greater than or equal to every other dot product of the plurality of surface normals and the depth-based surface normal.

14. The system (1000) of claim 13, wherein the polarimetry measurement comprises a Mueller matrix image.

15. The system(lOOO) of claim 14. wherein the Mueller matrix image is derived from a mixed diffuse and specular polarization bidirectional reflectance distribution function (pBRDF) model.

16. The system (1000) of claim 13, wherein the plurality of light scattering angles comprise a plurality of Rusinkiewicz angles.

17. The system (1000) of claim 16, wherein the plurality of Rusinkiewicz angles comprise an angle representing a deviation from specular reflection, an incident angle of light from the light source (110) onto the object (200), an azimuth angle, or a combination thereof.

18. The system (1000) of claim 13, wherein identifying, based on the polarimetry measurement, the plurality of light scattering angles over the object (200) comprises a least squares operation.

19. The system (1000) of claim 13, wherein selecting a surface normal of the plurality of surface normals with a dot product greater than or equal to every other dot product of the plurality of surface normals comprises using a binary mask.

20. The system (1000) of claim 13, wherein the computer-readable instructions further comprise smoothing, by a smoothing filter, the rough depth estimate.

Images