Anti-spoofing face recognition through lighting and imaging

Abstract

Disclosed herein is a system and method for implementing anti-spoofing object recognition. In an embodiment, the system for object recognition includes an illumination device configured to illuminate an object, a sensor device, the sensor device receiving the illumination light reflected from the object, the illumination light including polarization information, a processor, and a memory including a program executable by the processor to calculate the polarization information from the illumination light and use the polarization information to determine whether the object is a real 3D object. It has been discovered that the polarization information can be utilized to determine whether the object is a 3D object or a flat (2D) object. Thus, the polarization information can be utilized to distinguish between the image of the 3D object and the light image of the object.

Description

[Technical field]

[0001] (CROSS REFERENCE TO RELATED APPLICATIONS) This application claims the benefit of and priority under 35 USC Section 119(e) of U.S. Provisional Patent Application No. 63 / 265,617, filed December 17, 2021, and entitled “Spoof-Resistant Facial Recognition Through Illumination and Imaging Engineering,” which is incorporated herein by reference in its entirety for all purposes.

[0002] FIELD OF THEINVENTION The present invention relates generally to anti-spoofing object recognition using polarized light. [Background technology]

[0003] (background) Metasurface elements are diffractive optical elements in which the individual waveguide elements have subwavelength spacing and a planar profile. Metasurface elements have been developed in recent years for applications in the UV-IR band (300-10,000 nm). Compared to conventional refractive optics, Metasurface elements introduce abrupt phase shifts onto the bright field. This allows Metasurface elements to have thicknesses on the order of the wavelength of light they are designed to operate with, while conventional refractive surfaces have thicknesses 10-100 times (or more) larger than the wavelength of light they are designed to operate with. In addition, Metasurface elements do not have any thickness variations in the constituent elements and are therefore capable of shaping light without any curvature, as required for refractive optics. In comparison to conventional diffractive optical elements (DOEs), e.g., binary diffractive optics, metasurface elements have the ability to impart a range of phase shifts on the incident light field; at a minimum, metasurface elements can have a phase shift of 0 to 2π, with at least five distinct values ​​from that range, whereas binary DOEs can impart only two distinct values ​​of phase shift, often limited to a phase shift of either 0 or 1π. In comparison to multilevel DOEs, metasurface elements do not require height variation of their constituent elements along the optical axis, only the in-plane geometry of the metasurface element features varies. Metasurface elements may be utilized in polarization sensing devices. Summary of the Invention [Means for solving the problem]

[0004] (Summary of disclosure) Systems and methods according to various embodiments of the present invention may include a system for object recognition, the system including an illumination device configured to illuminate an object, a sensor device that receives illumination light reflected from the object including polarization information, a processor, and a memory including programming executable by the processor to calculate the polarization information from the illumination light and use the polarization information to determine whether the object is a real 3D object.

[0005] In various other embodiments, the sensor device and / or the lighting device comprise one or more Metasurface optical elements.

[0006] In yet various other embodiments, the memory further includes programming configured to use the polarization information to determine whether the object is a real 3D object or whether the object is a photograph of an object, a mask of a 3D object, or a video of a 3D object.

[0007] In still various other embodiments, the polarization information is 1 Polarization and S 2 Polarization and S 3 At least one selected from the group consisting of polarization, degree of polarization (DoP), and azimuth angle.

[0008] In still various other embodiments, S 1 Polarized, S 2 Polarized, S 3 Polarization is

number

[0009] In still various other embodiments, DoP is

number

[0010] In still various other embodiments, the memory further includes programming executable by the processor to compare the original optical image of the object with the subsequent optical image of the object to perform object recognition.

[0011] In still various other embodiments, the original optical image and the subsequent optical image are 2D near-infrared images.

[0012] In still various other embodiments, the original optical image and the subsequent optical image are polarized images.

[0013] In yet various other embodiments, the memory further includes programming executable by the processor to determine whether a portion of the original optical image and / or the subsequent optical image is not real, and comparison of the original optical image and the subsequent optical image of the object is not performed for a portion of the original optical image and / or the subsequent optical image that is determined to be not real.

[0014] In still various other embodiments, the portions of the original optical image and / or subsequent optical image that are not real are of a mask, sunglasses, and / or a beard.

[0015] In still various other embodiments, determining whether portions of the original optical image and / or the subsequent optical image are not real is performed based on polarization information from the illumination light.

[0016] In still various other embodiments, the object includes a face.

[0017] In still various other embodiments, the memory further includes programming configured to implement an object detection algorithm.

[0018] In still various other embodiments, the memory further includes programming configured to utilize results from an object detection algorithm to separate object pixels from non-object pixels.

[0019] In still various other embodiments, the sensor device further receives an initial image and a subsequent image, and the memory further includes programming configured to compare the initial image and the subsequent image and determine whether the initial image and the subsequent image are substantially identical.

[0020] In still various other embodiments, the comparing of the initial image to the subsequent image is performed only on object pixels of the subsequent image.

[0021] In still various other embodiments, the illumination source includes an LED or VCSEL array.

[0022] In still various other embodiments, the illumination source further includes sunlight.

[0023] In still various other embodiments, the system further includes an ambient light sensor configured to determine an amount of sunlight, and the memory further includes programming executable by the processor to modify the amount of light output by the LED or VCSEL array based on the amount of sunlight.

[0024] In still various other embodiments, the system further includes a bandpass filter configured to pass certain wavelengths of light from sunlight.

[0025] In still various other embodiments, the illumination source includes sunlight.

[0026] Additionally, systems and methods according to various embodiments of the present invention may include a method for verifying object recognition, the method including illuminating an object with a light source, sensing light reflected from the object with respect to polarization information, and using the polarization information to determine whether the object is a real 3D object.

[0027] In various other embodiments, illuminating the object is performed by an illumination device and sensing light reflected from the object is performed by a sensor device, and the sensor device and / or the illumination device include one or more Metasurface elements.

[0028] In yet various other embodiments, the method further includes using the polarization information to determine whether the object is a real 3D object or whether the object is a photograph of an object, a mask of a 3D object, or a video of a 3D object.

[0029] In still various other embodiments, the polarization information is 1 Polarization and S 2 Polarization and S 3 At least one selected from the group consisting of polarization, degree of polarization (DoP), and azimuth angle.

[0030] In still various other embodiments, S 1 Polarized, S 2 Polarized, S 3 Polarization is

number

[0031] In still various other embodiments, DoP is

number

[0032] In still various other embodiments, the method further includes comparing the original optical image of the object with the subsequent optical image of the object to perform object recognition.

[0033] In still various other embodiments, comparing the original optical image to the subsequent optical image is performed prior to determining whether the object is a real 3D object.

[0034] In still various other embodiments, comparing the original optical image with the subsequent optical image is performed after determining whether the object is a real 3D object.

[0035] In still various other embodiments, the method further includes determining whether a portion of the original optical image and / or the subsequent optical image is not real, and the comparison of the original optical image and the subsequent optical image of the object is not performed for a portion of the original optical image and / or the subsequent optical image that is determined to be not real.

[0036] In still various other embodiments, the portions of the original optical image and / or subsequent optical image that are not real are of a mask, sunglasses, and / or a beard.

[0037] In still various other embodiments, determining whether portions of the original optical image and / or the subsequent optical image are not real is performed based on polarization information from the illumination light.

[0038] In still various other embodiments, the object includes a face.

[0039] In still various other embodiments, the method further includes implementing an object detection algorithm.

[0040] In still various other embodiments, the method further includes utilizing results from an object detection algorithm to separate object pixels from non-object pixels.

[0041] In still various other embodiments, the method further includes receiving an initial image and a subsequent image, and comparing the initial image and the subsequent image to determine whether the initial image and the subsequent image are substantially identical.

[0042] In still various other embodiments, the comparing of the initial image to the subsequent image is performed only on object pixels of the subsequent image.

[0043] In still various other embodiments, the light source includes an LED or VCSEL array.

[0044] In still various other embodiments, the light source further comprises sunlight.

[0045] In still various other embodiments, the method further includes determining an amount of sunlight and modifying the amount of light output by the LED or VCSEL array based on the amount of sunlight.

[0046] In still various other embodiments, the method further includes filtering out certain wavelengths of light from the sunlight.

[0047] In still various other embodiments, the light source includes sunlight. [Brief description of the drawings]

[0048] The present description will be more fully understood with reference to the following figures, which are presented as exemplary embodiments of the invention and should not be construed as a complete recitation of the scope of the invention.

[0049] [Figure 1A] 1A and 1B are various flow charts illustrating a method for utilizing the polarization state of reflected light to verify face recognition, according to an embodiment of the present invention. [Figure 1B] 1A and 1B are various flow charts illustrating a method for utilizing the polarization state of reflected light to verify face recognition, according to an embodiment of the present invention.

[0050] [Figure 2A] FIG. 2A illustrates a method of object recognition utilizing polarization states, including preliminary object detection, according to one embodiment of the present invention.

[0051] [Figure 2B] FIG. 2B illustrates a method of object recognition utilizing polarization states, including preliminary object detection, according to one embodiment of the present invention.

[0052] [Diagram 3]FIG. 3 is a block diagram of a computing system for verifying facial recognition according to one embodiment of the invention.

[0053] [Figure 4A] FIG. 4A illustrates an exemplary polarized image capture module, according to one embodiment of the present invention.

[0054] [Figure 4B] FIG. 4B shows various SEM images including different polarizing metasurfaces according to an embodiment of the present invention.

[0055] [Diagram 5] FIG. 5 illustrates a comparison between a real 2D image of an individual and a photographic 2D image of the individual, according to an embodiment of the present invention.

[0056] [Figure 6] FIG. 6 illustrates a comparison between a real 2D image of an individual, a 2D image of a photograph of the individual, and a 2D image of a 3D mask of the individual. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0057] (Detailed Description) In many embodiments, the two-dimensional facial recognition algorithm involves taking a first 2D optical image of an individual that is verified to be a real person, and then comparing this 2D optical image with subsequent optical images of the individual for key features from the original (e.g., distance between the eyes, shape of the mouth, etc.). The comparison may be performed through machine learning algorithms. If the two images are substantially identical, the system will verify the authenticity of the individual and, in the case of a security system, provide the individual access. These facial recognition algorithms may be implemented within smart devices such as smartphones. Because these systems may only work on 2D information, such systems are susceptible to being fooled or spoofed through various techniques, such as a non-authentic individual holding up a high-resolution printout of the authentic individual, or playing a video of the authentic individual, in which case the 2D system also searches for the person's movements to verify identity.

[0058] In contrast, 3D authentication systems, which include complementary techniques such as structured light or time-of-flight in addition to 2D images, aim to verify using additional information such as the depth of an individual's features. In such cases, the system may be robust against attacks using simple 2D similarities of the authentic individual. These 3D systems may still be susceptible to high-performance attacks using masks with similarities of the authentic individual. These 3D systems may also add substantial complexity, cost, and size compared to their 2D counterparts. In some embodiments, 3D systems may be added in addition to 2D imaging systems.

[0059] Various embodiments of the present disclosure include systems that utilize only 2D information but remain robust against attacks that trick or spoof the simple 2D face recognition systems typically described above. Some embodiments of the presently disclosed systems use imaging and illumination systems that are capable of completely and uniquely identifying the polarization state of reflected light from a scene. By analyzing only 2D images and / or the corresponding polarization state parameters of the images, the system can be robust against spoofing such as 2D light images, videos, and 3D masks. In general, the disclosed invention operates as described through the flow charts in the figures described below.

[0060] 1A and 1B are various flow charts illustrating methods of utilizing polarization state parameters of reflected light to verify face recognition, according to embodiments of the present invention. These methods may be performed by a system that includes a polarization camera, and in some embodiments, utilizes a metasurface and produces a polarization state image.

[0061] FIG. 1A is a flow chart of a method 100a of object recognition utilizing polarization states according to an embodiment of the present invention. In FIG. 1A, the method 100a includes capturing a first image (e.g., an initial image) including polarization of a scene, face, or object (102). In some embodiments, the polarization of the first image may be utilized similar to block 112 to determine whether the first image is “real.” If the first image is not determined to be “real,” the method may prompt the user to capture another first image. The method 100a further includes capturing a subsequent image (104) of the same scene, face, or object at a later time. The first image and / or subsequent image may be a set of images that are closely spaced in time, e.g., the same scene or object, with one or more images captured less than 1 second, less than 5 seconds, or less than 10 seconds apart. In some embodiments, the first image and / or subsequent image are captured closely followed by a polarization image or multiple snapshots of the same image that are closely spaced in time. In some embodiments, the first image and the subsequent image may be from a 2D near infrared (NIR) light / camera image in conjunction with a separate polar image (e.g., for intensity, etc.). However, in some embodiments, the first image and the subsequent image may be a polarized image (e.g., without a 2D NIR image). Thus, the subsequent image may be utilized in block 112 to determine whether the scene, face, or object is a 3D scene, face, or object without taking another polarized image. In a subsequent step, the polarized images may be compared to verify the authenticity of the object. Method 100a further includes comparing (106) the first image and the subsequent image to determine whether they are substantially identical. In some embodiments, the comparison involves a 2D image or polarized image comparison when utilizing a polarized image for the first image and the subsequent image. In some embodiments, the first image may be a first polarized image, and a subsequent image may also be a polarized image, and if they are compared and found to be identical, another polarized image may be utilized to determine whether the first image and the subsequent image are of a real 3D object.If the images are not identical, the system denies the user access (110). If the images are identical, the system then examines the captured polarization information to ensure that the object is a real object and not a 2D light image, a 2D video, a 2D mask, and / or a person wearing a mask (112). This may include assessing texture, depth, and / or other "real" parameters, and / or relative intensity of pixels or portions of the image in the polarization image. In some embodiments, examining the captured polarization information to ensure that the object is a real object (112) includes analyzing parameters, metrics, and / or benchmarks to determine that the image is a "real" 3D image. For example, the polarization information and / or the 2D image may be analyzed to determine if the object has the correct texture or polarization parameters. If the polarization information confirms that the object is a 3D object, access is granted. In some embodiments, the object may be a face or a human. Method 100a may perform facial recognition.

[0062] FIG. 1B is a flow chart of a method 100b of object recognition using polarization states, according to one embodiment of the present invention. FIG. 1B shares many of the steps of FIG. 1A. The description of these steps is applicable to the method of FIG. 1B and will not be repeated in detail. In FIG. 1B, the method 100b includes first inspecting the polarization information to figure out whether the image is of a real 3D object (112) before proceeding to compare the two images (106).

[0063] In some embodiments, the method of object recognition may include preliminary object detection prior to comparing the two images. The object detection may crop out pixels that are not part of the object, which may make the remaining steps more accurate and reduce processing time.

[0064] FIG. 2A illustrates a method 250a of object recognition using polarization states, including preliminary object detection, according to an embodiment of the present invention. FIG. 2A shares many of the steps of FIGS. 1A and 1B. The description of these steps is applicable to the method of FIG. 2A and will not be repeated in detail. In FIG. 2A, the method 250a includes detecting an object in a subsequent image (252) before performing a comparison (106) of the first and second images. The object detection may be a face detection algorithm. An example of an object detection algorithm is described in Liu, Wei, et al., "Ssd: Single shot multibox detector," European conference on computer vision. Springer, Cham (2016), which is incorporated herein by reference in its entirety.

[0065] FIG. 2B illustrates a method 250b of object recognition utilizing polarization states, including preliminary object detection, according to an embodiment of the present invention. FIG. 2B shares many of the steps of FIGS. 1A and 1B. The description of these steps is applicable to the method of FIG. 2B and will not be repeated in detail. FIG. 2B is similar to FIG. 2A, except that before proceeding to comparing the two images (106), the polarization information is examined to figure out whether the image is of a real 3D object (112). Although object detection (252) is illustrated and described before the image comparison (106) and the polarization information examination (112), in some embodiments, object detection (252) may be performed after these two steps as well. Object detection (252) may be used to separate pixels that belong to the object from pixels that do not belong to the object (e.g., non-object pixels), which may make the rest of the steps more accurate and reduce processing time.

[0066] FIG. 3 is a block diagram of a computing system for verifying face recognition according to an embodiment of the present invention. The system 200 may be a separate computing system or a system implemented on a device that captures 2D images, such as a smartphone. The system 200 includes an input / output 204 that is capable of receiving a 2D image including polarized light. The system 200 further includes a processor 202 and a memory 206. The memory 206 includes programming, including a 2D image comparator 208 and a polarization analyzer 210, executable by the processor. The image comparator 208 may compare an initial 2D image with a later captured 2D image, as discussed above. The polarization analyzer 210 may analyze the polarization of the captured polarized light of the later captured 2D image to determine whether the polarization signature is consistent with a 3D object. The image comparator 208 and the polarization analyzer 210 may be used to verify face recognition, as discussed in connection with FIGS. 1A, 1B, 2A, 2B. The input / output 204 may output a result of the verification. The memory 206 may further include programming, including an object recognizer (not shown), configured to perform the object recognition step (252) of FIGS. 2A and 2B.

[0067] In an embodiment, the imaging system includes both an active illumination source and an imaging sensor or camera. The illumination source may be an illumination device, and the imaging sensor or camera may be a sensor device. FIG. 4A illustrates an exemplary polarization image capture module according to an embodiment of the present invention. The module includes an active illumination source and a camera that can resolve polarization. In an embodiment, the camera may resolve the full Stokes vector in a single frame. In some embodiments, the camera includes one or more metasurfaces configured to produce one or more polarization images with different polarization states. The illumination source may be a light source, such as an LED, a VCSEL array, or any illumination source that provides a known illumination condition that can be turned on and off.

[0068] The polarized image capture module of FIG. 4A includes an active illumination source, but the illumination source may also be sunlight, ambient light, or sunlight or ambient light supplemented with a light source such as an LED or VCSEL array. The sunlight or ambient light may have random polarization states therein and also include a range of wavelengths (e.g., 940 nm and / or typical IR wavelengths). An ambient light sensor may be utilized to turn the light source on or off or modify the intensity or pattern of the light source depending on the sunlight / ambient conditions (e.g., the amount of sunlight present). For example, the device may use only sunlight in some conditions or both a light source and sunlight in some conditions. The device may modify the amount of light from the light source depending on the sunlight conditions. A bandpass filter may be included to filter the wavelengths of sunlight and pass only one wavelength or a narrow band of wavelengths.

[0069] The illumination source can be either preferentially polarized or unpolarized. In some embodiments, the light source can be unpolarized or have a variable polarization, while in other embodiments, the light source can have a fixed polarization state. For example, the light source can be a VCSEL array with a fixed VCSEL polarization. The fixed VCSEL polarization can be achieved through a patterned metasurface aperture on the VCSEL array to achieve a uniform polarization state out of the VCSEL array. In some embodiments, a light source with a fixed polarization state can provide better results than a light source without polarization. The fixed polarization state can be a linear, circular, and / or elliptical polarization of light, or any combination of these polarizations of light.

[0070] In some embodiments, the illumination source may switch between two or more polarizations in a time-resolved manner or when it emits multiple different polarizations within the field of interest. For example, either alternating polarization states may be presented at different times or two polar states may be presented simultaneously. The two polarization states may be orthogonal or non-orthogonal to each other and they may be either two or more of linear, circular, or elliptical states. The polarization may be presented in different patterns of polarization states, such as, for example, flood, dotted, batwing, top hat, super-Gaussian, or other illumination patterns. In some embodiments, one or more metasurfaces may be used to produce various polarized illumination patterns, either simultaneously or alternating, and / or overlapping or physically separated, and / or in different patterns.

[0071] In some embodiments, the illumination source and / or the camera may include a metasurface. Examples of illumination sources and cameras including metasurfaces are described in U.S. Patent Publication No. 2019 / 0064532, filed August 31, 2018, entitled "TRANSMISSIVE METASURFACE LENS INTEGRATION," which is incorporated by reference in its entirety for all purposes. As described in this patent publication, the illumination source and the camera may be functionally included within a single unit. FIG. 4A illustrates one implementation of an illumination source and a camera integrated into a single unit.

[0072] In some embodiments, the imaging system may be any imaging system capable of recovering full polarization information. In a more specific case, however, the imaging system may include one or more Metasurface optical elements, a standard refractive lens, and a standard CMOS image sensor. The one or more Metasurface optical elements may split the scene into two or more polarization states and form two or more partial images on the CMOS sensor, which, when appropriately computationally recombined, can provide the polarization state of the object being imaged. The imaging system may be a full Stokes polarization camera. One example of a full-Stokes polarization camera incorporating one or more metasurface optical elements is discussed in Rubin NA, D'Aversa G, Chevalier P, Shi Z, Chen WT, Capasso F. Matrix Fourier optics enables a compact full-Stokes polarization camera. Science (2019 Jul 5);365(6448):eaax1839. doi: 10.1126 / science.aax1839. PMID: 31273096, which is incorporated by reference in its entirety.

[0073] 4B are various SEM images of an example of a polarization metasurface that may be incorporated into the polarization image capture module of FIG. 4A. Further examples of polarization metasurfaces are illustrated in U.S. Patent Publication No. 20210286188, filed February 22, 2019, entitled "Arbitrary polarization-switchable metasurfaces," which is incorporated herein by reference in its entirety. The polarization metasurface may include silicon pillars of one or more shapes, including rectangular, square, and / or elliptical in various configurations. The polarization metasurface may be in an asymmetric configuration.

[0074] 5 illustrates a comparison between a real 2D image of an individual and a photographic 2D image of the individual, according to an embodiment of the present invention. The real 2D image and the photographic 2D image may be captured using a polarized imaging system, as described in connection with FIG. 4A. 0 Column 502 is the unfiltered black and white image. 1 Columns 504 and S 2 Column 506 corresponds to different Stokes parameters. In some embodiments, the polarization imaging system may also be used to verify face recognition, S 3 In some embodiments, the S 1 Polarization parameter, S 2 Polarization parameters, and S 3 The polarization parameters, in combination, may be used to assess whether an object is a 2D or 3D object. The Stokes parameters are:

number

number

number

[0075] In some embodiments, the degree of polarization (DoP) can be used to assess whether an object is a 2D or 3D object. The DoP column 508 in FIG. 3 corresponds to the portion of each portion of the image that is polarized. The DoP can be calculated through the following formula:

number

[0076] The azimuth array 510 is made up of different polarized light S 1 , S 2 , and S 3 For example, S 1 and S. 2 The azimuth angle between may be calculated using the following equation:

number

[0077] In some embodiments, linear polarization, including two orthogonal polarizations, may be used to verify facial recognition or to identify fake objects or images, i.e., objects or images that are intended to be spoofed. For example, S 1 The image may be an image filtered for P polarization, and S 2The image may be an image filtered for S-polarized light. Non-orthogonal polarizations may also be used. Different polarization states and their polarization parameters may also be used to disambiguate or distinguish between real or natural 3D objects and fake, i.e., spoofed, objects, and / or to recognize or match faces or objects at two different times. Different polarization states and their polarization parameters may also be used to determine the exact similarity of faces and / or skin texture or naturalness or surface texture. Any one or more of the different polarization state parameters may be used individually or in combination. In some embodiments, these different polarization state parameters may be used in conjunction with algorithms or machine learning to determine the identity of images and / or faces.

[0078] Returning to FIG. 5, the top image is of a real 2D image of an individual, including standard monochrome intensities. In comparison, the bottom image is of a light image of an individual. As shown, S 1 , S 2 The contrast between each of the S, DoP, and / or azimuth angles may be used to determine whether the image is an image of a real 3D shape corresponding to a real individual, or an image of a light image of the individual, which may have less contrast. 1 , S 2 A combination of the DoP and / or azimuth angle may be used to determine whether an image is an image of a real 3D shape corresponding to a real individual or an image of a light image of the individual. In some embodiments, the real 2D image of the individual is 1 , S 2 , DoP, and / or azimuth, respectively, may contain more contrast between high and low polarizations than a 2D image of a light portrait of the individual. In some embodiments, a machine learning algorithm, such as a neural network, may be used in conjunction with training data to determine whether an image is an image of a real 3D shape corresponding to a real individual or an image of a light portrait of the individual. As shown, a monochrome image S0 does not, by itself, reveal enough information to determine whether a 2D image is of a real 3D individual or a light image of an individual. In some embodiments, the relative intensities or other metrics of the various polarization state images may be compared on a pixel-by-pixel or region-by-region basis as part of the comparison process, and / or image processing or machine learning algorithms may be applied.

[0079] FIG. 6 illustrates a comparison between a real 2D image of an individual, a 2D image of a photograph of the individual, and a 2D image of a 3D mask of the individual. 0 , S 1 , S 2 , and S 3 The columns are described above in connection with FIG. 0 Column 502, S 1 Column 504, and S 2 The column is S in Figure 6. 0 Column 602, S 1 Column 604, and S 2 Column 606. FIG. 3 Corresponding to the polarization measurement, S 3 As shown, S 3 The polarization measurement is S 1 and S 2 The polarization measurement may be different from the polarization measurement.

[0080] The top row of images 610 corresponds to a real 2D image of the individual. The middle row of images 612 corresponds to a 2D image of a photograph of the individual. The bottom row of images 614 corresponds to a 2D image of a mask of the individual. As shown, S 1 , S 2 , and / or S 3 The contrast between each of S may be used to determine whether the image is an image of a real 3D shape corresponding to a real individual, or an image of a light image of the individual, or an image of a mask of the individual, which may have less contrast. 1 , S 2 , and / or S 3A combination of these may be used to determine whether an image is an image of a real 3D shape corresponding to a real individual, or an image of a light image of an individual, or a mask of an individual. 0 alone does not reveal enough information to determine whether a 2D image is of a real 3D individual, a photo-portrait of an individual, or a mask of an individual.

[0081] In some embodiments, parts of an image are sensed as being "unreal" or "unnatural." For example, if a person has a mask (e.g., surgical mask, N95 mask, KN-95 mask, dust mask, cosmetic mask), sunglasses, beard, or the like, the system may recognize these features based on the polarization state parameters of that region of the face image or images, especially compared to a conventional image without such added features. For example, the texture of the face portion covered by the beard or surgical mask may appear altered or "not skin-like" based on the difference in polarization parameters, either compared to a benchmark or compared to a conventional image or images of the face without the added features. In some embodiments, the texture aspects sensed through the polarization state parameters of one or more polarized images may be used to identify a feature (e.g., beard, glasses, mask, etc.) and / or to determine identity or match with respect to a subsequent comparison against a similar set of images, and / or to remove the identified feature from the subsequent comparison. For example, if a first set of images of a human face without a surgical mask is taken using the systems and processes described above, and then the same human face is similarly imaged, but now with a surgical mask, comparison and analysis of the two sets of images may include identifying the surgical mask either through comparison and / or texture variation. In some cases where the first set of images has the added features of a beard or mask and the second set of features does not have the added features, the systems and processes described herein may be used to distinguish the added features. In some cases, the analysis or comparison may continue to only compare the two sets of images in areas without the added mask to determine a match or identity.Although the examples herein of facial features and features added to the face (masks, beards, glasses, etc.) are described as examples, it should be understood that the same concepts, systems, and processes may be applied to other body parts or inanimate objects to identify features and / or to make comparisons of portions that remain unaltered or uncovered by the added features.

[0082] (The Doctrine of Equivalents) Although the above description contains many specific embodiments of the present invention, these should not be interpreted as limitations on the scope of the present invention, but rather as one example of one embodiment thereof. It is therefore to be understood that the present invention may be practiced in ways other than as specifically described without departing from the scope and spirit of the present invention. The present embodiments should therefore be considered in all respects as illustrative and not restrictive. Thus, the scope of the present invention should be determined not by the embodiments exemplified, but by the appended claims and their equivalents.

Claims

1. 1. A system for object recognition, comprising: an illumination device configured to illuminate an object; a sensor device that receives illumination light reflected from the object, the illumination light including polarization information; A processor; Including programming, memory and Equipped with The programming, by the processor, calculating the polarization information from the illumination light; using said polarization information to determine whether said object is a real 3D object; The system is operable to:

2. The system of claim 1 , wherein the sensor device and / or the lighting device comprises one or more Metasurface optical elements.

3. 10. The system of claim 1, wherein the memory further comprises programming configured to use the polarization information to determine whether the object is a real 3D object or whether the object is a photograph of an object, a mask of a 3D object, or a video of a 3D object.

4. The polarization information is S １ Polarization and S ２ Polarization and S ３ The system of claim 1 , comprising at least one selected from the group consisting of polarization, degree of polarization (DoP), and azimuth angle.

5. The S １ Polarized light, S ２ Polarized light, S ３ Polarization is ##EQU00011## and said S ０ The system of claim 4 , wherein the polarized light corresponds to an original monochrome image.

6. The DoP is ##EQU00012## The system of claim 5 , wherein the calculation is performed by:

7. The system of claim 1 , wherein the memory further comprises programming executable by the processor to compare an original optical image of the object with a subsequent optical image of the object to perform object recognition.

8. The system of claim 7 , wherein the original optical image and the subsequent optical image are 2D near-infrared images.

9. The system of claim 7 , wherein the original optical image and the subsequent optical image are polarized images.

10. 8. The system of claim 7, wherein the memory further comprises programming executable by the processor to determine whether a portion of the original optical image and / or the subsequent optical image is not real, and wherein comparing the original optical image and the subsequent optical image of the object is not performed for the portion of the original optical image and / or the subsequent optical image that is determined to be not real.

11. The system of claim 10 , wherein the portion of the original optical image and / or the subsequent optical image that is not real is of a mask, sunglasses, and / or a beard.

12. The system of claim 7 , wherein determining whether a portion of the original optical image and / or the subsequent optical image is not real is performed based on the polarization information from the illumination light.

13. The system of claim 1 , wherein the object comprises a face.

14. The system of claim 1 , wherein the memory further comprises programming configured to implement an object detection algorithm.

15. 15. The system of claim 14, wherein the memory further comprises programming configured to utilize results from the object detection algorithm to separate object pixels from non-object pixels.

16. 16. The system of claim 15, wherein the sensor device further receives an initial image and a subsequent image, and the memory further includes programming configured to compare the initial image and the subsequent image and determine whether the initial image and the subsequent image are substantially identical.

17. The system of claim 16 , wherein comparing the initial image and the subsequent image is performed only on the object pixels of the subsequent image.

18. The system of claim 1 , wherein the lighting device comprises an LED or VCSEL array.

19. The system of claim 18 , wherein the lighting device further comprises sunlight.

20. 20. The system of claim 19, further comprising an ambient light sensor configured to determine the amount of sunlight, the memory further comprising programming executable by the processor to modify an amount of light output by the LED or VCSEL array based on the amount of sunlight.

21. 20. The system of claim 19, further comprising a bandpass filter configured to pass certain wavelengths of light from the sunlight.

22. The system of claim 1 , wherein the lighting device comprises sunlight.

Images