Multiple lighting conditions for biometric authentication

Abstract

The invention relates to various lighting conditions for biometric authentication. Methods and apparatus are disclosed for illuminating a biometric aspect, such as a periorbital region feature or an iris feature, with multiple illumination configurations to create multiple illumination channels for computer analysis. Multiple lighting channels may be used for quality assurance by comparing the actual lighting channel properties of a given image to the lighting channels activated for image capture. By providing additional information for comparison with a storage template generated from images taken at the plurality of lighting channels during registration, images captured at the plurality of lighting channels may also be used in biometric authentication. Biometric authentication using a quality assurance process may resist attempts to bypass biometric authentication.

Description

[0001] This patent application claims the benefit of priority to U.S. Provisional Application Serial No. 63 / 585,681, filed on September 27, 2023, entitled “Multiple Illumination Conditions for Biometric Authentication,” and the entire text of that application is incorporated herein by reference. Technical Field

[0002] The present disclosure relates to various lighting conditions for biometric authentication. Background Art

[0003] Biometric authentication techniques use one or more biometric aspects of a person to identify that person, such as for secure, authenticated access to a device, system, etc. Typically, during an enrollment process, one or more images of the biometric aspects to be tracked are captured (e.g., an image of a person's iris may be captured) and processed to generate a set of indicators that are unique to that person, thereby uniquely identifying that person. At a later time, when the person attempts to access the device, system, etc., images of the person's biometric aspects are again captured and processed using algorithms similar to those used during enrollment. The indicators determined from the data captured at the later time are compared to the stored indicators from enrollment, and if the match between the two is good enough, the person is authenticated. Summary of the invention

[0004] A biometric authentication system that uses multiple illumination channels to capture images of biometric aspects may be able to derive more information than a biometric authentication system that does not use multiple illumination channels. For example, the biometric authentication system may use multiple illumination channels in a quality assurance algorithm to identify situations where a non-authentic representation of a biometric aspect is being presented instead of an authentic biometric aspect, such as a printout of a picture of an eye is being presented instead of an image of a real person interacting with the system to capture a person's eye. In addition, multiple illumination channels may provide an additional stream of information that may be used to improve the robustness of the biometric authentication process. For example, images captured under different illumination conditions may be considered images of "different illumination channels." In this way, not only may a biometric aspect such as an iris match be evaluated for matching, but the biometric authentication process may also evaluate matching properties of several versions of an iris image captured using different illumination channels. This may improve the security of the biometric authentication process. In addition, the biometric authentication process may track or otherwise be aware of the order in which the illumination channels were applied, and the biometric authentication process may not only verify an iris match, but may also verify a match of an iris image illumination channel to the illumination channel applied to a given image capture. Furthermore, as described above, using multiple illumination channels in quality assurance algorithms can save computer resources by bypassing further processing of images with mismatched illumination channel properties.

[0005] In some embodiments, a machine learning-assisted quality assurance algorithm can be used to further reduce the computer resources consumed in identifying matched and unmatched illumination channel properties. For example, a machine learning model can be trained to generate inferences about the illumination channel properties of a given image, which can then be compared to the known illumination channel that was applied when the given image was captured.

[0006] In some embodiments, multiple illumination channels may be used to evaluate multiple types of biometric aspects, such as biometric aspects in the periocular region and / or in the eye (such as the iris). For example, images of the periocular region captured using multiple illumination channels may provide information about the three-dimensional (3D) structure of the periocular region. Other biometric aspects may include the face, ears, and hands. A biometric authentication system may use such information about the 3D structure of the biometric aspects to perform authentication, or such information may be used in conjunction with an iris-based biometric authentication process or other type of authentication system. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1A is a front view of an illumination source having multiple independent light emitting elements according to some embodiments.

[0008] Figure 1Bis a side view of an illumination source and controller, camera, and biometric aspects of using an illumination configuration to generate an illumination channel according to some embodiments.

[0009] Figure 1C According to some embodiments, the Figure 1B Side view of the illumination sources of different illumination configurations as well as the controller, camera and biometric features used in the illumination configurations.

[0010] Figure 1D is a graphical representation of three lighting configurations and three lighting channels that may be generated by the corresponding lighting configurations according to some embodiments.

[0011] Figure 2A is a block diagram illustrating components of a multi-illumination channel input capture system that may be implemented by a controller according to some embodiments for capturing images for evaluation by a multi-illumination channel analysis system (such as with respect to the illumination channels applied when the images were captured).

[0012] Figure 2B is a block diagram illustrating components of an example multi-illumination channel analysis system that may be implemented by a controller according to some embodiments.

[0013] Figure 3 is a block diagram illustrating a remote training system that trains a machine learning model for use in a quality assurance system and a set of devices that use the trained machine learning model to evaluate lighting channels according to some embodiments.

[0014] Figure 4 is a process flow diagram illustrating an example illumination channel consistency determination process according to some embodiments.

[0015] Figure 5 is a process flow diagram illustrating a possible biometric authentication process using a 3D structure in conjunction with iris-based biometric authentication according to some embodiments.

[0016] FIG. 6A to FIG. 6D Examples of information about 3D structure and other biometric surface information that may be used by a multi-illumination channel analysis system are shown according to some embodiments.

[0017] 7A to 7C An example device that may be used to implement a multi-illumination channel analysis system according to some embodiments is shown.

[0018] Figure 8 is a block diagram illustrating an example computing device that may be used in accordance with some embodiments.

[0019] This specification includes references to "one embodiment" or "an embodiment." The appearance of the phrase "in one embodiment" or "in an embodiment" does not necessarily refer to the same embodiment. The particular features, structures or characteristics may be combined in any suitable manner consistent with the present disclosure.

[0020] The term "comprising" is open ended. As used in the claims, the term does not exclude additional structures or steps. Consider the following cited claim: "an apparatus comprising one or more processor units..." Such a claim does not exclude the apparatus from including additional components (e.g., a network interface unit, a graphics circuit, etc.).

[0021] "Configured to", various units, circuits or other components may be described or recited as "configured to" perform one or more tasks. In such contexts, "configured to" is used to imply a structure (e.g., a circuit) that performs the one or more tasks during operation by indicating that the unit / circuit / component includes the structure. In this way, the unit / circuit / component is said to be configured to perform the task even when the specified unit / circuit / component is currently inoperable (e.g., not turned on). The units / circuits / components used with the "configured to" language include hardware-such as circuits, memories storing executable program instructions to implement operations, etc. Reference to a unit / circuit / component "configured to" perform one or more tasks is explicitly intended not to invoke the sixth paragraph of 35 USC § 112 for the unit / circuit / component. In addition, "configured to" may include a general structure (e.g., a general circuit) manipulated by software or firmware (e.g., an FPGA or a general processor executing software) to operate in a manner capable of performing one or more tasks to be solved. "Configured to" may also include adapting a manufacturing process (eg, a semiconductor fabrication facility) to produce a device (eg, an integrated circuit) suitable for implementing or performing one or more tasks.

[0022] "First," "Second," etc. As used herein, these terms act as labels for the nouns that precede them, and do not imply any type of ordering (e.g., spatial, temporal, logical, etc.). For example, a buffer circuit may be described herein as performing a write operation of a "first" value and a "second" value. The terms "first" and "second" do not necessarily imply that the first value must be written before the second value.

[0023] "Based on" or "depending on", as used herein, these terms are used to describe one or more factors that influence a determination. These terms do not exclude additional factors that may influence the determination. That is, the determination may be based solely on these factors or at least in part on these factors. Consider the phrase "A is determined based on B." In this case, B is a factor that influences the determination of A, and such phrases do not exclude that the determination of A may also be based on C. In other examples, A may be determined based solely on B.

[0024] "Or", when used in the claims, the term "or" is used as an inclusive or rather than an exclusive or. For example, the phrase "at least one of x, y or z" means any one of x, y and z and any combination thereof. DETAILED DESCRIPTION

[0025] Various embodiments of systems and methods for capturing and using multiple illumination channel images for quality assurance and biometric authentication are described. An illumination channel may correspond to a configuration of an illumination element used to capture an image of a biometric aspect, such as an iris of an eye or a periocular region around an eye. Such a configuration of an illumination element may vary across multiple image captures to produce a set of multiple illumination channel images of the same biometric aspect. For example, images of the same iris may be captured under a set of varying illumination configurations to generate a set of multiple illumination channel images of the iris. Quality assurance using multiple illumination channels may identify attempts to satisfy biometric authentication without actually presenting the actual biometric aspect of a user, such as the periocular region of a registered user or the iris of a registered user. Furthermore, images captured using different illumination configurations include different illumination channels, and thus images captured using different illumination configurations provide a more robust data set about a given biometric aspect than images captured using the same illumination configuration. A biometric authentication system may use the additional information generated by multiple illumination channels to make the biometric authentication process more robust, such as by avoiding false positives and false negatives. A biometric authentication system may obtain information about the three-dimensional (3D) structure of the periocular region by capturing images using different illumination configurations. Information about the 3D structure and other characteristics of the periocular region captured using different illumination channels may be used to perform biometric authentication. The generated biometric template may be an enrollment template, and the enrollment template may be stored so that a processor may subsequently compare the enrollment template with subsequently received biometric input, which may be processed into a matching template associated with an authentication request. Biometric authentication may include biometric identification of a user, for example, identifying a registered user from a group of registered users based on a comparison of the matching template with an enrollment template corresponding to a user in the group of users.

[0026] The biometric authentication system can generate different lighting configurations and corresponding different lighting channels by changing the wavelength, intensity and / or direction of light emitted by one or more illumination sources. In some embodiments, the biometric authentication system can use different lighting channels in a quality assurance system, which may include a trained machine learning model that checks whether the lighting channel that the captured image appears to have as an image attribute corresponds to a recorded or otherwise known lighting configuration used to capture the image. The biometric authentication system can determine that the captured image is not an image of the biometric aspects of a real person presented to the sensor or is otherwise forged or manipulated based on a mismatch in the quality assurance system. For example, if the light is pointed in a particular direction under a given lighting configuration, the shadows produced in the image may indicate the lighting configuration used, thereby indicating the lighting channel attributes of the image. If another image that was not captured using the same lighting configuration is presented, the shadows in the other image may not match the expected lighting channel, so the biometric authentication system can reject the captured image as a low-quality image (e.g., a forged image capture, etc.).

[0027] In addition, multiple illumination channel images can increase the entropy of the data set used for biometric authentication, thereby making the biometric authentication system more robust. For example, if two people have very similar biometric aspects that are difficult to distinguish (as an example for illustrative purposes only), the biometric authentication system may make a false positive authentication. To avoid this, the similarity score threshold used by the biometric authentication system can be set high enough to avoid false positives. However, although the two people have similar biometric aspects, the biometric authentication system can use differences to distinguish between the two people. These differences may appear differently in images captured under different lighting conditions. Therefore, the biometric authentication system captures multiple images of a given biometric aspect under different lighting conditions so that the biometric authentication system has multiple illumination channel images, thereby providing more information about the biometric aspect and capturing more differences (e.g., greater entropy). By having more information, the biometric authentication system can adjust the threshold used to determine the authentication to avoid both false positives and false negatives. In this way, a biometric authentication system using multiple illumination channel images can improve the robustness of the biometric authentication system.

[0028] Biometric authentication systems using multiple illumination channels can be included in various devices, such as mobile user devices (e.g., smart phones and smart watches), vehicles (e.g., cars), stationary devices (e.g., monitors, televisions, doorbells, smart home control units, and smart speakers), and head-mounted display devices (e.g., head-mounted display devices and glasses display devices). Biometric authentication systems using multiple illumination channels can analyze biometric aspects such as the iris, periocular area, face, ears, hands, and other biometric aspects of the user.

[0029] Figure 1A is a front view of an illumination source having multiple independent light emitting elements according to some embodiments.

[0030] Figure 1A An example illumination source 110 is shown that includes a plurality of light emitting elements 102 and 104. Figure 1A , there are eight light emitting elements arranged in a ring. However, it should be noted that the number and arrangement of the light emitting elements in the illumination source 110 may be different. In some embodiments, the light emitting elements may be light emitting diodes (LEDs), and in some embodiments, other light emitting elements besides LEDs may be used, or a combination of LEDs and other light emitting elements may be used. In some embodiments, some light emitting elements may be configured to emit light of a first wavelength, and other light emitting elements may be configured to emit light of different wavelengths. It should be noted that in some embodiments, more than two wavelengths may be used. In addition, in some embodiments, each light emitting element may be configured to selectively emit light of two or more different wavelengths. Each light emitting element may also be configured to emit light of various intensities.

[0031] Figure 1B is a side view of an illumination source and controller, camera, and biometric aspects of using an illumination configuration to generate an illumination channel according to some embodiments.

[0032] Figure 1B An example imaging system including an illumination source according to some embodiments is shown. The imaging system may include, but is not limited to, one or more cameras 140, an illumination source 110, and a controller 160. Figure 1B In the embodiment, the eye camera 140 is directed toward the eye used as the biometric aspect 150 to receive reflected light from the illumination source 110 that has been reflected by the biometric aspect 150. In some embodiments, the eye camera 140 may alternatively or additionally capture images of the periocular area and facial portions. Figure 1B, controller 160 has sent an illumination configuration 120 for a first illumination channel to illumination source 110, causing one light emitting element 102 to be activated. Camera 140 has captured an image 122 of a biometric aspect 150 illuminated using the first illumination channel and has sent the image to controller 160. In some embodiments, more than one camera may be used to capture images of the biometric aspect. The camera may be a visible light camera or a non-visible light camera, such as an infrared or near infrared camera.

[0033] Figure 1C According to some embodiments, the Figure 1B Side view of the illumination sources of different illumination configurations as well as the controller, camera and biometric features used in the illumination configurations.

[0034] exist Figure 1C , the controller 160 has selected a second illumination configuration 130 for a second illumination channel and has sent control instructions to the illumination source 110 such that different light emitting elements 104 are activated. The camera 140 has captured an image 132 of the biometric aspect 150 illuminated using the second illumination channel and has sent the image to the controller 160. The controller 160 can use the images captured under different illumination configurations in a multi-illumination channel analysis system.

[0035] Figure 1D is a graphical representation of three lighting configurations and three lighting channels that may be generated by the corresponding lighting configurations according to some embodiments.

[0036] Figure 1D The upper pair of images shows Figure 1B 1 and 2 , an illumination source 110 in a first illumination configuration 120 in which the light emitting elements 102 are active at a particular wavelength and intensity, and a first illumination channel 124 generated by the first illumination configuration 120. The illumination channel 124 for the periocular region 190 has a shadow 194 around the region of the eye 192 where the depth variation exists.

[0037] The second pair of images shows Figure 1B and Figure 1C 104 is an illustration of a third illumination configuration 170 not seen in the illustration, in which two light emitting elements 102 and 104 are active, and a third illumination channel 174 generated by the third illumination configuration 170. Because more than one light emitting element 102 and 104 is active, there are different shadows 196 caused by the depth variation around the eye 192. An illumination channel may include multiple active light emitting elements that may be active at different wavelengths and intensities from each other.

[0038] The third pair of images shows a fourth illumination configuration 180 in which a previously unused light emitting element 106 is active, and a fourth illumination channel 184 produced by the fourth illumination configuration 180. Because this is a different illumination channel 184, it has generated a shadow 198 that is different from the shadows 194 and 196 produced by the other illumination channels 124 and 174. The three illumination channels 124, 174, and 184 shown may be used as part of a biometric analysis system and / or a quality assurance system.

[0039] Figure 2A is a block diagram illustrating components of a multi-illumination channel input capture system that may be implemented by a controller according to some embodiments for capturing images for evaluation by a multi-illumination channel analysis system (such as with respect to the illumination channels applied when the images were captured).

[0040] Figure 2A Possible components of a multi-illumination channel input capture system 200 that may be implemented by the controller 160 are shown. The multi-illumination channel input capture system 200 may be used to generate input using multiple illumination channels for use in a multi-illumination channel analysis system 210. The multi-illumination channel input capture system 200 may include an illumination configuration memory 202 and an illumination configuration selector 204, both of which may communicate with an output interface 206 to light emitting elements to send a selected illumination configuration to control the light emitting elements. The multi-illumination channel input capture system 200 may also include a captured image input interface 208 to receive an image taken using the selected illumination channel. The captured image input interface 208 may provide a captured image to a multi-illumination channel analysis system 210, which may use the captured image and a sequence of recorded or otherwise known illumination configurations sent by the output interface 206 to the light emitting elements. Figure 2B The multi-illumination channel analysis system 210 is further described in .

[0041] Figure 2B is a block diagram illustrating components of an example multi-illumination channel analysis system that may be implemented by a controller according to some embodiments.

[0042] Figure 2B 1 shows possible components of a multi-illumination channel analysis system 210 implemented by the controller 160. The multi-illumination channel analysis system 210 may include a quality assurance system 220. The quality assurance system 220 may include an illumination channel record maintenance module 230 to record the illumination channels of captured images. There may also be an illumination channel consistency determination module 240, which includes a trained machine learning model 250. The illumination channel consistency determination module is Figure 4The multi-illumination channel analysis system 210 may also include a biometric authentication system 260. The biometric authentication system 260 may include a multi-illumination channel comparison system 270, which includes a module 272 for analyzing 3D structure, a module 274 for analyzing reflectivity, and a module 276 for analyzing color. The multi-illumination channel comparison system 270 may include one or more trained machine learning models to derive information from the captured images.

[0043] The multi-illumination channel comparison system 270 can compare information about the 3D structure, reflectivity, and color of the biometric aspect in the image at the illumination channel with information about the 3D structure, reflectivity, and color of the biometric aspect of the registered user from a stored template generated from one or more images captured at the same illumination channel during enrollment. Using multiple illumination channels in a single authentication process can increase the certainty that the person is the registered user by increasing the number of possibilities of detecting anomalies by the multi-illumination channel comparison system and increasing the difficulty of impersonating the registered user by using illumination channels that result in different light interactions with the biometric aspect (such as the periocular area). The multi-illumination channel comparison system 270 can use a similarity scoring algorithm in the comparison to determine whether the 3D structure, reflectivity, and color of the periocular area matches the 3D structure, reflectivity, and color of the periocular area of ​​the registered user. Various thresholds can be used, such as a similarity score that is required to indicate a match greater than X% (e.g., 90%, 95%, 99%, etc.) to consider an image a match with a stored image captured at the same illumination channel. In some embodiments, the biometric authentication system 260 may require a threshold to be met at one or more illumination channels. The biometric authentication system 260 may also include an iris-based biometric authentication process 280, which may be used in conjunction with, for example, Figure 5 The illustrated multiple illumination channel comparison system 270 interacts with the iris image. For example, as an example, determining a 3D structure match can be a prior or subsequent step to determining an iris match. In some embodiments, an iris-based biometric authentication process can be an integral part of the multiple illumination channel comparison system 270, where an image of the iris is compared to a stored image of the iris captured in the same illumination channel.

[0044] Figure 3 is a block diagram illustrating a remote training system that trains a machine learning model for use in a quality assurance system and a set of devices that use the trained machine learning model to evaluate lighting channels according to some embodiments.

[0045] For example, in Figure 3In the training process 380, the machine learning model (310) is trained so that it can be used in the run-time evaluation process 390. In some embodiments, the two training data sets 320 and 330 are sets of training data that can be used to create the trained machine learning model 250. The machine learning model training 310 can use one or more sets of training data. A set of training data (320) including images of the periocular region labeled according to the illumination channel used to capture the image can be used to train the machine learning model to understand the appearance when the illumination channel is applied to the periocular region. In some embodiments, the training data can be information derived from the image, for example, the training data can include 3D structure data, reflectivity, and wavelength of reflected light.

[0046] A set of training data (330) including images labeled according to whether the image is a valid image that can be used for authentication can be used to train a machine learning model: some images are useless and can be ignored without the controller calculating an objective criterion for image quality. Valid images can be images that are known to be biometric and have been captured under a known lighting configuration. Valid images can include information that can be used in a biometric authentication system. Invalid images can be images that do not have biometric aspects and can include images of objects (such as images of the area around the eye), images with significant blur or occlusion, and images captured under uninformative lighting configurations (such as no light at all). In some embodiments, machine learning model training 310 can use more than one set of training data, or machine learning model training 310 can use a set of training data that uses a combination of labeling systems.

[0047] In some embodiments, the trained machine learning model 250 can be sent to various devices (such as devices 350, 352, and 354) via network 340, and the trained machine learning model can be used in a runtime evaluation process 390 at the respective devices to determine the illumination channel properties of an image by analyzing images (360) captured using varying illumination source configurations and generating inferences (such as embedded features) 370 about the properties of the illumination channel of the image. Figure 4 As further described in , the generated inferences may be used by illumination channel consistency determination module 240 to determine whether the inferred illumination channel matches the recorded illumination channel.

[0048] Figure 4 is a process flow diagram illustrating an example illumination channel consistency determination process according to some embodiments.

[0049] For example, Figure 4The illumination channel consistency determination module 240 implemented by the controller 160 is shown. At 410, a trained machine learning model trained on images can estimate which illumination channel was used when capturing the images, which can be information derived from images labeled according to the illumination channel used to capture the images. At 420, the illumination channel consistency determination module 240 can compare the estimated illumination channel with the recorded or otherwise known illumination channel. At 430, the illumination channel consistency determination module 240 determines whether the changes in the image are consistent with the illumination channel indicated by the recorded order or the order otherwise known to the illumination channel consistency module 240. In some embodiments, the illumination channel consistency determination module 240 can make such a determination by determining whether the percentage of correctly estimated illumination channels meets a threshold percentage. In some embodiments, a minimum number of images from different illumination channels can be analyzed. In some embodiments, the order of the illumination channels used can be randomized. At 440, if the changes in the image are consistent with the illumination channel indicated by the recorded order, the illumination channel consistency determination module 240 can determine that the image is of sufficient quality. For example, if images of the same biometric aspect are captured using five illumination channels, and it is determined that four or more illumination channel attributes of the images match the recorded or otherwise known illumination channels used, then the set of images may pass the quality check. However, it should be understood that a different number of illumination channels and different thresholds may be used in the quality check. At 450, if the changes in the image are inconsistent with the illumination channels indicated by the recorded or otherwise known sequence, then the illumination channel consistency determination module 240 may determine that the image is not of sufficient quality. In some embodiments, determining that the image is not of sufficient quality may trigger additional authentication requirements.

[0050] Figure 5 is a process flow diagram illustrating a possible biometric authentication process using a 3D structure in conjunction with iris-based biometric authentication according to some embodiments.

[0051] exist Figure 5In the example biometric authentication system 260 shown, the biometric authentication system 260 begins at 510 by causing an input image of the periocular region to be captured. The biometric authentication system 260 then determines at 512 whether there are open eyes in the image. If there are open eyes in the image, the biometric authentication system 260 can proceed to 280, i.e., an iris-based biometric authentication process. If there are no open eyes in the image, the biometric authentication system 260 can proceed to 270, i.e., a multi-illumination channel comparison system. The biometric authentication system can derive information about the image, such as whether the image includes an open eye, by using a trained machine learning model. The biometric authentication system can use one or more trained machine learning models to derive additional information from the image, such as information indicating a three-dimensional structure, reflectivity, and wavelength of reflected light.

[0052] In some embodiments, a similarity scoring algorithm may be used to generate a match score. Various thresholds may be used, such as requiring a match score indicating a match greater than X% (e.g., 90%, 95%, 99%, etc.) to determine that a match is sufficiently "high" and requiring a match score indicating a match less than Y% (e.g., 90%, 85%, 80%, etc.) to determine that a match is sufficiently "low". In some embodiments, X% may be higher than Y%. A similarity scoring algorithm may be used in both an iris-based biometric authentication process and a multi-illumination channel comparison system.

[0053] At 516, a match score is generated in the iris-based authentication process by comparing information about the iris in the input image with stored information about the iris of the registered user. At 522, if the match score is high enough, the biometric authentication system 260 can authenticate 522. At 524, if the match score is low enough, the biometric authentication system 260 may not authenticate. If the match score of the iris-based authentication process is not high enough or low enough, the match score may be uncertain, and the biometric authentication system 260 may proceed to 270, i.e., the multi-illumination channel comparison system. At 514, a match score 514 is generated in the multi-illumination channel comparison system by comparing information derived from the input image (such as the periocular region 3D structure, reflectivity, and color) with a stored multi-illumination channel template (including information derived from an image marked as corresponding to the illumination channel of the input image, such as the periocular region 3D structure, reflectivity, and color). At 522, if the match score is high enough, the biometric authentication system 260 may authenticate. At 524, if the match score is low enough, the biometric authentication system 260 may not authenticate. At 526, if the match score of the multi-illumination channel comparison system is inconclusive, the biometric authentication system 260 may retry by returning to the image capture step of 510. The multi-illumination channel comparison system may prevent the biometric authentication system 260 from failing due to a lack of open eyes or an inconclusive iris match.

[0054] exist Figure 5 In the embodiment of the present invention, the biometric authentication system 260 uses the iris-based authentication process 280 if possible, but it should be noted that some embodiments can save computer resources by using the multi-illumination channel comparison system 270 before or instead of the iris-based authentication process 280. In some embodiments, other types of authentication systems (such as fingerprint-based authentication systems, for example) can be used in addition.

[0055] FIG. 6A to FIG. 6D Examples of information about 3D structure and other biometric surface information that may be used by a multi-illumination channel analysis system are shown according to some embodiments.

[0056] Fig. 6AA portion of a periocular region (e.g., such as a nose) is depicted that has been divided into four planes 612, each plane having a surface normal 610 associated therewith. The surface normal 610 is a vector perpendicular to the plane 612 of the surface. The surface normal 610 can be information about a 3D structure that can be used in a biometric authentication system. In some embodiments, the surface normal can be generated by first dividing the image into several segments and determining the surface normals of these segments or by looking up the surface normal at a set frequency over the entire image. The biometric authentication system can use a trained machine learning model to determine the surface normal, such as by using shading information from a single image and one or more of the perspective views from multiple images captured at different angles. The biometric authentication system can also use the surface normal as information indicating a three-dimensional structure. The three-dimensional structure can include information about the size and shape of the biometric aspect.

[0057] Figure 6B A contour map that can represent a 3D structure is depicted. The contour map has four depth ranges 620, 622, 624, and 626. The outer side of the contour map is uniformly in the second depth range 622. Moving inward from the outer edge, the depth increases to the deepest depth range, i.e., the first depth range 620. Moving further inward, the depth decreases again to the second depth range 622, and then continues to decrease through the third depth range 624 to the highest depth range, i.e., the fourth depth range 626 at the center. A contour map like this can represent an approximation of a 3D structure in a two-dimensional format. Figure 6B The example contour map in is a contour map of the eye and the area immediately surrounding the eye, but it should be understood that this example is not limiting and the contour map used to represent the 3D structure can have other biometric aspects or include biometric aspects of the eye such as the periocular area. The biometric authentication system can use a trained machine learning model to determine a depth map (such as a contour map) as information derived from one or more images. The biometric authentication system can use a depth sensor to obtain the depth information. The biometric authentication system can use the depth map as information indicative of a three-dimensional structure.

[0058] Figure 6C Emitted light 630 is depicted being reflected off surface 636 and becoming reflected light 632. When emitted light 630 is reflected off surface 636, some of the light may be absorbed 634, so the intensity of reflected light 632 may be lower than emitted light 630. The change in intensity from emitted light 630 to reflected light 632 may allow calculation of the reflectivity of surface 636. Some of the absorbed light 634 may be light of a particular wavelength, such as Fig.6D shown.

[0059] Fig.6DAlso shown are changes that may occur when light is reflected off of surface 636, with the emphasis on some wavelengths of emitted light 630 being absorbed 634 by surface 626, such that reflected light 632 may not include some wavelengths or may partially include some wavelengths. In some embodiments, the wavelengths of reflected light of the biometric authentication system may correspond to one or more of wavelengths in the visible spectrum and wavelengths not present in the visible spectrum.

[0060] exist Fig.6D 644B. ​​In FIG. 644B, after the emitted light 630 interacts with the surface 636, the first wavelength 644A of the emitted light 640 remains unchanged after interacting with the surface 636 and becoming the reflected light 642. The second wavelength 646A is completely absorbed by the surface 636 and is not present in the wavelength of the reflected light 642. The third wavelength 648 is partially absorbed by the surface 636 and is reduced in intensity to be the reflected light 648B. These changes in wavelength between the emitted light 640 and the reflected light 642 can allow the color of the surface 636 to be calculated. In some embodiments, the color and reflectivity can be information that can be used to supplement the 3D structure in the multi-illumination channel comparison system 260.

[0061] In some embodiments, the lighting configuration may vary with respect to additional characteristics of the light, such as the lighting distribution. The lighting distribution indicates a combination of the position and intensity of one or more lighting elements, thereby indicating the uniformity or lack of uniformity of the lighting intensity across the surface (e.g., biometrically).

[0062] 7A to 7C An example device that may be used to implement a multi-illumination channel analysis system according to some embodiments is shown.

[0063] Fig. 7A An example head mounted device (HMD) according to some embodiments is shown, which may include components as shown in Figures 1 to 6 and implement methods as shown in Figures 1 to 6. Fig. 7A As shown, the HMD can be positioned on the user's head 790 so that the display is disposed in front of the user's eyes. The user looks toward the display through the eyepieces.

[0064] 7A to 7C 1 to 6 may be implemented according to some embodiments. 7A to 7C The illustrated device 350 is given by way of example and is not intended to be limiting. In various embodiments, the shape, size, and other features of the HMD may vary, and the location, number, type, and other features of the components of the HMD and eye imaging system may vary. Fig. 7A shows a side view of an example HMD, and Figure 7B and Figure 7CAn alternative front view of an example HMD is shown, wherein Figure 7B A device 350 is shown having one lens 730 covering both eyes 740, and Figure 7C Device 350 is shown having a right lens 730A and a left lens 730B.

[0065] The HMD may include a lens 730 mounted in a wearable housing or frame 710. The HMD may be worn on the head of a user ("wearer") such that the lens is disposed in front of the wearer's eyes 740. In some embodiments, the HMD may implement any of a variety of types of display technologies or display systems. For example, the HMD may include a display system that directs light that forms an image (virtual content) through one or more waveguide layers in the lens 720; the output coupler of the waveguide (e.g., an embossed grating or a volume holographic grating) may output light toward the wearer to form an image at or near the wearer's eyes 740.

[0066] As another example, the HMD may include a retinal direct projection system that directs light toward a reflective component of a lens; the reflective lens is configured to redirect the light to form an image at the wearer's eye 740. In some embodiments, for the purpose of generating or updating a template eye feature representation, the display system may change the displayed content to at least partially affect the condition and characteristics of the eye 740. For example, the display may increase brightness to change the condition of the eye 740, such as affecting the illumination of the eye 740. As another example, the display may change the distance at which an object appears on the display to affect the condition of the eye 740, such as the accommodation distance of the eye 740.

[0067] In some embodiments, the HMD may also include one or more sensors (e.g., eye sensors or gaze sensors) that collect information about the wearer's environment (video, depth information, lighting information, etc.) and information about the wearer. The sensors may include one or more of the following, but are not limited to the following: one or more eye cameras 140 (e.g., infrared (IR) cameras) that capture views of the user's eyes 740, one or more world-facing or PoV cameras 750 (e.g., RGB cameras) that can capture images or videos of the real-world environment in the field of view in front of the user, and one or more ambient light sensors that capture lighting information of the environment. The cameras 140 and 750 may be integrated into or attached to the frame 710. The HMD may also include one or more illumination sources 110, such as LEDs or infrared point light sources, that emit light (e.g., light in the IR portion of the spectrum) toward one or both eyes 740 of the user.

[0068] The controller 160 for the multi-illumination channel analysis system 210 may be implemented in the HMD, or alternatively may be implemented at least in part by an external device (e.g., a computing system or a portable device) communicatively coupled to the HMD via a wired or wireless interface. The controller 160 may include one or more of various types of processors, image signal processors (ISPs), graphics processing units (GPUs), encoders / decoders (codecs), systems on chips (SOCs), CPUs, and / or other components for processing and rendering video and / or images.

[0069] The memory 770 for the multi-illumination channel analysis system 210 may be implemented in the HMD, or alternatively may be implemented at least in part by an external device (e.g., a computing system) communicatively coupled to the HMD via a wired or wireless interface. The memory 770 may be used, for example, to record video or images captured by one or more cameras 750 integrated in or attached to the frame 710. The memory 770 may include any type of memory, such as dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate (DDR, DDR2, DDR3, etc.), SDRAM (including mobile versions of SDRAM, such as mDDR3, etc., or low power versions of SDRAM, such as LPDDR2, etc.), RAMBUS DRAM (RDRAM), static RAM (SRAM), etc.

[0070] In some embodiments, one or more memory devices may be coupled to a circuit board to form a memory module such as a single inline memory module (SIMM), a dual inline memory module (DIMM), etc. Alternatively, these devices may be mounted with an integrated circuit implementation system in a chip stack configuration, a package stack configuration, or a multi-chip module configuration. In some embodiments, DRAM may be used as temporary storage for images or video for processing, but other storage options may also be used in the HMD to store processed data, such as flash memory or other "hard drive" technology. This other storage device may be separate from the externally coupled storage device mentioned below.

[0071] Although Fig. 7A Only the illumination source 110 and cameras 140 and 750 for one eye are shown, but embodiments may include an illumination source 110 and cameras 140 and 750 for each eye, and user authentication may be performed for both eyes. In addition, the illumination source 110, camera 140, and PoV camera 750 may be located in other locations than shown. The HMD may have an opaque display, or a transparent display may be used, which allows the user to see the real environment through the display while displaying virtual content overlaid on the real environment.

[0072] Figure 8is a block diagram illustrating an example computing device that may be used in accordance with some embodiments.

[0073] In at least some embodiments, a computing device that implements part or all of one or more of the techniques described herein may include a general-purpose computer system that includes or is configured to access one or more computer-accessible media. Figure 8 Such a general purpose computing device 800 is shown. In the illustrated embodiment, the computing device 800 includes one or more processors 810 coupled to a main memory 840 (which may include non-volatile and volatile memory modules, also referred to as system memory) via an input / output (I / O) interface 830. The computing device 800 also includes a network interface 870 coupled to the I / O interface 830, and additional I / O devices 820, which may include various types of sensors.

[0074] In various embodiments, the computing device 800 may be a single processor system including one processor 810, or a multiprocessor system including several processors 810 (e.g., two, four, eight, or other suitable number). The processor 810 may be any suitable processor capable of executing instructions. For example, in various embodiments, the processor 810 may be a general-purpose or embedded processor that implements any of a variety of instruction set architectures (ISAs) such as x86, PowerPC, SPARC, or MIPS ISAs or any other suitable ISAs. In a multiprocessor system, each of the processors 810 may typically, but not necessarily, implement the same ISA. In some implementations, a graphics processing unit (GPU) may be used instead of a conventional processor, or a graphics processor may be used in addition to a conventional processor.

[0075] The memory 840 may be configured to store instructions and data accessible by the processor 810. In at least some embodiments, the memory 840 may include volatile and non-volatile portions; in other embodiments, only volatile memory may be used. In various embodiments, the volatile portion of the system memory 840 may be implemented using any suitable memory technology, such as static random access memory (SRAM), synchronous dynamic RAM, or any other type of memory. For the non-volatile portion of the system memory (e.g., which may include one or more NVDIMMs), in some embodiments, flash-based memory devices may be used, including NAND flash devices. In at least some embodiments, the non-volatile portion of the system memory may include a power source, such as a supercapacitor or other power storage device (e.g., a battery). In various embodiments, memristor-based resistive random access memory (ReRAM), three-dimensional NAND technology, ferroelectric RAM, magnetoresistive RAM (MRAM), or any of various types of phase change memory (PCM) may be used for at least the non-volatile portion of the system memory. In the illustrated embodiment, executable program instructions 850 and data 860 that implement one or more desired functions, such as those methods, techniques, and data described above, are shown stored in main memory 840 .

[0076] In one embodiment, the I / O interface 830 may be configured to coordinate I / O traffic between the processor 810, the main memory 840, and various peripheral devices, including the network interface 870 or other peripheral interfaces, such as various types of persistent and / or volatile storage devices, sensor devices, etc. In some embodiments, the I / O interface 830 may perform any necessary protocol, timing, or other data transformations to convert data signals from one component (e.g., the main memory 840) into a format suitable for use by another component (e.g., the processor 810). In some embodiments, the I / O interface 830 may include support for devices attached, for example, via various types of peripheral device buses (such as a variation of the peripheral component interconnect (PCI) bus standard or the universal serial bus (USB) standard). In some embodiments, the functions of the I / O interface 830 may be divided into two or more separate components, such as a north bridge and a south bridge, for example. In addition, in some embodiments, some or all of the functions of the I / O interface 830 (such as an interface to the memory 840) may be directly incorporated into the processor 810.

[0077] The network interface 870 may be configured to allow data to be exchanged between the computing device 800 and other devices 890 (such as other computer systems or devices) attached to one or more networks 880. In various embodiments, the network interface 870 may support communication via any suitable wired or wireless general-purpose data network (e.g., such as an Ethernet network type). In addition, the network interface 870 may support communication via a telecommunications / telephone network such as an analog voice network or a digital fiber-optic communication network, via a storage area network such as a Fiber Channel SAN, or via any other suitable type of network and / or protocol.

[0078] In some embodiments, the main memory 840 can be an embodiment of a computer-accessible medium that is configured to store program instructions and data as described above with respect to Figures 1 to 7 for implementing embodiments of corresponding methods and apparatus. However, in other embodiments, program instructions and / or data may be received, sent, or stored on different types of computer-accessible media. In general, a computer-accessible medium may include a non-transitory storage medium or a storage medium such as a magnetic or optical medium, such as a disk or DVD / CD coupled to the computing device 800 via an I / O interface 830. Non-transitory computer-accessible storage media may also include any volatile or non-volatile media, such as RAM (e.g., SDRAM, DDR SDRAM, RDRAM, SRAM, etc.), ROM, etc., which may be included in some embodiments of the computing device 800 as the main memory 840 or other types of memory. In addition, the computer-accessible medium may include a transmission medium or a signal such as an electrical signal, an electromagnetic signal, or a digital signal, transmitted via a communication medium such as a network and / or a wireless link, such as may be implemented via a network interface 870. Part or all of multiple computing devices, such as Figure 8 The computing devices shown in the figure can be used to implement the functions described in various embodiments; for example, software components running on various different devices and servers can cooperate to provide functions. In some embodiments, in addition to or instead of using a general-purpose computer system to implement, a storage device, a network device, or a special-purpose computer system can be used to implement a portion of the described functions. As used herein, the term "computing device" refers to at least all of these types of devices and is not limited to these types of devices.

[0079] In different embodiments, the methods described herein can be implemented in software, hardware or a combination thereof. In addition, the order of the frames of the method can be changed, and various elements can be added, reordered, combined, omitted, modified, etc. For those skilled in the art who benefit from the present disclosure, various modifications and changes can obviously be made. The various embodiments described herein are intended to be illustrative and not restrictive. Many variations, modifications, additions and improvements are possible. Therefore, multiple examples can be provided for the components described as a single example in this article. The boundaries between various components, operations and data repositories are arbitrary to a certain extent, and specific operations are shown in the context of a specific exemplary configuration. Other allocations of functions are expected, and they may fall within the scope of the appended claims. Finally, the structure and function presented as discrete components in the example configuration may be implemented as a combined structure or component. These and other variations, modifications, additions and improvements may fall within the scope of the embodiments defined in the following claims.

Claims

1. A system comprising: an illumination source comprising a plurality of light emitting elements, wherein one or more individual ones of the light emitting elements are configured to be controlled independently of one or more other light emitting elements to emit light toward one or more biometric aspects; a camera configured to capture light reflected from the one or more biometric features; and A controller comprising one or more processors, wherein the controller is configured to: directing the illumination source to emit light using a plurality of selected configurations for the light emitting elements, thereby causing the emitted light to be emitted according to a plurality of illumination channels; directing the camera to capture a corresponding image of an object when a corresponding one of the illumination channels is applied; as well as The respective captured images are analyzed for use in biometric authentication, wherein the analysis accounts for changes in the respective captured images due to application of respective ones of the illumination channels.

2. The system of claim 1 , wherein the selected plurality of illumination channels comprises a particular combination of light emitted from the light emitting elements, the particular combination being selected: Wavelength combination; Strength; and direction.

3. The system of claim 1 , wherein to perform the analysis of the corresponding captured images, the controller is further configured to: causing a record to be maintained of the order in which illumination channels of the respective ones of the images were applied when capturing the respective ones of the images; A determination is made as to whether the changes observed in corresponding ones of the images are consistent with the illumination channels being applied according to the recorded sequence.

4. The system of claim 3, wherein the analysis of the corresponding captured images is performed using a trained machine learning model.

5. The system of claim 4, wherein the trained machine learning model is trained using a dataset comprising images labeled according to the respective illumination channels in which the respective images were captured.

6. The system of claim 1 , wherein to perform the analysis of the corresponding captured images, the controller is further configured to: Information derived from the one or more corresponding captured images is compared to information derived from one or more corresponding enrollment images corresponding to the user, wherein the same set of illumination channels is applied to the one or more corresponding captured images and the one or more corresponding enrollment images.

7. A system according to claim 6, wherein the comparison of information derived from the one or more corresponding captured images with information derived from the one or more corresponding registered images is performed as part of a user authentication process, wherein the user authentication process also includes an iris-based biometric authentication process.

8. The system of claim 6, wherein information considered in the comparison of information derived from the one or more corresponding captured images with information derived from the one or more corresponding registered images includes one or more of the following: information indicative of a three-dimensional structure of the one or more biometric aspects; a reflectivity of the surface of the one or more biometric aspects; and The wavelength of light reflected from the one or more biometric features.

9. The system of claim 6, wherein the controller is configured to perform the comparison of information derived from the one or more corresponding captured images with information derived from the one or more corresponding enrolled images in response to: It is determined that observed changes in information derived from corresponding ones of the images are consistent with illumination channels applied according to the order in which the corresponding ones of the images were captured.

10. The system of claim 6, wherein information derived from one or more of the corresponding captured images and information derived from one or more corresponding registered images are determined by one or more trained machine learning models.

11. The system according to any one of claims 1 to 10, wherein the system further comprises a head mounted device.

12. The system according to any one of claims 1 to 10, wherein the system further comprises: Mobile user equipment; Transportation; Fixed equipment.

13. One or more non-transitory computer-readable storage media storing program instructions that, when executed on or across one or more computing devices, cause the one or more computing devices to: selecting a plurality of configurations for an illumination source, the illumination source comprising a plurality of light emitting elements, wherein one or more individual ones of the light emitting elements are configured to be controlled independently of one or more other light emitting elements to emit light toward one or more biometric aspects; directing the illumination source to emit light using a plurality of selected configurations for the light emitting elements, thereby causing the emitted light to be emitted according to a plurality of illumination channels; directing the camera to capture a corresponding image of an object when a corresponding one of the illumination channels is applied; as well as The respective captured images are analyzed for use in biometric authentication, wherein the analysis accounts for changes in the respective captured images due to application of respective ones of the illumination channels.

14. The computer-readable storage medium of claim 13, wherein the plurality of illumination channels comprises a particular combination of light emitted from the light emitting elements, the particular combination being selected: Wavelength combination; Lighting distribution; Strength; and direction.

15. The computer-readable storage medium of claim 13 or claim 14, wherein the analysis of the corresponding captured images is performed by: selecting the corresponding one or more intended illumination channels based on the corresponding one or more captured images; and A determination is made as to whether the corresponding one or more expected illumination channels are the same as the corresponding one or more illumination channels applied when capturing the corresponding one or more images.

16. The computer-readable storage medium of claim 13 or claim 14, wherein the analysis of the corresponding captured images is performed by comparing information derived from one or more corresponding captured images to corresponding templates corresponding to a user and the corresponding one or more lighting channels of the corresponding one or more images.

17. A method comprising: selecting a plurality of configurations for an illumination source, the illumination source comprising a plurality of light emitting elements, wherein one or more individual ones of the light emitting elements are configured to be controlled independently of one or more other light emitting elements to emit light toward one or more biometric aspects; directing the illumination source to emit light using a plurality of selected configurations for the light emitting elements, thereby causing the emitted light to be emitted according to a plurality of illumination channels; directing the camera to capture a corresponding image of an object when a corresponding one of the illumination channels is applied; as well as The respective captured images are analyzed for use in biometric authentication, wherein the analysis accounts for changes in the respective captured images due to application of respective ones of the illumination channels.

18. The method of claim 17, further comprising performing a randomization process to select the plurality of lighting configurations.

19. The method of claim 17 or claim 18, wherein the analysis of the corresponding captured images is performed by: selecting the corresponding one or more intended illumination channels based on the corresponding one or more captured images; and A determination is made as to whether the corresponding one or more expected illumination channels are the same as the corresponding one or more illumination channels applied when capturing the corresponding one or more images.

20. The method of claim 17 or claim 18, wherein the analysis of the corresponding captured images is performed by comparing information derived from one or more of the corresponding captured images with corresponding templates corresponding to a user and corresponding one or more lighting channels of the corresponding one or more images.

Images