Imaging device and authentication device

Abstract

To acquire clear wavelength separation image data.SOLUTION: An imaging apparatus comprises: an irradiation unit which irradiates a living body with light in plural wavelengths having different wavelengths; an imaging unit which images the living body irradiated by the irradiation unit and generates image data of the living body; an image processing unit which generates a plurality of pieces of wavelength separation image data obtained by separating the plural wavelengths on the basis of the image data generated by the imaging unit; and a control unit which controls an irradiation light amount of the light in the plural wavelengths on the basis of the image data and the plurality of pieces of wavelength separation image data generated by the image processing unit.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an imaging device for imaging a living body and an authentication device for imaging and authenticating a living body.

Background Art

[0002] There is a biometric authentication technique in which light from a light source is irradiated onto a living body, and biometric authentication is performed using a biological image obtained by photographing the reflected light. When near-infrared light is selected as the irradiation light, biometric authentication using a blood vessel image obtained by photographing, utilizing the difference in the absorption characteristics of near-infrared light between hemoglobin in blood vessels and other biological tissues, can be realized. Also, by selecting light in the visible light wavelength bands of green or blue as the irradiation light, biometric authentication using an epidermal image obtained by photographing irregularities present on the surface of the skin such as fingerprints and wrinkles of joints that enable individual identification can be realized.

[0003] There is a reflection type method as a biological imaging method for realizing an authentication technique using a biological image such as a finger blood vessel image or an epidermal image with the same device configuration. The reflection type method is a method in which a light source and an imaging unit are arranged close to each other, and the irradiation light from the light source is irradiated onto a living body and the reflected light is imaged to obtain a biological image.

[0004] Patent Document 1 discloses a subject information acquisition device that reduces the influence of components outside the measurement target in the information obtained by photoacoustic imaging (PAI). This subject information acquisition device includes a light source that irradiates a subject with a first light having a first wavelength λ1 and a second light having a second wavelength λ2, a detection means that converts photoacoustic waves generated from the subject into detection signals, a signal processing means that acquires characteristic information from the detection signals, and a light intensity acquisition means that acquires the intensity of incident light irradiated on the subject. The signal processing means acquires characteristic information by subtracting the signal generated when the first light is absorbed by hemoglobin from the signal generated when the second light is absorbed by hemoglobin. The first and second wavelengths are 780 - 810 nm and 840 - 920 nm respectively. When the incident light intensities of the first and second lights are Φ(λ1) and Φ(λ2) respectively, Φ(λ1) ≤ Φ(λ2) is satisfied, and the difference between Φ(λ1) and Φ(λ2) is adjusted to be within a predetermined range.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] When trying to take a clear biological image by irradiating a living body with lights of multiple wavelengths simultaneously, if the luminance of any component of the RGB image data saturates or is too small at the biological site of the image data, biological information is lost at that biological site. Therefore, there is a possibility that biological information of the biological site is also lost in the wavelength-separated image data generated by spectral processing of the RGB image data. Also, even if the light amounts of lights of multiple wavelengths are adjusted so that the RGB image data has an appropriate brightness (luminance), the generated wavelength-separated image data does not necessarily have an appropriate brightness (luminance).

[0007] An object of the present invention is to acquire clear wavelength-separated image data.

Means for Solving the Problems

[0008] An imaging device according to one aspect of the invention disclosed in the present application includes an irradiation unit that irradiates a living body with light of a plurality of different wavelengths, an imaging unit that images the living body irradiated by the irradiation unit and generates image data of the living body, and an image processing unit that generates a plurality of wavelength-separated image data obtained by separating the plurality of wavelengths based on the image data generated by the imaging unit, so that the luminance of the image data and the luminances of the plurality of wavelength-separated image data generated by the image processing unit fall within a predetermined range and a control unit that controls the irradiation light amount of the light of the plurality of wavelengths.

Effects of the Invention

[0009] According to a typical embodiment of the present invention, clear wavelength-separated image data can be acquired. Problems, configurations, and effects other than those described above will be clarified by the description of the following examples.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

[0011] <Biometric authentication example> FIG. 1 is an explanatory diagram showing an example of biometric authentication. FIG. 1 shows an example of performing biometric authentication by capturing an image of three fingers. The authentication device captures an image of a finger illuminated by a light source that outputs light of multiple wavelengths, and generates captured image data Is of the finger. The captured image data Is of the finger is RGB image data including R component image data Ir, G component image data Ig, and B component image data Ib. Here, as an example, a case will be described in which the light emitted from the light source includes light of a near-infrared wavelength and light of a green wavelength, the luminance of the G component image data Ig is the highest, and luminance saturation is likely to occur.

[0012] For this reason, the authentication device generates near-infrared wavelength separated image data Ii and green light wavelength separated image data Ic from the captured image data Is by wavelength separation. The authentication device controls the light amount of the light source so that the luminance of each of the near-infrared wavelength separated image data Ii and the green light wavelength separated image data Ic becomes the target luminance, using the G component image data Ig that has the highest luminance among the R component image data Ir, the G component image data Ig, and the B component image data Ib.

[0013] Specifically, for example, the authentication device (A) Relationship between the luminance of the R component image data Ir and the luminance of each of the near-infrared wavelength separation image data Ii and the green wavelength separation image data Ic (B) The relationship between the luminance of the G component image data Ig and the luminance of each of the near-infrared wavelength separation image data Ii and the green wavelength separation image data Ic. (C) Relationship between the luminance of the B - component image data Ib and the luminance of each of the near - infrared wavelength - separated image data Ii and the green - wavelength - separated image data Ic Based on this, the light amount of the light source is adjusted so that the luminance of the G - component image data Ig is within a predetermined range, and the luminance of each of the near - infrared wavelength - separated image data Ii and the green - wavelength - separated image data Ic becomes the target luminance. Thereby, both suppression of luminance saturation of the captured image data Is and stabilization of the luminance of each of the near - infrared wavelength - separated image data Ii and the green - wavelength - separated image data Ic are achieved.

[0014] <Configuration example of the imaging device and the authentication device> FIG. 2 is a block diagram showing a configuration example of the imaging device and the authentication device according to the first embodiment. The imaging device 200 captures the fingers of the hand 210 held above the upper - surface portion 200B of the housing 200A as a subject. In the first embodiment, as an example, the index finger 211, the middle finger 212, and the ring finger 213 are used as the subjects (imaging targets). However, the fingers 211 to 213 serving as the subjects may include two or more of the ten fingers of both hands 210. The back side of the hand 210 of the fingers 211 to 213 is referred to as the front side of the fingers 211 to 213, and the palm side of the hand 210 of the fingers 211 to 213 is referred to as the back side of the fingers 211 to 213.

[0015] In FIG. 2, the imaging device 200 includes a housing 200A, an imaging unit 201, a light source 202, and a data memory 206. The authentication device 208 is a device in which a controller 207 is connected to the imaging device 200. The housing 200A is attached or placed (hereinafter, collectively referred to as "installed") on, for example, the installation surface 220. The installation surface 220 may be a surface of a table parallel to the ground such as the ground, the ceiling surface, or a desk, or a surface perpendicular to the ground such as a wall. An axis orthogonal to the installation surface 220 is defined as the Z - axis, the direction away from the installation surface 220 on the Z - axis is defined as the + Z direction, and the direction approaching the installation surface 220 is defined as the - Z direction. Also, the installation surface 220 is parallel to the XY plane. The XY plane is a plane spanned by the X - axis and the Y - axis.

[0016] As shown in FIG. 2, the imaging device 200 and the authentication device 208 are installed such that the hand 210 is held over the upper panel portion 200B. In this case, the X-axis is the longitudinal direction of the finger when the hand 210 is presented. The Y-axis is the arrangement direction of the fingers 211 to 213.

[0017] The housing 200A contains an imaging unit 201 and a plurality of light sources 202 (in FIG. 1, light sources 202-1 and 202-2). When not distinguishing between the light sources 202-1 and 202-2, they are simply denoted as the light source 202. Further, a first optical filter 203 is provided between the imaging unit 201 and the upper panel portion 200B of the housing 200A.

[0018] The imaging unit 201 receives the subject light that has passed through the first optical filter 203. The subject light is the light (reflected light) obtained by reflecting the irradiation light from the light source 202 on the subject. The first optical filter 203 transmits only light of a specific wavelength. This prevents the imaging unit 201 from receiving unnecessary light and suppresses the generation of noise in the captured image data Is. The imaging unit 201 and the upper panel portion 200B of the housing 200A face the presented hand 210.

[0019] Further, in the region of the upper panel portion 200B existing in the +Z direction from the imaging unit 201, a light-transmitting plate 205 is provided that allows the light reflected by a living body such as the fingers 211 to 213 to pass through the irradiation light of the light source 202. The light-transmitting plate 205 is composed of a transparent member such as acrylic or glass, for example. Also, a film that allows only light of a specific wavelength to pass through may be attached to the light-transmitting plate 205. This can make it difficult to visually recognize the inside of the imaging device 200 from the outside.

[0020] In addition, the first optical filter 203 and the second optical filter 204 may be polarizing filters. Thereby, among the light components irradiated on a living body such as the fingers 211 to 213 and reflected, the specular reflection component on the skin surface can be reduced. Therefore, the imaging device 200 can image the blood vessel image of the living body more clearly. Further, the second optical filter 204 may be a band-pass filter that transmits only a specific wavelength of the irradiation light from the light source 202. Thereby, the imaging unit 201 can receive light of a specific wavelength more efficiently.

[0021] The imaging unit 201 is composed of, for example, a color imaging device such as a CMOS (Complementary Metal Oxide Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and generates image data of a plurality of colors such as RGB. The imaging surface of the imaging unit 201 faces the upper panel portion 200B.

[0022] The imaging unit 201 receives the light incident from outside the housing 200A through the light-transmitting plate 205 and the first optical filter 203 of the upper panel portion 200B on the imaging surface, and performs photoelectric conversion. The imaging unit 201 is connected to the data memory 206, and stores the image data Is subjected to photoelectric conversion in the data memory 206.

[0023] The image data Is is finger image data including a plurality of biological information such as blood vessels of the finger, unevenness of the skin surface such as fingerprints, and color information of the skin surface based on differences in light absorption characteristics of skin tissues such as melanin and keratin. It may be image data Is (finger blood vessel image data) showing only the blood vessels of the finger, or image data Is (finger surface image data) representing only the unevenness of the skin surface such as fingerprints and the color information of the skin surface based on differences in light absorption characteristics of skin tissues such as melanin and keratin. Hereinafter, the finger image data, the finger blood vessel image data, and the finger surface image data are collectively referred to as finger image data Is. The data memory 206 is connected to the controller 207.

[0024] The light source 202 irradiates light onto a subject existing in the +Z direction from the upper panel portion 200B through the second optical filter 204. When photographing the blood vessels of a finger, the irradiation light from the light source 202 is, for example, near-infrared light. Also, when photographing the skin surface of a finger, the irradiation light from the light source 202 is, for example, visible light such as green or blue.

[0025] The light source 202 is connected to a controller 207 outside the housing 200A. The controller 207 controls the amount of light irradiated from the light source 202. Also, the controller 207 detects the positions of the fingers 211 to 213, extracts features of blood vessels and fingerprints within the fingers 211 to 213 from the finger image data. Further, the controller 207 may authenticate a plurality of finger image data stored in the data memory 206.

[0026] Specifically, for example, the controller 207 acquires two pieces of finger image data Is from the data memory 206, and authenticates whether the index fingers 211, middle fingers 212, and ring fingers 213 of both pieces of finger image data Is are the index fingers 211, middle fingers 212, and ring fingers 213 of the same person based on the features of the blood vessels of the fingers and the features of the skin surface of the fingers.

[0027] FIG. 3 is a block diagram showing a first block configuration example of the photographing device 200 and the authentication device 208 according to the first embodiment. The photographing device 200 includes a light source control unit 300. The light source control unit 300 controls the amount of light irradiated from the light source 202. The light source control unit 300 is included in the controller 207 shown in FIG. 2. The computer 310 includes an authentication function. The computer 310 is included in the controller 207 shown in the figure 2 and is included in the controller 207 shown in FIG.

[0028] Computer 310 includes a processor 311, a memory device 312, an input device 313, an output device 314, and a communication interface (communication IF) 315. The processor 311, the memory device 312, the input device 313, the output device 314, and the communication IF 315 are connected by a bus 316. The processor 311 controls the computer 310. The memory device 312 serves as the working area of the processor 311. Also, the memory device 312 is a non - temporary or temporary recording medium that stores various programs and data. Examples of the memory device 312 include a ROM (Read Only Memory), a RAM (Random Access Memory), an HDD (Hard Disk Drive), and a flash memory. The input device 313 inputs data. Examples of the input device 313 include a keyboard, a mouse, a touch panel, a numeric keypad, and a scanner. The output device 314 outputs data. Examples of the output device 314 include a display, a printer, and a speaker. The communication IF 315 connects to a network and transmits and receives data.

[0029] Examples of the programs stored in the memory device 312 described above include an image processing program, a light source control program, and an authentication program. The image processing program is a program that causes the processor 311 to generate image data based on the output signal from the imaging unit 201. The light source control program is a program that causes the processor 311 to increase or decrease the amount of irradiation light from the light source 202. The authentication program is a program that causes the processor 311 to authenticate the identity of the fingerprint image data Is stored in the memory device 312. Although examples of software implementation of each function of image processing, light source control, and authentication have been described, each function of image processing, light source control, and authentication may be implemented by a dedicated circuit.

[0030] That is, the imaging device 200 that does not include the light source control unit 300 is the imaging device 200 shown in FIG. 2, and the imaging device 200 that includes the light source control unit 300 is the imaging device 200 shown in FIG. 3. Further, the authentication device 208 having the functions of image processing, light source control, and authentication includes the light source control unit 300 and the computer 310, and corresponds to the authentication device 208 in FIGS. 2 and 3.

[0031] FIG. 4 is a block diagram showing a second block configuration example of the imaging device 200 and the authentication device 208 according to the first embodiment. The imaging device 200 and the authentication device 208 shown in FIG. 4 incorporate a computer 310. The data memory 206 is realized by a storage device 312. The light source control unit 300 is realized by causing the processor 311 to execute a program stored in the storage device 312. Further, the authentication function is realized by causing the processor 311 to execute a program stored in the storage device 312. If the computer 310 does not have an authentication function, it is the imaging device 200, and if the computer 310 has an authentication function, it is the authentication device 208.

[0032] Before authentication, the computer 310 may register the user ID and the password in the storage device 312 in association with the finger image data Is of the user by receiving the user ID and the password by the input device 313 or wirelessly receiving the user ID and the password from an IC chip or a communication terminal possessed by the user by the communication IF 315.

[0033] Further, the computer 310 may identify the fingerprint image data Is associated with the user ID and password stored in the storage device 312 by obtaining the user ID, password, and fingerprint image data Is from the input device 313 or the communication IF 315 as described above, and authenticate both pieces of fingerprint image data Is (so-called one-to-one authentication). By identifying the user ID with the password in the pre-authentication stage and authenticating it with the fingerprint image data Is associated with the user ID, more accurate authentication becomes possible. Note that the computer 310 may identify the fingerprint image data Is that matches the currently input fingerprint image data Is from the group of fingerprint image data stored in the storage device 312 (so-called one-to-N authentication).

[0034] The irradiation light from the light source 202 in FIGS. 2 to 4 includes light of a plurality of different wavelengths. When photographing the blood vessels of a finger, the irradiation light from the light source 202 is, for example, near-infrared light. Also, as light for photographing biometric information on the skin surface such as a fingerprint of a finger, light having a wavelength such as blue or green can be used. The irradiation light of a plurality of wavelengths from the light source 202 can independently control the amount of light irradiated. Therefore, the irradiation light of a plurality of wavelengths from the light source 202 can be irradiated simultaneously, or each wavelength can be irradiated individually at different timings.

[0035] <Biometric Image Capturing Process> FIG. 5 is a flowchart showing an example of the biometric image capturing process procedure according to the first embodiment. In FIG. 5, as an example, the execution entity is the authentication device 208, but it may also be the imaging device 200.

[0036] When the imaging process is started in step S501, the authentication device 208 controls the light source 202 by the light source control in step S502 to perform light irradiation. At this time, the light source 202 irradiates light of a plurality of wavelengths simultaneously.

[0037] Next, the authentication device 208 performs image capturing in step S503, and performs wavelength separation processing on the acquired image data Is in step S504. As a result, the image data Is is separated into a plurality of wavelength separation image data Ii and Ic corresponding to the light components of the respective irradiated wavelengths. Here, an example of separating into two wavelength separation image data will be described, but another wavelength can be set and it can also be separated into three wavelength separation image data. For example, by using the wavelength separation image data corresponding to the light of the blue wavelength, it is possible to increase the types of biometric features and improve the authentication accuracy.

[0038] Next, in step S505, the authentication device 208 performs finger detection processing by performing image processing on the image data Is acquired in step S503 and the wavelength separation image data Ii and Ic obtained in step S504. generate As an example of the finger detection processing, there is a process of binarizing the hand area by threshold processing on the luminance of each pixel of the image data, and detecting a finger based on the shape of the contour line of the binarized image data.

[0039] In the finger detection determination in step S506, the authentication device 208 determines whether a finger has been detected based on the result of the finger detection processing in step S505. If it is determined in the finger detection determination (step S506) that a finger has not been detected (step S506: No), the process returns to the light source control in step 402, and the finger detection processing is repeated.

[0040] On the other hand, if it is determined that a finger has been detected (step S506: Yes), the process proceeds to the process of calculating the luminance of the finger area in step S507, and the authentication device 208 calculates the luminance within the finger area of the image data Is acquired in step S503 and the wavelength separation image data Ii and Ic generated in S504.

[0041] In step S508, the authentication device 208 calculates each light quantity value of the plurality of light sources 202 at the time of the next imaging based on the luminance of the finger area calculated in step S507.

[0042] In the determination of the end of shooting in step S509, the authentication device 208 makes a determination on whether to end shooting, such as whether authentication has been completed or whether a time-out of the shooting time has occurred. When it is determined that shooting has ended (step S509: Yes), the authentication device 208 ends shooting in step S510. When it is determined that shooting has not ended (step S509: No), it returns to step S502 and repeats the light source control.

[0043] In the light source control of step S502, when the light quantity values of the respective light sources 202 have been calculated in step S508, the authentication device 208 irradiates the respective light sources 202 with the calculated light quantity values and performs image acquisition in step S503.

[0044] In the luminance calculation of the finger region in step S507, the authentication device 208 obtains the luminance information of the region for extracting biometric features among the finger regions detected by the finger detection process in step S505 for the image data Is including RGB components captured in step S503 and the plurality of wavelength-separated image data Ii, Ic generated in step S504.

[0045] The luminance information is, for example, the average luminance value of the pixels in the region for extracting biometric features within the finger region. When there are multiple detected fingers, the average luminance of all the detected finger regions may be used as the luminance information. Also, due to large individual differences in the living body and large fluctuations in hand postures, the luminance of some finger regions may be too large or too small.

[0046] In this way, when it is difficult to uniformly adjust the luminance of all fingers to be within an appropriate range, the authentication device 208 can suppress a decrease in authentication accuracy by calculating the luminance after excluding some fingers. For example, among the multiple fingers detected, the authentication device 208 excludes fingers with a luminance equal to or higher than the upper limit luminance at which luminance saturation occurs or fingers with a luminance equal to or lower than the lower limit luminance, and uses the average luminance of the finger regions of the remaining fingers to calculate the light quantity values of the respective light sources 202 in step S508. Thereby, even when it is difficult to adjust the luminance of all finger regions to be within an appropriate range, by setting the luminance of the remaining fingers excluding some fingers to be within an appropriate range, high authentication accuracy can be maintained.

[0047] Note that the appropriate range is, for example, a predetermined range from the upper limit value of the luminance at which luminance saturation does not occur (upper limit luminance) to the lower limit value of the luminance at which biometric information is not missing (lower limit luminance). As an example of the lower limit luminance, it can be set to the lower limit luminance within a range where the contrast between light and dark of the luminance can be sufficiently observed in the blood vessel region in the image data.

[0048] Regarding the calculation of the light quantity values of the respective light sources 202 in step S508, the luminances Lr, Lg, Lb of the respective color component image data Ir, Ig, Ib of the RGB image data Is captured by simultaneously irradiating light of multiple wavelengths and the luminances Li, Lc of the wavelength-separated image data Ii, Ic are used as luminance information.

[0049] The authentication device 208 adjusts the light quantity so that the luminance Lg of the color component image data (for example, the image data Ig) with the maximum sum of sensitivities to the light of each wavelength irradiated simultaneously among the captured RGB finger image data Is is within the appropriate range of luminance. Then, similarly for each wavelength-separated image data Ii, Ic, the authentication device 208 adjusts the light quantity values of the respective light sources 202 so that the luminances Li, Lc are within the appropriate range of luminance.

[0050] Hereinafter, the irradiation light of the light source 202 shall be near-infrared light and green light. An example of calculating the light quantity value of each light source 202 when irradiating the finger simultaneously with the light of these two wavelengths from the light source 202 and using the RGB image data Is acquired by the imaging unit 201 having sensitivity to near-infrared light and visible light is shown (step S508).

[0051] The luminances Lr, Lg, and Lb of the pixels in the finger region of the respective color component image data Ir, Ig, and Ib of the captured RGB image data Is can be expressed by the following formulas (1) to (3) as functions of the luminances Li and Lc of the pixels of the wavelength-separated image data Ii of the near-infrared light and the wavelength-separated image data Ic of the green light irradiated by the light source 202.

[0052] Lr = Fr(Li, Lc) ··· (1) Lg = Fg(Li, Lc) ··· (2) Lb = Fb(Li, Lc) ··· (3)

[0053] The functions Fr, Fg, and Fb are functions that take as inputs the luminances Li and Lc of the pixels in the finger region of the wavelength-separated image data Ii and the wavelength-separated image data Ic. Also, the wavelength-separated image data Ii and the wavelength-separated image data Ic are obtained by the authentication device 208 performing wavelength separation processing (S504) based on the luminances Lr, Lg, and Lb of the pixels in the finger region of the respective color component image data Ir, Ig, and Ib of the RGB image data Is. The luminances Li and Lc of the pixels in the finger region of the wavelength-separated image data Ii and the wavelength-separated image data Ic can be expressed by the following formulas (4) and (5) as functions of the near-infrared light quantity Qi and the green light quantity Qc irradiated by the light source 202, respectively.

[0054] Li = Fi(Qi) ··· (4) Lc = Fc(Qc) ··· (5)

[0055] The functions Fi and Fc are functions that take the values of the near-infrared light quantity Qi and the green light quantity Qc as inputs, respectively. The authentication device 208 solves the system of simultaneous equations (1) to (3) above using the luminances Lr, Lg, and Lb of the pixels within the finger region of the color component image data Ir, Ig, and Ib of the RGB image data Is, thereby obtaining the luminances Li and Lc of the pixels within the finger region of the wavelength-separated image data Ii and the wavelength-separated image data Ic, which are two unknowns.

[0056] The authentication device 208 obtains the relationship between the irradiation light quantities Qi and Qc of the light of the two wavelengths, and the luminances Li and Lc of the wavelength-separated image data Ii and the wavelength-separated image data Ic, from the above formulas (4) and (5). Then, the authentication device 208 adjusts the irradiation light quantities (Qi, Qc) of each light source 202 such that the luminances Lr, Lg, and Lb of each pixel of the observed color component image data Ir, Ig, and Ib of R, G, and B are within an appropriate range, and the luminances Li and Lc of the wavelength-separated image data Ii and Ic are each a predetermined luminance. As the predetermined luminance, for example, it can be a luminance between the upper limit luminance at which the luminance does not saturate and the lower limit luminance at which the contrast of the light and dark of the luminance can be sufficiently observed in the blood vessel region and other biological regions.

[0057] The luminances Li and Lc in the above formulas (4) and (5) may be representative values (average value, maximum value, minimum value, median value, mode value) of the pixel group within the finger region.

[0058] For example, assume that the color component image data in which the sum of the sensitivities of the near-infrared light and the green light simultaneously irradiated on the finger from the light source 202 is the largest among the color component image data Ir, Ig, and Ib is the G component image data Ig. When shooting by simultaneously irradiating the finger with near-infrared light and green light from the light source 202, the luminance Lg of the G component image data Ig is the largest, and it is easy to enter a state where biological information is missing, such as luminance saturation. In a state where the biological information of the G component image data Ig is missing, there may also be a possibility that the biological information is missing in the wavelength-separated image data Ii and Ic generated using the G component image data Ig.

[0059] Therefore, the authentication device 208 adjusts the irradiation light amounts (Qi, Qc) of the respective light sources 202 while ensuring that the luminance of the color component image data (in this example, G component image data Ig) at which the sum of the sensitivities of the plurality of lights irradiated by the light source 202 is maximized falls within an appropriate range, and that the luminances Li, Lc of the wavelength-separated image data Ii, Ic become predetermined luminances. As a result, clear wavelength-separated image data Ii, Ic can be obtained. The predetermined luminance can be, for example, a luminance between the upper limit luminance at which luminance does not saturate and the lower limit luminance at which the contrast of light and dark of luminance can be sufficiently observed in the blood vessel region and other biological regions.

[0060] In the example where the sum of the sensitivities of the near-infrared light and green light simultaneously irradiated on the finger is maximized in the case of the G component image data Ig, the authentication device 208 adjusts the light amount of the light source 202 using at least the luminance Lg of the G component image data Ig. In this case, the authentication device 208 may also adjust the light amount of the light source 202 using the luminances Lr, Lb of the R component image data Ir and B component image data Ib in combination. By keeping the luminances Lr, Lg, Lb of the respective color component image data Ir, Ig, Ib of the RGB image data Is required for generating the wavelength-separated image data Ii, Ic within an appropriate range, clearer wavelength-separated image data Ii, Ic can be obtained.

[0061] Next, an example of a light source control method will be described on the assumption that the luminances Lr, Lg, Lb of the color component image data Ir, Ig, Ib obtained from the observed RGB image data Is can be expressed by a linear combination of the luminances Li, Lc of the plurality of wavelength-separated image data Ii, Ic. Assuming that the luminances Lr, Lg, Lb of the color component image data Ir, Ig, Ib have linearity with the luminances Li, Lc of the wavelength-separated image data Ii, Ic corresponding to the components of the near-infrared light irradiated by the light source 202, they can be expressed by the following formulas (6) to (8).

[0062] Lr = Ar × (αR × Li + βR × Lc) ··· (6) Lg = Ag × (αG × Li + βG × Lc) ··· (7) Lb = Ab × (αB × Li + βB × Lc) ··· (8)

[0063] In the above formulas (6) to (8), Ar, Ag, and Ab are coefficients that take arbitrary values. αR, αG, and αB are known coefficients representing the light reception sensitivities of the respective color components R, G, and B of the RGB image data Is captured by the imaging unit 201 at the wavelength of near-infrared light. βR, βG, and βB are known coefficients representing the light reception sensitivities of the respective color components R, G, and B of the RGB image data Is at the wavelength of green light. Further, the relationship between the luminance Li of the near-infrared light wavelength-separated image data Ii and the near-infrared light quantity Qi can be represented by the following formula (9), and the relationship between the luminance Lc of the green light wavelength-separated image data Ic and the green light quantity Qc can be represented by the following formula (10).

[0064] Li = αi × Qi ··· (9) Lc = βc × Qc ··· (10)

[0065] αi is a coefficient representing the relationship between the near-infrared light quantity Qi and the luminance Li of the wavelength-separated image data Ii. βc is a coefficient representing the relationship between the green light quantity Qc and the luminance Lc of the wavelength-separated image data Ic. The near-infrared light quantity Qi and the green light quantity Qc are known values that can be read as the values of the light quantity of the light source 202 set in the authentication device 208 when the luminance Li and the luminance Lc are observed. The coefficient αi is obtained from the relationship between the near-infrared light quantity Qi and the observed luminance Li by the above formula (9), and the coefficient βc is obtained from the relationship between the green light quantity Qc and the observed luminance Lc by the above formula (10).

[0066] The authentication device 208 can obtain the luminances Li and Lc of the wavelength-separated image data Ii and Ic, which are two unknowns, by solving the system of simultaneous equations (6) to (8) above using the luminances Lr, Lg, and Lb of the color component image data Ir, Ig, and Ib of the observed RGB image data Is and the known light reception sensitivities αR, αG, αB, βR, βG, and βB of the respective color components R, G, and B of the RGB image data Is.

[0067] Further, the authentication device 208 obtains the coefficient αi from the relationship between the amount of near-infrared light Qi set in the current captured frame and the luminance Li obtained by solving the simultaneous equations of the above (6) to (8) based on the relationship of the above equations (9) and (10). Then, the authentication device 208 obtains the coefficient βc from the relationship between the amount of green light Qc and the luminance Lc obtained by solving the simultaneous equations of the above (6) to (8).

[0068] In the next captured frame, the authentication device 208 adjusts the amount of near-infrared light Qi and the amount of green light Qc so that the luminances Li and Lc of the calculated wavelength-separated image data Ii and Ic become a predetermined luminance. The predetermined luminance is, for example, a value at which the contrast between the brightness and darkness of the luminance in the blood vessel region or the living body region can be sufficiently observed. The authentication device 208 is within an appropriate range so that luminance saturation or the like does not occur in the G component image data Ig ((αG + βG) is the largest) among the sums of the light reception sensitivities for each color component of the light of the wavelengths of at least two light sources 202 {(αR + βR), (αG + βG), (αB + βB)}. The irradiation light amounts (Qi, Qc) of each light source 202 in the next captured frame are adjusted so that the luminance becomes the luminance within the appropriate range and the luminances Li and Lc of the wavelength-separated image data Ii and Ic are each within the appropriate range. The luminance within the appropriate range is, for example, a luminance between the upper limit luminance at which luminance saturation does not occur and the lower limit of the luminance at which the contrast between the brightness and darkness of the luminance in the blood vessel region or the living body region can be sufficiently observed.

[0069] Thereby, it is possible to suppress the occurrence of luminance saturation in the G component image data Ig where luminance saturation is likely to occur and make the luminances Li and Lc of the wavelength-separated image data Ii and Ic be luminance values within an appropriate range. Furthermore, by performing light source control so that the luminances Lr and Lb of the R component image data Ir and the B component image data Ib are also within an appropriate range, not only the G component image data Ig but also the sharpness of the wavelength-separated image data Ii and Ic can be further enhanced.

[0070] Next, an example of a method for more efficiently controlling the light source by simplifying the relationship between the light of a plurality of wavelengths irradiated by the light source 202 and the captured RGB image data Is will be described. Similar to the previous description, the authentication device 208 irradiates the finger simultaneously with near-infrared light and green light as irradiation light by the light source 202, and the imaging unit 201 captures the RGB image data Is with an imaging unit 201 that is sensitive to near-infrared light and visible light.

[0071] As a characteristic of the sensitivity characteristics of the RGB color component image data Ir, Ig, and Ib of the imaging unit 201, assume a case where all of the R component image data Ir, G component image data Ig, and B component image data Ib have substantially the same sensitivity to near-infrared light.

[0072] Also, assume that the G component image data Ig has the highest sensitivity to green light and the R component image data Ir has almost no sensitivity.

[0073] Based on these assumptions, in the above equations (6) and (7), αR can be regarded as approximately equal to αG, and in the above equation (6), βR can be regarded as approximately equal to 0. Also, assuming Ar = Ag = 1 for arbitrary variables, the luminance Lr of the R component image data Ir represented by the above equation (6) can be expressed only by the luminance Li of the wavelength-separated image data Ii as shown in the following equation (11), and the G component image data Ig represented by the above equation (7) can be expressed as a linear combination of the luminances Li and Lc of the wavelength-separated image data Ii and Ic as shown in the following equation (12).

[0074] Lr = αR × Li ··· (11) Lg = αR × Li + βG × Lc ··· (12)

[0075] When the finger is irradiated with near-infrared light and green light simultaneously, in the G component image data Ig that is sensitive to both wavelengths of light and has a higher luminance than the R component image data Ir, the authentication device 208 controls the light amount of the light source 202 so that the luminances Li and Lc of the wavelength-separated image data Ii and Ic are within an appropriate range where luminance saturation and the like do not occur, and resets the near-infrared light amount Qi and the green light amount Qc.

[0076] In this example, since the R-component image data Ir has sensitivity αR only to near-infrared light, the authentication device 208 first obtains the luminance Li from the observed luminance Lr and the known coefficient αR based on the above formula (11). Next, the authentication device 208 obtains the coefficient αi from the relationship between the amount of near-infrared light Qi set in the current capture frame and the luminance Li obtained by the above formula (11) based on the above formula (9). The authentication device 208 determines the amount of near-infrared light Qi in the next capture frame so that the luminance Li of the wavelength-separated image data Ii and the luminance Lr of the R-component image data Ir become luminances within a predetermined range based on the coefficient αi obtained by the above formula (9) and the above formula (11).

[0077] By first obtaining the sensitivity αR indicating the relationship between the luminance Lr of the R-component image data Ir and the luminance Li of the wavelength-separated image data Ii by the above formula (11), the authentication device 208 can obtain the sensitivity βG indicating the relationship between the luminance Lg of the G-component image data Ig and the luminance Lc of the wavelength-separated image data Ic after subtracting the influence of the wavelength-separated image data Ii in the above formula (12).

[0078] Next, based on the above (12) and the above formula (10), the authentication device 208 calculates the amount of green light using the luminance Lg of the G-component image data Ig and the luminance Lc of the wavelength-separated image data Ic. Specifically, for example, the authentication device 208 calculates the amount of green light Qc when the luminance Lg of the G-component image data Ig is within an appropriate range and the luminance Lc of the wavelength-separated image data Ic is a predetermined luminance within an appropriate range based on the above formula (12) and the above formula (10). As the predetermined luminance, for example, it can be a luminance between the upper limit luminance at which the luminance does not saturate and the lower limit luminance at which the contrast of light and dark of the luminance can be sufficiently observed in the blood vessel region and the living body region.

[0079] Thus, even when photographing a subject by irradiating light of two wavelengths simultaneously, by simplifying the relationship between the RGB image data Is and the light of the irradiated wavelengths, the authentication device 208 can perform individual light source control of near-infrared light and green light, and can efficiently control the light amount of each wavelength light.

[0080] The relationships between the luminances Lr, Lg, Lb of the color component image data Ir, Ig, Ib and the luminances Li, Lc of the wavelength-separated image data Ii, Ic described by the above formulas (1) to (12), and the irradiation light amount of the light source 202 are considered to vary in real time with changes in the position and posture of the finger presented to the authentication device 208.

[0081] Therefore, the relationships of the above formulas (1) to (12) are calculated for each of the color component image data Ir, Ig, Ib and the wavelength-separated image data Ii, Ic obtained from continuously captured same frame (RGB image data Is). Then, after calculating the relationships of the above formulas (1) to (12) for each frame, in the shooting of the next frame, the authentication device 208 adjusts the light amount of each light source 202 so that clear color component image data Ir, Ig, Ib and wavelength-separated image data Ii, Ic can be obtained.

[0082] In this way, by calculating while updating the relationships between the luminances Lr, Lg, Lb of the color component image data Ir, Ig, Ib and the luminances Li, Lc of the wavelength-separated image data Ii, Ic and the irradiation light amount of the light source 202 in real time, it becomes possible to acquire clear color component image data Ir, Ig, Ib and wavelength-separated image data Ii, Ic corresponding to changes in the position and posture of the finger.

[0083] Next, the relationships between the luminances Lr, Lg, Lb of the color component image data Ir, Ig, Ib and the luminances Li, Lc of the wavelength-separated image data Ii, Ic and the irradiation light amount of the light source 202 are obtained for each continuously captured frame, and the calculation method of the irradiation light amount of each light source 202 at the time of shooting the next frame will be described.

[0084] Here, a case where the luminance Lr of the R-component image data Ir can be expressed by the above formula (11) and the luminance Lg of the G-component image data Ig can be expressed by the above formula (12) will be described as an example. From the above formula (11), the luminance Lr of the R-component image data Ir is proportional to the luminance Li of the wavelength-separated image data Ii of near-infrared light. Since, from the above formula (9), the luminance Li of the wavelength-separated image data Ii is in a proportional relationship with the near-infrared light quantity Qi, the luminance Lr of the R-component image data Ir is in a proportional relationship with the near-infrared light quantity Qi.

[0085] Therefore, when calculating the average luminance Lrav of the luminance Lr in the finger region in the R-component image data Ir, the coefficient a (proportionality constant) can be obtained from the relationship Lrav = a × near-infrared light quantity Qi. From the above formula (11), the luminance Lr of the R-component image data Ir and the luminance Li of the wavelength-separated image data Ii have a linear relationship. Therefore, a target luminance TLi of the luminance Li of the wavelength-separated image data Ii is set in advance, and from the relationship between the luminance Lr of the R-component image data Ir and the luminance Li of the wavelength-separated image data Ii, the luminance Lr of the R-component image data Ir when the luminance Li of the wavelength-separated image data Ii becomes the target luminance TLi is determined as the target luminance TR.

[0086] FIG. 6 is a graph showing the relationship between the near-infrared light quantity Qi and the luminance Lr of the R-component image data Ir. As shown in FIG. 6, the relationship between the near-infrared light quantity Qi = IR1 and the average luminance Lrav = R1 of the finger region of the R-component image data Ir observed when irradiated with the near-infrared light quantity IR1 can be linearly approximated (R1 = a × IR1). The authentication device 208 can estimate the near-infrared light quantity Qi = TIR for making the luminance Lr of the R-component image data Ir the target luminance TR from this relationship, and use it as the irradiation light quantity at the next shooting.

[0087] Next, the authentication device 208 adjusts the green light quantity Qc. Here, the following formula (13) is obtained by subtracting the above formula (11) from the above formula (12).

[0088] Lg - Lr = βG × Lc ··· (13)

[0089] The difference (Lg - Lr) between the luminance Lg of the G - component image data Ig and the luminance Lr of the R - component image data Ir is proportional to the luminance Lc of the green - light wavelength - separated image data Ic. From the above formula (10), the luminance Lc of the wavelength - separated image data Ic is proportional to the green - light quantity Qc. Therefore, it can be understood that the difference (Lg - Lr) between the luminance Lg of the G - component image data Ig and the luminance Lr of the R - component image data Ir is proportional to the green - light quantity Qc.

[0090] That is, if the luminance difference (Lg - Lr) within the finger region between the G - component image data Ig and the R - component image data Ir is defined as Dav, the coefficient b (proportionality constant) can be obtained from the relationship Dav = b×green - light quantity Qc. Thus, based on the above formula (13) and the above formula (10), the relationship between the observed difference (Lg - Lr) in luminance between the G - component image data Ig and the R - component image data Ir and the green - light quantity Qc is obtained. Thereby, the authentication device 208 can adjust the green - light quantity Qc so that the luminance Lg of the G - component image data Ig and the luminance Lc of the wavelength - separated image data Ic are of appropriate magnitudes.

[0091] Specifically, for example, in advance, while the luminance Lg of the G - component image data Ig is within an appropriate range, a target luminance TLc within an appropriate range is set for the luminance Lc of the wavelength - separated image data Ic. Also, from the relationship between the luminance difference Dav between the luminance Lg of the G - component image data Ig and the luminance Lr of the R - component image data Ir and the luminance Lc of the wavelength - separated image data Ic, when the luminance Lc of the wavelength - separated image data Ic becomes the target luminance TLc, the luminance difference Dav between the luminance Lg of the G - component image data Ig and the luminance Lr of the R - component image data Ir is defined as the luminance - difference target value TD.

[0092] Figure 7 is a graph showing the relationship between the green - light quantity Qc and the luminance difference Dav. As shown in Figure 7, the relationship between the luminance - difference value D1 of the finger region between the G - component image data Ig and the R - component image data Ir observed when the green - light quantity Qc is Qc = G1 and the green - light quantity G1 can be approximated by a straight line (D1 = b×G1). From this relationship, the authentication device 208 estimates the green - light quantity Qc = TG for making the luminance difference Dav between the luminance Lg of the G - component image data Ig and the luminance Lr of the R - component image data Ir the luminance - difference target value, and uses it as the irradiation light quantity at the next shooting.

[0093] In this way, the authentication device 208 obtains the light amount values at the time of the next frame shooting in the order of the near-infrared light amount Qi and the green light amount Qc, and performs light source control. Even when the position and posture of the finger fluctuate, clear color component image data Ir, Ig, Ib and wavelength separation image data Ii, Ic can be obtained corresponding to the fluctuations.

[0094] In this example, the case where the luminances Lr, Lg, Lb of the color component image data Ir, Ig, Ib and the luminances Li, Lc of the wavelength separation image data Ii, Ic have linearity and the light amount calculation can be simplified by several approximations has been described. However, even when the relationships between the luminances Lr, Lg, Lb of the color component image data Ir, Ig, Ib and the luminances Li, Lc of the wavelength separation image data Ii, Ic in the above formulas (1) to (5) do not have linearity, if the relationship between the light amount value of the light source 202 and the luminances Lr, Lg, Lb of the color component image data Ir, Ig, Ib and the luminances Li, Lc of the wavelength separation image data Ii, Ic is obtained for each frame, it goes without saying that light source control corresponding to the fluctuations in the position and posture of the finger can be performed.

Example

[0095] Example 2 is an example in which, in Example 1, when all the wavelength separation image data Ii, Ic cannot be adjusted to appropriate luminance values at the same time, light source control is performed to achieve high authentication accuracy. The same components as those in Example 1 are denoted by the same reference numerals, and the description thereof is omitted.

[0096] In Example 1, the authentication device 208 controls the irradiation light amount of each light source so that the plurality of wavelength separation image data Ii, Ic generated from the RGB image data Is taken by simultaneously irradiating light of a plurality of wavelengths on the finger becomes clear. Here, if all the wavelength separation image data Ii, Ic are to be set to the target luminance at the same time, luminance saturation may occur in some of the color component image data in the RGB image data Is due to individual differences in the living body and fluctuations in the finger posture.

[0097] When luminance saturation occurs, biometric information is missing in the finger regions of the wavelength-separated image data Ii and Ic, resulting in a decrease in authentication accuracy. Therefore, in order to maintain high authentication accuracy, priority is given to appropriately adjusting the luminance of the wavelength-separated image data that contributes significantly to improving accuracy among the plurality of wavelength-separated image data Ii and Ic. Even if it is not possible to appropriately adjust the luminance of all of the plurality of wavelength-separated image data Ii and Ic, high authentication accuracy can be maintained by preferentially setting the luminance of the wavelength-separated image data that contributes more to improving accuracy to an appropriate value.

[0098] FIG. 8 is a flowchart showing an example of a biometric image capturing process procedure according to the second embodiment. In FIG. 8, as an example, the execution entity is the authentication device 208, but it may also be the imaging device 200.

[0099] When the imaging process is started in step S801, the authentication device 208 controls the light source 202 by the light source control in step S802 to perform light irradiation. At this time, the light source 202 irradiates light of a plurality of wavelengths simultaneously. Next, the authentication device 208 performs imaging in step S803 to acquire image data Is, and performs wavelength separation processing on the image data Is in step S804. As a result, the authentication device 208 separates the image data Is obtained by the imaging in step S803 into each wavelength component of the irradiated light of a plurality of wavelengths, and generates a plurality of wavelength-separated image data Ii and Ic.

[0100] Next, the authentication device 208 performs a finger region detection process by performing image processing on the image data Is acquired in step S803 or the wavelength-separated image data Ii and Ic generated in step S804 in step S805.

[0101] In the finger detection determination in step S806, the authentication device 208 determines whether a finger has been detected based on the result of the finger detection process in step S805. If it is determined in the finger detection determination (step S805) that a finger has not been detected (step S806: No), the process returns to the light source control in step 602 to repeat the finger detection process.

[0102] In the finger detection determination (step S805), when it is determined that a finger has been detected (step S806: Yes), the process proceeds to the luminance calculation of the finger region in step S807. The authentication device 208 calculates the luminances Lr, Lg, Lb within the finger region of the color component image data Ir, Ig, Ib of the image data Is acquired in step S803, and the luminances Li, Lc within the finger region of the wavelength-separated image data Ii, Ic generated in step S804.

[0103] In step S808, the authentication device 208 determines whether the luminance of the image data can be adjusted based on the luminance within the finger region calculated in step S807 and the light quantity value of the light source 202. Then, the authentication device 208 determines whether all the wavelength-separated image data Ii, Ic can be adjusted to appropriate luminance ranges simultaneously by light source control.

[0104] If it is determined that all the wavelength-separated image data Ii, Ic can be adjusted (step S808: Yes), the process proceeds to the calculation of the first light quantity value of each light source in step S809. Then, the authentication device 208 calculates the light quantity value of each of the plurality of light sources 202 at the next shooting so that the image data Is acquired in step S803 and all the wavelength-separated image data Ii, Ic are within the appropriate luminance ranges based on the luminance of the finger region calculated in step S807.

[0105] If it is determined that the image data Is and all the wavelength-separated image data Ii, Ic cannot be adjusted (step S808: No), the process proceeds to the calculation of the second light quantity value of each light source 202 in step S810. Then, the authentication device 208 calculates the light quantity value of each of the plurality of light sources 202 at the next shooting so that the wavelength-separated image data with a high contribution rate to the improvement of accuracy is prioritized and within the appropriate luminance range.

[0106] The wavelength-separated image data to be prioritized is determined based on the contribution rate to the improvement of accuracy obtained in advance from experiments. For example, the false authentication rate when using the wavelength-separated image data Ii and the false authentication rate when using the wavelength-separated image data Ic are obtained through experiments and stored in the storage device 312.

[0107] The authentication device 208 acquires the false authentication rate from the storage device 312 and gives priority to the wavelength-separated image data with the lower false authentication rate. Further, the authentication device 208 may calculate the false authentication rate for each of the wavelength-separated image data Ii and Ic using the past authentication results (history information), and overwrite the storage device 312. Thereby, the authentication device 208 acquires the latest false authentication rate from the storage device 312 and gives priority to the wavelength-separated image data with the lower false authentication rate. Thereby, it becomes possible to realize more highly accurate authentication.

[0108] In the shooting end determination in step S811, the authentication device 208 makes a shooting end determination such as whether authentication has been completed or whether a shooting time timeout has occurred. When it is determined that shooting has ended (step S811: Yes), shooting ends in step S812. On the other hand, when it is determined that shooting has not ended (step S811: No), the process returns to step S802, and the authentication device 208 performs light source control.

[0109] In the light source control in step S802, in step S80 9 or step S810 when the light quantity value of each light source 202 has been calculated, the authentication device 208 irradiates with the calculated light quantity value of each light source 202 and performs image acquisition in step S803.

[0110] In the determination of whether the brightness of the image data can be adjusted in step S808, when the image data Is acquired in step S803 is RGB image data, the authentication device 208 determines that, for example, when brightness saturation occurs in the finger region in any of the color component image data Ir, Ig, Ib or when the brightness becomes below the threshold value, it is not possible to simultaneously adjust all the wavelength-separated image data Ii and Ic generated in step S804 to an appropriate brightness range.

[0111] Even if any of the color component image data Ir, Ig, and Ib of the RGB image data Is obtained in step S803 has a luminance within an appropriate range, when each light source 202 is irradiated with the light quantity value calculated by the first light quantity value calculation in step S809, luminance saturation may occur in any of the color component image data Ir, Ig, and Ib of the RGB image data Is obtained in the next photographing, or the luminance may become equal to or lower than the threshold value. Even in such a case, the authentication device 208 can determine that the luminance adjustment of the image data is impossible.

[0112] Also, in the determination of whether the luminance of the image data can be adjusted in step S808, the authentication device 208 can more accurately determine whether the luminance can be adjusted by making the determination over a plurality of frames that are continuously photographed. For example, when the first light quantity value calculation of each light source in step S809 is continuously performed for a plurality of frames, the RGB image data Is to be photographed and the wavelength-separated image data Ii and Ic generated are stabilized with approximately the same luminance, but there may be cases where luminance saturation occurs in some of the color component image data of the RGB image data Is or the luminance becomes equal to or lower than the threshold value. In such a case, the authentication device 208 can determine that the luminance adjustment of the image data is impossible.

[0113] The first light quantity value calculation of each light source 202 in step S809 is the light quantity value calculation when it is determined that all the wavelength-separated image data Ii and Ic can be adjusted to a luminance within an appropriate range. Therefore, it is the same as the light quantity value calculation of each light source in S508 in the flowchart of FIG. 4 in the first embodiment, and the description thereof is omitted in the second embodiment.

[0114] The calculation of the second light quantity value of each light source 202 in step S810 is a process of preferentially adjusting the light quantity value of the light source 202 so that the luminance of the wavelength-separated image with a high contribution rate to the improvement of accuracy is within an appropriate range among the plurality of wavelength-separated image data Ii and Ic. Specifically, in order to generate the priority wavelength-separated image data (priority image data) and the wavelength-separated image data Ii and Ic, the authentication device 208 adjusts the light quantity of each light source 202 so that the luminances Lr, Lg, and Lb of the color component image data Ir, Ig, and Ib of the RGB image data Is as the generation source are within an appropriate range.

[0115] Regarding the luminance of the wavelength-separated image data other than the priority image data, on the condition that the priority image data and the RGB image data Is do not deviate outside a predetermined range from the target luminance that is within an appropriate range, the authentication device 208 adjusts the light quantity of each light source 202 so that the luminance is within a predetermined range from the target luminance that is within an appropriate range.

Example

[0116] Example 3 is an example in which, in Example 1 and Example 2, when the influence of external light such as sunlight and indoor lighting is large, light source control is performed to capture a biological image more clearly. The same components as those in Example 1 are denoted by the same reference numerals, and the description thereof is omitted.

[0117] In Example 1 and Example 2, among the color component image data Ir, Ig, and Ib of the captured RGB image data Is, the method of performing light source control to capture a clear image using the color component image data with the maximum sum of the light reception sensitivities {(αR + βR), (αG + βG), (αB + βB)} for each color component of the light of a plurality of wavelengths irradiated from the light source 202 was described.

[0118] Since the sensitivity characteristics of the light with the wavelength irradiated by the imaging unit 201 are known in advance, it is premised that it is possible to determine before shooting which color component image data among the RGB image data Is is likely to have a higher luminance. However, due to the influence of very strong indoor lighting or the like of light in a specific wavelength band, it is conceivable that the color component image data that becomes the brightest among the color component image data Ir, Ig, and Ib of the captured RGB image data Is may vary. A light source control method in consideration of the influence of ambient light (external light) at the time of shooting will be described below.

[0119] FIG. 9 is a flowchart showing an example of a biological image capturing process procedure according to the third embodiment. In FIG. 9, as an example, the execution entity is the authentication device 208, but it may be the imaging device 200. From the start of shooting in step S901 to the calculation of the luminance of the finger region from the image data in step S907, the same processing as steps S501 to S507 in the flowchart of FIG. 5 in the first embodiment is performed, and thus the description thereof is omitted.

[0120] In step S908, the authentication device 208 calculates luminance information such as the average luminance of the finger region of each color component image data Ir, Ig, and Ib of the RGB image data Is calculated in step S907. Then, the authentication device 208 determines the color component image data having the highest luminance due to the influence of the irradiation light of the light source 202 and the external light.

[0121] In step S909, the authentication device 208 uses at least the luminance of the color component image data having the highest luminance determined in step S90 8 and calculates the light quantity values of the plurality of light sources 202 at the time of the next shooting based on the luminance of the finger region calculated in step S907.

[0122] Regarding the light source control in step S902 and the calculation of the light quantity values of the respective light sources 202 in step S909, the authentication device 208 uses the luminances Lr, Lg, and Lb of the respective color component image data Ir, Ig, and Ib of the RGB image data Is captured by simultaneously irradiating light of a plurality of wavelengths, and the luminances Li and Lc of the wavelength-separated image data Ii and Ic.

[0123] Among the captured finger image data Is, the light quantity is adjusted so that the luminances Lr, Lg, and Lb of the color component image data Ir, Ig, and Ib, which are the maximum among the sums of the light reception sensitivities for each color component of the light of each wavelength irradiated simultaneously {(αR + βR), (αG + βG), (αB + βB)}, are neither too large nor too small but within an appropriate range. Then, similarly for each wavelength-separated image data Ii and Ic, the light quantity values of each light source 202 are adjusted so that the luminances Li and Lc are within an appropriate range.

[0124] In Example 3, the irradiation light of the light source 202 is near-infrared light and green light. Then, an example of the calculation of the light quantity values of each light source 202 in step S909 is shown, which uses the RGB image data Is obtained by simultaneously irradiating these two lights on the finger and using an imaging unit 201 that is sensitive to near-infrared light and visible light.

[0125] The luminances Lr, Lg, and Lb of the captured color component image data Ir, Ig, and Ib can be expressed by the following formulas (14) to (16) as functions of the luminance Li of the wavelength-separated image data Ii corresponding to the component of the near-infrared light irradiated by the light source 202, the luminance Lc of the wavelength-separated image data Ic corresponding to the component of the green light, and the luminance Lo of the wavelength-separated image data Io corresponding to the component of the light outside a specific wavelength.

[0126] Lr = Fr2(Li, Lc, Lo) ··· (14) Lg = Fg2(Li, Lc, Lo) ··· (15) Lb = Fb2(Li, Lc, Lo) ··· (16)

[0127] The functions Fr2, Fg2, and Fb2 are functions that take as inputs the luminance Li of the wavelength-separated image data Ii, the luminance Lc of the wavelength-separated image data Ic, and the luminance Lo of the wavelength-separated image data Io. By solving the system of simultaneous equations of the above formulas (14) to (16) based on the luminances Lr, Lg, and Lb of the observed color component image data Ir, Ig, and Ib of each R, G, and B, the luminances Li, Lc, and Lo of each wavelength-separated image data Ii, Ic, and Io, which are three unknowns, can be obtained. Further, the luminance Li of the wavelength-separated image data Ii of near-infrared light and the luminance Lc of the wavelength-separated image data Ic of green light obtained by the wavelength separation process can be expressed by the following formulas (17) and (18) as functions of the near-infrared light quantity Qi and the green light quantity Qc, respectively.

[0128] Li = Fi2(Qi) ··· (17) Lc = Fc2(Qc) ··· (18)

[0129] The functions Fi2 and Fc2 are functions that take as inputs the near-infrared light quantity Qi and the green light quantity Qc, respectively. Further, since the external light is limited to a specific wavelength band, the luminance Lo of the wavelength-separated image data Io separated into the wavelength band of the external light component based on the luminances Lr, Lg, and Lb of the observed color component image data Ir, Ig, and Ib of each R, G, and B can be expressed by the following formula (19) as a function of the external light.

[0130] Lo = Fn(Lo) ··· (19)

[0131] The function Fn is a function that takes as input the light quantity value of the external light. However, basically, the external light is unknown and its light quantity cannot be controlled. Therefore, it is not possible to obtain the relationship of the above formula (19) or adjust the luminance Lo of the wavelength-separated image data I o of the external light component. Therefore, the authentication device 208 controls the irradiation light quantity of the light of each wavelength of the light source 202 and the external light the amount of lightThen, the relationships between the luminances Lr, Lg, Lb of the color component image data Ir, Ig, Ib and the luminances Li, Lc, Lo of the wavelength-separated image data Ii, Ic, Io (the above formulas (14) to (18)) are obtained. Then, the authentication device 208 adjusts the irradiation light amount of each light source 202 so that the luminances Lr, Lg, Lb of the observed color component image data Ir, Ig, Ib are within an appropriate range and the luminances Li, Lc of the wavelength-separated image data Ii, Ic are within an appropriate range of luminances.

[0132] At this time, assuming that the color component image data with the largest sum of the sensitivities of the near-infrared light, green light, and external light simultaneously irradiated on the finger among the color component image data Ir, Ig, Ib of the RGB image data Is is the B-component image data Ib ((ab + bb + cb) is the maximum value). ar, br, cr, ag, bg, cg, ab, bb, cb will be described later by the following formulas (20) to (22).

[0133] In this case, the luminance Lb of the B-component image data Ib is the largest, and it is likely to be in a state where biological information is missing, such as the luminance Lb being saturated. Therefore, while making the luminance Lb of the B-component image data Ib, for which the sum of the sensitivities of at least the near-infrared light and green light irradiated by the light source 202 and the sensitivity of the external light is the largest, fall within an appropriate range, the irradiation light amount of each light source 202 is adjusted so that the luminances Li, Lc of the wavelength-separated image data Ii, Ic are within an appropriate range of luminances, thereby obtaining clear wavelength-separated image data Ii, Ic.

[0134] In the case of the B - component image data Ib where the sum of the sensitivities of the near - infrared light and green light irradiated on the finger and the sensitivity of the external light is maximized, at least the luminance Lb of the B - component image data Ib is used to adjust the light amount of the light source 202. However, the authentication device 208 may also use the luminances Lr and Lg of the other R - component image data Ir and G - component image data Ig in combination to adjust the light amount of the light source 202. By keeping the luminances Lr, Lg, and Lb of the respective color - component image data Ir, Ig, and Ib of the RGB image data Is necessary for generating the wavelength - separated image data Ii and Ic within an appropriate range, clearer wavelength - separated image data Ii and Ic can be obtained.

[0135] Next, an example of a light - source control method will be described when it is assumed that the luminances Lr, Lg, and Lb of the color - component image data Ir, Ig, and Ib of the observed RGB image data Is can be expressed by a linear combination of a plurality of wavelength - separated image data Ii and Ic. Each of the photographed R, G, B color - component image data Ir, Ig, Ib the luminances Lr, Lg, Lb corresponds to the luminance Li of the wavelength - separated image data Ii corresponding to the component of the near - infrared light irradiated by the light source 202, the luminance Lc of the wavelength - separated image L data Ic corresponding to the component of the green light, and the luminance Lo of the wavelength - separated image data Io corresponding to the external - light component, and can be expressed by the following (20) to (22).

[0136] Lr = Br×(ar×Li + br×Lc + cr×Lo) ··· (20) Lg = Bg×(ag×Li + bg×Lc + cg×Lo) ··· (21) Lb = Bb×(ab×Li + bb×Lc + cb×Lo) ··· (22)

[0137] Br, Bg, and Bb are coefficients that take arbitrary values. ar, ag, and ab are known coefficients representing the light - receiving sensitivities of the color - component image data Ir, Ig, and Ib of the RGB image data Is photographed by the imaging unit 201 at the wavelength of the near - infrared light. br, bg, and bb are known coefficients representing the light - receiving sensitivities of the color - component image data Ir, Ig, and Ib of the RGB image data Is at the wavelength of the green light.

[0138] Cr, Cg, and Cb are known coefficients representing the light reception sensitivities of the color component image data Ir, Ig, and Ib of the RGB image data Is at the wavelength of external light. By solving the system of simultaneous equations of the above formulas (20) to (22), the unknown luminances Li, Lc, and Lo can be obtained. Also, the relationship between the luminance Li of the near-infrared light wavelength-separated image data Ii and the near-infrared light quantity Qi can be expressed by the following formula (23), and the relationship between the luminance Lc of the green light wavelength-separated image data Ic and the green light quantity Qc can be expressed by the following formula (24).

[0139] Li = Ai × Qi ··· (23) Lc = Bc × Qc ··· (24)

[0140] Ai is a coefficient representing the relationship between the near-infrared light quantity Qi and the luminance Li of the near-infrared light wavelength-separated image data Ii. Bc is a coefficient representing the relationship between the green light quantity Qc and the luminance Lc of the green light wavelength-separated image data Ic.

[0141] In this way, while considering the influence of the luminance Lo of the external light component, the luminances Lr, Lg, and Lb of the color component image data Ir, Ig, and Ib at which the luminance of the RGB image data Is represented by the above (20) to (22) is maximized fall within an appropriate range, and by adjusting the irradiation light quantity of each light source 202 so that the luminances Li and Lc of the wavelength-separated image data Ii and Ic are within appropriate ranges respectively, clear wavelength-separated image data Ii and Ic can be obtained.

Example

[0142] Example 4 will describe a method for verifying a plurality of biometric data using each wavelength-separated image data obtained by adjusting the irradiation light quantity of each light source 202 by light source control in Examples 1 to 3.

[0143] The wavelength-separated image data generated from the RGB image data Is captured by adjusting the irradiation light amount of each light source 202 contains different biological information for each wavelength. The authentication device 208 includes an extraction unit that extracts different biometric data from each wavelength-separated image data, and a verification unit that verifies a plurality of biometric data (multimodal biometric data) extracted by the extraction unit. The extraction unit and the verification unit are functions realized by causing the processor 311 to execute an extraction program and a verification program stored in the storage device 312, respectively. Thereby, higher-precision authentication is realized compared to the case of using single biometric data.

[0144] In the fourth embodiment, the verification unit calculates a reliability for each wavelength-separated image data (modality) based on the luminance of each wavelength-separated image data obtained after adjusting the light amount of the plurality of light sources 202. For example, a luminance that can more accurately capture biological features such as the contrast between the light and dark of the luminance in the blood vessel region without saturation of the luminance for each wavelength-separated image data is stored in the storage device 312 in advance as a target luminance.

[0145] It is set such that the smaller the difference between the luminance of each wavelength-separated image data obtained after adjusting the light amount and the above-described target luminance, the higher the reliability. Then, the verification unit determines the ratio when synthesizing the verification results of the biometric data of each modality based on the ratio of the reliability calculated for each modality. For example, when there are two wavelength-separated image data, the verification unit verifies the biometric data 1 and the biometric data 2 extracted from each wavelength-separated image data with the pre-registered biometric features, and calculates a verification score 1 and a verification score 2 corresponding to the similarity.

[0146] When combining two matching scores 1 and 2 into one matching score by linear combination or the like, the weight of the matching score at the time of linear combination is determined based on the ratio of the magnitudes of the reliability calculated for each wavelength-separated image data. As a result, the matching score of the wavelength-separated image data with higher reliability can be reflected in the combined score. Therefore, the reliability of the wavelength-separated image data that captures more accurately the biological features such as the contrast of light and dark in the blood vessel region is set high, and more accurate authentication is realized.

[0147] FIG. 10 is a flowchart showing an example of a biological image capturing process procedure according to Example 4. In FIG. 10, as an example, the execution entity is the authentication device 208, but it may be the imaging device 200. From the start of imaging in step S1001 to the calculation of the light quantity value of each light source in step S1008, since it is the same process as steps S501 to S508 in the flowchart of FIG. 5 in Example 1, the description is omitted.

[0148] In step S1009, the authentication device 208 determines the end of light source control based on the light quantity value of each light source 202, the captured RGB image data Is, and the luminance of each wavelength-separated image data. When it is determined that the light source control has ended (step S1009: Yes), the authentication device 208 makes an imaging end determination in the next step S1010. When it is determined that the light source control has not ended (step S1009: No), the process returns to the light source control in step S1002.

[0149] In the imaging end determination in step S1010, the authentication device 208 determines, for example, whether or not the timeout time has been reached since the start of imaging. When it is determined that the imaging has ended, such as when the timeout time has been reached (step S1010: Yes), the authentication device 208 ends the imaging in step S1011. When it is determined that the imaging has not ended (step S1010: No), the authentication device 208 extracts biological feature data for performing matching from each finger region of each wavelength-separated image data by extracting biological feature data in the extraction unit (step S1012).

[0150] Next, as described above, the authentication device 208 calculates the reliability of each wavelength-separated image data by the collation unit (step S1013). The authentication device 208 collates the multimodal biometric data extracted from each finger region of each wavelength-separated image data (near-infrared image data and green light image data) in step S1012 with the multimodal biometric data registered in the database in advance by the collation unit, and calculates a collation score (step S1014). The database is realized by the storage device 312 or the storage device 312 of another computer communicable with the computer 310. The collation score is calculated by collation such as template matching or feature point matching, for example.

[0151] In the authentication process of step S1015, the authentication device 208 synthesizes the collation scores of the biometric data of each modality of the multimodal biometric data calculated in step S1014 based on the reliability of each wavelength-separated image data (modality) obtained in step S1013, and determines whether authentication is possible for the synthesized collation score.

[0152] In the authentication determination, the authentication device 208 determines that authentication is successful if the collation score is greater than a preset threshold, and determines that authentication has failed if it is less than or equal to the threshold. The synthesis of the collation scores is, for example, a linear combination of the collation scores of each modality. The coefficient of the collation score of each modality in this linear combination is determined based on the reliability obtained in step S1013 as described above. Thereby, the ratio of the collation score of the modality with high reliability can be increased at the time of synthesis, and the authentication accuracy can be improved.

[0153] In the authentication completion determination of step S1016, the authentication device 208 determines whether authentication is completed as a result of performing the authentication process based on the collation score in step S1015. If it is determined that authentication is completed (step S1016: Yes), the authentication device 208 ends the photographing in step S1011. If it is determined that authentication has not yet been completed (step S1016: No), the process returns to the light source control in step S1002.

[0154] In the end determination of the light source control in step S1009, the authentication device 208 can make a determination based on whether the luminance of the captured RGB image data Is and the luminance of each wavelength-separated image data are within an appropriate range. Also, the authentication device 208 may perform the end determination of the light source control over a plurality of consecutive frames. By ending the light source control in a state where the luminance of each wavelength-separated image is within an appropriate range continuously over a plurality of frames, it becomes possible to make a more accurate and stable end determination of the light source control. Also, even when the luminance of all the wavelength-separated image data does not fall within an appropriate range, it is also possible to set a state where the luminance value is stable continuously as the end condition of the light source control.

[0155] In the calculation of the reliability of each wavelength-separated image data in step S1013, the authentication device 208 calculates the reliability of each wavelength-separated image data based on indicators such as whether the luminance of each wavelength-separated image data is close to an appropriate value. When there are a plurality of detected finger counts, the calculated reliability of each wavelength-separated image data can be applied to all the detected fingers. Also, it may be different reliabilities for each detected finger. In this way, the authentication device 208 can calculate the reliability based on luminance information such as the average luminance of each finger region detected for each wavelength-separated image data.

[0156] Also, there may be large individual differences in the living body and large fluctuations in hand postures, and the luminance of some finger regions may be too large or too small. Thus, when it is difficult to uniformly adjust the luminance of all fingers within an appropriate range, the authentication device 208 can improve the authentication accuracy by increasing the reliability of fingers with luminance within a more appropriate range and decreasing the reliability of fingers with luminance outside the appropriate range.

[0157] Note that the present invention is not limited to the above-described embodiments, and includes various modifications and equivalent configurations within the scope of the appended claims. For example, the above-described embodiments have been described in detail for easy understanding of the present invention, and the present invention is not necessarily limited to those having all the configurations described. Also, a part of the configuration of one embodiment may be replaced with the configuration of another embodiment. Further, the configuration of another embodiment may be added to the configuration of one embodiment. Also, for a part of the configuration of each embodiment, addition, deletion, or replacement with other configurations may be made.

[0158] In addition, each of the above-described configurations, functions, processing units, processing means, etc. may be realized in hardware, for example, by designing a part or all of them with an integrated circuit, or may be realized in software by a processor interpreting and executing a program for realizing each function.

[0159] Information such as programs, tables, files, etc. for realizing each function can be stored in a storage device such as a memory, a hard disk, an SSD (Solid State Drive), or a recording medium such as an IC (Integrated Circuit) card, an SD card, or a DVD (Digital Versatile Disc).

[0160] Also, the control lines and information lines show those considered necessary for explanation, and do not necessarily show all the control lines and information lines necessary for implementation. In practice, it may be considered that almost all the configurations are interconnected.

Explanation of Reference Numerals

[0161] 200 Imaging device 200A Housing 200B Top panel part 201 Imaging unit 202 Light source 203 First optical filter 204 Second optical filter 205 Translucent plate 206 Data memory 207 Controller 208 Authentication device 220 Installation surface 300 Light source control unit 310 Computer 311 Processor 312 Memory device 313 Input device 314 Output device

Claims

1. An irradiation unit that irradiates a living body with light of a plurality of different wavelengths; A photographing unit that photographs the living body irradiated by the irradiation unit and generates image data of the living body; An image processing unit that generates a plurality of wavelength-separated image data obtained by separating the plurality of wavelengths based on the image data generated by the photographing unit; A control unit that controls the irradiation light amount of the light of the plurality of wavelengths so that the luminance of the image data and the luminance of the plurality of wavelength-separated image data generated by the image processing unit are within a predetermined range; A photographing apparatus, comprising the above.

2. The photographing apparatus according to Claim 1, wherein the image data is composed of a plurality of attribute image data having different attributes, The photographing apparatus is characterized by this.

3. The photographing apparatus according to Claim 2, The control unit controls the irradiation light amount of the light of the plurality of wavelengths so that the luminance of specific attribute image data, among the plurality of attribute image data, in which the total sensitivity of the light of the plurality of wavelengths irradiated by the irradiation unit is maximized is within the predetermined range. The photographing apparatus is characterized by this.

4. The photographing apparatus according to Claim 1, The control unit controls the irradiation light amount of the light of the plurality of wavelengths so that the luminance of specific wavelength-separated image data, among the plurality of wavelength-separated image data, having a high contribution rate to the authentication accuracy is within a predetermined range. The photographing apparatus is characterized by this.

5. The photographing apparatus according to Claim 1, wherein the living body is a plurality of fingers, The control unit controls the irradiation light amount of the light of the plurality of wavelengths so that the luminance of the image data for the plurality of fingers and the luminance of regions of some fingers included in the plurality of wavelength-separated image data are within the predetermined range. The photographing apparatus is characterized by this.

6. The imaging device according to claim 3, wherein the control unit controls the irradiation light amounts of the lights of the plurality of wavelengths so that the luminance of specific attribute image data, among the plurality of attribute image data, in which the sum of the sensitivities of the lights of the respective wavelengths of the plurality of wavelengths and the sensitivity of the light of other wavelengths received by the imaging unit without being irradiated from the irradiation unit is maximized, falls within the predetermined range. An imaging device characterized by the above.

7. An irradiation unit that irradiates a living body with lights of a plurality of different wavelengths, an imaging unit that images the living body irradiated by the irradiation unit and generates image data of the living body, an image processing unit that generates a plurality of wavelength-separated image data obtained by separating the plurality of wavelengths based on the image data generated by the imaging unit, a control unit that controls the irradiation light amounts of the lights of the plurality of wavelengths so that the luminance of the image data and the luminance of the plurality of wavelength-separated image data generated by the image processing unit fall within a predetermined range, an extraction unit that extracts biometric data from each of the plurality of wavelength-separated image data, a collation unit that collates each of the plurality of biometric data extracted by the extraction unit with registered biometric data, An authentication device characterized by comprising the above.

Images