Identifying device, identifying method and program
The identification device improves biometric authentication accuracy by using pulsed light to differentiate between surface and internal reflections, enhancing feature detection for more reliable user identification.
Patent Information
- Application Number
- JP2024130062
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2018-05-30
- Filing Date
- 2024-08-06
- Publication Date
- 2024-11-06
- Estimated Expiration
- 2039-05-08
AI Technical Summary
Existing biometric authentication methods, such as those combining fingerprint and vein authentication, can be improved for higher accuracy.
An identification device that uses pulsed light with a specific pulse width to distinguish between surface reflection and internal scattering components, employing an image sensor to detect and process these components for enhanced biometric authentication.
Enhances biometric authentication accuracy by clearly distinguishing surface and internal biological features, reducing false acceptance and rejection rates.
Smart Images

Figure 2024156912000001_ABST
Abstract
Description
[Technical field]
[0001] The present disclosure relates to an identification device, an identification method, and a program. [Background technology]
[0002] In the field of personal identification, authentication is transitioning from password authentication to biometric authentication. In password authentication, authentication is based on a password entered by a user. On the other hand, in biometric authentication, authentication is based on information about a person's physical characteristics. Biometric authentication has the advantage of having less risk of forgetting, leaking, or brute force attacks. In biometric authentication, for example, personal identification information is obtained by shining light on a part of the user's body and observing the reflected light.
[0003] Patent Document 1 discloses a method for improving the accuracy of biometric authentication of a user. In the method described in Patent Document 1, a user's biometric authentication is performed by combining fingerprint authentication and vein authentication. Specifically, the surface of the user's finger is irradiated with light of a wavelength of 900 nm for fingerprint authentication and light of 750 nm for vein authentication. The user is authenticated based on the reflected light. [Prior art documents] [Patent documents]
[0004] [Patent Document 1] JP 2008-020942 A Summary of the Invention [Problem to be solved by the invention]
[0005] The present disclosure provides a technique that can further improve authentication accuracy. [Means for solving the problem]
[0006] An identification device according to an embodiment of the present disclosure includes a light source, an image sensor, a memory storing biometric data indicating a physical feature of a user, and a processor. The processor causes the light source to emit pulsed light having a pulse width of 0.2 ns to 1 μs to be irradiated to the user, causes the image sensor to detect at least a portion of the reflected pulsed light returning from the user, and outputs a signal corresponding to a two-dimensional distribution of the intensity of the at least a portion of the reflected pulsed light, and identifies the user by comparing the signal with the biometric data. Effect of the Invention
[0007] According to the technology of the present disclosure, more accurate biometric authentication is possible. [Brief description of the drawings]
[0008] [Figure 1A] FIG. 1A is a diagram illustrating an example of an identification device according to the present embodiment. [Figure 1B] FIG. 1B is a diagram showing an example of a change over time in the intensity of light reaching an image sensor. [Figure 1C] FIG. 1C is a diagram in which the horizontal axis represents the width of the input pulse light, and the vertical axis represents the amount of light detected by the sensor. [Figure 1D] FIG. 1D is a diagram showing an example of a schematic configuration of one pixel of an image sensor. [Figure 1E] FIG. 1E is a diagram illustrating an example of the configuration of an image sensor. [Figure 1F] FIG. 1F is a diagram showing an example of an operation within one frame. [Figure 1G] FIG. 1G is a flow chart outlining the operation of the control circuit. [Figure 2A] FIG. 2A is a diagram illustrating an example of an optical signal in which rectangular pulse light is emitted from a light source and returned from a user to an image sensor. [Figure 2B] FIG. 2B is a diagram illustrating another example of an optical signal in which rectangular pulse light is emitted from a light source and returned from a user and reaches an image sensor. [Figure 3A] FIG. 3A is a diagram showing an example of a timing chart when detecting a surface reflection component. [Figure 3B] FIG. 3B is a diagram showing an example of a timing chart when the internal scattering component is detected. [Figure 4A] FIG. 4A is a diagram illustrating an example of a distribution of a two-dimensional pattern light projected from a light source to a user. [Figure 4B] FIG. 4B is a diagram illustrating another example of the distribution of two-dimensional pattern light projected from the light source to the user. [Figure 4C] FIG. 4C is a diagram illustrating still another example of the distribution of two-dimensional pattern light projected from the light source to the user. [Figure 5A] FIG. 5A is a diagram that illustrates a user being photographed by a general camera. [Figure 5B] FIG. 5B is a diagram illustrating an example of capturing an image of a user using surface reflection components in this embodiment. [Figure 5C] FIG. 5C is a diagram illustrating an example of capturing an image of a user using internal scattering components in this embodiment. [Figure 6] FIG. 6 is a diagram for explaining the principle of measuring the round-trip time of pulsed light by the indirect TOF method. [Figure 7] FIG. 7 is a diagram showing an example of capturing an image of a user by the TOF method in this embodiment. [Figure 8] FIG. 8 is a flowchart illustrating an example of a process executed by the control circuit. [Figure 9] FIG. 9 is a diagram showing a schematic example of an identification device according to this embodiment mounted in a vehicle interior. [Figure 10] FIG. 10 is a diagram showing an example in which the identification device according to this embodiment is applied to a mobile terminal. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0009] The embodiments described below are all comprehensive or specific examples. The numerical values, shapes, materials, components, and arrangement positions of the components shown in the following embodiments are merely examples and are not intended to limit the present disclosure. In addition, among the components in the following embodiments, components that are not described in the independent claims showing the highest concept are described as optional components.
[0010] In the present disclosure, all or part of a circuit, unit, device, member or part, or all or part of a functional block in a block diagram may be implemented by one or more electronic circuits including a semiconductor device, a semiconductor integrated circuit (IC), or an LSI (large scale integration). The LSI or IC may be integrated into one chip, or may be configured by combining multiple chips. For example, functional blocks other than memory elements may be integrated into one chip. Here, although it is called an LSI or an IC, the name may change depending on the degree of integration, and it may be called a system LSI, a VLSI (very large scale integration), or an ULSI (ultra large scale integration). A Field Programmable Gate Array (FPGA), which is programmed after the manufacture of an LSI, or a reconfigurable logic device, which can reconfigure the connection relationship inside the LSI or set up the circuit partition inside the LSI, may also be used for the same purpose.
[0011] Furthermore, all or part of the functions or operations of the circuit, unit, device, member, or part can be executed by software processing. In this case, the software is recorded in one or more non-transitory recording media such as ROMs, optical disks, hard disk drives, etc., and when the software is executed by a processor, the functions specified in the software are executed by the processor and peripheral devices. The system or device may include one or more non-transitory recording media on which the software is recorded, a processor, and necessary hardware devices, such as interfaces.
[0012] Hereinafter, the embodiment will be specifically described with reference to the drawings.
[0013] (Embodiment) [1. Identification device] First, the configuration of an identification device 100 according to an embodiment of the present disclosure will be described with reference to FIGS. 1A to 1G.
[0014] 1A is a diagram illustrating an example of an identification device 100 according to the present embodiment. The identification device 100 includes a light source 10, an image sensor 20, a memory 25, a control circuit 30, and a signal processing circuit 40. The image sensor 20 includes a plurality of photoelectric conversion elements 22 and a plurality of charge accumulation units 24. In this specification, the control circuit 30 and the signal processing circuit 40 may be collectively referred to as a "processor."
[0015] The light source 10 emits pulsed light that is irradiated onto the head of the user 1. The image sensor 20 detects at least a portion of the pulsed light returning from the head of the user 1. The control circuit 30 controls the light source 10 and the image sensor 20. The signal processing circuit 40 processes the signal output from the image sensor 20.
[0016] In this embodiment, the control circuit 30 includes a light source control unit 32 that controls the light source 10, and a sensor control unit 34 that controls the image sensor 20. The light source control unit 32 controls the intensity, pulse width, emission timing, and / or wavelength of the pulsed light emitted from the light source 10. The sensor control unit 34 controls the timing of signal accumulation in each pixel of the image sensor 20.
[0017] Each component will be described in more detail below.
[0018] [1-1.Light source 10] The light source 10 irradiates light on the head of the user 1, for example, on the forehead. The light emitted from the light source 10 and reaching the user 1 is divided into a surface reflection component I1 that is reflected on the surface of the user 1 and an internal scattering component I2 that is scattered inside the user 1. The internal scattering component I2 is a component that is reflected or scattered once or multiple times inside the living body. When the forehead of the user 1 is irradiated with light, the internal scattering component I2 refers to a component that reaches a site about 8 mm to 16 mm deep from the surface of the forehead, for example, the brain, and returns to the identification device 100 again. The surface reflection component I1 includes three components: a direct reflection component, a diffuse reflection component, and a scattered reflection component. The direct reflection component is a reflection component whose angle of incidence and angle of reflection are equal. The diffuse reflection component is a component that is diffusely reflected due to the uneven shape of the surface. The scattered reflection component is a component that is scattered and reflected by internal tissue near the surface. When the forehead of the user 1 is irradiated with light, the scattered reflection component is a component that is scattered and reflected inside the epidermis. Hereinafter, in this disclosure, the surface reflection component I1 reflected on the surface of the user 1 includes these three components. The surface reflection component I1 and the internal scattering component I2 change their directions due to reflection or scattering, and some of them reach the image sensor 20.
[0019] The light source 10 generates pulsed light multiple times at a predetermined time interval or at a predetermined timing in accordance with an instruction from the control circuit 30. The pulsed light generated by the light source 10 may be, for example, a rectangular wave with a fall period close to zero. The fall period is the period from when the intensity of the pulsed light starts to decrease until the decrease ends. The component in the fall period of the pulsed light is also referred to as the rear end of the pulsed light. In general, light incident on the user 1 propagates through the user 1 via various paths and is emitted from the surface of the user 1 with a time difference. For this reason, the rear end of the internal scattering component I2 of the pulsed light has a spread. When the test part is the forehead, the spread of the rear end of the internal scattering component I2 is about 4 ns. In consideration of this, the fall period of the pulsed light may be set to, for example, 2 ns or less, which is half of that. The fall time may be further half that, 1 ns or less. On the other hand, the component in the rise period of the pulsed light may be used to detect the surface reflection component I1. The rise period is the period from when the intensity of the pulsed light starts to increase until the increase ends. The component in the rising period of the pulsed light is also called the front end of the pulsed light.
[0020] The light source 10 may be formed by combining a light source such as a laser, such as an LD, with a diffuser. By using a laser, it is possible to output light with high time response. A light output with high time response has a waveform with a sharp rising characteristic or falling characteristic. The rising characteristic and the falling characteristic are also called Tr characteristic and Tf characteristic, respectively. By placing a diffuser in front of the light source 10, it is also possible to illuminate the user 1 with light in a two-dimensional manner.
[0021] The wavelength of the light emitted by the light source 10 may be any wavelength within the wavelength range of, for example, 650 nm to 950 nm. This wavelength range is included in the wavelength range from red to near-infrared. In this specification, the term "light" is used for not only visible light but also infrared light. The above wavelength range is called the "biological window" and has the property of being relatively difficult to be absorbed by moisture and skin in a living body. When a living body is to be detected, the detection sensitivity can be increased by using light in the above wavelength range.
[0022] In the identification device 100 of this embodiment, the light source 10 designed in consideration of the effect on the retina may be used in order to measure the user 1 in a non-contact manner. For example, the light source 10 may satisfy Class 1 of the laser safety standards established in various countries. When Class 1 is satisfied, the user 1 is irradiated with light of such low illuminance that the exposed emission limit (AEL) is below 1 mW. Note that the light source 10 itself does not have to satisfy Class 1. For example, a diffuser plate or an ND filter may be installed in front of the light source 10 to diffuse or attenuate the light, thereby satisfying Class 1 of the laser safety standards.
[0023] Conventionally, a streak camera has been used to distinguish and detect information such as absorption coefficients or scattering coefficients at different locations in the depth direction inside a living body. For example, Japanese Patent Application Laid-Open No. 4-189349 discloses an example of such a streak camera. In these streak cameras, ultrashort pulsed light with a pulse width of femtoseconds or picoseconds is used to measure with a desired spatial resolution. In this specification, the "pulse width" means the full width at half maximum of the pulse. In contrast to the conventional method using a streak camera, the identification device 100 in this embodiment can distinguish and detect the surface reflection component I1 and the internal scattering component I2. Therefore, the pulsed light emitted by the light source 10 does not need to be ultrashort pulsed light, and the pulse width can be selected arbitrarily.
[0024] When irradiating the forehead with light, the amount of light of the internal scattering component I2 can be an extremely small value, about several thousandths to several tens of thousandsths of the amount of light of the surface reflection component I1. Furthermore, considering the safety standards for lasers, the amount of light that can be irradiated becomes extremely small, making it very difficult to detect the internal scattering component I2. Even in this case, if the light source 10 generates pulsed light with a relatively large pulse width, it is possible to increase the accumulated amount of the internal scattering component I2, which is accompanied by a time delay. This increases the amount of detected light and improves the signal-to-noise ratio.
[0025] The light source 10 emits pulsed light with a pulse width of, for example, 3 ns or more. In general, the time spread of light scattered in biological tissues such as the brain is about 4 ns. FIG. 1B shows an example of the change in the amount of light reaching the image sensor 20 over time when the width of the input pulsed light is 0 ns, 3 ns, and 10 ns. As shown in FIG. 1B, as the width of the pulsed light from the light source 10 is widened, the amount of light of the internally scattered component I2 that appears at the rear end of the pulsed light returning from the user 1 increases. FIG. 1C is a diagram showing the width of the input pulsed light on the horizontal axis and the amount of light detected by the image sensor 20 on the vertical axis. The image sensor 20 is equipped with an electronic shutter. The result of FIG. 1C was obtained under the condition that the electronic shutter was opened 1 ns after the time when the rear end of the pulsed light was reflected by the surface of the user 1 and reached the image sensor 20. The reason for selecting this condition is that the ratio of the surface reflection component I1 to the internally scattered component I2 is high immediately after the arrival of the rear end of the pulsed light. As shown in FIG. 1C, when the pulse width of the pulsed light emitted by the light source 10 is set to 3 ns or more, the amount of light detected by the image sensor 20 can be maximized.
[0026] Furthermore, the resolution of the timing control by the drive circuit for the light source 10 and the electronic shutter is about 0.2 ns, so the pulse width of the light source 10 is set to, for example, 0.2 ns or more.
[0027] The light source 10 may emit a pulsed light having a pulse width of 5 ns or more, or even 10 ns or more. On the other hand, if the pulse width is too large, the amount of unused light increases and is wasted. Therefore, the light source 10 generates a pulsed light having a pulse width of, for example, 50 ns or less. Alternatively, the light source 10 may emit a pulsed light having a pulse width of 30 ns or less, or even 20 ns or less.
[0028] The irradiation pattern of the light source 10 may have a uniform intensity distribution within the irradiation region. This is for the following reason. In the identification device 100 of this embodiment, the surface reflection component I1 is separated and reduced over time. This makes it possible to detect the internal scattering component I2 even at a null point directly below the irradiation point on the user 1. Therefore, in the identification device 100 of this embodiment, the measurement resolution can be improved by irradiating the test part of the user 1 over a spatially wide range.
[0029] [1-2. Image sensor 20] The image sensor 20 receives light emitted from the light source 10 and reflected or scattered by the user 1. The image sensor 20 has a plurality of photodetection cells arranged two-dimensionally, and obtains two-dimensional information of the user 1 at once. This makes it possible to obtain two-dimensional information of the user 1 in a relatively short time, compared to a line sensor that detects the subject of the user 1 by sliding it. In this specification, the photodetection cells are also referred to as "pixels." The image sensor 20 is, for example, any imaging element, such as a CCD image sensor or a CMOS image sensor.
[0030] The image sensor 20 has an electronic shutter. The electronic shutter is a circuit that controls the timing of imaging. In this embodiment, the sensor control unit 34 in the control circuit 30 has the function of the electronic shutter. The electronic shutter controls the period of one signal accumulation in which the received light is converted into an effective electrical signal and accumulated, and the period in which the signal accumulation is stopped. The signal accumulation period can also be called an "exposure period." In the following description, the width of the exposure period may be called a "shutter width." The time from the end of one exposure period to the start of the next exposure period may be called a "non-exposure period." Hereinafter, the exposed state may be called "OPEN," and the stopped exposure state may be called "CLOSE."
[0031] The image sensor 20 can adjust the exposure period and non-exposure period by an electronic shutter in the range of sub-nanoseconds, for example, 30 ps to 1 ns. A conventional TOF (Time Of Flight) camera for distance measurement detects all light emitted from the light source 10 and reflected by the subject and returned in order to correct the effect of the brightness of the subject. Therefore, in the conventional TOF camera, the shutter width needs to be larger than the pulse width of the light. In contrast, in the identification device 100 of the present embodiment, there is no need to correct the amount of light of the subject. Therefore, the shutter width does not need to be larger than the pulse width. Therefore, the shutter width can be set to a value of, for example, 1 ns or more and 30 ns or less. According to the identification device 100 of the present embodiment, the shutter width can be reduced, thereby reducing the effect of dark current contained in the detection signal.
[0032] When the forehead of the user 1 is irradiated with light, the internal attenuation rate of the light is very large. For example, the outgoing light may attenuate to about one millionth of the incoming light. For this reason, the amount of light may be insufficient with only one pulse of irradiation to detect the internal scattering component I2. The amount of light is particularly weak in the case of irradiation under Class 1 laser safety standards. In this case, the light source 10 emits pulsed light multiple times, and the image sensor 20 also exposes the light multiple times using an electronic shutter accordingly, thereby accumulating the detection signal and improving the sensitivity.
[0033] An example of the configuration of the image sensor 20 will now be described.
[0034] The image sensor 20 may include a plurality of pixels arranged two-dimensionally on the imaging surface. Each pixel may include a photoelectric conversion element such as a photodiode, and one or more charge accumulation units. Hereinafter, an example will be described in which each pixel includes a photoelectric conversion element that generates a signal charge according to the amount of received light by photoelectric conversion, a charge accumulation unit that accumulates a signal charge generated by a surface reflection component I1 of the pulsed light, and a charge accumulation unit that accumulates a signal charge generated by an internal scattering component I2 of the pulsed light. In the following example, the control circuit 30 causes the image sensor 20 to detect the surface reflection component I1 by detecting a portion of the pulsed light returned from the head of the user 1 before the start of the fall. The control circuit 30 also causes the image sensor 20 to detect the internal scattering component I2 by detecting a portion of the pulsed light returned from the head of the user 1 after the start of the fall. The light source 10 emits light of, for example, two types of wavelengths.
[0035] Fig. 1D is a diagram showing an example of a schematic configuration of one pixel 201 of the image sensor 20. Note that Fig. 1D shows a schematic configuration of one pixel 201 and does not necessarily reflect an actual structure. In this example, the pixel 201 includes a photodiode 203 that performs photoelectric conversion, a first floating diffusion layer 204, a second floating diffusion layer 205, a third floating diffusion layer 206, and a fourth floating diffusion layer 207 that are charge storage units, and a drain 202 that discharges signal charges.
[0036] Photons incident on each pixel due to the emission of one pulse light are converted into signal electrons, which are signal charges, by the photodiode 203. The converted signal electrons are either discharged to the drain 202 or distributed to any one of the first floating diffusion layer 204, the second floating diffusion layer 205, the third floating diffusion layer 206, and the fourth floating diffusion layer 207, in accordance with a control signal input from the control circuit 30.
[0037] The emission of pulsed light from the light source 10, the accumulation of signal charges in the first floating diffusion layer 204, the second floating diffusion layer 205, the third floating diffusion layer 206, and the fourth floating diffusion layer 207, and the discharge of the signal charges to the drain 202 are repeated in this order. This repetitive operation is fast and can be repeated, for example, tens of thousands to hundreds of millions of times within the time of one frame. The time of one frame is, for example, about 1 / 30 seconds. The pixel 201 finally generates and outputs four image signals based on the signal charges accumulated in the first floating diffusion layer 204, the second floating diffusion layer 205, the third floating diffusion layer 206, and the fourth floating diffusion layer 207.
[0038] In this example, the control circuit 30 causes the light source 10 to repeatedly emit a first pulsed light having a first wavelength and a second pulsed light having a second wavelength in sequence. By selecting two wavelengths having different absorptivities for the internal tissue of the user 1 as the first wavelength and the second wavelength, the condition of the user 1 can be analyzed.
[0039] The control circuit 30 first causes the light source 10 to emit a first pulsed light. The control circuit 30 causes the first floating diffusion layer 204 to accumulate signal charges during a first period during which the surface reflection component I1 of the first pulsed light is incident on the photodiode 203. Then, the control circuit 30 causes the second floating diffusion layer 205 to accumulate signal charges during a second period during which the internal scattering component I2 of the first pulsed light is incident on the photodiode 203. Next, the control circuit 30 causes the light source 10 to emit a second pulsed light. The control circuit 30 causes the third floating diffusion layer 206 to accumulate signal charges during a third period during which the surface reflection component I1 of the second pulsed light is incident on the photodiode 203. Then, the control circuit 30 causes the fourth floating diffusion layer 207 to accumulate signal charges during a fourth period during which the internal scattering component I2 of the second pulsed light is incident on the photodiode 203.
[0040] In this way, the control circuit 30 sequentially accumulates the signal charge from the photodiode 203 in the first floating diffusion layer 204 and the second floating diffusion layer 205 with a predetermined time difference after starting the emission of the first pulsed light. After that, the control circuit 30 sequentially accumulates the signal charge from the photodiode 203 in the third floating diffusion layer 206 and the fourth floating diffusion layer 207 with the predetermined time difference after starting the emission of the second pulsed light. The above operation is repeated multiple times. In order to estimate the amount of disturbance light and environmental light, a period may be provided in which signal charge is accumulated in other floating diffusion layers (not shown) with the light source 10 turned off. By subtracting the amount of signal charge in the other floating diffusion layers from the amount of signal charge accumulated in the first floating diffusion layer 204, the second floating diffusion layer 205, the third floating diffusion layer 206, and the fourth floating diffusion layer 207, a signal from which disturbance light and environmental light components have been removed can be obtained.
[0041] In this embodiment, the number of charge accumulation units per pixel is set to 4, but may be designed to any number of 1 or more depending on the purpose. For example, when only one type of wavelength is used, the number of charge accumulation units may be 2. When only one type of wavelength is used and only the surface reflection component I1 or only the internal scattering component I2 is detected, the number of charge accumulation units per pixel may be 1. When two or more types of wavelengths are used, the number of charge accumulation units may be 1 if imaging using each wavelength is performed in a different frame. As described later, when the detection of the surface reflection component I1 and the detection of the internal scattering component I2 are performed in different frames, the number of charge accumulation units may be 1.
[0042] FIG. 1E is a diagram showing an example of the configuration of the image sensor 20. In FIG. 1E, an area surrounded by a two-dot chain line frame corresponds to one pixel 201. The pixel 201 includes one photodiode. Although FIG. 1E shows only four pixels arranged in two rows and two columns, a larger number of pixels may be arranged in practice. The pixel 201 includes a first floating diffusion layer 204, a second floating diffusion layer 205, a third floating diffusion layer 206, and a fourth floating diffusion layer 207. The signals accumulated in the first floating diffusion layer 204, the second floating diffusion layer 205, the third floating diffusion layer 206, and the fourth floating diffusion layer 207 are treated as if they were signals of four pixels of a general CMOS image sensor, and are output from the image sensor 20.
[0043] Each pixel 201 has four signal detection circuits. Each signal detection circuit includes a source follower transistor 309, a row selection transistor 308, and a reset transistor 310. In this example, the reset transistor 310 corresponds to the drain 202 shown in FIG. 1D, and a pulse input to the gate of the reset transistor 310 corresponds to a drain discharge pulse. Each transistor is, for example, but not limited to, a field effect transistor formed on a semiconductor substrate. As shown in the figure, one of the input terminal and output terminal of the source follower transistor 309 is connected to one of the input terminal and output terminal of the row selection transistor 308. One of the input terminal and output terminal of the source follower transistor 309 is typically a source. One of the input terminal and output terminal of the row selection transistor 308 is typically a drain. The gate, which is the control terminal of the source follower transistor 309, is connected to the photodiode 203. The signal charge generated by the photodiode 203 is accumulated in a floating diffusion layer, which is a charge accumulation portion between the photodiode 203 and the source follower transistor 309. The signal charges are holes or electrons.
[0044] Although not shown in FIG. 1E, the first floating diffusion layer 204, the second floating diffusion layer 205, the third floating diffusion layer 206, and the fourth floating diffusion layer 207 are connected to the photodiode 203. A switch may be provided between the photodiode 203 and the first floating diffusion layer 204, the second floating diffusion layer 205, the third floating diffusion layer 206, and the fourth floating diffusion layer 207. The switch switches the conduction state between the photodiode 203 and each of the first floating diffusion layer 204, the second floating diffusion layer 205, the third floating diffusion layer 206, and the fourth floating diffusion layer 207 in response to a signal accumulation pulse from the control circuit 30. This controls the start and stop of accumulation of signal charges in each of the first floating diffusion layer 204, the second floating diffusion layer 205, the third floating diffusion layer 206, and the fourth floating diffusion layer 207. The electronic shutter in this embodiment has a mechanism for such exposure control.
[0045] The signal charges stored in the first floating diffusion layer 204, the second floating diffusion layer 205, the third floating diffusion layer 206, and the fourth floating diffusion layer 207 are read out by turning on the gate of the row selection transistor 308 by the row selection circuit 302. At this time, the current flowing from the source follower power supply 305 to the source follower transistor 309 and the source follower load 306 is amplified according to the signal potential of the first floating diffusion layer 204, the second floating diffusion layer 205, the third floating diffusion layer 206, and the fourth floating diffusion layer 207. The analog signal due to this current read out from the vertical signal line 304 is converted into digital signal data by an analog-digital (AD) conversion circuit 307 connected to each column. This digital signal data is read out for each column by the column selection circuit 303 and output from the image sensor 20. After reading out one row, the row selection circuit 302 and the column selection circuit 303 read out the next row, and so on, reading out information on the signal charges of the floating diffusion layers of all rows. After reading out all the signal charges, the control circuit 30 resets all the floating diffusion layers by turning on the gate of the reset transistor 310. This completes the imaging of one frame. Similarly, high-speed imaging of frames is repeated thereafter, completing the imaging of a series of frames by the image sensor 20.
[0046] In the present embodiment, an example of the CMOS type image sensor 20 has been described, but the image sensor 20 may be another type of imaging element. The image sensor 20 may be, for example, a CCD type, a single photon counting element, or an amplified image sensor such as an EMCCD or ICCD.
[0047] 1F is a diagram showing an example of an operation in one frame in this embodiment. As shown in FIG. 1F, the emission of the first pulsed light and the emission of the second pulsed light may be alternately switched multiple times in one frame. In this way, the time difference between the acquisition timing of the detection images by the two types of wavelengths can be reduced, and even if the user 1 is moving, it is possible to capture images by the first and second pulsed lights almost simultaneously.
[0048] [1-3.Memory 25] The memory 25 stores biometric data indicating the physical characteristics of the user 1 that have been acquired and registered in advance. This biometric data is called a template. The memory 25 may be built into the identification device 100 or may be provided externally. The memory 25 may be, for example, one or more ROMs, optical disks, or hard disk drives.
[0049] The biometric data stored in the memory 25 may be, for example, an image of the user 1 itself, or data characterizing the image. The data characterizing the image may include, for example, information indicating the texture of the skin surface of the user 1, information indicating the distribution of the blood vessels of the user 1, or information indicating the unevenness of the characteristic parts of the user 1. The "information indicating the texture of the skin surface of the user 1" may be, for example, a two-dimensional distribution of frequency components obtained by Fourier transforming pixel values of a two-dimensional distribution of wrinkles or minute unevenness at a certain point on the skin surface of the user 1. This transformation is effective when the wrinkles or minute unevenness show a repetition of a similar two-dimensional pattern. The "information indicating the distribution of the blood vessels of the user 1" is, for example, image information indicating the distribution of the veins of the user 1. The "information indicating the unevenness of the characteristic parts of the user 1" is, for example, image information of a two-dimensional distribution of distances indicating unevenness in at least one of the eye sockets, nose, cheeks, cheekbones, mouth, chin, and under the ears of the user 1. By storing data characterizing an image of user 1, not only can redundant information be removed, improving recognition accuracy, but the amount of data to be stored can also be reduced.
[0050] [1-4. Control circuit 30 and signal processing circuit 40] The control circuit 30 causes the light source 10 to emit pulsed light to be irradiated to the user 1. The control circuit 30 causes the image sensor 20 to detect at least a portion of the reflected pulsed light returning from the user 1. The control circuit 30 adjusts the time difference between the timing of emitting pulsed light from the light source 10 and the shutter timing of the image sensor 20. Hereinafter, this time difference may be referred to as "phase" or "phase delay." The "light emission timing" of the light source 10 is the timing at which the pulsed light emitted by the light source 10 starts to rise. The "shutter timing" is the timing at which exposure starts. The control circuit 30 may adjust the phase by changing the light emission timing, or may adjust the phase by changing the shutter timing.
[0051] The control circuit 30 causes the image sensor 20 to output a signal indicating a two-dimensional image of the user 1 according to the intensity distribution of the detected light. The control circuit 30 causes the signal processing circuit 40 to process the signal output from the image sensor 20. The control circuit 30 may be configured to remove an offset component from the signal detected by each pixel of the image sensor 20. The offset component is a signal component due to ambient light such as sunlight or fluorescent light, or disturbance light. The offset component due to ambient light and disturbance light is estimated by the image sensor 20 detecting a signal when the light source 10 is not emitting light, that is, when the drive of the light source 10 is turned off.
[0052] The control circuit 30 may be, for example, a combination of a processor and a memory, or an integrated circuit such as a microcontroller having a built-in processor and memory. The control circuit 30 adjusts the light emission timing and the shutter timing, estimates the offset component, and removes the offset component, for example, by the processor executing a program recorded in the memory.
[0053] The signal processing circuit 40 is a circuit that processes the image signal output from the image sensor 20. The signal processing circuit 40 performs arithmetic processing such as image processing. The signal processing circuit 40 can be realized by, for example, a digital signal processor (DSP), a programmable logic device (PLD) such as a field programmable gate array (FPGA), or a combination of a central processing unit (CPU) or an image processing arithmetic processor (GPU) and a computer program. The control circuit 30 and the signal processing circuit 40 may be an integrated single circuit, or may be separate individual circuits. The signal processing circuit 40 may also be a component of an external device such as a server installed in a remote location. In this case, the external device such as the server has a communication means and transmits and receives data to and from the light source 10, the image sensor 20, and the control circuit 30.
[0054] FIG. 1G is a flowchart showing an outline of the operation by the control circuit 30. The control circuit 30 generally executes the operation shown in FIG. 1G. Here, as an example, the operation in the case where only the internal scattering component I2 is detected will be described. The control circuit 30 first causes the light source 10 to emit a pulsed light for a predetermined time (step S101). At this time, the electronic shutter of the image sensor 20 is in a state where charge accumulation is stopped. The control circuit 30 causes the electronic shutter to stop charge accumulation until the period in which a part of the pulsed light is reflected by the surface of the user 1 and reaches the image sensor 20 is completed. Next, the control circuit 30 causes the electronic shutter to start charge accumulation at the timing when another part of the pulsed light is scattered inside the user 1 and reaches the image sensor 20 (step S102). After the predetermined time has elapsed, the control circuit 30 causes the electronic shutter to stop charge accumulation (step S103). Next, the control circuit 30 determines whether the number of times the above charge accumulation has been executed has reached a predetermined number (step S104). The predetermined number can be, for example, more than 1,000 times and less than 100,000 times. If the result of the determination in step S104 is No, the control circuit 30 repeats steps S101 to S103 until the result of the determination is Yes. If the result of the determination in step S104 is Yes, the control circuit 30 causes the image sensor 20 to generate and output a signal indicating an image of a two-dimensional distribution based on the total amount of signal charges accumulated in each floating diffusion layer for each pixel (step S105).
[0055] By the above operations, the components of light scattered inside the object to be measured can be detected with high sensitivity. Note that multiple light emission and exposure are not essential and are performed as necessary. Note that the operations of steps S101 to S105 shown in FIG. 1G can also be applied to the case where only the surface reflection component I1 is detected.
[0056] The control circuit 30 authenticates the user 1 by comparing the signal output from the image sensor 20 with the biometric data stored in the memory 25. The operation of the control circuit 30 is as follows. The control circuit 30 causes the signal processing circuit 40 to generate image data from the information obtained by the image sensor 20. The control circuit 30 compares the image data with the biometric data stored in the memory 25. If data characterizing an image of the user 1 is stored in the memory 25, the signal processing circuit 40 extracts data characterizing the image from the generated image of the user 1, and the control circuit 30 compares the data characterizing the image of the user 1 with each other. Alternatively, the control circuit 30 may simply compare the correlation between the information obtained by the image sensor 20 and information registered in advance. The control circuit 30 may also authenticate the user 1 based on the user's features obtained by deep learning or machine learning such as a support vector machine.
[0057] [1-5.Optical system] The identification device 100 may include an imaging optical system that forms a two-dimensional image of the user 1 on the light receiving surface of the image sensor 20. The optical axis of the imaging optical system is approximately perpendicular to the light receiving surface of the image sensor 20. The imaging optical system may include a zoom lens. When the position of the zoom lens is changed, the magnification ratio of the two-dimensional image of the user 1 changes, and the resolution of the two-dimensional image on the image sensor 20 changes. Therefore, even if the distance to the user 1 is far, it is possible to enlarge the area to be measured and observe it in detail.
[0058] The identification device 100 may also include a bandpass filter between the user 1 and the image sensor 20 that passes only light in or near the band of wavelengths emitted from the light source 10. This can reduce the influence of disturbance components such as ambient light. The bandpass filter is made of a multilayer film filter or an absorption filter. Taking into account the temperature of the light source 10 and band shifts caused by oblique incidence on the filter, the bandwidth of the bandpass filter may be about 20 to 100 nm.
[0059] Furthermore, the identification device 100 may include a polarizing plate between the light source 10 and the user 1, and between the image sensor 20 and the user 1. In this case, the polarization directions of the polarizing plate arranged on the light source 10 side and the polarizing plate arranged on the image sensor side are in a crossed Nicol relationship. This makes it possible to prevent the specular reflection component of the surface reflection component I1 of the user 1, i.e., the component whose incidence angle and reflection angle are the same, from reaching the image sensor 20. In other words, the amount of light that the surface reflection component I1 reaches the image sensor 20 can be reduced.
[0060] [2. Time-resolved imaging operation] The identification device 100 in this embodiment can distinguish and detect the surface reflection component I1 and the internal scattering component I2 by time-resolved imaging.
[0061] An example of the operation of the identification device 100 in this embodiment will now be described.
[0062] As shown in FIG. 1A, when the light source 10 irradiates the user 1 with pulsed light, a surface reflection component I1 and an internal scattering component I2 are generated. A part of the surface reflection component I1 and the internal scattering component I2 reach the image sensor 20. The internal scattering component I2 passes through the inside of the user 1 before being emitted from the light source 10 and reaching the image sensor 20. That is, the optical path length of the internal scattering component I2 is longer than the optical path length of the surface reflection component I1. Therefore, the internal scattering component I2 reaches the image sensor 20 later on average than the surface reflection component I1.
[0063] In time-resolved imaging, the surface reflection component I1 is obtained by the following operations.
[0064] FIG. 2A is a diagram showing an example of an optical signal in which rectangular pulse light is emitted from the light source 10 and returned from the user 1 and reaches the image sensor 20. The horizontal axis represents time (t) in all of signals (a) to (d), and the vertical axis represents intensity in signals (a) to (c), and the OPEN or CLOSE state of the electronic shutter in signal (d). Signal (a) shows the surface reflection component I1. Signal (b) shows the internal scattering component I2. Signal (c) shows the combined component of the surface reflection component I1(a) and the internal scattering component I2(b). Signal (d) shows the electronic shutter timing for acquiring the surface reflection component I1 of the user 1.
[0065] As shown by the signal (d) in FIG. 2A, by closing the shutter, it is possible to efficiently collect the component that arrives earlier among the reflected light that returns to the image sensor 20. The component that arrives earlier means that there is less scattering at the user 1, and contains the surface information of the user 1. The time during which the light is actually accumulated is a short time at the front end of the pulse wave, but the shutter does not necessarily have to be open for that period. As shown by the signal (d) in FIG. 2A, if the charge is accumulated just before CLOSE, the shutter may be opened at a stage earlier than the front end of the pulse wave reaches the image sensor 20. This eliminates the need for an expensive high-time resolution imaging device on the order of picoseconds. The identification device 100 in this embodiment can be configured with an inexpensive image sensor 20.
[0066] 2A, the control circuit 30 causes the image sensor 20 to detect at least a portion of the rising edge component of the reflected pulsed light, and to output a signal indicating a two-dimensional image of the user 1. In this embodiment, the signal output from the image sensor 20 may include a signal obtained from at least a portion of the rising edge component of the reflected pulsed light.
[0067] The light source 10 emits a rectangular pulse wave. At this time, the pulse width does not need to be on the order of ps, but can be several ns. Therefore, an inexpensive light source can be used. If the Tr characteristic at the front end of the pulse wave is steep, it is possible to minimize the inclusion of unnecessary internal scattering components I2 with a time delay in the acquired image when only the front end is photographed with a shutter.
[0068] In time-resolved imaging, the internal scattering component I2 is obtained by the following operations.
[0069] Fig. 2B is a diagram showing another example of an optical signal in which rectangular pulse light is emitted from the light source 10 and returned from the user 1 to reach the image sensor 20. Signals (a) to (c) in Fig. 2B show the same time changes as signals (a) to (c) in Fig. 2A, respectively. Signal (d) in Fig. 2B shows the electronic shutter timing for acquiring the internal scattering component I2.
[0070] As shown in signal (a) in FIG. 2B, the surface reflection component I1 maintains a rectangular shape. On the other hand, as shown in signal (b) in FIG. 2B, the internal scattering component I2 is a sum of light that has passed through various optical path lengths, and therefore exhibits a characteristic of trailing at the rear end of the pulsed light. That is, the falling period is longer than that of the surface reflection component I1. In order to increase the proportion of the internal scattering component I2 from the optical signal of signal (c) in FIG. 2B and extract it, the electronic shutter starts to accumulate charges after the rear end of the surface reflection component I1, as shown in signal (d) in FIG. 2B. After the rear end of the surface reflection component I1 means when or after the surface reflection component I1 falls. This shutter timing is adjusted by the control circuit 30. As described above, the identification device 100 in this embodiment distinguishes between the surface reflection component I1 and the internal scattering component I2 that has reached the depths of the user 1 and detects them, so the pulsed light width and the shutter width are arbitrary. Therefore, unlike the conventional method using a streak camera, the internal scattering component I2 can be obtained with a simple configuration, and the cost can be significantly reduced.
[0071] 2B, the control circuit 30 causes the image sensor 20 to detect at least a portion of the component in the falling period of the reflected pulsed light, and to output a signal indicating a two-dimensional image of the user 1. In this embodiment, the signal output from the image sensor 20 may include a signal obtained from at least a portion of the component in the falling period of the reflected pulsed light.
[0072] In the signal (a) in FIG. 2B, the rear end of the surface reflection component I1 falls vertically. In other words, the time from when the surface reflection component I1 starts to fall until it ends is zero. However, in reality, the pulsed light itself irradiated by the light source 10 may not be completely vertical, the surface of the user 1 may have fine irregularities, or the rear end of the surface reflection component I1 may not fall vertically due to scattering within the epidermis. In addition, since the user 1 is an opaque object, the surface reflection component I1 has a much larger amount of light than the internal scattering component I2. Therefore, even if the rear end of the surface reflection component I1 slightly protrudes from the vertical falling position, the internal scattering component I2 may be buried. In addition, due to a time delay associated with the movement of electrons during the readout period of the electronic shutter, it may not be possible to realize an ideal binary readout as shown in the signal (d) in FIG. 2B. Therefore, the control circuit 30 may delay the shutter timing of the electronic shutter slightly from immediately after the falling edge of the surface reflection component I1. For example, it may be delayed by about 0.5 ns to 5 ns. Instead of adjusting the shutter timing of the electronic shutter, the control circuit 30 may adjust the light emission timing of the light source 10. The control circuit 30 adjusts the time difference between the shutter timing of the electronic shutter and the light emission timing of the light source 10. If the shutter timing is delayed too much, the internal scattering component I2, which is originally small, will be further reduced. For this reason, the shutter timing may be kept near the rear end of the surface reflection component I1. Since the time delay due to scattering by user 1, which is assumed from the sensor sensitivity, is 4 ns, the maximum amount of delay in the shutter timing is about 4 ns.
[0073] The light source 10 may emit a pulse of light multiple times, and exposure may be performed multiple times with each pulse of light at the same phase shutter timing, thereby amplifying the detected light amount of the internal scattering component I2.
[0074] Instead of or in addition to placing a bandpass filter between the user 1 and the image sensor 20, the control circuit 30 may estimate the offset component by capturing an image with the same exposure time without emitting light from the light source 10. The estimated offset component is subtracted from the signal detected by each pixel of the image sensor 20. This makes it possible to remove dark current components and / or ambient light generated on the image sensor 20.
[0075] Next, an example of a method for detecting the surface reflection component I1 and the internal scattering component I2 per frame will be described.
[0076] FIG. 3A shows an example of a timing chart for detecting the surface reflection component I1. For example, as shown in FIG. 3A, in order to detect the surface reflection component I1, the shutter may be opened before the pulsed light reaches the image sensor 20, and closed before the trailing end of the pulsed light reaches the image sensor 20. By controlling the shutter in this way, it is possible to reduce the mixing of the internal scattering component I2. It is possible to increase the proportion of the light that passes through the vicinity of the surface of the user 1. In particular, the timing of closing the shutter may be immediately after the light reaches the image sensor 20. This enables signal detection with an increased proportion of the surface reflection component I1, which has a relatively short optical path length. As another method of acquiring the surface reflection component I1, the image sensor 20 may acquire the entire pulsed light, or continuous light may be irradiated from the light source 10 to detect it.
[0077] 3B shows an example of a timing chart for detecting the internal scattering component I2. By opening the shutter during the period when the trailing end portion of the pulse reaches the image sensor 20, the signal of the internal scattering component I2 can be acquired.
[0078] As in this embodiment, if time-division imaging is performed using the same camera or the same sensor, temporal and spatial deviations are unlikely to occur. When acquiring signals of both the surface reflection component I1 and the internal scattering component I2 using the same sensor, the components to be acquired may be switched for each frame as shown in Figs. 3A and 3B. Alternatively, as described with reference to Figs. 1D to 1F, the components to be acquired at high speed within one frame may be alternately switched. In this case, the detection time difference between the surface reflection component I1 and the internal scattering component I2 can be reduced.
[0079] Furthermore, the signals of the surface reflection component I1 and the internal scattering component I2 may be acquired using light of two wavelengths. When acquiring the surface reflection component I1 and the internal scattering component I2 using two wavelengths, a method of switching between four types of charge accumulation at high speed within one frame may be used, as described with reference to Figures 1D to 1F. Such a method can reduce the time lag of the detection signals.
[0080] [3. Spatially resolved imaging operation] Each of the surface reflection component I1 and the internal scattering component I2 can be acquired by spatially resolved imaging in addition to the above-mentioned time-resolved imaging.
[0081] 4A to 4C are diagrams showing examples of the distribution of two-dimensional pattern light projected from the light source 10 to the user 1. In the examples shown in FIG. 4A to 4C, white parts represent bright parts where light exists, and black parts represent dark parts where light does not exist. FIG. 4A shows a checker pattern in which multiple bright parts and multiple dark parts are periodically distributed. FIG. 4B shows a dot pattern in which multiple dark parts are randomly distributed in a bright part. FIG. 4C shows a dot pattern in which multiple bright parts are randomly distributed in a dark part.
[0082] The control circuit 30 causes the light source 10 to emit a two-dimensional pattern light that projects a two-dimensional pattern onto the user 1. As shown in FIG. 4A to FIG. 4C, the two-dimensional pattern light includes at least one bright portion and at least one dark portion. The two-dimensional pattern light is obtained, for example, by placing a light-shielding mask having a two-dimensional distribution in front of the light source 10. Alternatively, the two-dimensional pattern light may be formed using a digital micromirror device (DMD) or a spatial light modulator (SLM). The two-dimensional pattern light may be continuous light or pulsed light.
[0083] As shown in Figures 4A to 4C, a two-dimensional pattern light with spatially modulated intensity, such as a checkered pattern or a dot pattern, is emitted from a light source 10. When the two-dimensional pattern light is projected onto a user 1, most of the light returns from bright areas, and a small amount of light returns from dark areas. The reflected light returning from the dark areas contains an internal scattering component I2 scattered inside the user 1, and contains almost no surface reflection component I1.
[0084] In order to detect the internal scattering component I2 by the two-dimensional pattern light, the control circuit 30 causes the image sensor 20 to detect at least a portion of the reflected light returning from at least one dark area of the user 1 onto which the two-dimensional pattern light is projected, and to output a signal according to the intensity distribution of the at least a portion of the detected reflected light. In this way, if the two-dimensional pattern light is pulsed light, it is possible to obtain a portion from the front end to the rear end of the internal scattering component I2 shown in signal (b) in FIG. 2A or FIG. 2B. In this embodiment, the signal output from the image sensor 20 may include the above signal obtained from at least one dark area.
[0085] On the other hand, the reflected light returning from the bright area contains both the surface reflection component I1 and the internal scattering component I2. Therefore, the surface reflection component I1 can be calculated by subtracting the detection data in the nearby dark area from the detection data in the bright area. In this case, the spatial resolution decreases. As a countermeasure, instead of acquiring a signal in a single shot, it is possible to acquire signals multiple times by, for example, shifting the distribution of the two-dimensional pattern light or changing the distribution of the two-dimensional pattern light itself. This makes it possible to acquire the surface reflection component I1 without reducing the spatial resolution.
[0086] In order to detect the surface reflection component I1 by the two-dimensional pattern light, the control circuit 30 causes the image sensor 20 to detect at least a part of the reflected light returning from at least one dark part and at least a part of the reflected light returning from at least one bright part of the user 1 on which the two-dimensional pattern light is projected. The control circuit 30 causes the image sensor 20 to output a signal according to the intensity distribution of at least a part of the reflected light returning from at least one dark part and a signal according to the intensity distribution of at least a part of the reflected light returning from at least one bright part. The control circuit 30 causes the signal processing circuit 40 to calculate the surface reflection component I1 by subtracting the signal obtained from at least one dark part from the signal obtained from at least one bright part. As a result, if the two-dimensional pattern light is pulsed light, it is possible to obtain a portion from the front end to the rear end of the surface reflection component I1 shown in the signal (a) in FIG. 2A or FIG. 2B. In this embodiment, the signal output from the image sensor 20 may include the signal obtained from at least one bright part in addition to the signal obtained from at least one dark part.
[0087] Spatially resolved imaging may be combined with time resolved imaging to obtain each of the surface reflection component I1 and the internal scattering component I2.
[0088] When the two-dimensional pattern light is a pulsed light, the surface reflection component I1 may be obtained by detecting at least a part of the rising edge of the reflected pulsed light returning from at least one bright portion. Similarly, the internal scattering component I2 may be obtained by detecting at least a part of the falling edge of the reflected pulsed light returning from at least one bright portion or at least one dark portion. The operations of the control circuit 30 and the signal processing circuit 40 when detecting the rising edge and falling edge components of the reflected pulsed light are as described above.
[0089] [4. Biometric authentication using surface reflection component I1 and internal scattering component I2] A specific example of biometric authentication using the identification device 100 of this embodiment will be described below in comparison with a method using a general camera.
[0090] FIG. 5A is a diagram showing a typical image of a user 1 captured by a general camera 90. The light irradiating the user 1 penetrates within a few millimeters of the surface of the user 1. Therefore, in the conventional camera 90, both the surface reflection component I1 and the internal scattering component I2 contained in the reflected light returning from the user 1 are detected. An image in which both the surface reflection component I1 and the internal scattering component I2 are mixed may be slightly blurred. Therefore, in face recognition using the general camera 90, there is a risk that the false acceptance rate may increase or the false rejection rate may increase. For example, there is a risk that twin brothers may be mistaken for the user himself, or that a person who has changed his hairstyle may be mistaken for another person. In other words, the accuracy of face recognition may be deteriorated in a method using the general camera 90.
[0091] FIG. 5B is a diagram showing an example of capturing an image of user 1 using surface reflection component I1 in this embodiment. As described above, in identification device 100 in this embodiment, surface reflection component I1 can be detected by time-resolved imaging or space-resolved imaging. This allows the texture of the skin surface of user 1, such as wrinkles or minute irregularities, to be detected more clearly. By using the result of comparing information acquired from surface reflection component I1 with information indicating the texture of the skin surface of user 1 contained in the biometric data in memory 25, the authentication accuracy can be improved. As a result, the false rejection rate or false acceptance rate is reduced.
[0092] FIG. 5C is a diagram showing an example of capturing an image of the user 1 using the internal scattering component I2 in this embodiment. As described above, the identification device 100 in this embodiment can detect the internal scattering component I2 by time-resolved imaging or spatially-resolved imaging. This allows the distribution of blood vessels such as veins of the user 1 to be detected more clearly. The accuracy of authentication can be improved by using the result of comparing the information acquired from the internal scattering component I2 with the information indicating the distribution of blood vessels of the user 1 contained in the biometric data in the memory 25. As a result, the false rejection rate or false acceptance rate is reduced.
[0093] Authentication using the surface reflection component I1 and authentication using the internal scattering component I2 may be combined. For example, depending on the application, authentication may be successful only after both are authenticated, or authentication may be successful if only one of the two is authenticated. When the internal scattering component I2 is used for vein authentication, near-infrared light that easily penetrates a living body may be used. On the other hand, when the surface reflection component I1 is used for face print authentication, light with a short wavelength such as a blue range that does not easily penetrate a living body may be used. In this way, light of different wavelengths may be used to detect the surface reflection component I1 and the internal scattering component I2.
[0094] The internal scattering component I2 includes biometric information from deeper parts of the user 1 than the surface reflection component I1. Therefore, information different from the surface reflection component I1, such as vein information or internal cells, can be obtained. Therefore, it is possible to perform multiple different types of authentication, such as performing authentication mainly based on face patterns with the surface reflection component I1 and performing authentication mainly based on veins with the internal scattering component I2. Even if one authentication fails, the failure can be compensated for by the other authentication. This leads to stable and highly accurate authentication, and high security can be ensured. In addition, if another person masquerades as the user 1, both authentications must be forged, making tampering extremely difficult. In this way, the identification device 100 in this embodiment can realize an authentication system that is more difficult to forge or tamper with.
[0095] In addition to performing multiple types of authentication independently, authentication may be performed by machine learning using data that combines the surface reflection component I1 and the internal scattering component I2. By including different features in the data, the amount of information increases, and the accuracy rate of recognition by machine learning can be improved.
[0096] In addition to the above, user 1 may be authenticated by a TOF method that calculates the distance to an object from the round-trip time of pulsed light. The TOF method makes it possible to obtain a two-dimensional distribution of the distance between image sensor 20 and the surface of user 1. The two-dimensional distribution of the distance between image sensor 20 and the surface of user 1 can also be called a three-dimensional distribution of the surface of user 1.
[0097] The distance between the image sensor 20 and the surface of the user 1 is calculated as follows. The round-trip time for the pulsed light emitted from the light source 10 to be reflected by the surface of the user 1 and detected by the image sensor 20 is denoted as τ, and the speed of light in air is denoted as c. In this case, the distance d between the image sensor 20 and the surface of the user 1 is d=cτ / 2.
[0098] There are two methods for acquiring the round trip time τ: the direct TOF method and the indirect TOF method.
[0099] In the direct TOF method, the distance d to an object is calculated by directly measuring the round trip time τ of light. In the direct TOF method, the time resolution to achieve a distance resolution of Δd = 1 mm is Δτ = 2Δd / c ≒ 6.6 picoseconds. Thus, in the direct TOF method, high-speed imaging with a time resolution in the picosecond range is used to achieve a distance resolution in the millimeter range.
[0100] On the other hand, in the indirect TOF method, which is often used in practice, the distance d to an object is calculated by measuring the round-trip time τ of the pulsed light from the phase difference of the reflected pulsed light. The phase difference of the reflected pulsed light corresponds to the time lag between the emitted pulsed light and the reflected pulsed light. In the indirect TOF method, the shorter the pulse width Δt, the higher the measurement accuracy. In the indirect TOF method, the light intensity of the emitted light is generally modulated by a square wave or a sine wave to detect the phase difference.
[0101] Fig. 6 is a diagram explaining the principle of measuring the round trip time τ of pulsed light by the indirect TOF method. Signal a) in Fig. 6 shows the time change in the light intensity of the emitted pulsed light, and signals (b) and (c) in Fig. 6 show the time change in the intensity of the reflected pulsed light.
[0102] In the example shown by signal (a) in Fig. 6, the light intensity of the output pulse light is modulated by a square wave. In order to measure the phase difference with high accuracy, the light intensity of the output pulse light is basically modulated at a speed on the same order as the round-trip time τ of light.
[0103] The phase difference of the light pulses can be detected by opening the electronic shutter in the image sensor 20 and measuring the intensity. In the example shown in FIG. 6 (b), the image sensor 20 detects the phase difference of the light pulses at time t 0 After that, the electronic shutter is opened to start detecting the reflected pulse light. 0 is longer than the pulse width Δt of the pulsed light and shorter than the round-trip time τ of the light. That is, Δt <t 0After the electronic shutter is opened, the image sensor 20 accumulates a signal charge according to the amount of reflected pulse light every time Δt. The amount of signal charge accumulated during the first time Δt is S 1 Then, the amount of signal charge accumulated during the next time Δt is S 2 The round-trip time τ of light and the start time t 0 The difference between 2 / (S 1 +S 2 )]Δt. Therefore, the round trip time of light is τ=t 0 +[S 2 / (S 1 +S 2 )]Δt. The amount of accumulated signal charge S 1 , S 2 The distance d to the object is calculated from the round trip time τ of the light obtained indirectly from
[0104] In the example shown by signal (c) in FIG. 6, the reflected pulsed light returns to the image sensor 20 before the emitted pulsed light has finished being emitted. In other words, τ<Δt. In this case, the image sensor 20 opens the electronic shutter at the same time as the light source 10 emits the pulsed light, and accumulates a signal charge according to the amount of the reflected pulsed light every time Δt. The round-trip time of light is τ=[S 2 / (S 1 +S 2 )]Δt.
[0105] In the TOF method, the control circuit 30 causes the image sensor 20 to detect at least a portion of the reflected pulsed light and output a signal indicating a three-dimensional distribution of the surface of the user 1. In this embodiment, the signal output from the image sensor 20 may include a signal indicating the three-dimensional distribution of the surface of the user 1.
[0106] By using the surface reflection component I1 contained in at least a portion of the reflected pulse light in the TOF method, it is possible to more clearly detect the three-dimensional distribution of the surface of user 1, including wrinkles or minute irregularities, compared to conventional TOF methods.
[0107] In the direct TOF method, the distance to an object can be calculated by detecting the surface reflection component I1 contained in the rising period of the reflected pulse light. In the indirect TOF method, the distance to an object is calculated by detecting the portion from the front end to the rear end of the reflected pulse light, as shown by signals b) and (c) in Figure 6. By using spatially resolved imaging, the portion from the front end to the rear end of the surface reflection component I1 can be separated from the reflected pulse light.
[0108] In addition, by using spatially resolved imaging, it is possible to obtain the portion from the front end to the rear end of the internal scattering component I2 from the reflected pulsed light. By using the portion from the front end to the rear end of the internal scattering component I2 in the indirect TOF method, it is also possible to obtain the two-dimensional distribution of the distance between the image sensor 20 and the blood vessels of the user 1, that is, the three-dimensional distribution of the blood vessels of the user 1.
[0109] FIG. 7 is a diagram showing a schematic example of an image of user 1 captured by the TOF method in this embodiment. In the example shown in FIG. 7, a three-dimensional distribution of the surface of user 1 is calculated from the surface reflection component I1. By using the result of comparing information acquired from the three-dimensional distribution of the surface of user 1 with information indicating the concaves and convexes of the characteristic parts of user 1 contained in the biometric data in memory 25, the authentication accuracy can be improved. As a result, the false rejection rate or false acceptance rate is reduced. The three-dimensional distribution of the surface of user 1 may be calculated from the reflected pulse light including both the surface reflection component I1 and the internal scattering component I2.
[0110] Next, a series of processes for biometric authentication of user 1 using identification device 100 in this embodiment will be described.
[0111] FIG. 8 is a flowchart showing an example of the process executed by the control circuit 30.
[0112] In step S201, the control circuit 30 causes the light source 10 to irradiate the user 1 with light. In the case of time-resolved imaging, the light is pulsed light, and in the case of spatially resolved imaging, the light is two-dimensional pattern light. In the case of a combination of time-resolved imaging and spatially resolved imaging, the light is two-dimensional pattern light and pulsed light.
[0113] In step S202, the control circuit 30 causes the image sensor 20 to detect at least a portion of the reflected light returning from the user 1 and output a signal indicating a two-dimensional image of the user 1 according to the intensity distribution of the detected light. If the reflected light is pulsed light, at least a portion of the reflected light is, for example, a component in the rising period or a component in the falling period of the pulsed light. The surface reflection component I1 is obtained from the component in the rising period, and the internal scattering component I2 is obtained from the component in the falling period. When the three-dimensional distribution of the surface of the user 1 is obtained by the indirect TOF method, a portion of the reflected pulsed light from the front end to the rear end of the surface reflection component I1, or a portion from the front end to the rear end of the combined component of the surface reflection component I1 and the internal scattering component I2 is detected.
[0114] In step S203, the control circuit 30 causes the signal processing circuit 40 to generate an image of the user 1 from the output signal, and extracts data characterizing the image. The data characterizing the image generated from the surface reflection component I1 includes information indicating the texture of the skin surface of the user 1. The data characterizing the image generated from the internal scattering component I2 includes information indicating the distribution of blood vessels of the user 1. In the case of the TOF method, the data characterizing the image generated from the surface reflection component I1, or the image generated from the composite component of the surface reflection component I1 and the internal scattering component I2 includes information indicating the three-dimensional distribution of the surface of the user 1, i.e., the unevenness of the characteristic parts of the user 1.
[0115] In step S204, control circuitry 30 compares the data characterizing the generated image of user 1 extracted from that image with data characterizing the image of user 1 stored in memory.
[0116] In step S205, the control circuit 30 judges whether the difference between the two pieces of data to be compared is equal to or less than a reference value. For example, when comparing information on the three-dimensional distribution of the surface of the user 1 acquired by the TOF method with information indicating the unevenness of the characteristic portion of the user 1 contained in the biometric data, an example of the judgment criterion is as follows: The control circuit 30 evaluates whether the difference between the two pieces of data is equal to or less than a reference value for all measurement points on the surface of the user 1, or for a certain percentage or more of the measurement points, such as 80%.
[0117] If the difference is equal to or smaller than the reference value, in step S206, the control circuit 30 determines that the user 1 is the real user. If the difference is not equal to or smaller than the reference value, in step S207, the control circuit 30 determines that the user 1 is a different person.
[0118] The identification device 100 in this embodiment can realize contactless biometric authentication of the user 1. Therefore, the burden on the user 1 during authentication is small.
[0119] Among the parts of the body of the user 1, other than the face, such as the hand, may be used. For example, the surface reflection component I1 may be obtained from a fingerprint, or the internal scattering component I2 may be obtained from the veins of the finger. A fingerprint or vein specific to an individual is likely to be clearly visible on the hand, improving the recognition accuracy. When the hand is used as the subject of the user 1, the body movement may be suppressed by placing the hand on glass. In this case, if there is a space between the glass and the identification device 100, it is advantageous that the camera can be focused. In addition, in the case of hand authentication, the signal-to-noise ratio of the detection signal on the image sensor 20 can be improved by emitting an amount of light exceeding class 1 from the light source 10.
[0120] Next, an application example of the identification device 100 in this embodiment will be described.
[0121] FIG. 9 is a diagram showing an example in which the identification device 100 according to the present embodiment is mounted in the cabin of an automobile. As shown in FIG. 9, the driver, i.e., the user 1, may be identified by the identification device 100 provided in the cabin. Instead of a car key, the user 1 may be identified by the identification device 100 and the car may be started. During automatic driving, the passenger may be identified by the identification device 100, and the identification result may be fed back to automatic driving customized according to the passenger. The feedback may be, for example, gentle braking for an elderly passenger, or exhilarating driving for a passenger who likes driving. This enables automatic driving adapted to the individual.
[0122] 10 is a diagram illustrating an example in which the identification device 100 of this embodiment is applied to a mobile terminal. The mobile terminal is, for example, a smartphone, a tablet, or a personal computer. As the image sensor 20 in the identification device 100, an image sensor built into the mobile terminal may be used.
[0123] The identification device 100 in this embodiment may be applied to, for example, an ATM installed in a bank or a convenience store, or at the entrance of a house, other than an automobile or a mobile terminal.
[0124] The identification device 100 in this embodiment enables highly accurate authentication by combining two or more authentications. The identification device 100 in this embodiment can be used for terminal-based security authentication in addition to terminal-based login authentication. The terminal-based security authentication includes, for example, authentication of logging into a bank account or logging into accounts for various services after logging into a terminal. A system may be created in which conventional password authentication is also used in combination, with the authentication in this disclosure being an additional authentication. This reduces the risk of hacking and ensures higher security than a conventional system in which only password authentication is performed.
[0125] The present disclosure also includes methods that include operations performed by control circuitry 30 and signal processing circuitry 40.
[0126] As described above, the present disclosure includes the identification devices and identification methods described in the following items.
[0127] [Item 1] The identification device according to the first aspect includes a light source, an image sensor, a memory in which biometric data indicating the physical characteristics of a user is stored, and a processor. The processor causes the light source to emit pulsed light having a pulse width of 0.2 ns to 1 μs to be irradiated to the user, causes the image sensor to detect at least a portion of the reflected pulsed light returning from the user, and outputs a signal according to a two-dimensional distribution of the intensity of the at least a portion of the reflected pulsed light, and identifies the user by comparing the signal with the biometric data.
[0128] [Item 2] In the identification device according to the first aspect, the pulse width of the pulsed light may be not less than 3 ns and not more than 50 ns.
[0129] [Item 3] In the identification device according to the first or second item, the image sensor may include a plurality of pixels, and the processor may cause the image sensor to repeatedly accumulate, for each of the plurality of pixels, an electric charge corresponding to the intensity of at least a portion of the reflected pulsed light, between 1,000 and 100,000 times, and the signal may indicate a two-dimensional distribution of the total amount of electric charge accumulated in the plurality of pixels.
[0130] [Item 4] In the identification device according to any one of the first to third items, the biometric data may include first information indicating a texture of a skin surface of the user.
[0131] [Item 5] In the identification device related to the fourth item, the processor may cause the image sensor to output a first signal obtained by detecting at least a portion of a rise period, which is a period from when the intensity of the reflected pulsed light starts to increase to when the increase ends, and identify the user by comparing information obtained from the first signal with the first information.
[0132] [Item 6] In the identification device according to any one of the first to fifth aspects, the biometric data may include second information indicating a distribution of blood vessels of the user.
[0133] [Item 7] In the identification device related to the sixth item, the processor may cause the image sensor to output a second signal obtained by detecting at least a portion of a component of a falling period, which is a period from when the intensity of the reflected pulsed light starts to decrease to when the decrease ends, and identify the user by comparing information obtained from the second signal with the second information.
[0134] [Item 8] In the identification device according to any one of the first to seventh aspects, the biometric data may include third information indicating concaves and convexes of a characteristic portion of the user. In the identification device according to any one of the first to seventh aspects, the biometric data may include fourth information indicating a shape of a characteristic part of the user. In the identification device according to any one of the first to seventh aspects, the biometric data may include fifth information indicating a position of a characteristic part of the user.
[0135] [Item 9] In the identification device related to the eighth item, the processor may cause the image sensor to output a third signal indicating a two-dimensional distribution of the distance between the image sensor and the surface of the user, and identify the user by comparing information obtained from the third signal with the third information.
[0136] [Item 10] In the identification device related to the ninth item, the processor may cause the image sensor to detect at least a portion of a component of a rise period of the reflected pulsed light, which is a period from when the intensity of the reflected pulsed light starts to increase to when the increase ends, and calculate the two-dimensional distribution of the distances from the component.
[0137] [Item 11] The identification device according to the eleventh item includes a light source, an image sensor, a memory in which biometric data indicating the physical characteristics of a user is stored, and a processor. The biometric data includes first information indicating the texture of the skin surface of the user. The processor causes the light source to emit irradiation light that projects a two-dimensional pattern including at least one light area and at least one dark area onto the user, causes the image sensor to detect at least a portion of the reflected light returning from the user, and outputs a signal according to a two-dimensional distribution of the intensity of the at least a portion of the reflected light, and identifies the user by comparing the signal with the biometric data stored in the memory.
[0138] [Item 12] In the identification device according to an eleventh item, the biometric data may further include second information indicating a distribution of blood vessels of the user.
[0139] [Item 13] In the identification device related to the twelfth item, the processor may identify the user by causing the image sensor to detect at least a portion of a first reflected light returning from at least one dark area of the user onto which the two-dimensional pattern is projected, outputting a first signal corresponding to a two-dimensional distribution of the intensity of the at least a portion of the first reflected light, and comparing information obtained from the first signal with the second information.
[0140] [Item 14] In the identification device related to the 13th item, the processor may identify the user by causing the image sensor to detect at least a portion of a second reflected light returning from at least one bright area of the user onto which the two-dimensional pattern is projected, outputting a second signal corresponding to a two-dimensional distribution of the intensity of the at least a portion of the second reflected light, and comparing information obtained by subtracting the first signal from the second signal with the first information.
[0141] [Item 15] In the identification device related to the 15th item, the processor may cause the image sensor to detect at least a portion of a first reflected light returning from the at least one dark area of the user onto which the two-dimensional pattern is projected, detect at least a portion of a second reflected light returning from the at least one bright area, output a first signal corresponding to a two-dimensional distribution of intensity of the at least a portion of the first reflected light, and output a second signal corresponding to a two-dimensional distribution of intensity of the at least a portion of the second reflected light, and identify the user by comparing information obtained by subtracting the first signal from the second signal with the first information.
[0142] [Item 16] In the identification device according to the eleventh aspect, the irradiation light may be pulsed light.
[0143] [Item 17] In the identification device according to the sixteenth aspect, a pulse width of the pulsed light may be not less than 3 ns and not more than 50 ns.
[0144] [Item 18] In the identification device according to the 16th or 17th item, the image sensor may include a plurality of pixels, and the processor may cause the image sensor to repeatedly accumulate, for each of the plurality of pixels, an electric charge corresponding to the intensity of at least a portion of the reflected light, between 1,000 times and 100,000 times, and the signal may indicate a two-dimensional distribution of the total amount of electric charge accumulated in the plurality of pixels.
[0145] [Item 19] In the identification device according to any one of the sixteenth to eighteenth items, the biometric data may further include third information indicating concavities and convexities of a characteristic portion of the user.
[0146] [Item 20] In the identification device related to the 19th item, the processor may cause the image sensor to detect at least a portion of a first reflected pulsed light returning from the at least one dark area of the user onto which the two-dimensional pattern is projected, detect at least a portion of a second reflected pulsed light returning from the at least one bright area, output a first signal corresponding to a two-dimensional distribution of intensity of the at least a portion of the first reflected pulsed light, and output a second signal corresponding to a two-dimensional distribution of intensity of the at least a portion of the second reflected pulsed light, and identify the user by comparing information obtained by subtracting the first signal from the second signal with the third information.
[0147] [Item 21] In the identification device according to any one of the sixteenth to eighteenth items, the biometric data may further include second information indicating a distribution of blood vessels of the user.
[0148] [Item 22] In the identification device related to the 21st item, the processor may cause the image sensor to detect at least a portion of a fall period, which is the period from when the intensity of a first reflected pulse light returning from at least one dark area starts to decrease to when the decrease ends, of the user onto whom the two-dimensional pattern is projected, output a first signal corresponding to a two-dimensional distribution of the intensity of at least a portion of the fall period of the first reflected pulse light, and identify the user by comparing information obtained from the first signal with the second information.
[0149] [Item 23] In the identification device according to any one of items 16 to 18, the processor may cause the image sensor to detect at least a portion of a rise period of the user onto whom the two-dimensional pattern is projected, which is the period from when the intensity of the second reflected pulse light returning from the at least one bright area starts to increase to when the increase ends, output a second signal corresponding to a two-dimensional distribution of the intensity of at least a portion of the rise period of the second reflected pulse light, and identify the user by comparing information obtained from the second signal with the first information.
[0150] [Item 24] The identification method according to the 24th item is an identification method using an identification device equipped with a light source and an image sensor, and includes causing the light source to emit pulsed light having a pulse width of 0.2 ns or more and 1 μs or less to be irradiated to a user, causing the image sensor to detect at least a portion of the reflected pulsed light returning from the user and output a signal corresponding to a two-dimensional distribution of intensity of the at least a portion of the reflected pulsed light, and identifying the user by comparing the signal with biometric data indicating physical characteristics of the user.
[0151] [Item 25] The identification method according to the 25th item is an identification method using an identification device having a light source and an image sensor, and includes causing the light source to emit irradiation light that projects a two-dimensional pattern including at least one bright area and at least one dark area onto the user, causing the image sensor to detect at least a portion of the reflected light returning from the user and output a signal corresponding to a two-dimensional distribution of intensity of the at least a portion of the reflected light, and identifying the user by comparing the signal with biometric data indicating the texture of the user's skin surface. [Industrial Applicability]
[0152] The identification device according to the present disclosure is useful for a camera or an authentication device that identifies a user in a contactless manner, and can be applied to security services. [Explanation of symbols]
[0153] 1 User 10 light source 20 Image Sensor 22 Photoelectric conversion unit 24 Charge storage section 25 Memory 30 Control circuit 32 Light source control unit 34 Sensor control unit 40 Signal Processing Circuit 100 Identification device I1 Surface reflection component I2 internal scattering component
Claims
1. A light source; An image sensor; A processor; Equipped with The processor, causing the light source to emit light for projecting a dot pattern onto a user's face; causing the image sensor to detect light from the face and generate an image signal; A first discrimination process is executed based on a surface reflection component of light from the face based on the image signal; A second classification process is performed based on the image signal and is based on an internal scattering component of light from the face. An apparatus that authenticates the user based on a result of the first identification process and a result of the second identification process.
2. the processor determines that authentication of the user has been successful when it determines that the first identification process has been successful and also determines that the second identification process has been successful.
2. The apparatus of claim 1.
3. the light source and the image sensor are provided in a vehicle; 3. The apparatus of claim 1 or 2, wherein the processor starts the vehicle if it determines that authentication of the user is successful.
4. The processor, generating information indicative of the contours of the face based on the surface reflection components; executing the first identification process based on the information indicating the concaves and convexes; 3. Apparatus according to claim 1 or 2.
5. The device according to claim 1 or 2, wherein the wavelength of the irradiating light is within the red to near infrared wavelength range.
6. A processing method executed by at least one processor, comprising: causing a light source to emit light that projects a dot pattern onto a user's face; causing an image sensor to detect light from the face and generate an image signal; executing a first identification process based on a surface reflection component of light from the face based on the image signal; executing a second classification process based on an internal scattering component of light from the face based on the image signal; and authenticating the user based on a result of the first identification process and a result of the second identification process.
7. A program for carrying out the method according to claim 6.
Citation Information
Patent Citations
Vein image collation device, and personal identification device and personal identification system using the same
JP2008123312A
Biometric authentication system
JP2010146095A
Authentication device, crime prevention system, authentication method, and program
JP2017049867A
Imaging apparatus
JP2017202328A
Biological information detecting device
JP2018089369A
