Image degradation factor estimation device, image degradation factor estimation method, and image degradation factor estimation program

Abstract

To estimate a factor causing degradation of a biological image captured in a non-contact state and support acquisition of a biological image suitable for authentication.SOLUTION: An image degradation factor estimation device includes: an imaging part which images a part of a subject's body; an evaluation part which determines whether the biological image captured by the imaging part is suitable for biometric authentication; an estimation part which estimates at least one factor causing the determination that biological image is unsuitable for biometric authentication when the biological image is determined to be unsuitable for biometric authentication; and an output part which outputs the estimated factor.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present disclosure relates to an apparatus for estimating image degradation factors, a method for estimating image degradation factors, and a program for estimating image degradation factors.

Background Art

[0002] Patent Document 1 discloses a biometric authentication device having a light source that irradiates a finger with light, an imaging unit that images the light transmitted through the finger, and an authentication unit that extracts a blood vessel pattern from the captured image and collates it with a pre-registered blood vessel pattern, wherein the optical axis of the light from the light source and the imaging axis of the imaging unit intersect each other.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In recent years, the demand for biometric authentication has been increasing. However, among biometric authentications, conventional fingerprint authentication is generally fingerprint authentication (so-called contact-type fingerprint authentication) in which a finger used for authentication is pressed against a glass surface or the like for authentication as in the biometric authentication device disclosed in Patent Document 1. Therefore, there is a demand for a technique for acquiring a fingerprint image or fingerprint information without contact (so-called non-contact type fingerprint authentication) from the viewpoints of infection control and public health.

[0005] In view of the above-described conventional circumstances, the present disclosure has been devised, and an object thereof is to provide an apparatus for estimating image degradation factors, a method for estimating image degradation factors, and a program for estimating image degradation factors that estimate the degradation factors of a biological image captured in a non-contact state and assist in acquiring a biological image suitable for authentication.

Means for Solving the Problems

[0006] The present disclosure provides an image degradation factor estimation device including an imaging unit that images a part of a subject's living body, an evaluation unit that determines whether a biological image captured by the imaging unit is suitable for biometric authentication, an estimation unit that estimates at least one factor for which the biological image is determined not to be suitable for the biometric authentication when it is determined that the biological image is not suitable for the biometric authentication, and an output unit that outputs the estimated factor.

[0007] The present disclosure also provides an image degradation factor estimation method performed by at least one computer, including imaging a part of a subject's living body, determining whether a biological image of the part of the subject's living body that has been imaged is suitable for biometric authentication, and estimating and outputting at least one factor for which the biological image is determined not to be suitable for the biometric authentication when it is determined that the biological image is not suitable for the biometric authentication.

[0008] The present disclosure further provides an image degradation factor estimation program for causing at least one computer to execute steps of imaging a part of a subject's living body, determining whether a biological image of the part of the subject's living body that has been imaged is suitable for biometric authentication, and estimating and outputting a factor for which the biological image is determined not to be suitable for the biometric authentication when it is determined that the biological image is not suitable for the biometric authentication.

Advantages of the Invention

[0009] According to the present disclosure, it is possible to estimate the degradation factors of a biological image captured in a non-contact state and assist in obtaining a biological image suitable for authentication.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Embodiments for Carrying Out the Invention

[0011] (Background Leading to the Present Disclosure) Conventionally, when a personal authentication device performs fingerprint registration for fingerprint authentication, the user has had to visit a facility (such as a store, office, etc.) that uses fingerprint authentication to register their fingerprint, resulting in low user convenience. As a method of solving such problems, there is a fingerprint registration method in which the user performs non-contact fingerprint registration on a terminal device owned by the user. However, when imaging a fingerprint in a non-contact state, the user has had to image the fingertip held in a free posture (angle) in the air such that the feature amount of the user's fingerprint can be extracted.

[0012] When imaging a fingerprint in a non-contact state, depending on the imaging environment such as temperature, humidity, brightness of illumination, or arrangement of illumination, the fingertip may become dry or a part of the fingertip may become dark. Even if the fingertip is imaged repeatedly, a biometric image suitable for fingerprint authentication may not be obtained. In such a case, although it may be possible to image a fingerprint image suitable for fingerprint authentication by wetting the fingertip or changing the position where the fingertip is held, since the user does not know the reason (factor) why the imaged biometric image is not suitable for fingerprint authentication, it has been difficult to obtain a fingerprint image suitable for fingerprint authentication.

[0013] Therefore, in each of the embodiments described below, an example of an image degradation factor estimation device, an image degradation factor estimation method, and an image degradation factor estimation program that estimate the degradation factors of a biological image captured in a non-contact state and assist in obtaining a biological image suitable for authentication will be described.

[0014] Hereinafter, with reference to the drawings as appropriate, each embodiment specifically disclosing the configuration and operation of the image degradation factor estimation device, the image degradation factor estimation method, and the image degradation factor estimation program according to the present disclosure will be described in detail. However, a more detailed description than necessary may be omitted. For example, a detailed description of well-known matters and a redundant description of substantially the same configuration may be omitted. This is to avoid making the following description unnecessarily redundant and to facilitate the understanding of those skilled in the art. Note that the accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.

[0015] In the following Embodiment 1 and Embodiment 2, as an example of a biological image, a method for assisting the acquisition of a fingerprint image used for fingerprint authentication will be described, but the present disclosure is not limited thereto. The method for acquiring a biological image disclosed in Embodiment 1 and Embodiment 2 may be applied to and used for the acquisition of, for example, a palmprint image used for palmprint authentication, a face image used for face authentication, and the like.

[0016] (Embodiment 1) With reference to FIG. 1, an example of the internal configuration of the terminal device P1 according to Embodiment 1 will be described. FIG. 1 is a diagram showing an example of the internal configuration of the terminal device P1 in Embodiment 1.

[0017] The terminal device P1 is realized by, for example, a smartphone, a tablet terminal, or the like. The terminal device P1 can receive an operation by a user and acquire a fingerprint image or fingerprint data of the user for use in fingerprint authentication. The fingerprint data here refers to data indicating the feature amounts of fingerprints extracted using known techniques (for example, the minutiae method or the frequency feature analysis method).

[0018] Note that the terminal device P1 in Embodiment 1 will be described as an example that can execute registration of the acquired fingerprint image or fingerprint data and fingerprint authentication based on comparison between the acquired fingerprint image or fingerprint data and the fingerprint data registered in advance, but it is not limited thereto. For example, the terminal device P1 may be a device that only executes acquisition of the acquired fingerprint image or fingerprint data, or may be a device that executes registration of the acquired fingerprint image or fingerprint data.

[0019] The terminal device P1 includes a communication unit 10, a processor 11, a memory 12, a camera 13, a monitor 14, and a fingerprint database DB1. Note that the fingerprint database DB1 may be an external recording medium or an external storage device that is connected to be capable of data communication with the terminal device P1.

[0020] The communication unit 10 is connected to be capable of data communication with another device (not shown) via a network (not shown). The communication unit 10 outputs various data acquired via the network (not shown) to the processor 11.

[0021] The processor 11 is configured using, for example, a Central Processing Unit (hereinafter referred to as "CPU") or a Field Programmable Gate Array (hereinafter referred to as "FPGA"), and performs various processes and controls in cooperation with the memory 12. Specifically, the processor 11 refers to the programs and data held in the memory 12 and executes the programs to realize various functions such as fingerprint imaging and acquisition of fingerprint data, or fingerprint authentication using the acquired fingerprint data.

[0022] The memory 12 includes, for example, a Random Access Memory (hereinafter referred to as "RAM") used as a work memory when each process of the processor 11 is executed, and a Read Only Memory (hereinafter referred to as "ROM") that stores a program and data defining the operation of the processor 11. In the RAM, data or information generated or acquired by the processor 11 is temporarily stored. In the ROM, a program defining the operation of the processor 11 is written. The memory 12 stores a deterioration estimation model 12A.

[0023] The deterioration estimation model 12A is a learned model that has been learned, for example, to be able to estimate deterioration factors of the quality of a fingerprint image, such as the inclination of the fingerprint shown in the fingerprint image, the dry state of the fingerprint part, the wet state of the fingerprint part, the brightness of the fingerprint, or the presence or absence of an obstacle covering the fingerprint. The deterioration estimation model 12A may be data that can be acquired or updated from any other device via the communication unit 10. Further, the deterioration factors that can be estimated by the deterioration estimation model 12A are not limited to the above-described examples and may be arbitrarily added or deleted.

[0024] The deterioration estimation model 12A is used by the processor 11 to estimate the deterioration factors (causes) of the quality of a fingerprint image determined to be unsuitable for fingerprint authentication. The deterioration estimation model 12A analyzes the fingerprint image and estimates the probability that each deterioration factor can be a deterioration factor of the quality of the fingerprint image. When the deterioration estimation model 12A determines that the probability of each estimated deterioration factor is equal to or greater than the probability of being determined to be a deterioration factor of the quality of the fingerprint image, it is estimated to be a deterioration factor of the quality of the fingerprint image. The deterioration estimation model 12A associates the information on the probability of each calculated deterioration factor, the information on the estimated deterioration factor result of the fingerprint image, and the fingerprint image and stores them in the memory 12.

[0025] Note that the quality of the fingerprint image referred to here does not refer to the image quality of the fingerprint image itself, but indicates the quality required to acquire fingerprint data suitable for fingerprint authentication.

[0026] The camera 13 is configured to include at least a lens (not shown) and an image sensor (not shown). The image sensor is a solid-state imaging device such as a Charged-Coupled Device (CCD) or a Complementary Metal Oxide Semiconductor (CMOS), and converts an optical image formed on the imaging surface into an electrical signal. The camera 13 starts imaging according to a control instruction for starting imaging input by the processor 11 or an imaging operation by the user. The camera 13 outputs the captured imaging image to the processor 11.

[0027] The monitor 14 is configured using, for example, a Liquid Crystal Display (LCD) or an organic Electroluminescence (EL). The monitor 14 outputs and displays the imaging image captured by the camera 13 or various screens generated by the processor 11. Further, the monitor 14 may be a user interface configured by a touch panel. In such a case, the monitor 14 has a function as an operation unit 15, receives an input operation by the user, and outputs the result of the input operation by the user to the processor 11.

[0028] The operation unit 15 can receive a user operation and outputs the content of the input operation based on the user operation to the processor 11. The operation unit 15 may be realized as the touch panel of the monitor 14 described above. Further, the operation unit 15 may include a microphone (not shown) and receive a voice input operation based on the user's voice.

[0029] The fingerprint database DB1 is a so-called storage, and is configured using a storage medium such as a flash memory, a Hard Disk Drive (hereinafter referred to as "HDD"), or a Solid State Drive (hereinafter referred to as "SSD"). The fingerprint database DB1 stores (registers) and manages the fingerprint data output from the processor 11.

[0030] Next, with reference to FIG. 2, a method for acquiring a fingerprint image executed by the terminal device P1 will be described. FIG. 2 is a flowchart showing an example of the operation procedure of the terminal device P1 according to Embodiment 1.

[0031] The processor 11 activates an application capable of acquiring a fingerprint by a user operation. When the processor 11 receives an imaging start operation by a user operation, it generates and outputs a control instruction to start imaging to the camera 13 (St11).

[0032] The camera 13 images the finger of the user held toward the camera 13, and outputs the captured image to the processor 11 (St12).

[0033] The processor 11 detects at least one fingertip from the captured image, cuts out the area where the detected fingertip is reflected, and generates a fingerprint image. The processor 11 calculates an evaluation value indicating the degree to which the generated fingerprint image is suitable for fingerprint authentication (St13), and determines whether the calculated evaluation value is equal to or greater than a threshold value (St14). Note that the threshold value here is the value of the evaluation value indicating that the fingerprint image is suitable for fingerprint authentication.

[0034] In the process of step St14, when the processor 11 determines that the calculated evaluation value is equal to or greater than the threshold value (St14, YES), it acquires the generated fingerprint image as a fingerprint image to be used for acquisition or registration of fingerprint data (St15).

[0035] On the other hand, in the process of step St14, when the processor 11 determines that the calculated evaluation value is not equal to or greater than the threshold value (St14, NO), it determines whether the current number of imaging times n (n: an integer of 1 or more) is equal to or greater than the upper limit number of times m (m: an integer of 1 or more) (St16). Note that the upper limit number of times m here is the upper limit number of times of repositioning the fingertip. Also, the upper limit number of times m is desirably 2 or more, but may be set to 1 time when repositioning is not permitted.

[0036] In the process of step St16, when the processor 11 determines that the current number of imaging times n is less than the upper limit number of times m (St16, NO), it uses the degradation estimation model 12A to estimate the degradation factors of the fingerprint image quality, that is, the factors for which the fingerprint image is determined to be unsuitable for fingerprint authentication (St17). In the present disclosure, as an example, the degradation factors that can be estimated by the degradation estimation model 12A are five degradation factors: "tilt", "dryness", "wetness", "brightness", and "obstacle". However, the degradation factors that can be estimated by the degradation estimation model 12A are not limited to the above five, and can be arbitrarily changed, deleted, or added.

[0037] Based on the estimation result of the degradation factor, the processor 11 generates a degradation factor estimation result screen SC21 (see FIG. 3) for notifying the user of the estimated degradation factor, and outputs it to the monitor 14 for display (St18).

[0038] Based on the estimated degradation factor, the processor 11 generates a user guide screen (for example, the user guide screens SC31, SC41, etc. shown in FIG. 3) for eliminating each of the estimated degradation factors, and outputs it to the monitor 14 for display (St19). The processor 11 executes the degradation factor prevention guide by displaying the user guide screen, eliminates each of the degradation factors, and then captures the user's finger again (St19). Note that the imaging timing of the user's finger may be any timing. For example, it may be executed based on receiving an imaging start operation by the user operation, or may be executed after the display of all user guide screens is completed.

[0039] On the other hand, in the process of step St16, when the processor 11 determines that the current number of imaging times n is greater than or equal to the upper limit number of times m (St16, YES), it uses the degradation estimation model 12A to estimate the degradation factors of the fingerprint image quality, that is, the factors for which the fingerprint image is determined to be unsuitable for fingerprint authentication.

[0040] In the (m + 1)-th imaging, the processor 11 generates a user guide screen (for example, the user guide screens SC37, SC41 to SC45 shown in FIG. 7, etc.) for eliminating all factors that can cause degradation of the fingerprint image, outputs it to the monitor 14, and causes it to be displayed (St20). The processor 11 executes a degradation factor prevention guide by displaying the user guide screen, and after eliminating each degradation factor, captures the user's finger again (St20). Note that the imaging timing of the user's finger may be any timing. For example, it may be executed based on acceptance of an imaging start operation by a user operation, or may be executed after the display of all user guide screens is completed.

[0041] The processor 11 detects at least one fingertip from the re-captured imaging image, cuts out the area where the detected fingertip appears, and generates a fingerprint image. The processor 11 calculates an evaluation value indicating the degree to which the generated fingerprint image is suitable for fingerprint authentication (St21). Further, the processor 11 estimates the degradation factor of the quality of the fingerprint image generated using the re-captured imaging image by using the degradation estimation model 12A (St21).

[0042] The processor 11 determines and acquires the fingerprint image to be used for acquisition or registration of fingerprint data based on the evaluation value or degradation factor of each of the (m + 1) fingerprint images acquired by the (m + 1) times of imaging (St22).

[0043] Here, the processor 11 may determine, as the fingerprint image to be used for acquisition or registration of fingerprint data, the fingerprint image with the largest evaluation value among each of the (m + 1) fingerprint images. Further, based on the estimated evaluation value or the probability of the degradation factor, the processor 11 may select the fingerprint image with the smallest average value of the evaluation values, the fingerprint image with the smallest maximum value of the evaluation values, or the fingerprint image with the smallest number of degradation factors estimated to be the degradation factor (i.e., the number of degradation factors whose probability of the degradation factor is equal to or greater than the threshold for estimating the degradation factor) as the fingerprint image to be used for acquisition or registration of fingerprint data. The processor 11 extracts and acquires fingerprint data from the determined fingerprint image, and performs registration of the acquired fingerprint data or fingerprint authentication using the acquired fingerprint data.

[0044] As described above, the terminal device P1 according to the first embodiment estimates the degradation factor of the fingerprint image, and by executing a user guide (i.e., a degradation factor prevention guide) for preventing the degradation factor based on the estimated degradation factor, can more effectively acquire a fingerprint image more suitable for fingerprint authentication.

[0045] Also, generally, since the estimation process of the degradation factor using the degradation estimation model 12A (i.e., the learned model) is a complex process, the processing load on the processor 11 is large and the power consumption is large. Therefore, the terminal device P1 according to the first embodiment performs an evaluation (step St14) as to whether the fingerprint image is a fingerprint image suitable for fingerprint authentication, and estimates the degradation factor of the fingerprint image only when it is determined that the fingerprint image is not a fingerprint image suitable for fingerprint authentication. Thereby, the terminal device P1 can reduce the processing load on the processor 11 in acquiring fingerprint data, and can suppress heat generation or power consumption of the terminal device P1.

[0046] Next, with reference to FIG. 3, an example of the n-th degradation factor prevention guide will be described. FIG. 3 is a diagram showing an example of the n-th degradation factor prevention guide. It goes without saying that the various screen examples shown in FIG. 3 are merely examples and are not limited thereto.

[0047] Based on the fingerprint image FG11, the processor 11 estimates the probability of each degradation factor. While the processor 11 is estimating the probability of each degradation factor, it displays a degradation factor estimation screen SC11 on the monitor 14. The degradation factor estimation screen SC11 is generated to include the fingerprint image FG11 that is the object of degradation factor estimation.

[0048] Based on the probability of each estimated degradation factor, the processor 11 estimates the degradation factor of the fingerprint image FG11. When the estimation of the probability of each degradation factor is completed, the processor 11 generates a degradation factor probability screen SC12 and outputs it to the monitor 14. The degradation factor probability screen SC12 is generated to include information on the probability of each degradation factor. Note that the display of the degradation factor probability screen SC12 is not essential and may be omitted.

[0049] Also, when the estimation of the degradation factor is completed, the processor 11 generates a degradation factor estimation result screen SC21 and outputs it to the monitor 14. The degradation factor estimation result screen SC13 is generated to include information RS21 regarding the estimation result of the degradation factor.

[0050] For example, in the example shown in FIG. 3, the processor 11 estimates that the probability of the degradation factor due to the inclination of the finger (fingerprint) is "90 / 100", the probability of the degradation factor due to the dryness of the finger (fingerprint) is "3 / 100", the probability of the degradation factor due to the wetness of the finger (fingerprint) is "4 / 100", the probability of the degradation factor due to the brightness of the finger (fingerprint) is "87 / 100", and the probability of the degradation factor due to the shielding object that shields the finger (fingerprint) is "0 / 100". In such a case, the processor 11 estimates the degradation factors "inclination" and "brightness" whose estimated probabilities are equal to or greater than the threshold value as the degradation factors of the fingerprint image FG11. The processor 11 generates a degradation factor estimation result screen SC21 notifying that the degradation factors of the fingerprint image FG11 are "inclination" and "brightness".

[0051] Note that different values may be set for the thresholds for estimating the degradation factors for each degradation factor. For example, the probability that the degradation factor "tilt" is estimated as a degradation factor of the fingerprint image and the probability that the degradation factor "brightness" is estimated as a degradation factor of the fingerprint image may be the same or different. Further, the threshold for estimating the degradation factor may be arbitrarily changeable based on the environment (e.g., time zone, season, etc.) in which the fingerprint image is acquired.

[0052] The processor 11 generates a user guide screen SC30 that requests re-capture of the fingerprint and outputs it to the monitor 14. The user guide screen SC30 is generated to include a message requesting re-capture of the fingerprint. For example, in the example shown in FIG. 3, the user guide screen SC30 includes a message Msg30 "Please re-capture because the tilt and brightness of the finger are poor" that requests re-capture of the fingerprint and the degradation factors of the fingerprint image FG11 are "tilt" and "brightness".

[0053] The processor 11 generates user guide screens SC41 and SC42 for assisting in eliminating all the degradation factors estimated to be degradation factors and outputs them to the monitor 14. The user guide screens SC41 and SC42 are generated to include assistance information for eliminating the corresponding degradation factors, respectively.

[0054] For example, in the example shown in FIG. 3, when the estimation result of the degradation factor is "tilt" and "brightness", the processor 11 generates a user guide screen SC41 for assisting in eliminating the degradation factor "brightness" and a user guide screen SC42 for assisting in eliminating the degradation factor "tilt". The user guide screen SC41 includes, as assistance information for assisting in eliminating the degradation factor "brightness", a guide message Msg41 "Please capture an image in a bright environment" and a guide image GD41 notifying that the brightness of the current fingerprint image FG11 is low. Further, the user guide screen SC42 includes, as assistance information for assisting in eliminating the degradation factor "tilt", a guide message Msg42 "Please tilt your finger slightly to the right" and a guide image GD42 indicating that the finger shown in the current fingerprint image FG11 is tilted to the left.

[0055] Note that the user guide screen SC41 and the user guide screen SC42 may be generated on one screen. Further, the processor 11 executes a switching process between the user guide screen SC41 and the user guide screen SC42 based on a user operation, and after executing all the guides for the user by the user guide screens SC41 and SC42, re-captures the user's finger.

[0056] Next, with reference to each of FIGS. 4 to 6, other examples of user guide screens will be described. FIG. 4 is a diagram showing an example of a degradation factor prevention guide. FIG. 5 is a diagram showing an example of a degradation factor prevention guide. FIG. 6 is a diagram showing an example of a degradation factor prevention guide. Needless to say, the various screen examples shown in FIGS. 4 to 6 are merely examples and are not limited thereto.

[0057] When the estimation result of the degradation factor is "dryness", the processor 11 generates user guide screens SC33 and SC43 for assisting in eliminating the degradation factor "dryness" and outputs them to the monitor 14.

[0058] For example, in the example shown in FIG. 4, the user guide screen SC33 is generated to include a guide message Msg33 "Since the finger is dry, please apply hand cream, water, etc. and image again" as support information for assisting in eliminating the deterioration factor "dryness". Also, the user guide screen SC43 includes, as support information for assisting in eliminating the deterioration factor "dryness", a guide message Msg43 "Please apply hand cream, water, etc. and image again" and a guide image GD43 indicating that the fingertips shown in the current fingerprint image FG11 are dry. Note that the symbol "·" in the above-described guide messages Msg33 and Msg43 is used to mean "or".

[0059] When the estimation result of the deterioration factor is "wet", the processor 11 generates user guide screens SC34 and SC44 for assisting in eliminating the deterioration factor "wet" and outputs them to the monitor 14.

[0060] For example, in the example shown in FIG. 5, the user guide screen SC34 is generated to include a guide message Msg34 "Since the finger is wet, please wipe off the water and image again" as support information for assisting in eliminating the deterioration factor "wetness". Also, the user guide screen SC44 includes, as support information for assisting in eliminating the deterioration factor "wetness", a guide message Msg44 "Please wipe off the water and image" and a guide image GD44 indicating that the fingertips shown in the current fingerprint image FG11 are wet.

[0061] When the estimation result of the deterioration factor is "obstacle", the processor 11 generates user guide screens SC35 and SC45 for assisting in eliminating the deterioration factor "obstacle" and outputs them to the monitor 14. Here, the "obstacle" refers to a state where the fingerprint of the fingertip is shielded by dust, dirt, an object, etc. and cannot be acquired.

[0062] For example, in the example shown in FIG. 6, the user guide screen SC35 is generated to include a guide message Msg35 "There is an obstacle, so It is generated including "Please clean your finger and image again." Also, the user guide screen SC45 includes a guide message Msg45 "Please clean your finger and image" and a guide image GD45 indicating that the fingerprint shown in the current fingerprint image FG11 is blocked (hidden) by an obstacle GM as support information for assisting in eliminating the degradation factor "obstacle".

[0063] As described above, the terminal device P1 can more effectively obtain a fingerprint image suitable for fingerprint authentication by generating and displaying a user guide screen corresponding to the degradation factor of the fingerprint image.

[0064] Next, with reference to FIG. 7, an example of the (m + 1)-th degradation factor prevention guide will be described. FIG. 7 is a diagram showing an example of the (m + 1)-th degradation factor prevention guide. Needless to say, the various screen examples shown in FIG. 7 are just examples and are not limited to this.

[0065] When the processor 11 determines that the current imaging count n is equal to or greater than the reimaging upper limit count m, it executes a degradation factor prevention guide for eliminating all possible degradation factors that can cause degradation of the fingerprint image and performing the (m + 1)-th imaging. Specifically, regardless of the degradation factors estimated from the fingerprint images captured so far, the processor 11 generates and displays user guide screens SC41 to SC45 corresponding to all possible degradation factors in order to eliminate all possible degradation factors that can cause degradation of the fingerprint image before performing the (m + 1)-th imaging.

[0066] For example, when the total number of possible degradation factors that can cause degradation of the fingerprint image is five ("tilt", "dryness", "wetness", "brightness", and "obstacle"), user guide screens SC41 to SC45 for eliminating each of these five degradation factors are generated and output to the monitor 14. Note that the guides for eliminating each of the five degradation factors may be executed collectively on one or more user guide screens.

[0067] After displaying the user guide screens SC41 to SC45, when the processor 11 determines based on a user operation that the deterioration factors caused by the user have been eliminated, the (m + 1)-th imaging is executed.

[0068] As described above, when the terminal device P1 determines that the current number of imaging times n is equal to or greater than the preset maximum number of reimaging times m, by generating and displaying a user guide screen corresponding to all deterioration factors that can cause deterioration of the fingerprint image, a fingerprint image more suitable for fingerprint authentication can be obtained more effectively.

[0069] (Embodiment 2) The terminal device P1 according to Embodiment 1 has been described as an example of executing fingerprint imaging, determination as to whether a fingerprint image is a fingerprint image suitable for fingerprint authentication, estimation of deterioration factors of the fingerprint image, and a deterioration factor prevention guide based on the estimation result of the deterioration factors. The biometric information acquisition system 100 according to Embodiment 2 will be described as an example in which the terminal device P1A executes fingerprint imaging and a deterioration factor prevention guide based on the estimation result of the deterioration factors, and the server S1 executes determination as to whether a fingerprint image is a fingerprint image suitable for fingerprint authentication and estimation of deterioration factors of the fingerprint image.

[0070] Note that the terminal device P1A of the biometric information acquisition system 100 in Embodiment 2 has a configuration partially similar to that of the terminal device P1 according to Embodiment 1. Therefore, in the following description of the biometric information acquisition system 100 according to Embodiment 2, the same reference numerals are given to the same configurations as those of the terminal device P1 in Embodiment 1, and the description thereof will be omitted.

[0071] With reference to FIG. 8, a configuration example of the biometric information acquisition system 100 according to Embodiment 2 will be described. FIG. 8 is a diagram showing an internal configuration example of the terminal device P1A and the server S1 according to Embodiment 2.

[0072] The terminal device P1A captures the user's fingertip or executes a deterioration factor prevention guide based on the deterioration estimation result. The terminal device P1A includes a communication unit 10A, a camera 13, a monitor 14, and an operation unit 15. Note that the camera 13, the monitor 14, and the operation unit 15 of the terminal device P1A may be separate bodies or composed of a plurality of devices respectively.

[0073] The communication unit 10A is communicably connected to the communication unit 20 of the server S1 via a network. Note that the communication unit 10 may be communicably connected to the communication unit 20 of the server S1 by wireless or wired communication. The wireless communication here is, for example, short-range wireless communication such as Bluetooth (registered trademark) or NFC (registered trademark), or communication via a wireless LAN (Local Area Network) such as Wi-Fi (registered trademark).

[0074] The communication unit 10A transmits various data output from the processor 11 to the server S1. Also, the communication unit 10A outputs various data transmitted from the server S1 to the processor 11.

[0075] The processor 11A transmits the captured image captured by the camera 13 or the fingerprint image obtained by cutting out the area where the fingerprint appears from the captured image to the server S1. Note that the transmitted captured image or fingerprint image may be encrypted.

[0076] Also, the processor 11A generates a user guide screen (see FIGS. 3 to 7) corresponding to the deterioration estimation result based on the information on the deterioration factor estimation result of the fingerprint image transmitted from the server S1, outputs it to the monitor 14, and executes the deterioration factor prevention guide.

[0077] The server S1 includes a communication unit 20, a processor 21, a memory 22, and a fingerprint database DB2. Note that the fingerprint database DB2 may be an external recording medium or an external storage device communicably connected to the server S1.

[0078] The communication unit 20 is communicably connected to the terminal device P1A via a network. The communication unit 20 outputs various data acquired via the network to the processor 21.

[0079] The processor 21 is configured using, for example, a CPU or an FPGA, and performs various processes and controls in cooperation with the memory 22. Specifically, the processor 21 refers to the programs and data held in the memory 22, and by executing those programs, realizes various functions such as estimation of the deterioration factors of fingerprint images using the deterioration estimation model 12A, registration of fingerprint data, or fingerprint authentication using fingerprint data.

[0080] The memory 22 has, for example, a RAM as a work memory used when executing each process of the processor 21, and a ROM that stores programs and data defining the operation of the processor 21. Data or information generated or acquired by the processor 21 is temporarily stored in the RAM. A program defining the operation of the processor 21 is written in the ROM. The memory 22 stores the deterioration estimation model 12A.

[0081] Next, with reference to FIG. 9, the correspondence between the operation procedure shown in FIG. 2 and the operation procedure executed by the biometric information acquisition system 100 according to Embodiment 2 will be described. FIG. 9 is a table showing the correspondence between the processes of the terminal device P1A and the server S1 according to Embodiment 2 and the respective processes shown in FIG. 2.

[0082] The terminal device P1A according to Embodiment 2 executes the processes of steps St11 to St12 and the processes of steps St18 to St20 among the operation procedures (steps St11 to St22) shown in FIG. 2.

[0083] Also, the server S1 according to Embodiment 2 executes the processes of steps St13 to St17 and the processes of steps St21 to St22 among the operation procedures (steps St11 to St22) shown in FIG. 2.

[0084] As described above, the biological information acquisition system 100 according to the second embodiment can execute the operation procedure shown in FIG. 2 by the terminal device P1A and the server S1.

[0085] (Appendix) From the descriptions of the above embodiments, the following technologies are disclosed.

[0086] (Technology 1) An imaging unit (camera 13) that images a part of the living body (e.g., fingertip, etc.) of a subject (user), An evaluation unit (processor 11) that determines whether the biological image captured by the imaging unit (camera 13) is suitable for biometric authentication, When it is determined that the biological image is not suitable for the biometric authentication, an estimation unit (processor 11) that estimates at least one factor for which the biological image is determined not to be suitable for the biometric authentication, An output unit (monitor 14) that outputs the estimated factor, and An apparatus (terminal device P1) for estimating image degradation factors. With this configuration, the apparatus (terminal device P1) for estimating image degradation factors can support the user in improving the degradation factors by outputting the degradation factors of the fingerprint image, and can more effectively acquire a biological image suitable for fingerprint authentication.

[0087] (Technology 2) The output unit (monitor 14) generates and outputs a guide screen (see FIGS. 3 to 6) that supports the elimination of the estimated factor based on the estimated factor. The apparatus (terminal device P1) for estimating image degradation factors according to (Technology 1). With this configuration, the apparatus (terminal device P1) for estimating image degradation factors can support the user in improving the degradation factors by generating and displaying a user guide screen (see FIGS. 3 to 6) corresponding to the degradation factors of the fingerprint image, and can more effectively acquire a biological image suitable for fingerprint authentication.

[0088] (Technology 3) The imaging unit (camera 13) When it is determined that the biometric image is not suitable for the biometric authentication, after the output of the estimated factor, a part of the living body is imaged again, The estimation unit (processor 11) is, When it is determined that the biometric image captured again is not suitable for the biometric authentication, it is determined whether the number of times n of imaging a part of the living body by the imaging unit (camera 13) is equal to or more than a predetermined number of times (the upper limit number of times m of re-taking), When it is determined that the number of times n of imaging by the imaging unit (camera 13) is less than the predetermined number of times (the upper limit number of times m of re-taking), the factor for which it is determined that the biometric image captured again is not suitable for the biometric authentication is estimated, The output unit (monitor 14) is, Output the estimated factor, The image degradation factor estimation device (terminal device P1) described in (Technology 1) or (Technology 2). With this configuration, the image degradation factor estimation device (terminal device P1) supports the user to improve the degradation factor by generating and displaying a user guide screen corresponding to the degradation factor of the fingerprint image, and can more effectively acquire a biometric image more suitable for fingerprint authentication.

[0089] (Technology 4) The output unit (monitor 14) is, When it is determined that the number of times n of imaging by the imaging unit (camera 13) is equal to or more than the predetermined number of times (the upper limit number of times m of re-taking), a guide screen (see FIG. 7) for assisting in eliminating all factors that can be estimated by the estimation unit (processor 11) and for which it can be determined that the biometric image is not suitable for the biometric authentication is generated and output, The image degradation factor estimation device (terminal device P1) described in any one of (Technology 1) to (Technology 3). With this configuration, when the image degradation factor estimation device (terminal device P1) determines that the current number of imaging times n is equal to or more than the preset upper limit number of re-taking times m, a user guide screen (see FIG. 7) corresponding to all degradation factors that can cause degradation of the fingerprint image is generated and displayed, so that a fingerprint image more suitable for fingerprint authentication can be more effectively acquired.

[0090] (Technology 5) When it is determined that the biological image is not suitable for the biometric authentication, after the output of the estimated factor, a part of the living body is imaged again by the imaging unit (camera 13). When it is determined that the biological image is not suitable for the biometric authentication, after the output of the estimated factor, a part of the living body is imaged again, The evaluation unit (processor 11) calculates an evaluation value indicating the degree to which the biological image is suitable for the biometric authentication for each biological image, The output unit (monitor 14) determines and outputs, as the biological image to be used for the biometric authentication, the biological image having the largest evaluation value among the plurality of biological images. An image degradation factor estimation device (terminal device P1) according to any one of (Technology 1) to (Technology 4). With this configuration, the image degradation factor estimation device (terminal device P1) can acquire a fingerprint image suitable for fingerprint authentication.

[0091] (Technology 6) The estimation unit (processor 11) For a plurality of factor candidates that can be estimated by the estimation unit (processor 11) and for which it can be determined that the biological image is not suitable for the biometric authentication (for example, the degradation factors "tilt", "dryness", "wetness", "brightness", and "obstacle" in the present disclosure), the probability that each biological image is the factor not suitable for the biometric authentication is estimated, Based on the probabilities of the respective plurality of factor candidates, the factor for which it is determined that the biological image is not suitable for the biometric authentication is estimated. An image degradation factor estimation device (terminal device P1) according to any one of (Technology 1) to (Technology 4). With this configuration, the image degradation factor estimation device (terminal device P1) can estimate a degradation factor with a high probability of contributing as a factor for which the fingerprint image is not suitable for fingerprint authentication.

[0092] (Technology 7) The imaging unit (camera 13) When it is determined that the biological image is not suitable for the biometric authentication, after the output of the estimated factor, a part of the living body is imaged again, The output unit (monitor 14) Based on the probabilities of the plurality of factor candidates (for example, the degradation factors "tilt", "dryness", "wetness", "brightness", and "obstacle" in the present disclosure) of each of the plurality of biological images, any one of the plurality of biological images is determined as a biological image to be used for the biological authentication and output. The image degradation factor estimation device (terminal device P1) according to (Technology 6). With this configuration, the image degradation factor estimation device (terminal device P1) can acquire a fingerprint image suitable for fingerprint authentication based on the probabilities of the factor candidates that can be factors unsuitable for fingerprint authentication.

[0093] (Technology 8) When it is determined that the biological image is not suitable for the biological authentication, the estimation unit (processor 11) estimates the factor using a degradation estimation model for estimating the factor. The image degradation factor estimation device (terminal device P1) according to any one of (Technology 1) to (Technology 7). With this configuration, the image degradation factor estimation device (terminal device P1) generally performs a complicated process, and the processing load on the processor 11 is large and the power consumption is large. By executing the degradation factor estimation process using the degradation estimation model 12A (that is, the learned model) only when it is determined that the fingerprint image is not a fingerprint image suitable for fingerprint authentication, the processing load on the processor 11 in acquiring fingerprint data can be reduced, and heat generation or power consumption of the terminal device P1 can be suppressed.

[0094] (Technology 9) The estimation unit (processor 11) When it is determined that the biological image is suitable for the biological authentication, the estimation of the factor is omitted. The output unit (monitor 14) Determines and outputs the biological image determined to be suitable for the biological authentication as the biological image to be used for the biological authentication. The image degradation factor estimation device (terminal device P1) according to any one of (Technology 1) to (Technology 8). With this configuration, the image degradation factor estimation device (terminal device P1) can acquire a fingerprint image suitable for fingerprint authentication.

[0095] (Technology 10) An image degradation factor estimation method performed by at least one computer (terminal device P1), imaging a part of the living body of a subject (user), determining whether the biometric image of the part of the living body of the subject (user) is suitable for biometric authentication, when it is determined that the biometric image is not suitable for the biometric authentication, estimating and outputting at least one factor for which it is determined that the biometric image is not suitable for the biometric authentication. Image degradation factor estimation method. With this configuration, the computer (terminal device P1) can support the user in improving the degradation factors by outputting the degradation factors of the fingerprint image, and can more effectively acquire a biometric image suitable for fingerprint authentication. Note that the above-described image degradation factor estimation method may be executed by a plurality of computers (terminal devices P1A and server S1).

[0096] (Technology 11) In at least one computer (terminal device P1), a step of imaging a part of the living body of a subject (user), a step of determining whether the biometric image of the part of the living body of the subject (user) is suitable for biometric authentication, when it is determined that the biometric image is not suitable for the biometric authentication, a step of estimating and outputting the factor for which it is determined that the biometric image is not suitable for the biometric authentication, for Image degradation factor estimation program. With this configuration, the computer (terminal device P1) can support the user in improving the degradation factors by outputting the degradation factors of the fingerprint image, and can more effectively acquire a biometric image suitable for fingerprint authentication. Note that the above-described image degradation factor estimation program may be executed by a plurality of computers (terminal devices P1A and server S1).

[0097] The various embodiments have been described above with reference to the accompanying drawings, but the present disclosure is not limited to such examples. It is obvious that those skilled in the art can conceive of various change examples, modification examples, substitution examples, addition examples, deletion examples, and equivalent examples within the scope described in the claims, and it is understood that they also belong to the technical scope of the present disclosure. Further, within the scope not departing from the gist of the invention, the respective components in the above-described various embodiments may be arbitrarily combined.

Industrial Applicability

[0098] The present disclosure is useful as an apparatus for estimating factors degrading a biological image captured in a non-contact state, a method for estimating factors degrading an image, and a program for presenting a program for estimating factors degrading an image, which assist in acquiring a biological image suitable for authentication.

Explanation of Signs

[0099] 10, 10A, 20 Communication unit 11, 11A, 21 Processor 12, 22 Memory 12A Degradation estimation model 13 Camera 14 Monitor 15 Operation unit 100 Biological information acquisition system DB1, DB2 Fingerprint database FG11 Fingerprint image GD41, GD42, GD43, GD44, GD45 Guide image P1, P1A Terminal device S1 Server SC11 Degradation factor estimation screen SC12 Degradation factor probability screen SC13, SC21 Degradation factor estimation result screen SC30, SC31, SC33, SC34, SC35, SC37, SC41, SC42, SC43, SC44, SC45 User guide screen

Claims

1. An imaging unit that images a part of a subject's living body; An evaluation unit that determines whether a biological image captured by the imaging unit is suitable for biometric authentication; When it is determined that the biological image is not suitable for the biometric authentication, an estimation unit that estimates at least one factor for which it is determined that the biological image is not suitable for the biometric authentication; An output unit that outputs the estimated factor, and An apparatus for estimating an image degradation factor.

2. Based on the estimated factor, the output unit generates and outputs a guidance screen that supports elimination of the estimated factor. The apparatus for estimating an image degradation factor according to claim 1.

3. The imaging unit: When it is determined that the biological image is not suitable for the biometric authentication, after outputting the estimated factor, images a part of the living body again; The estimation unit: When it is determined that the biological image captured again is not suitable for the biometric authentication, determines whether the number of times of imaging a part of the living body by the imaging unit is equal to or more than a predetermined number of times; When it is determined that the number of times of imaging by the imaging unit is less than the predetermined number of times, estimates a factor for which it is determined that the biological image captured again is not suitable for the biometric authentication; The output unit: Outputs the estimated factor. The apparatus for estimating an image degradation factor according to claim 1.

4. The output unit: When it is determined that the number of times of imaging by the imaging unit is equal to or more than the predetermined number of times, generates and outputs a guidance screen that supports elimination of all factors that can be estimated by the estimation unit and for which it can be determined that the biological image is not suitable for the biometric authentication. The apparatus for estimating an image degradation factor according to claim 3.

5. The imaging unit: When it is determined that the biological image is not suitable for the biometric authentication, after outputting the estimated factor, images a part of the living body again; The evaluation unit: Calculates an evaluation value indicating the degree to which the biological image is suitable for the biometric authentication for each biological image; The output unit: Determines and outputs, as the biological image to be used for the biometric authentication, the biological image having the largest evaluation value among the plurality of biological images. The apparatus for estimating an image degradation factor according to claim 1.

6. The estimation unit: Among a plurality of factor candidates that can be estimated by the estimation unit and for which it can be determined that the biological image is not suitable for the biometric authentication, estimates the probability that each factor candidate is the factor for which the biological image is not suitable for the biometric authentication. estimating the factor determined that the biological image is not suitable for the biometric authentication based on the probability of each of the plurality of factor candidates; The apparatus for estimating an image degradation factor according to claim 1.

7. The imaging unit, when it is determined that the biological image is not suitable for the biometric authentication, after outputting the estimated factor, imaging a part of the living body again, The output unit, based on the probability of each of the plurality of factor candidates of each of the plurality of biological images, determining and outputting any one of the plurality of biological images as a biological image to be used for the biometric authentication; The apparatus for estimating an image degradation factor according to claim 6.

8. When it is determined that the biological image is not suitable for the biometric authentication, the estimation unit estimates the factor using a degradation estimation model that estimates the factor. The apparatus for estimating an image degradation factor according to claim 1.

9. The estimation unit, when it is determined that the biological image is suitable for the biometric authentication, omits the estimation of the factor, The output unit, determines and outputs a biological image determined to be suitable for the biometric authentication as a biological image to be used for the biometric authentication; The apparatus for estimating an image degradation factor according to claim 1.

10. An image degradation factor estimation method performed by at least one computer, comprising: imaging a part of a subject's living body; determining whether a biological image of the part of the subject's living body that has been imaged is suitable for biometric authentication; when it is determined that the biological image is not suitable for the biometric authentication, estimating and outputting at least one factor determined that the biological image is not suitable for the biometric authentication; Image degradation factor estimation method.

11. On at least one computer, a step of imaging a part of a subject's living body; a step of determining whether a biological image of the part of the subject's living body that has been imaged is suitable for biometric authentication; when it is determined that the biological image is not suitable for the biometric authentication, a step of estimating and outputting a factor determined that the biological image is not suitable for the biometric authentication, for Image degradation factor estimation program.

Images