Image deterioration factor estimation device, image deterioration factor estimation method, and image deterioration factor estimation program

Abstract

This image deterioration factor estimation device is provided with: an imaging unit that captures an image of a part of a body of a subject; an evaluation unit that determines whether or not a biometric image captured by the imaging unit is suitable for biometric authentication; an estimation unit that, when it is determined that the biometric image is not suitable for biometric authentication, estimates at least one factor for which it is determined that the biometric image is not suitable for biometric authentication; and an output unit that outputs the estimated factor.

Description

Apparatus for estimating factors of image degradation, method for estimating factors of image degradation, and program for estimating factors of image degradation

[0001] The present disclosure relates to an apparatus for estimating a cause of image degradation, a method for estimating a cause of image degradation, and a program for estimating a cause of image degradation.

[0002] Patent Document 1 discloses a biometric authentication device that includes a light source that irradiates a finger with light, an imaging unit that captures an image of the light that has passed through the finger, and an authentication unit that extracts a blood vessel pattern from the captured image and compares it with a pre-registered blood vessel pattern, with the optical axis of the light from the light source and the imaging axis of the imaging unit intersecting each other.

[0003] Japanese Patent Application Publication No. 2009-134727

[0004] Demand for biometric authentication has been increasing in recent years. However, conventional fingerprint authentication, such as the biometric authentication device disclosed in Patent Literature 1, generally employs fingerprint authentication in which a finger used for authentication is pressed against a glass surface or the like (so-called contact-type fingerprint authentication). Therefore, from the perspective of infectious disease prevention, public health, and the like, there is a demand for technology that can acquire fingerprint images or fingerprint information without contact (so-called contactless fingerprint authentication).

[0005] The present disclosure has been devised in consideration of the above-described conventional circumstances, and aims to provide an image degradation factor estimation device, an image degradation factor estimation method, and an image degradation factor estimation program that estimate the degradation factors of a biometric image captured in a non-contact state and assist in obtaining a biometric image suitable for authentication.

[0006] The present disclosure provides an image degradation factor estimation device including an imaging unit that captures an image of a part of a subject's body, an evaluation unit that determines whether the biometric image captured by the imaging unit is suitable for biometric authentication, an estimation unit that, if the biometric image is determined to be unsuitable for the biometric authentication, estimates at least one factor that determined that the biometric image is unsuitable for the biometric authentication, and an output unit that outputs the estimated factor.

[0007] The present disclosure also provides a method for estimating factors of image degradation, performed by at least one computer, which captures an image of a part of a subject's body, determines whether the biometric image of the captured part of the subject's body is suitable for biometric authentication, and, if it is determined that the biometric image is not suitable for the biometric authentication, estimates and outputs at least one factor that determined that the biometric image is not suitable for the biometric authentication.

[0008] The present disclosure also provides a program for estimating factors of image degradation, which causes at least one computer to execute the steps of capturing an image of a part of a subject's body, determining whether the biometric image capturing the part of the subject's body is suitable for biometric authentication, and, if it is determined that the biometric image is not suitable for the biometric authentication, estimating and outputting the factors that determined that the biometric image is not suitable for the biometric authentication.

[0009] According to the present disclosure, it is possible to estimate the cause of degradation of a biometric image captured in a non-contact state and assist in obtaining a biometric image suitable for authentication.

[0010] FIG. 1 is an explanatory diagram of an internal configuration example of a terminal device according to the first embodiment. FIG. 2 is a flowchart showing an example of an operation procedure of a terminal device according to the first embodiment. FIG. 3 is a diagram showing an example of a guide to prevent deterioration factors for the nth time. FIG. 4 is a diagram showing an example of a guide to prevent deterioration factors for the nth time. FIG. 5 is a diagram showing an example of a guide to prevent deterioration factors for the nth time. FIG. 6 is a diagram showing an example of a guide to prevent deterioration factors for the mth time. FIG. 7 is a diagram showing an example of a guide to prevent deterioration factors for the mth time.

[0011] (Background to the present disclosure) Conventionally, when registering a fingerprint for fingerprint authentication with a personal authentication device, the user must visit a facility (e.g., a store, an office, etc.) that uses fingerprint authentication to register the fingerprint, resulting in low usability. To solve this problem, there is a fingerprint registration method in which a fingerprint is registered in a contactless manner using a terminal device owned by the user. However, when capturing a fingerprint in a contactless manner, the user must capture an image of the user's fingertip held in the air at any position (angle) so that the feature values ​​of the user's fingerprint can be extracted.

[0012] When capturing fingerprints in a contactless state, depending on the imaging environment, such as temperature, humidity, lighting brightness, or lighting placement, the fingertip may become dry or part of the fingertip may become dark, and even if the fingertip is captured repeatedly, a biometric image suitable for fingerprint authentication may not be obtained. In such cases, it may be possible to capture a fingerprint image suitable for fingerprint authentication by moistening the fingertip or changing the position where the fingertip is held over the sensor, but since the user does not know the reason (factor) why the captured biometric image is not suitable for fingerprint authentication, it has been difficult to capture a fingerprint image suitable for fingerprint authentication.

[0013] Therefore, in each of the embodiments shown below, examples of an image degradation factor estimation device, an image degradation factor estimation method, and an image degradation factor estimation program will be described, which estimate the degradation factors of a biometric image captured in a non-contact state and assist in obtaining a biometric image suitable for authentication.

[0014] Hereinafter, with reference to the drawings as appropriate, detailed descriptions will be given of embodiments that specifically disclose the configurations and operations of an image degradation factor estimation device, an image degradation factor estimation method, and an image degradation factor estimation program according to the present disclosure. However, unnecessary detailed descriptions may be omitted. For example, detailed descriptions of well-known matters or redundant descriptions of substantially identical configurations may be omitted. This is to avoid unnecessary redundancy in the following description and to facilitate understanding by those skilled in the art. Note that the accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter recited in the claims.

[0015] In the following first and second embodiments, a method for supporting the acquisition of a fingerprint image used in fingerprint authentication will be described as an example of a biometric image, but the method is not limited to this. The biometric image acquisition method disclosed in the first and second embodiments may be applied and used to acquire, for example, a palm print image used in palm print authentication, a face image used in face authentication, etc.

[0016] (First embodiment) An example of the internal configuration of a terminal device P1 according to the first embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of the internal configuration of a terminal device P1 according to the first embodiment.

[0017] The terminal device P1 is realized by, for example, a smartphone, a tablet terminal, etc. The terminal device P1 can accept operations by a user and acquire a fingerprint image or fingerprint data of the user to be used for fingerprint authentication. The fingerprint data here is data indicating fingerprint feature quantities extracted using known techniques (e.g., minutiae method, frequency feature analysis method, etc.).

[0018] Note that, although an example will be described in which the terminal device P1 in the first embodiment is capable of registering an acquired fingerprint image or fingerprint data and performing fingerprint authentication based on matching the acquired fingerprint image or fingerprint data with pre-registered fingerprint data, the present invention is not limited to this. For example, the terminal device P1 may be a device that only acquires an acquired fingerprint image or fingerprint data, or may be a device that registers the acquired fingerprint image or fingerprint data.

[0019] The terminal device P1 includes a communication unit 10, a processor 11, a memory 12, a camera 13, a monitor 14, and a fingerprint database DB1. The fingerprint database DB1 may be an external recording medium or an external storage device connected to the terminal device P1 so as to enable data communication therebetween.

[0020] The communication unit 10 is connected to another device (not shown) via a network (not shown) so as to be able to communicate data with the other device (not shown). The communication unit 10 outputs various data acquired via the network (not shown) to the processor 11.

[0021] The processor 11 is configured using, for example, a Central Processing Unit (hereinafter referred to as "CPU") or a Field Programmable Gate Array (hereinafter referred to as "FPGA"), and performs various processes and controls in cooperation with the memory 12. Specifically, the processor 11 references the programs and data stored in the memory 12 and executes the programs to realize various functions such as capturing an image of a fingerprint and acquiring fingerprint data, or fingerprint authentication using the acquired fingerprint data.

[0022] The memory 12 includes, for example, a random access memory (hereinafter referred to as "RAM") as a work memory used when the processor 11 executes each process, and a read only memory (hereinafter referred to as "ROM") that stores programs and data that define the operation of the processor 11. The RAM temporarily stores data or information generated or acquired by the processor 11. The ROM stores programs that define the operation of the processor 11. The memory 12 stores a deterioration estimation model 12A.

[0023] The deterioration estimation model 12A is a trained model that has been trained to be able to estimate factors that deteriorate the quality of a fingerprint image, such as the inclination of a fingerprint in a fingerprint image, the dryness or wetness of the fingerprint portion, the brightness of the fingerprint, or the presence or absence of an obstruction that blocks the fingerprint. The deterioration estimation model 12A may be data that can be acquired or updated from any other device via the communication unit 10. Furthermore, the deterioration factors that can be estimated by the deterioration estimation model 12A are not limited to the examples described above, and may be added or deleted as desired.

[0024] Degradation estimation model 12A is used by processor 11 to estimate factors (causes) of degradation of the quality of a fingerprint image determined to be unsuitable for fingerprint authentication. Degradation estimation model 12A performs image analysis of the fingerprint image and estimates the probability that each degradation factor could be a factor in degrading the quality of the fingerprint image. If degradation estimation model 12A determines that the probability of each estimated degradation factor is equal to or greater than the probability that it is determined that the factor could be a factor in degrading the quality of the fingerprint image, it estimates that the factor is a factor in degrading the quality of the fingerprint image. Degradation estimation model 12A associates information about the calculated probability of each degradation factor, information about the estimated degradation factor results for the fingerprint image, and the fingerprint image, and stores them in memory 12.

[0025] The quality of the fingerprint image referred to here does not refer to the image quality of the fingerprint image itself, but rather to the quality required to obtain fingerprint data suitable for fingerprint authentication.

[0026] The camera 13 is configured to have at least a lens (not shown) and an image sensor (not shown). The image sensor is, for example, a solid-state imaging element such as a charged-coupled device (CCD) or a complementary metal oxide semiconductor (CMOS), and converts an optical image formed on an imaging surface into an electrical signal. The camera 13 starts imaging in response to a control instruction to start imaging input by the processor 11 or an imaging operation by the user. The camera 13 outputs the captured image to the processor 11.

[0027] The monitor 14 is configured using, for example, a liquid crystal display (LCD) or an organic electroluminescence (EL). The monitor 14 outputs and displays images captured by the camera 13 or various screens generated by the processor 11. The monitor 14 may also be a user interface configured with a touch panel. In such a case, the monitor 14 has a function as an operation unit 15, accepts input operations by the user, and outputs the results of the user's input operations to the processor 11.

[0028] The operation unit 15 can accept user operations and outputs the contents of input operations based on the user operations to the processor 11. The operation unit 15 may be realized as a touch panel of the monitor 14. The operation unit 15 may also include a microphone (not shown) and may accept voice input operations based on the user's voice.

[0029] The fingerprint database DB1 is a so-called storage device, and is configured using a storage medium such as a flash memory, a hard disk drive (hereinafter referred to as an "HDD"), or a solid state drive (hereinafter referred to as an "SSD"), etc. The fingerprint database DB1 stores (registers) and manages the fingerprint data output from the processor 11.

[0030] Next, a method for acquiring a fingerprint image executed by the terminal device P1 will be described with reference to Fig. 2. Fig. 2 is a flowchart showing an example of an operation procedure of the terminal device P1 according to the first embodiment.

[0031] The processor 11 starts an application capable of acquiring a fingerprint through a user operation. When the processor 11 receives an image capture start operation through a user operation, the processor 11 generates and outputs a control instruction to the camera 13 to start capturing an image (St11).

[0032] The camera 13 captures an image of the user's fingers held up towards the camera 13, and outputs the captured image to the processor 11 (St12).

[0033] The processor 11 detects at least one fingertip from the captured image, cuts out the area containing the detected fingertip, and generates a fingerprint image. The processor 11 calculates an evaluation value indicating the degree to which the generated fingerprint image is suitable for fingerprint authentication (St13), and determines whether the calculated evaluation value is equal to or greater than a threshold (St14). The threshold here is the evaluation value indicating that the fingerprint image is suitable for fingerprint authentication.

[0034] If the processor 11 determines in the processing of step St14 that the calculated evaluation value is greater than or equal to the threshold value (St14, YES), it acquires the generated fingerprint image as a fingerprint image to be used for acquiring or registering fingerprint data (St15).

[0035] On the other hand, if the processor 11 determines in the processing of step St14 that the calculated evaluation value is not equal to or greater than the threshold value (NO in St14), it determines whether the current number of times n (n: an integer equal to or greater than 1) of images captured is equal to or greater than the upper limit number m (m: an integer equal to or greater than 1) (St16). Note that the upper limit number m here refers to the upper limit number of times the fingertip can be re-captured. Furthermore, the upper limit number m is preferably 2 or greater, but may be set to 1 if re-capture is not permitted.

[0036] If the processor 11 determines in the processing of step St16 that the current number of times n of imaging is not equal to or greater than the upper limit number m (NO in St16), it uses the degradation estimation model 12A to estimate the factor of deterioration in the quality of the fingerprint image, that is, the factor that determined that the fingerprint image is not suitable for fingerprint authentication (St17). Note that in the present disclosure, the degradation factors that can be estimated by the degradation estimation model 12A are, for example, five degradation factors: "tilt," "dryness," "humidity," "brightness," and "obstruction." However, the degradation factors that can be estimated by the degradation estimation model 12A are not limited to the above-mentioned five, and may be changed, deleted, or added as desired.

[0037] Based on the result of the degradation factor estimation, the processor 11 generates a degradation factor estimation result screen SC21 (see FIG. 3) that notifies the user of the estimated degradation factor, and outputs and displays it on the monitor 14 (St18).

[0038] The processor 11 generates a user guide screen (e.g., the user guide screens SC30 and SC41 shown in FIG. 3 ) for resolving each of the estimated deterioration factors based on the estimated deterioration factors, and outputs and displays the generated user guide screen on the monitor 14 (St19). The processor 11 executes a deterioration factor prevention guide by displaying the user guide screen, and after resolving each of the deterioration factors, captures an image of the user's fingers again (St19). Note that the timing of capturing an image of the user's fingers may be any timing, and may be performed based on, for example, receipt of an image capture start operation by a user operation, or may be performed after the display of all user guide screens has been completed.

[0039] On the other hand, if the processor 11 determines in the processing of step St16 that the current number of times n of imaging is greater than or equal to the upper limit number m (St16, YES), it uses the degradation estimation model 12A to estimate the factors that have caused the quality of the fingerprint image to deteriorate, i.e., the factors that have caused the fingerprint image to be determined to be unsuitable for fingerprint authentication.

[0040] The processor 11 generates a user guide screen (e.g., user guide screens SC37, SC41 to SC45, etc., shown in FIG. 7 ) for eliminating all factors that may cause deterioration of the fingerprint image during the (m+1)th imaging, and outputs and displays the generated screen on the monitor 14 (St20). The processor 11 executes a guide to prevent deterioration factors by displaying the user guide screen, and after eliminating each of the deterioration factors, it images the user's fingers again (St20). Note that the timing of capturing an image of the user's fingers may be any timing, and may be performed based on, for example, the acceptance of an imaging start operation by a user operation, or may be performed after the display of all user guide screens has been completed.

[0041] The processor 11 detects at least one fingertip from the re-captured image, cuts out an area showing the detected fingertip, and generates a fingerprint image. The processor 11 calculates an evaluation value indicating the degree to which the generated fingerprint image is suitable for fingerprint authentication (St21). The processor 11 also uses the degradation estimation model 12A to estimate factors that cause degradation in the quality of the fingerprint image generated using the re-captured image (St21).

[0042] The processor 11 determines and acquires the fingerprint images to be used for acquiring or registering fingerprint data based on the evaluation values ​​or degradation factors of each of the (m+1) fingerprint images acquired by capturing (m+1) images (St22).

[0043] Here, processor 11 may determine the fingerprint image with the highest evaluation value among the (m+1) fingerprint images as the fingerprint image to be used for fingerprint data acquisition or registration. Furthermore, processor 11 may determine the fingerprint image with the smallest average evaluation value, the smallest maximum evaluation value, or the smallest number of degradation factors estimated to be degradation factors (i.e., the number of degradation factors whose probability of being a degradation factor is equal to or greater than a threshold for estimating degradation factors) as the fingerprint image to be used for fingerprint data acquisition or registration, based on the estimated evaluation values ​​or probabilities of degradation factors. Processor 11 extracts and acquires fingerprint data from the determined fingerprint image, and performs registration of the acquired fingerprint data or fingerprint authentication using the acquired fingerprint data.

[0044] As described above, the terminal device P1 according to embodiment 1 can more effectively obtain a fingerprint image that is more suitable for fingerprint authentication by estimating the factors that cause deterioration of the fingerprint image and, based on the estimated factors, executing a user guide (i.e., a guide to prevent deterioration factors) that prevents the factors.

[0045] Furthermore, the process of estimating the cause of deterioration using the deterioration estimation model 12A (i.e., the trained model) is generally complex, resulting in a large processing load on the processor 11 and high power consumption. Therefore, the terminal device P1 according to the first embodiment evaluates whether the fingerprint image is suitable for fingerprint authentication (step St14), and estimates the cause of deterioration of the fingerprint image only when it is determined that the fingerprint image is not suitable for fingerprint authentication. This reduces the processing load on the processor 11 in acquiring fingerprint data, and can suppress heat generation or power consumption of the terminal device P1.

[0046] Next, an example of the guide for preventing the cause of deterioration for the nth time will be described with reference to Fig. 3. Fig. 3 is a diagram showing an example of the guide for preventing the cause of deterioration for the nth time. It goes without saying that the various screen examples shown in Fig. 3 are merely examples and are not limited to these.

[0047] The processor 11 estimates the probability of each deterioration factor based on the fingerprint image FG11. While the processor 11 is estimating the probability of each deterioration factor, it displays a deterioration factor estimation screen SC11 on the monitor 14. The deterioration factor estimation screen SC11 is generated to include the fingerprint image FG11, which is the target of degradation factor estimation.

[0048] The processor 11 estimates the deterioration factor of the fingerprint image FG11 based on the estimated probability of each deterioration factor. When the processor 11 has completed estimating the probability of each deterioration factor, it generates a deterioration factor probability screen SC12 and outputs it to the monitor 14. The deterioration factor probability screen SC12 is generated to include information on the probability of each deterioration factor. Note that displaying the deterioration factor probability screen SC12 is not essential and may be omitted.

[0049] Furthermore, when the estimation of the deterioration factors is completed, the processor 11 generates a deterioration factor estimation result screen SC21 and outputs it to the monitor 14. The deterioration factor estimation result screen SC21 is generated to include information RS21 relating to the estimation result of the deterioration factors.

[0050] 3 , processor 11 estimates that the probability of the degradation factor being the tilt of the finger (fingerprint) is "90 / 100," the probability of the degradation factor being the dry finger (fingerprint) is "3 / 100," the probability of the degradation factor being the wet finger (fingerprint) is "4 / 100," the probability of the degradation factor being the brightness of the finger (fingerprint) is "87 / 100," and the probability of the degradation factor being the obstruction blocking the finger (fingerprint) is "0 / 100." In such a case, processor 11 estimates that the degradation factors "tilt" and "brightness," whose estimated probabilities are equal to or greater than a threshold, are the degradation factors of fingerprint image FG11. Processor 11 generates a degradation factor estimation result screen SC21 notifying that the degradation factors of fingerprint image FG11 are "tilt" and "brightness."

[0051] The threshold for estimating the degradation factor may be set to a different value for each degradation factor. For example, the probability that the degradation factor "slope" is estimated as a degradation factor of a fingerprint image may be the same as or different from the probability that the degradation factor "brightness" is estimated as a degradation factor of a fingerprint image. Furthermore, the threshold for estimating the degradation factor may be arbitrarily changeable based on the environment (e.g., time of day, season, etc.) in which the fingerprint image is acquired.

[0052] The processor 11 generates a user guide screen SC30 requesting a re-imaging of the fingerprint and outputs it to the monitor 14. The user guide screen SC30 is generated including a message requesting a re-imaging of the fingerprint. For example, in the example shown in FIG. 3, the user guide screen SC30 indicates that the factors of deterioration of the fingerprint image FG11 are "tilt" and "brightness," and includes a message Msg30 requesting a re-imaging of the fingerprint, stating "The tilt and brightness of the finger are poor, so please re-imaging the fingerprint."

[0053] The processor 11 generates user guide screens SC41 and SC42 for supporting the elimination of all degradation factors estimated to be degradation factors, and outputs them to the monitor 14. The user guide screens SC41 and SC42 are generated including support information for eliminating the corresponding degradation factors.

[0054] 3 , when the estimated degradation factors are "tilt" and "brightness," the processor 11 generates a user guide screen SC41 for assisting in eliminating the degradation factor "brightness" and a user guide screen SC42 for assisting in eliminating the degradation factor "tilt." The user guide screen SC41 includes, as support information for assisting in eliminating the degradation factor "brightness," a guide message Msg41 "Take the image in a bright environment" and a guide image GD41 notifying the user that the brightness of the current fingerprint image FG11 is low. Furthermore, the user guide screen SC42 includes, as support information for assisting in eliminating the degradation factor "tilt," a guide message Msg42 "Tilt your finger slightly to the right" and a guide image GD42 indicating that the finger captured in the current fingerprint image FG11 is tilted to the left.

[0055] The user guide screen SC41 and the user guide screen SC42 may be generated on a single screen. The processor 11 also executes a process of switching between the user guide screen SC41 and the user guide screen SC42 based on a user operation, and after executing all guidance for the user using the user guide screens SC41 and SC42, re-images the user's fingers.

[0056] Next, other examples of user guide screens will be described with reference to Figures 4 to 6. Figure 4 is a diagram showing an example of a guide for preventing deterioration causes. Figure 5 is a diagram showing an example of a guide for preventing deterioration causes. Figure 6 is a diagram showing an example of a guide for preventing deterioration causes. It goes without saying that the various screen examples shown in Figures 4 to 6 are merely examples and are not limited to these.

[0057] When the estimated result of the deterioration cause is "dryness", the processor 11 generates user guide screens SC33 and SC43 for assisting in resolving the deterioration cause "dryness" and outputs them to the monitor 14.

[0058] 4, the user guide screen SC33 is generated by including a guide message Msg33 "Your fingers are dry. Please apply hand cream or water and retake the image" as support information for assisting in resolving the deterioration factor "dryness." The user guide screen SC43 also includes a guide message Msg43 "Please apply hand cream or water and retake the image" and a guide image GD43 indicating that the fingertip in the current fingerprint image FG11 is dry, as support information for assisting in resolving the deterioration factor "dryness." The symbol "・" in the above-mentioned guide messages Msg33 and Msg43 is used to mean "or."

[0059] When the estimated result of the deterioration cause is "moisture", the processor 11 generates user guide screens SC34 and SC44 for assisting in resolving the deterioration cause "moisture" and outputs them to the monitor 14.

[0060] 5, the user guide screen SC34 is generated by including a guide message Msg34 "Your finger is wet, please wipe off the water and take another image" as support information for supporting the resolution of the deterioration factor "wetness." The user guide screen SC44 also includes a guide message Msg44 "Please wipe off the water and take another image" and a guide image GD44 indicating that the fingertip in the current fingerprint image FG11 is wet, as support information for supporting the resolution of the deterioration factor "wetness."

[0061] When the estimated result of the deterioration cause is "obstruction," the processor 11 generates user guide screens SC35 and SC45 for assisting in eliminating the deterioration cause "obstruction," and outputs them to the monitor 14. Note that "obstruction" here refers to a state in which a fingerprint on a fingertip is obstructed by dust, dirt, an object, or the like, making it impossible to acquire the fingerprint.

[0062] 6, the user guide screen SC35 is generated by including a guide message Msg35 "There is an obstruction, please clean your finger and take another image" as support information for assisting in eliminating the deterioration factor "obstruction." The user guide screen SC45 also includes, as support information for assisting in eliminating the deterioration factor "obstruction," a guide message Msg45 "Please clean your finger and take another image" and a guide image GD45 indicating that the fingerprint in the current fingerprint image FG11 is obstructed (hidden) by an obstruction GM.

[0063] As described above, the terminal device P1 can more effectively acquire a fingerprint image that is more suitable for fingerprint authentication by generating and displaying a user guide screen that corresponds to the cause of deterioration of the fingerprint image.

[0064] Next, an example of the (m+1)th deterioration cause prevention guide will be described with reference to Fig. 7. Fig. 7 is a diagram showing an example of the (m+1)th deterioration cause prevention guide. It goes without saying that the various screen examples shown in Fig. 7 are merely examples and are not limited to these.

[0065] When processor 11 determines that the current number of imaging attempts n is equal to or greater than the upper limit number of retakes m, it executes a degradation factor prevention guide for eliminating all degradation factors that could be factors that could cause degradation of the fingerprint image and executing the (m+1)th imaging. Specifically, processor 11 generates and displays user guide screens SC41 to SC45 corresponding to all degradation factors before executing the (m+1)th imaging, in order to eliminate all degradation factors that could cause degradation of the fingerprint image, regardless of the degradation factors estimated from fingerprint images captured up to that point.

[0066] For example, if there are a total of five possible degradation factors that can cause a fingerprint image to deteriorate ("tilt," "dryness," "wetness," "brightness," and "obstruction"), user guide screens SC41 to SC45 for resolving each of these five degradation factors are generated and output to the monitor 14. Note that the guidance for resolving each of the five degradation factors may be executed collectively on one or more user guide screens.

[0067] After displaying the user guide screens SC41 to SC45, if the processor 11 determines that the cause of deterioration caused by the user has been resolved based on the user's operation, it executes the (m+1)th imaging.

[0068] As a result, when the terminal device P1 determines that the current number of times n images have been taken is equal to or greater than the preset upper limit number m of retakes, it generates and displays a user guide screen that addresses all possible factors that could cause deterioration of the fingerprint image, thereby enabling the terminal device P1 to more effectively obtain a fingerprint image that is more suitable for fingerprint authentication.

[0069] (Embodiment 2) An example has been described in which the terminal device P1 according to embodiment 1 captures a fingerprint, determines whether the fingerprint image is suitable for fingerprint authentication, estimates factors that cause deterioration of the fingerprint image, and provides guidance to prevent deterioration factors based on the results of the estimation of the deterioration factors. In the biometric information acquisition system 100 according to embodiment 2, an example will be described in which the terminal device P1A captures a fingerprint and provides guidance to prevent deterioration factors based on the results of the estimation of the deterioration factors, and the server S1 determines whether the fingerprint image is suitable for fingerprint authentication and estimates factors that cause deterioration of the fingerprint image.

[0070] The terminal device P1A of the biometric information acquisition system 100 according to the second embodiment has a configuration that is partially similar to that of the terminal device P1 according to the first embodiment. Therefore, in the following description of the biometric information acquisition system 100 according to the second embodiment, the same components as those of the terminal device P1 according to the first embodiment are denoted by the same reference numerals, and the description thereof will be omitted.

[0071] An example of the configuration of a biometric information acquisition system 100 according to the second embodiment will be described with reference to Fig. 8. Fig. 8 is a diagram showing an example of the internal configuration of a terminal device P1A and a server S1 according to the second embodiment.

[0072] The terminal device P1A captures an image of the user's fingertip and provides a guide to prevent deterioration factors based on the deterioration estimation result. The terminal device P1A includes a communication unit 10A, a camera 13, a monitor 14, and an operation unit 15. Note that the camera 13, the monitor 14, and the operation unit 15 of the terminal device P1A may be separate devices, or may be configured as multiple devices.

[0073] The communication unit 10A is connected to the communication unit 20 of the server S1 via a network so as to be able to communicate data with it. The communication unit 10A may be connected to the communication unit 20 of the server S1 so as to be able to communicate wirelessly or via a wire. The wireless communication here refers to, for example, short-range wireless communication such as Bluetooth (registered trademark) or NFC (registered trademark), or communication via a wireless LAN (Local Area Network) such as Wi-Fi (registered trademark).

[0074] The communication unit 10A transmits to the server S1 various data output from the processor 11. The communication unit 10A also outputs to the processor 11 various data transmitted from the server S1.

[0075] The processor 11A transmits to the server S1 a captured image captured by the camera 13 or a fingerprint image obtained by cutting out an area in which a fingerprint appears from the captured image. The captured image or fingerprint image to be transmitted may be encrypted.

[0076] In addition, based on the information on the estimated results of the deterioration factors of the fingerprint image sent from the server S1, the processor 11A generates a user guide screen (see Figures 3 to 7) corresponding to the estimated results of the deterioration factors, outputs it to the monitor 14, and executes a guide to prevent deterioration factors.

[0077] The server S1 includes a communication unit 20, a processor 21, a memory 22, and a fingerprint database DB2. Note that the fingerprint database DB2 may be an external recording medium or an external storage device connected to the server S1 so as to be able to communicate data with the server S1.

[0078] The communication unit 20 is connected to the terminal device P1A via a network so as to be able to communicate data with the terminal device P1A. The communication unit 20 outputs various data acquired via the network to the processor 21.

[0079] The processor 21 is configured using, for example, a CPU or FPGA, and performs various processes and controls in cooperation with the memory 22. Specifically, the processor 21 references the programs and data stored in the memory 22 and executes the programs to realize various functions such as estimating the cause of deterioration of a fingerprint image using the deterioration estimation model 12A, registering fingerprint data, or performing fingerprint authentication using fingerprint data.

[0080] The memory 22 includes, for example, a RAM as a work memory used when the processor 21 executes each process, and a ROM for storing programs and data that define the operation of the processor 21. The RAM temporarily stores data or information generated or acquired by the processor 21. The ROM stores programs that define the operation of the processor 21. The memory 22 stores the deterioration estimation model 12A.

[0081] Next, with reference to Fig. 9, a description will be given of the correspondence between the operation procedure shown in Fig. 2 and the operation procedure executed by the biometric information acquisition system 100 according to embodiment 2. Fig. 9 is a table showing the correspondence between the processes of the terminal device P1A and the server S1 according to embodiment 2 and each process shown in Fig. 2.

[0082] The terminal device P1A according to the second embodiment executes the processes of steps St11 to St12 and steps St18 to St20 of the operation procedure (steps St11 to St22) shown in FIG.

[0083] Furthermore, the server S1 according to the second embodiment executes the processes of steps St13 to St17 and steps St21 to St22 of the operation procedure (steps St11 to St22) shown in FIG.

[0084] As described above, the biometric information acquisition system 100 according to the second embodiment can execute the operation procedure shown in FIG. 2 using the terminal device P1A and the server S1.

[0085] (Additional Notes) The above description of each embodiment discloses the following techniques.

[0086] (Technology 1) An image degradation factor estimation device (terminal device P1) comprising: an imaging unit (camera 13) that captures an image of a part of a subject's (user's) biometric body (for example, a fingertip, etc.); an evaluation unit (processor 11) that determines whether the biometric image captured by the imaging unit (camera 13) is suitable for biometric authentication; an estimation unit (processor 11) that, if the biometric image is determined to be unsuitable for the biometric authentication, estimates at least one factor that determined the biometric image to be unsuitable for the biometric authentication; and an output unit (monitor 14) that outputs the estimated factor. With this configuration, the image degradation factor estimation device (terminal device P1) outputs the degradation factor of a fingerprint image to support the user in improving the degradation factor, thereby making it possible to more effectively acquire a biometric image that is more suitable for fingerprint authentication.

[0087] (Technology 2) The image degradation factor estimation device (terminal device P1) described in (Technology 1) is configured such that the output unit (monitor 14) generates and outputs a guide screen (see FIGS. 3 to 6) that supports the user in eliminating the estimated factor based on the estimated factor. With this configuration, the image degradation factor estimation device (terminal device P1) generates and displays a user guide screen (see FIGS. 3 to 6) that corresponds to the degradation factor of the fingerprint image, thereby supporting the user in improving the degradation factor and more effectively acquiring a biometric image that is more suitable for fingerprint authentication.

[0088] (Technology 3) The image degradation factor estimation device (terminal device P1) according to (Technology 1) or (Technology 2), wherein the imaging unit (camera 13), when it is determined that the biometric image is not suitable for the biometric authentication, re-images a part of the biometric image after outputting the estimated factor; when it is determined that the re-imaged biometric image is not suitable for the biometric authentication, the estimation unit (processor 11) determines whether the number of times n that the imaging unit (camera 13) has imaged the part of the biometric image is equal to or greater than a predetermined number (upper limit number m of re-images); when it is determined that the number of times n that the imaging unit (camera 13) has imaged is not equal to or greater than the predetermined number (upper limit number m of re-images), the estimation unit estimates a factor that determined that the re-imaged biometric image is not suitable for the biometric authentication; and the output unit (monitor 14) outputs the estimated factor. (Technology 1) With this configuration, the image degradation factor estimation device (terminal device P1) generates and displays a user guide screen corresponding to the degradation factors of the fingerprint image, thereby assisting the user in improving the degradation factors and more effectively obtaining biometric images that are more suitable for fingerprint authentication.

[0089] (Technology 4) The image degradation factor estimation device (terminal device P1) according to any one of (Technology 1) to (Technology 3), wherein the output unit (monitor 14) generates and outputs a guide screen (see FIG. 7) that supports the elimination of all factors that the estimation unit (processor 11) can estimate and that may cause the biometric image to be determined to be unsuitable for biometric authentication when it is determined that the current number of images captured n by the imaging unit (camera 13) is equal to or greater than the predetermined number (maximum number of retakes m). With this configuration, when it is determined that the current number of images captured n is equal to or greater than the predetermined maximum number of retakes m, the image degradation factor estimation device (terminal device P1) generates and displays a user guide screen (see FIG. 7) that corresponds to all degradation factors that may cause the fingerprint image to be degraded, thereby making it possible to more effectively acquire a fingerprint image that is more suitable for fingerprint authentication.

[0090] (Technology 5) The image degradation factor estimation device (terminal device P1) according to any one of (Technology 1) to (Technology 4), wherein if the biometric image is determined to be unsuitable for the biometric authentication, the imaging unit (camera 13) re-images a part of the biometric image after outputting the estimated factor, the evaluation unit (processor 11) calculates an evaluation value indicating the degree to which the biometric image is suitable for the biometric authentication for each of the biometric images, and the output unit (monitor 14) determines and outputs the biometric image with the largest evaluation value among the plurality of biometric images as the biometric image to be used for the biometric authentication. With this configuration, the image degradation factor estimation device (terminal device P1) can acquire a fingerprint image that is more suitable for fingerprint authentication.

[0091] (Technology 6) The estimation unit (processor 11) estimates the probability that each of a plurality of candidate factors that the estimation unit (processor 11) can estimate and that may cause the biometric image to be determined unsuitable for the biometric authentication (for example, degradation factors "tilt," "dryness," "wetness," "brightness," and "obstruction" in the present disclosure) is the factor that causes the biometric image to be unsuitable for the biometric authentication, and estimates the factor that caused the biometric image to be determined unsuitable for the biometric authentication based on the probability of each of the plurality of candidate factors. With this configuration, the image degradation factor estimation device (terminal device P1) can estimate degradation factors that are highly likely to contribute to a fingerprint image being unsuitable for fingerprint authentication.

[0092] (Technology 7) The image degradation factor estimation device (terminal device P1) according to (Technology 6), wherein the imaging unit (camera 13) re-images a part of the biometric body after outputting the estimated factor if the biometric image is determined to be unsuitable for the biometric authentication, and the output unit (monitor 14) determines and outputs one of the biometric images to be the biometric image to be used for the biometric authentication based on the probability of each of the multiple candidate factors (for example, degradation factors "tilt," "dryness," "wetness," "brightness," and "obstruction" in the present disclosure) for each of the multiple biometric images. With this configuration, the image degradation factor estimation device (terminal device P1) can acquire a fingerprint image more suitable for fingerprint authentication based on the probability of each candidate factor that may be a factor unsuitable for fingerprint authentication.

[0093] (Technology 8) The image degradation factor estimation device (terminal device P1) according to any one of (Technology 1) to (Technology 7), wherein, when it is determined that the biometric image is not suitable for the biometric authentication, the estimation unit (processor 11) estimates the factor using a degradation estimation model that estimates the factor. With this configuration, the image degradation factor estimation device (terminal device P1) executes the degradation factor estimation process using the degradation estimation model 12A (i.e., a trained model), which is generally a complex process that places a large processing load on the processor 11 and consumes a lot of power, only when it is determined that the fingerprint image is not suitable for fingerprint authentication, thereby reducing the processing load on the processor 11 in acquiring fingerprint data and suppressing heat generation or power consumption of the terminal device P1.

[0094] (Technology 9) The device for estimating a cause of image degradation (terminal device P1) according to any one of (Technology 1) to (Technology 8), wherein the estimation unit (processor 11) omits estimating the cause when it is determined that the biometric image is suitable for the biometric authentication, and the output unit (monitor 14) determines the biometric image determined to be suitable for the biometric authentication as the biometric image to be used for the biometric authentication and outputs it. With this configuration, the device for estimating a cause of image degradation (terminal device P1) can acquire a fingerprint image that is more suitable for fingerprint authentication.

[0095] (Technology 10) A method for estimating factors of image degradation performed by at least one computer (terminal device P1), comprising: capturing an image of a portion of a subject's (user's) biological body; determining whether a biometric image of the captured portion of the subject's (user's) biological body is suitable for biometric authentication; and, if the biometric image is determined to be unsuitable for the biometric authentication, estimating and outputting at least one factor that determined the biometric image to be unsuitable for the biometric authentication. With this configuration, the computer (terminal device P1) outputs the factors of fingerprint image degradation to support the user in improving the factors of degradation, thereby more effectively obtaining a biometric image that is more suitable for fingerprint authentication. Note that the above-described method for estimating factors of image degradation may be executed by multiple computers (terminal device P1A and server S1).

[0096] (Technology 11) An image degradation factor estimation program that causes at least one computer (terminal device P1) to execute the following steps: capturing an image of a portion of a subject's (user's) biological body; determining whether the biometric image capturing the portion of the subject's (user's) biological body is suitable for biometric authentication; and, if the biometric image is determined to be unsuitable for the biometric authentication, estimating and outputting the factors that determined the biometric image to be unsuitable for the biometric authentication. With this configuration, the computer (terminal device P1) outputs the factors that cause fingerprint image degradation, thereby supporting the user in improving the factors, and enabling a biometric image that is more suitable for fingerprint authentication to be more effectively acquired. Note that the above-mentioned image degradation factor estimation program may be executed by multiple computers (terminal device P1A and server S1).

[0097] Although various embodiments have been described above with reference to the accompanying drawings, the present disclosure is not limited to such examples. It is clear that those skilled in the art can conceive of various modifications, alterations, substitutions, additions, deletions, and equivalents within the scope of the claims, and it is understood that these also fall within the technical scope of the present disclosure. Furthermore, the components of the various embodiments described above may be combined in any manner without departing from the spirit of the invention.

[0098] This application is based on a Japanese patent application (Patent Application No. 2023-212151) filed on December 15, 2023, the contents of which are incorporated herein by reference.

[0099] The present disclosure is useful as presenting an image degradation factor estimation device, an image degradation factor estimation method, and an image degradation factor estimation program that estimate the degradation factors of a biometric image captured in a non-contact state and assist in obtaining a biometric image suitable for authentication.

[0100] 10, 10A, 20 Communication unit 11, 11A, 21 Processor 12, 22 Memory 12A Deterioration estimation model 13 Camera 14 Monitor 15 Operation unit 100 Biometric information acquisition system DB1, DB2 Fingerprint database FG11 Fingerprint image GD41, GD42, GD43, GD44, GD45 Guide image P1, P1A Terminal device S1 Server SC11 Deterioration factor estimation screen SC12 Deterioration factor probability screen SC21 Deterioration factor estimation result screen SC30, SC33, SC34, SC35, SC37, SC41, SC42, SC43, SC44, SC45 User guide screen

Claims

1. An image degradation factor estimation device comprising: an imaging unit that captures an image of a part of a subject's body; an evaluation unit that determines whether or not the biometric image captured by the imaging unit is suitable for biometric authentication; an estimation unit that, if it is determined that the biometric image is not suitable for the biometric authentication, estimates at least one factor that determined that the biometric image is not suitable for the biometric authentication; and an output unit that outputs the estimated factor.

2. The image degradation cause estimating device according to claim 1, wherein said output unit generates and outputs a guide screen for assisting in eliminating said estimated cause based on said estimated cause.

3. The device for estimating causes of image degradation as described in claim 1, wherein the imaging unit, when it is determined that the biometric image is not suitable for the biometric authentication, re-images a part of the biometric image after outputting the estimated factor; when it is determined that the re-imaged biometric image is not suitable for the biometric authentication, the estimation unit, when it is determined that the re-imaged biometric image is not suitable for the biometric authentication, determines whether the number of times that the imaging unit has imaged a part of the biometric image is a predetermined number or more, and when it is determined that the number of times that the imaging unit has imaged is not the predetermined number or more, estimates the factor for which the re-imaged biometric image is determined to be unsuitable for the biometric authentication; and the output unit outputs the estimated factor.

4. The image degradation cause estimation device of claim 3, wherein the output unit, when it is determined that the number of times the imaging unit has captured images is equal to or greater than the predetermined number of times, generates and outputs a guide screen that assists in eliminating all factors that the estimation unit can estimate and that may cause the biometric image to be determined to be unsuitable for biometric authentication.

5. The device for estimating causes of image degradation as described in claim 1, wherein when the biometric image is determined to be unsuitable for the biometric authentication, the imaging unit re-images a portion of the biometric image after outputting the estimated cause, the evaluation unit calculates an evaluation value for each of the biometric images indicating the degree to which the biometric image is suitable for the biometric authentication, and the output unit determines and outputs the biometric image with the largest evaluation value among the multiple biometric images as the biometric image to be used for the biometric authentication.

6. The image degradation factor estimation device of claim 1, wherein the estimation unit estimates the probability that each of a plurality of candidate factors that the estimation unit can estimate and that may cause the biometric image to be determined to be unsuitable for the biometric authentication is a factor that makes the biometric image unsuitable for the biometric authentication, and estimates the factor that caused the biometric image to be determined to be unsuitable for the biometric authentication based on the respective probabilities of the plurality of candidate factors.

7. The device for estimating causes of image degradation as described in claim 6, wherein the imaging unit, when it is determined that the biometric image is not suitable for the biometric authentication, re-images a part of the biometric image after outputting the estimated cause, and the output unit determines and outputs one of the multiple biometric images as the biometric image to be used for the biometric authentication based on the probability of each of the multiple candidate causes for the multiple biometric images.

8. The image degradation cause estimation device according to claim 1, wherein, when the biometric image is determined to be unsuitable for the biometric authentication, the estimation unit estimates the cause using a degradation estimation model that estimates the cause.

9. The image degradation cause estimation device of claim 1, wherein the estimation unit omits estimating the cause when the biometric image is determined to be suitable for the biometric authentication, and the output unit determines the biometric image determined to be suitable for the biometric authentication as the biometric image to be used for the biometric authentication and outputs it.

10. A method for estimating factors of image degradation performed by at least one computer, comprising: capturing an image of a part of a subject's body; determining whether the biometric image of the captured part of the subject's body is suitable for biometric authentication; and, if it is determined that the biometric image is not suitable for the biometric authentication, estimating and outputting at least one factor that determined that the biometric image is not suitable for the biometric authentication.

11. A program for estimating factors of image degradation, which causes at least one computer to execute the steps of: capturing an image of a part of a subject's body; determining whether the biometric image capturing the part of the subject's body is suitable for biometric authentication; and, if it is determined that the biometric image is not suitable for the biometric authentication, estimating and outputting the factors that determined that the biometric image is not suitable for the biometric authentication.

Images