Biometric authentication device and biometric authentication method

Abstract

A biometric authentication device according to the present invention, which performs biometric authentication based on a blood vessel pattern at a predetermined authentication region of a human body, is provided with: an image acquisition unit that acquires an image obtained by photographing the authentication region; a first feature extraction unit that extracts a first feature representing a structural feature of the blood vessel pattern from the image acquired by the image acquisition unit; a second feature extraction unit that extracts a second feature representing a local feature of the blood vessel pattern on the basis of the first feature extracted by the first feature extraction unit; a feedback mechanism for feeding back the second feature extracted by the second feature extraction unit; and a classification unit that classifies the blood vessel pattern by using a computation result obtained by inputting information based on the first feature and the second feature fed back by the feedback mechanism to the second feature extraction unit. The biometric authentication is performed on the basis of the result of classification of the blood vessel pattern by the classification unit.

Description

Biometric authentication device and biometric authentication method

[0001] The present invention relates to an apparatus and method for biometric authentication using blood vessel patterns.

[0002] Conventionally, a technology for biometric authentication of an individual has been realized by capturing an image of a finger or palm as an authentication target body part, and recognizing a blood vessel (vein) pattern in the authentication target body part contained in the captured image. For example, Patent Literature 1 describes a finger vein identification method that uses an image of a region of interest detected from a finger image as an input image, obtains a difference image between the input image and a pre-registered image, and inputs the obtained difference image into a trained convolutional neural network (CNN) to obtain a result, thereby identifying finger veins.

[0003] Chinese Patent Application Publication No. 110738174

[0004] In biometric authentication, when photographing a finger or palm, which is a body part to be authenticated, with a camera, if excessive pressure is applied to the body part to be authenticated or if the position or orientation of the body part to be authenticated relative to the camera is misaligned, the captured image may include an area where blood vessels are not clearly visible and have disappeared (hereinafter referred to as a "missing area"), even though blood vessels are actually present. When such a missing area exists in the captured image, it becomes difficult to accurately identify the blood vessel pattern with conventional methods such as those described in Patent Document 1, and there is a risk that a correct biometric authentication result will not be obtained.

[0005] The present invention has been made against this background, and aims to improve authentication accuracy in biometric authentication using the blood vessel patterns of the human body when a missing area exists in an image of the part to be authenticated.

[0006] A biometric authentication device according to the present invention is a device that performs biometric authentication using a vascular pattern at a predetermined part of the human body to be authenticated, and includes: an image acquisition unit that acquires an image of the part to be authenticated; a first feature extraction unit that extracts a first feature representing a structural feature of the vascular pattern from the image acquired by the image acquisition unit; a second feature extraction unit that extracts a second feature representing a local feature of the vascular pattern based on the first feature extracted by the first feature extraction unit; a feedback mechanism that feeds back the second feature extracted by the second feature extraction unit; and a classification unit that classifies the vascular pattern using a calculation result obtained by inputting information based on the first feature and the second feature fed back by the feedback mechanism to the second feature extraction unit, and performs biometric authentication based on the classification result of the vascular pattern by the classification unit. A biometric authentication method according to the present invention is a method for performing biometric authentication using a vascular pattern in a predetermined part of the human body to be authenticated, comprising the steps of: acquiring an image of the part to be authenticated; extracting a first feature from the image that represents a structural feature of the vascular pattern; extracting a second feature that represents a local feature of the vascular pattern based on the first feature; feeding back the second feature; extracting a third feature that represents a comprehensive feature of the vascular pattern based on the first feature and the fed-back second feature; classifying the vascular pattern based on the third feature; and performing the biometric authentication based on the vascular pattern classification result.

[0007] According to the present invention, in biometric authentication using the blood vessel pattern of the human body, it is possible to improve authentication accuracy when a missing region exists in an image of a body part to be authenticated.

[0008] 1 is a configuration diagram of a biometric authentication device according to an embodiment of the present invention; a diagram showing an example of the hardware configuration of a calculation processing unit; a flowchart showing the flow of processing executed in a biometric authentication device according to an embodiment of the present invention; a flowchart showing the flow of learning processing; a flowchart showing the flow of authentication processing; an explanatory diagram of deep feature adjustment processing; an explanatory diagram of related feature generation processing; a diagram showing an example of the functional configuration of a related feature generation unit; a diagram showing examples of deep feature amounts; a diagram showing examples of shallow feature amounts; a diagram showing examples of related feature amounts.

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0010] Fig. 1 is a configuration diagram of a biometric authentication device according to one embodiment of the present invention. The biometric authentication device 1 shown in Fig. 1 is a device that identifies the blood vessel pattern of a human finger or palm as a body part to be authenticated from an image obtained by photographing the body part to be authenticated, and performs biometric authentication based on the blood vessel pattern. The biometric authentication device 1 is configured with an image acquisition unit 100, an arithmetic processing unit 200, and a learning control unit 300.

[0011] The image acquisition unit 100 acquires an image in which the vein pattern of the authentication part can be identified by capturing an image of the authentication part of the person to be authenticated. For example, an image including the vein pattern of the finger or palm can be acquired by irradiating near-infrared light onto the authentication part (finger or palm) and capturing the transmitted light or reflected light. Note that the image acquisition unit 100 may acquire an image by any other method as long as the vein pattern of the authentication part can be identified.

[0012] The arithmetic processing unit 200 identifies the blood vessel pattern in the area to be authenticated by performing a predetermined arithmetic processing on the image acquired by the image acquisition unit 100, and specifies the person to be authenticated from the identification result to perform biometric authentication. The arithmetic processing unit 200 has functional blocks of a shallow layer feature extraction unit 201, a deep layer feature extraction unit 202, a memory unit 203, a feature alignment unit 204, a related feature generation unit 205, an addition unit 206, a classification unit 207, and an authentication unit 208.

[0013] The shallow feature extraction unit 201 and the deep feature extraction unit 202 are each configured using a trained CNN. The shallow feature extraction unit 201 is configured by one or more layers on the lower side (input side) of the multiple layers that make up the CNN. On the other hand, the deep feature extraction unit 202 is configured by one or more layers on the upper side (output side) of the multiple layers that make up the CNN. Note that the CNN may have layers other than the shallow feature extraction unit 201 and the deep feature extraction unit 202.

[0014] The shallow layer feature extraction unit 201 receives a photographed image of the area to be authenticated acquired by the image acquisition unit 100. The shallow layer feature extraction unit 201 extracts shallow layer features F1 representing structural features of the vascular pattern in the area to be authenticated from the input image, and outputs the shallow layer features F1 to the deep layer feature extraction unit 202. The shallow layer features F1 extracted by the shallow layer feature extraction unit 201 are also output to the related feature generation unit 205 and the addition unit 206.

[0015] The deep feature extraction unit 202 receives the shallow feature F1 extracted by the shallow feature extraction unit 201. The deep feature extraction unit 202 extracts a deep feature F2 representing a local feature of the vascular pattern in the area to be authenticated based on the shallow feature F1, and outputs the deep feature F2 to the storage unit 203.

[0016] The storage unit 203 temporarily stores the deep feature F2 extracted by the deep feature extraction unit 202. The deep feature F2 stored in the storage unit 203 is read out by the feature alignment unit 204 and used in the processing performed by the feature alignment unit 204.

[0017] The feature alignment unit 204 adjusts the deep feature F2 read from the storage unit 203 to match the shallow feature F1 extracted by the shallow feature extraction unit 201. Here, the shallow feature F1 and the deep feature F2 are information output from different layers of the CNN, and therefore the number of dimensions (width, height, number of channels) constituting each piece of information are different. The feature alignment unit 204 adjusts the deep feature F2 by changing the number of dimensions of the information of the deep feature F2 so that it matches the shallow feature F1. The result of adjusting the deep feature F2 by the feature alignment unit 204 is input to the related feature generation unit 205 as an adjusted deep feature F2′. Note that details of the method of adjusting the deep feature F2 by the feature alignment unit 204 will be described later.

[0018] The related feature generation unit 205 generates a related feature Fr that represents the relationship between the shallow feature F1 and the deep feature F2, i.e., the relationship between the structural features and local features in the vascular pattern of the area to be authenticated, based on the shallow feature F1 input from the shallow feature extraction unit 201 and the adjusted deep feature F2' obtained by the feature alignment unit 204. This related feature Fr indicates an important area in the image acquired by the image acquisition unit 100 that should be noted when identifying the vascular pattern of the area to be authenticated.

[0019] As described above, the related feature generation unit 205 can generate related features Fr, which are features that span these layers, based on the shallow layer features F1 and deep layer features F2 extracted from different layers of the CNN.

[0020] The storage unit 203, the feature alignment unit 204, and the related feature generation unit 205 each perform the processes described above, thereby acting as a feedback mechanism that feeds back the deep feature F2 extracted by the deep feature extraction unit 202. Based on the deep feature F2 fed back by this feedback mechanism and the shallow feature F1 extracted by the shallow feature extraction unit 201, the related feature Fr is generated.

[0021] The adder 206 adds the related feature Fr generated by the related feature generator 205 to the shallow feature F1 to calculate a combined feature Fc by combining these features. The combined feature Fc calculated by the adder 206 is input to the deep feature extractor 202.

[0022] When the combined feature Fc is input from the adder 206, the deep feature extraction unit 202 performs the same arithmetic processing on the combined feature Fc as on the shallow feature F1, thereby extracting a combined feature F3 that represents the combined features of the vascular pattern in the area to be authenticated. The combined feature F3 extracted by the deep feature extraction unit 202 is output to the classification unit 207.

[0023] The classification unit 207 classifies the vascular patterns based on the comprehensive feature F3 input from the deep feature extraction unit 202. The classification unit 207 calculates a probability score based on the degree of match with each of the vascular patterns pre-registered for each person to be authenticated, for example, from the comprehensive feature F3. This makes it possible to identify which of the vascular patterns pre-registered for each person to which the vascular pattern of the part to be authenticated in the image acquired by the image acquisition unit 100 belongs. The classification unit 207 is configured, for example, using a trained CNN.

[0024] The authentication unit 208 performs biometric authentication based on the classification result of the blood vessel pattern by the classification unit 207. For example, from among authentication information preset for each person to be authenticated, the authentication unit 208 acquires authentication information corresponding to the person to be authenticated identified by the classification unit 207 and outputs the information to an external device as the biometric authentication result. This allows the external device connected to the biometric authentication device 1 to perform processing according to the authentication information of the person to be authenticated. Note that other processing may also be performed by the authentication unit 208.

[0025] The learning control unit 300 performs learning of the CNN constituting the shallow feature extraction unit 201 and the deep feature extraction unit 202 in the arithmetic processing unit 200. At this time, the learning control unit 300 uses images including vascular patterns of the body parts to be authenticated of various people as learning images, and causes each of the functional blocks of the arithmetic processing unit 200, namely the shallow feature extraction unit 201, the deep feature extraction unit 202, the memory unit 203, the feature alignment unit 204, the related feature generation unit 205, the addition unit 206, and the classification unit 207, to execute the above-described processing. Then, the classification results of the vascular patterns obtained for each learning image are reflected in each layer of the CNN model, thereby enabling learning of the CNN.

[0026] 2 is a diagram showing an example of the hardware configuration of the arithmetic processing unit 200 in the biometric authentication device 1. The arithmetic processing unit 200 is configured using a computer such as a server or a PC, and includes a processor 10, a memory 20, a recording device 30, an input / output interface 40, an input device 50, and an output device 60. In the arithmetic processing unit 200, these are connected to each other via a communication bus.

[0027] When the part 2 to be authenticated (e.g., a finger) of the person to be authenticated is placed in a predetermined position in the image acquisition unit 100, the image acquisition unit 100 photographs the part 2 to be authenticated and obtains a captured image including the vascular pattern of the part 2 to be authenticated.

[0028] The processor 10 executes a predetermined program to control the operation of the arithmetic processing unit 200 and cause a computer such as a server or PC to function as the arithmetic processing unit 200. The processor 10 is configured using, for example, a central processing unit (CPU), a micro processing unit (MPU), a graphics processing unit (GPU), etc. The memory 20 is a volatile storage medium and is used as a work area when the processor 10 executes a program. The recording device 30 is a recording medium capable of storing various programs and data and is configured using, for example, a hard disk drive (HDD), a solid state drive (SSD), etc. The input / output interface 40 is a part that performs interface processing of data input and output to the arithmetic processing unit 200. Data of captured images acquired by the image acquisition unit 100 is input to the arithmetic processing unit 200 via the input / output interface 40, recorded in the recording device 30, and used in processing by the processor 10. The input device 50 is a device that accepts data input from a user to the arithmetic processing unit 200 and is configured using, for example, a keyboard, a mouse, a touch panel, a microphone, etc. The output device 60 is a device that outputs information to the user, and is configured using, for example, a display, a printer, a speaker, and the like.

[0029] FIG. 3 is a flowchart showing the flow of processing executed by the biometric authentication device according to one embodiment of the present invention.

[0030] In step S100, a learning process is executed. Here, under the control of the learning control unit 300, a large number of learning images are acquired, and each learning image is processed by the respective functional blocks of the arithmetic processing unit 200, namely, the shallow feature extraction unit 201, the deep feature extraction unit 202, the memory unit 203, the feature alignment unit 204, the related feature generation unit 205, the adder 206, and the classification unit 207. Then, using the classification results of the vascular patterns of each learning image obtained, the CNNs constituting the shallow feature extraction unit 201 and the deep feature extraction unit 202 are trained. Details of the learning process executed in step S100 will be described later with reference to the flowchart of FIG. 4.

[0031] In step S150, it is determined whether or not an authentication request has been made. For example, if it is detected that the person to be authenticated has placed a finger or palm, which is the body part to be authenticated, at a predetermined position on the image acquisition unit 100, it is determined that an authentication request has been made and the process proceeds to step S200; otherwise, it is determined that an authentication request has not been made and the process remains in step S150. Note that the determination in step S150 may be made using other conditions as long as it can be determined that the image acquisition unit 100 is in a state where it can acquire a captured image including the vascular pattern of the body part to be authenticated.

[0032] In step S200, authentication processing is performed using the CNN that has been trained by the training processing in step S100. Here, a photographed image including the blood vessel pattern of the subject's finger or palm is acquired by the image acquisition unit 100 as the subject's body part to be authenticated, and each functional block of the arithmetic processing unit 200 performs its respective processing on this photographed image to acquire a biometric authentication result for the subject. Details of the authentication processing performed in step S200 will be described later with reference to the flowchart in FIG. 5.

[0033] In step S300, it is determined whether or not to retrain the CNN. Here, for example, if preset retraining conditions are met or if a user instructs retraining, it is determined that CNN retraining is to be performed, and the process returns to step S100. On the other hand, if these conditions are not met, it is determined that CNN retraining is not to be performed, and the process returns to step S150.

[0034] FIG. 4 is a flowchart showing the flow of the learning process executed in step S100 of FIG.

[0035] In step S101, a learning image is acquired. Here, for example, the learning image can be acquired by a user inputting images including blood vessel patterns of the body parts to be authenticated of various people prepared in advance to the arithmetic processing unit 200. Alternatively, a captured image acquired by the image acquisition unit 100 may be used as the learning image.

[0036] In step S102, the shallow layer feature extraction unit 201 extracts shallow layer features F1 for learning from the learning image acquired in step S101.

[0037] In step S103, the deep feature extraction unit 202 extracts deep features F2 for learning based on the shallow features F1 extracted in step S102.

[0038] In step S104, the deep feature F2 for learning extracted in step S103 is stored in the storage unit 203.

[0039] Through the processing of steps S102 to S104 described above, shallow layer features for learning F1 obtained by inputting the learning image acquired in step S101 into the shallow layer feature extraction unit 201, and deep layer features for learning F2 obtained by inputting the shallow layer features F1 into the deep layer feature extraction unit 202 are obtained, and the deep layer features for learning F2 are stored in the storage unit 203. This series of processing is also referred to as primary CNN calculation processing. Note that in the primary CNN calculation processing, image information input to the CNN flows forward from the shallow layer feature extraction unit 201 to the deep layer feature extraction unit 202.

[0040] In step S105, the shallow layer feature F1 for learning acquired in step S101 is acquired. Here, similar to step S102, the shallow layer feature F1 for learning may be acquired by inputting the learning image to the shallow layer feature extraction unit 201. Alternatively, the shallow layer feature F1 obtained in step S102 may be stored in the storage unit 203, and the shallow layer feature F1 for learning may be acquired by reading it from the storage unit 203.

[0041] In step S106, the deep feature F2 for learning stored in step S104 is read from the storage unit 203.

[0042] In step S107, the deep layer feature F2 read out in step S106 is adjusted to match the shallow layer feature F1 acquired in step S105 using the feature alignment unit 204. Here, as described above, the number of dimensions (width, height, number of channels) of the information of the deep layer feature F2 is changed so that they match the shallow layer feature F1, thereby adjusting the deep layer feature F2 to match the shallow layer feature F1, and an adjusted deep layer feature F2′ is acquired.

[0043] In step S108, based on the shallow feature F1 acquired in step S105 and the adjusted deep feature F2′ acquired in step S107, the related feature generation unit 205 generates a related feature Fr for learning that indicates the relationship between the shallow feature F1 for learning and the deep feature F2. This process is also called a relationship process. The method for generating the related feature Fr will be described in detail later.

[0044] In step S109, the adder 206 calculates a combined feature Fc for learning by adding the related feature Fr generated in step S108 to the shallow layer feature F1 acquired in step S105. This process is also called a combination process.

[0045] In the processing of steps S105 to S109 described above, information output from the deep feature extraction unit 202 in the CNN flows in the reverse direction toward the shallow feature extraction unit 201 via a feedback mechanism configured by the storage unit 203, the feature alignment unit 204, and the related feature generation unit 205.

[0046] In step S110, an overall feature F3 for learning is extracted based on the combined feature Fc calculated in step S109 using the deep feature extraction unit 202. Here, similar to step S103, the combined feature Fc is input to the deep feature extraction unit 202 and its output is obtained, thereby making it possible to extract the overall feature F3 for learning based on the combined feature Fc.

[0047] By the process of step S110 described above, a comprehensive feature for learning F3 is obtained by inputting the combined feature for learning Fc to the deep feature extraction unit 202. This process is also called a second-order CNN calculation process.

[0048] In step S111, the classification unit 207 classifies the vascular patterns based on the comprehensive feature F3 for learning extracted in step S110. As described above, the classification unit 207 calculates a probability score for each registered vascular pattern of each person to be authenticated based on the comprehensive feature F3 output from the deep feature extraction unit 202 and input to the classification unit 207, thereby classifying the vascular patterns contained in the learning images. This makes it possible to determine which person's vascular pattern the learning image acquired in step S101 corresponds to, along with the likelihood of this.

[0049] In step S112, a classification loss is calculated based on the difference between the classification result obtained in step S111 and the correct classification result. Here, the classification loss can be calculated based on, for example, the probability score for each vascular pattern calculated in step S111 and the correct classification result set in advance for the learning image.

[0050] In step S113, the classification loss value calculated in step S112 is fed back to the CNN that constitutes the shallow feature extraction unit 201 and the deep feature extraction unit 202 in the calculation processing unit 200, and the gradient value of the weight in each layer of the CNN is calculated.

[0051] In step S114, the weights in each layer of the CNN are updated based on the gradient values ​​calculated in step S113, thereby performing learning of the CNN model.

[0052] Through the processing of steps S112 to S114 described above, each layer of the CNN is trained based on the classification result of the vascular pattern for the training image obtained by inputting the training comprehensive feature F3 to the classification unit 207. As a result, the weight value in each layer of the CNN is updated so that the classification loss value for the classification result becomes small.

[0053] In step S115, it is determined whether or not to terminate the learning process. Here, for example, if a preset learning termination condition is met or if a user instructs to terminate learning, it is determined that the learning process should be terminated, and the learning process shown in the flowchart of Fig. 4 is terminated. On the other hand, if these conditions are not met, it is determined that the learning process should be continued without being terminated, and the process returns to step S101, and the processes from step S101 onwards, which have already been described, are repeated.

[0054] FIG. 5 is a flowchart showing the flow of the authentication process executed in step S200 of FIG.

[0055] In step S201, an image for authentication is acquired. As described above, a captured image including the blood vessel pattern of the part of the subject to be authenticated is acquired by the image acquisition unit 100, and this captured image is acquired as the image for authentication.

[0056] In steps S202 to S211, the authentication image acquired in step S201 is subjected to the same processes as steps S102 to S111 in Fig. 4. As a result, the vascular pattern contained in the authentication image is classified using the CNN trained by the learning process in step S100 in Fig. 3. If the probability score with all registered vascular patterns is below a predetermined value, the pattern is classified as an unregistered vascular pattern.

[0057] In step S212, the authentication unit 208 compares the classification result obtained in step S211 with authentication information preset for each person to be authenticated, and acquires the authentication information corresponding to that person to be authenticated.

[0058] In step S213, the authentication information acquired in step S212 is output from the authentication unit 208 as the biometric authentication result for the authentication image acquired in step S201.

[0059] After the process of step S213 is performed, the authentication process shown in the flowchart of FIG. 5 ends.

[0060] Next, the adjustment process of the deep feature F2 by the feature alignment unit 204, which is performed in steps S107 and S207 in FIGS. 4 and 5, respectively, will be described with reference to FIG. 6.

[0061] When the deep layer feature F2 consisting of information on a predetermined width, height, and number of channels is input to the feature alignment unit 204, the feature alignment unit 204 adjusts these values ​​by arithmetic processing using a neural network consisting of multiple layers so that they respectively match the width, height, and number of channels in the shallow layer feature F1. As a result, as shown in Figure 6, it is possible to acquire, as an adjusted deep layer feature F2', information that has information of the same number of dimensions (width, height, and number of channels) as the shallow layer feature F1 and that represents the same characteristics of the vascular pattern as the deep layer feature F2.

[0062] In the related feature generation unit 205, by using the adjusted deep feature F2′ obtained by the feature alignment unit 204 in this manner, it becomes possible to efficiently generate the related feature Fr.

[0063] Next, the generation process of the related feature quantity Fr by the related feature quantity generation unit 205, which is performed in steps S108 and S208 in FIGS. 4 and 5, respectively, will be described with reference to FIGS.

[0064] When the shallow layer feature F1 and the adjusted deep layer feature F2' are input to the related feature generation unit 205, the related feature generation unit 205 generates information reflecting the relationship between these feature values ​​through calculation processing using a neural network consisting of multiple layers. The information thus obtained is output as a related feature Fr based on the shallow layer feature F1 and the adjusted deep layer feature F2'. As a result, it is possible to generate a related feature Fr that represents the relationship between structural features and local features in the vascular patterns of the learning image and the authentication image, as shown in FIG. 7 .

[0065] Fig. 8 is a diagram showing an example of the functional configuration of the related feature generating unit 205. As shown in Fig. 8, the related feature generating unit 205 is configured by combining neural networks 2051, 2052, 2053, and 2054, for example.

[0066] The adjusted deep feature F2′ input to the related feature generation unit 205 is input to the neural network 2051. The neural network 2051 generates a query 2061 based on the adjusted deep feature F2′.

[0067] The shallow layer feature F1 input to the related feature generation unit 205 is input to neural networks 2052 and 2053. The neural network 2052 generates a key 2062 based on the shallow layer feature F1. The neural network 2053 generates a value 2063 based on the shallow layer feature F1.

[0068] The related feature generator 205 generates a correlation matrix 2064 by calculating the inner product of the query 2061 and the key 2062. The correlation matrix 2064 represents the relationship between the structural features of the vascular pattern represented by the shallow feature F1 and the local features of the vascular pattern represented by the adjusted deep feature F2', and can highlight relatively important areas in the vascular pattern. This correlation matrix 2064 represents important information for capturing non-local features of the vascular pattern.

[0069] The related feature generation unit 205 calculates an attention matrix 2065 that collectively encodes information represented by the shallow feature F1 and the adjusted deep feature F2′ by multiplying the correlation matrix 2064 by the value 2063. The calculated attention matrix 2065 is input to the neural network 2054.

[0070] The neural network 2054 adjusts the attention matrix 2065 so that the number of dimensions (width, height, and number of channels) of the attention matrix 2065 matches the shallow layer feature F1, and outputs the adjustment result as the related feature Fr.

[0071] FIG. 9 is a diagram showing an example of deep feature F2 extracted by the deep feature extraction unit 202 in steps S103 and S203 in FIGS. 4 and 5 . As shown in FIG. 9 , the deep feature F2 includes a region F2h with a high feature value and a region F2l with a low feature value. Region F2h indicates a region having a notable local feature in a learning image or an authentication image, and is an important region when classifying or matching vascular patterns. On the other hand, region F2l indicates a region with low attention or importance in a learning image or an authentication image. In the deep feature F2, region F2h generally corresponds to the foreground region, and region F2l corresponds to the background region.

[0072] Fig. 10 is a diagram showing an example of shallow layer features F1 extracted by the shallow layer feature extraction unit 201 in steps S102 and S202 in Fig. 4 and Fig. 5, respectively. The shallow layer features F1 are configured to include a region F1h having a high feature value, as shown in Fig. 10, for example. The region F1h indicates a region having notable structural features in the learning image or authentication image, and is an important region when classifying and matching vascular patterns.

[0073] 11 is a diagram showing an example of the related feature Fr generated by the related feature generation unit 205 in steps S108 and S208 in FIGS. 4 and 5 . As shown in FIG. 11 , the related feature Fr includes a region Frh with a high feature value, a region Frl with a low feature value, and a region Frm with a medium feature value. The regions Frh and Frl correspond to the foreground and background regions, respectively, in the learning image or the authentication image. Meanwhile, the region Frm indicates a region where the structural features and local features of the vascular pattern are highly correlated.

[0074] The related feature Fr shown in Fig. 11 includes both the local feature information of the deep feature F2 shown in Fig. 9 and the structural feature information of the shallow feature F1 shown in Fig. 10. Therefore, by using the related feature Fr, it is possible to more accurately identify the vascular pattern than when only the shallow feature F1 or the deep feature F2 is used. Therefore, it is possible to improve authentication accuracy when a missing region exists in the authentication image obtained by capturing the body part to be authenticated.

[0075] According to the embodiment of the present invention described above, the following advantageous effects can be obtained.

[0076] (1) The biometric authentication device 1 is a device that performs biometric authentication based on the blood vessel pattern of a predetermined part of the human body to be authenticated. The biometric authentication device 1 includes an image acquisition unit 100 that acquires an image of the area to be authenticated, a shallow feature extraction unit 201 that extracts shallow features F1 (first features) that represent structural features of the vascular pattern from the image acquired by the image acquisition unit 100, a deep feature extraction unit 202 that extracts deep features F2 (second features) that represent local features of the vascular pattern based on the shallow features F1 extracted by the shallow feature extraction unit 201, a feedback mechanism (a storage unit 203, a feature alignment unit 204, and a related feature generation unit 205) that feeds back the deep features F2 extracted by the deep feature extraction unit 202, and a classification unit 207 that classifies the vascular pattern using a calculation result (a comprehensive feature F3) obtained by inputting information (a composite feature Fc) based on the shallow features F1 and the deep features F2 fed back by the feedback mechanism into the deep feature extraction unit 202. Then, the authentication unit 208 performs biometric authentication based on the classification result of the blood vessel pattern by the classification unit 207. This makes it possible to improve the authentication accuracy in biometric authentication using the blood vessel pattern of the human body when a missing region exists in an image of the body part to be authenticated.

[0077] (2) The feedback mechanism includes a related feature generation unit 205 that generates a related feature Fr representing the relationship between structural features and local features in a vascular pattern based on the shallow feature F1 and the fed-back deep feature F2. The deep feature extraction unit 202 extracts a comprehensive feature F3 (third feature) representing the comprehensive feature of the vascular pattern based on a combined feature Fc calculated by the adder 206 by adding the related feature Fr generated by the related feature generation unit 205 to the shallow feature F1. The classification unit 207 classifies the vascular pattern based on the comprehensive feature F3 extracted by the deep feature extraction unit 202. This configuration allows for more accurate classification of vascular patterns by taking into account both the local features and structural features of the vascular pattern.

[0078] (3) The feedback mechanism includes a feature alignment unit 204 that adjusts the deep feature F2 to match the shallow feature F1. The related feature generation unit 205 generates the related feature Fr using the adjusted deep feature F2′ that corresponds to the deep feature F2 adjusted by the feature alignment unit 204. This configuration makes it possible to easily and efficiently generate the related feature Fr.

[0079] (4) The shallow feature F1 and the deep feature F2 each have information on a predetermined number of dimensions. The feature alignment unit 204 generates an adjusted deep feature F2' by adjusting the number of dimensions of the deep feature F2 to match the number of dimensions of the shallow feature F1. In this manner, the related feature generation unit 205 can adjust the deep feature F2 in a form appropriate for generating the related feature Fr, and generate an adjusted deep feature F2'.

[0080] (5) The shallow feature extraction unit 201 and the deep feature extraction unit 202 are each configured by a predetermined layer in a trained CNN, and the CNN layer corresponding to the shallow feature extraction unit 201 is located lower than the CNN layer corresponding to the deep feature extraction unit 202. This makes it possible to realize the shallow feature extraction unit 201 and the deep feature extraction unit 202 that can easily extract the shallow feature F1 and the deep feature F2 from an image, respectively.

[0081] (6) The biometric authentication device 1 includes a learning control unit 300 that performs CNN learning. In the learning process shown in the flowchart of Fig. 4, the learning control unit 300 performs a primary CNN calculation process (steps S102 to S104) to obtain shallow features for learning F1 obtained by inputting a predetermined learning image including a blood vessel pattern to the shallow feature extraction unit 201 and deep features for learning F2 obtained by inputting the shallow features for learning F1 to the deep feature extraction unit 202; an association process (step S108) to generate related features for learning Fr that represent the association between the shallow features for learning F1 and the deep features F2 obtained by the primary CNN calculation process; and a deep feature extraction process (step S109) to extract the related features for learning Fr generated by the association process from the shallow features for learning F1. The biometric authentication device 1 executes a synthesis process (step S109) for calculating a synthetic feature Fc for learning added to the layer feature F1, a secondary CNN calculation process (step S110) for inputting the synthetic feature Fc for learning calculated by the synthesis process into the deep feature extraction unit 202 to obtain an overall feature F3 for learning, and a learning process (steps S112 to S114) for training each layer of the CNN corresponding to the shallow feature extraction unit 201 and the deep feature extraction unit 202 based on the classification result of the vascular pattern for the learning image obtained by inputting the overall feature F3 for learning calculated by the secondary CNN calculation process into the classification unit 207. In this way, the trained CNN required to realize the shallow feature extraction unit 201 and the deep feature extraction unit 202 can be obtained at any timing by the processing of the biometric authentication device 1 itself.

[0082] (7) In the biometric authentication device 1, the image acquisition unit 100 and the calculation processing unit 200 execute the authentication process shown in the flowchart of Fig. 5. In this authentication process, the image acquisition unit 100 acquires an image of the area to be authenticated (step S201), the shallow layer feature extraction unit 201 extracts shallow layer feature F1 representing structural features of the vascular pattern from the image (step S202), and the deep layer feature extraction unit 202 extracts deep layer feature F2 representing local features of the vascular pattern based on the extracted shallow layer feature F1 (step S203). The extracted deep feature F2 is then fed back by a feedback mechanism consisting of the storage unit 203, feature alignment unit 204, and related feature generation unit 205 (steps S205-S209). Based on the shallow feature F1 and the fed-back deep feature F2, the deep feature extraction unit 202 extracts a comprehensive feature F3 representing the comprehensive features of the vascular pattern (step S210). The classification unit 207 then classifies the vascular pattern based on the comprehensive feature F3 (step S211). Based on the vascular pattern classification results obtained in this way, the authentication unit 208 performs biometric authentication (steps S212 and S213). This configuration improves authentication accuracy when a missing region exists in an image of the target body part during biometric authentication using the vascular pattern of the human body.

[0083] It goes without saying that the present invention is not limited to the above-described embodiments and can be modified in various ways without departing from the spirit of the present invention. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those having all of the described configurations. Furthermore, it is possible to add, delete, or replace part of the configuration of the above-described embodiments with other configurations.

[0084] Furthermore, the above-described configurations, functional units, processing units, processing means, etc. may be partially or entirely implemented in hardware, for example, by designing them as integrated circuits. The above-described configurations, functions, etc. may also be implemented in software, with a processor interpreting and executing a program that implements each function. Information such as the programs, tables, and files that implement each function can be stored in a memory, a recording device such as a hard disk or SSD, or a recording medium such as an IC card, an SD card, or a DVD.

[0085] In addition, in the above figures, the control lines and information lines shown are those that are considered necessary for explanation, and do not necessarily show all the control lines and information lines that are actually implemented. For example, it can be considered that almost all components are actually connected to each other.

[0086] 1: Biometric authentication device 2: Part to be authenticated 10: Processor 20: Memory 30: Recording device 40: Input / output interface 50: Input device 60: Output device 100: Image acquisition unit 200: Arithmetic processing unit 201: Shallow feature extraction unit 202: Deep feature extraction unit 203: Storage unit 204: Feature alignment unit 205: Related feature generation unit 206: Addition unit 207: Classification unit 208: Authentication unit 300: Learning control unit

Claims

1. A device for performing biometric authentication using a blood vessel pattern at a predetermined part of the human body to be authenticated, comprising: an image acquisition unit that acquires an image of the part to be authenticated; a first feature extraction unit that extracts a first feature representing a structural feature of the blood vessel pattern from the image acquired by the image acquisition unit; a second feature extraction unit that extracts a second feature representing a local feature of the blood vessel pattern based on the first feature extracted by the first feature extraction unit; a feedback mechanism that feeds back the second feature extracted by the second feature extraction unit; and a classification unit that classifies the blood vessel pattern using the calculation results obtained by inputting information based on the first feature and the second feature fed back by the feedback mechanism to the second feature extraction unit, wherein the biometric authentication device performs the biometric authentication based on the blood vessel pattern classification results obtained by the classification unit.

2. A biometric authentication device according to claim 1, wherein the feedback mechanism has a related feature generation unit that generates related features representing the relationship between structural features and local features in the vascular pattern based on the first feature and the fed-back second feature; the second feature extraction unit extracts third features representing comprehensive features of the vascular pattern based on a composite feature obtained by adding the related feature generated by the related feature generation unit to the first feature; and the classification unit classifies the vascular pattern based on the third feature extracted by the second feature extraction unit.

3. A biometric authentication device according to claim 2, wherein the feedback mechanism has a feature alignment unit that adjusts the second feature to match the first feature, and the related feature generation unit generates the related feature using the second feature adjusted by the feature alignment unit.

4. A biometric authentication device according to claim 3, wherein the first feature and the second feature each have information on a predetermined number of dimensions, and the feature alignment unit adjusts the number of dimensions in the second feature to match the number of dimensions in the first feature.

5. A biometric authentication device according to any one of claims 1 to 4, wherein the first feature extraction unit and the second feature extraction unit are each configured as a predetermined layer in a trained CNN (Convolutional Neural Network), and the layer of the CNN corresponding to the first feature extraction unit is lower than the layer of the CNN corresponding to the second feature extraction unit.

6. A biometric authentication device according to claim 5, further comprising a learning control unit for training the CNN, wherein the learning control unit performs a primary CNN calculation process to obtain first learning features obtained by inputting a predetermined learning image including the blood vessel pattern into the first feature extraction unit and second learning features obtained by inputting the first learning features into the second feature extraction unit; an association process to generate related learning features that represent the association between the first learning features obtained by the primary CNN calculation process and the second learning features; a synthesis process to calculate synthetic learning features by adding the related learning features generated by the association process to the first learning features; and a secondary CNN calculation process to obtain third learning features obtained by inputting the synthetic learning features calculated by the synthesis process into the second feature extraction unit. a learning process for learning each layer of the CNN corresponding to the first feature extraction unit and the second feature extraction unit, respectively, based on the classification result of the vascular pattern for the training image obtained by inputting the third training feature obtained by the secondary CNN arithmetic process to the classification unit.

7. A method for performing biometric authentication using a blood vessel pattern at a predetermined part of the human body to be authenticated, comprising: acquiring an image of the part to be authenticated; extracting a first feature amount representing a structural feature of the blood vessel pattern from the image; extracting a second feature amount representing a local feature of the blood vessel pattern based on the first feature amount; feeding back the second feature amount; extracting a third feature amount representing a comprehensive feature of the blood vessel pattern based on the first feature amount and the fed-back second feature amount; classifying the blood vessel pattern based on the third feature amount; and performing the biometric authentication based on the result of the blood vessel pattern classification.

Images