Biometric authentication device and biometric authentication method

Abstract

To improve the accuracy of biometric authentication using a blood vessel pattern of a human body in the presence of a vanishing region in a captured image of an authenticated portion.SOLUTION: A biometric authentication device 1 includes a storage unit 203, a feature alignment unit 204, and a related feature generation unit 205, as a feedback mechanism for feeding back deep features F2 extracted by a deep feature extraction unit 202. The biometric authentication device includes a classification unit 207 that classifies blood vessel patterns by using a computation result (integrated features 3) obtained by inputting composite features Fc which are information based on shallow features F1 extracted by a shallow feature extraction unit 201 and the deep features F2 fed back by the feedback mechanism, to the deep feature extraction unit 202.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an apparatus and method for biometric authentication using blood vessel patterns. [Background technology]

[0002] Conventionally, a technology for biometric authentication of an individual has been realized by capturing an image of a human finger or palm as a body part to be authenticated, and recognizing the blood vessel (vein) pattern of the body part to be authenticated contained in the captured image. For example, Patent Document 1 describes a finger vein identification method in which an image of a region of interest detected from a finger image is used as an input image, a difference image between this input image and a pre-registered image is obtained, and the difference image is input into a trained convolutional neural network (CNN), and finger vein identification is performed based on the obtained result. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Chinese Patent Application Publication No. 110738174 Summary of the Invention [Problem to be solved by the invention]

[0004] In biometric authentication, when photographing a finger or palm, which is the body part to be authenticated, with a camera, if excessive pressure is applied to the body part to be authenticated or if the position or orientation of the body part to be authenticated relative to the camera is misaligned, the captured image may include an area where blood vessels are not clearly visible and have disappeared (hereinafter referred to as a "missing area"), even though blood vessels are actually present. When such a missing area exists in the captured image, it becomes difficult to accurately identify the blood vessel pattern with conventional methods such as those described in Patent Document 1, and there is a risk that a correct biometric authentication result will not be obtained.

[0005] The present invention has been made based on this background, and aims to improve authentication accuracy in biometric authentication using the blood vessel patterns of the human body when a missing area exists in an image of the part to be authenticated. [Means for solving the problem]

[0006] A biometric authentication device according to the present invention is a device that performs biometric authentication using a vascular pattern at a predetermined part of the human body to be authenticated, and includes: an image acquisition unit that acquires an image of the part to be authenticated; a first feature extraction unit that extracts a first feature representing a structural feature of the vascular pattern from the image acquired by the image acquisition unit; a second feature extraction unit that extracts a second feature representing a local feature of the vascular pattern based on the first feature extracted by the first feature extraction unit; a feedback mechanism that feeds back the second feature extracted by the second feature extraction unit; and a classification unit that classifies the vascular pattern using a calculation result obtained by inputting information based on the first feature and the second feature fed back by the feedback mechanism to the second feature extraction unit, and performs biometric authentication based on the classification result of the vascular pattern by the classification unit. A biometric authentication method according to the present invention is a method for performing biometric authentication using a vascular pattern in a predetermined area of ​​the human body to be authenticated, which includes obtaining an image of the area to be authenticated, extracting a first feature from the image that represents a structural feature of the vascular pattern, extracting a second feature that represents a local feature of the vascular pattern based on the first feature, feeding back the second feature, extracting a third feature that represents a comprehensive feature of the vascular pattern based on the first feature and the fed-back second feature, classifying the vascular pattern based on the third feature, and performing the biometric authentication based on the vascular pattern classification result. [Effects of the Invention]

[0007] According to the present invention, in biometric authentication using the blood vessel pattern of the human body, it is possible to improve authentication accuracy when a missing region exists in an image of a body part to be authenticated. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a diagram illustrating the configuration of a biometric authentication device according to an embodiment of the present invention. [Figure 2] FIG. 2 is a diagram showing an example of the hardware configuration of a processing unit. [Figure 3] 4 is a flowchart showing the flow of processing executed by the biometric authentication device according to one embodiment of the present invention. [Figure 4] 10 is a flowchart showing the flow of a learning process. [Figure 5] 10 is a flowchart showing the flow of authentication processing. [Figure 6] FIG. 10 is an explanatory diagram of a deep feature adjustment process. [Figure 7] FIG. 10 is an explanatory diagram of a generation process of a related feature. [Figure 8] FIG. 4 is a diagram showing an example of the functional configuration of a related feature generating unit. [Figure 9] FIG. 10 is a diagram showing an example of deep features. [Figure 10] FIG. 10 is a diagram showing an example of shallow layer features. [Figure 11] FIG. 10 is a diagram showing an example of related feature amounts. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0010] Fig. 1 is a configuration diagram of a biometric authentication device according to one embodiment of the present invention. The biometric authentication device 1 shown in Fig. 1 is a device that identifies the blood vessel pattern of a human finger or palm as a body part to be authenticated from an image obtained by photographing the body part to be authenticated, and performs biometric authentication based on the blood vessel pattern. The biometric authentication device 1 is configured with an image acquisition unit 100, an arithmetic processing unit 200, and a learning control unit 300.

[0011] The image acquisition unit 100 acquires an image in which the blood vessel pattern of the authentication part can be identified by capturing an image of the authentication part of the person to be authenticated. For example, an image including the vein pattern of the finger or palm can be acquired by irradiating near-infrared light onto the authentication part (finger or palm) and capturing the transmitted light or reflected light. Note that the image acquisition unit 100 may acquire an image by other methods as long as the image is in which the blood vessel pattern of the authentication part can be identified.

[0012] The arithmetic processing unit 200 performs predetermined arithmetic processing on the image acquired by the image acquisition unit 100 to identify the blood vessel pattern in the area to be authenticated, and specifies the person to be authenticated from the identification result to perform biometric authentication. The arithmetic processing unit 200 has functional blocks of a shallow layer feature extraction unit 201, a deep layer feature extraction unit 202, a memory unit 203, a feature alignment unit 204, a related feature generation unit 205, an addition unit 206, a classification unit 207, and an authentication unit 208.

[0013] The shallow feature extraction unit 201 and the deep feature extraction unit 202 are each configured using a trained CNN. The shallow feature extraction unit 201 is configured by one or more layers on the lower side (input side) of the multiple layers that make up the CNN. On the other hand, the deep feature extraction unit 202 is configured by one or more layers on the upper side (output side) of the multiple layers that make up the CNN. Note that the CNN may have layers other than the shallow feature extraction unit 201 and the deep feature extraction unit 202.

[0014] The shallow layer feature extraction unit 201 receives an input of a photographed image of the authentication target region acquired by the image acquisition unit 100. The shallow layer feature extraction unit 201 extracts a shallow layer feature F1 representing a structural feature of the blood vessel pattern in the authentication target region from the input image, and outputs the shallow layer feature F1 to the deep layer feature extraction unit 202. The shallow layer feature F1 extracted by the shallow layer feature extraction unit 201 is also output to the related feature generation unit 205 and the addition unit 206.

[0015] The deep layer feature extraction unit 202 receives the shallow layer feature F1 extracted by the shallow layer feature extraction unit 201. The deep layer feature extraction unit 202 extracts a deep layer feature F2 representing a local feature of the vascular pattern in the area to be authenticated based on the shallow layer feature F1, and outputs the deep layer feature F2 to the storage unit 203.

[0016] The storage unit 203 temporarily stores the deep feature F2 extracted by the deep feature extraction unit 202. The deep feature F2 stored in the storage unit 203 is read out by the feature alignment unit 204 and used in the processing performed by the feature alignment unit 204.

[0017] The feature alignment unit 204 adjusts the deep feature F2 read from the storage unit 203 to match it with the shallow feature F1 extracted by the shallow feature extraction unit 201. Here, the shallow feature F1 and the deep feature F2 are information output from different layers of the CNN, and therefore have different numbers of dimensions (width, height, number of channels) constituting each piece of information. The feature alignment unit 204 adjusts the deep feature F2 by changing the number of dimensions of the information of the deep feature F2 so that it matches the shallow feature F1. The result of adjusting the deep feature F2 by the feature alignment unit 204 is input to the related feature generation unit 205 as an adjusted deep feature F2′. Note that details of the method of adjusting the deep feature F2 by the feature alignment unit 204 will be described later.

[0018] The related feature generation unit 205 generates a related feature Fr that represents the relationship between the shallow feature F1 and the deep feature F2, i.e., the relationship between the structural feature and the local feature in the vascular pattern of the part to be authenticated, based on the shallow feature F1 input from the shallow feature extraction unit 201 and the adjusted deep feature F2' obtained by the feature alignment unit 204. This related feature Fr indicates an important area in the image acquired by the image acquisition unit 100 that should be noted when identifying the vascular pattern of the part to be authenticated.

[0019] As described above, the related feature generation unit 205 can generate related features Fr, which are features that span these layers, based on shallow layer features F1 and deep layer features F2 extracted from different layers of the CNN.

[0020] The storage unit 203, the feature alignment unit 204, and the related feature generation unit 205 each perform the processes described above, thereby acting as a feedback mechanism that feeds back the deep feature F2 extracted by the deep feature extraction unit 202. Based on the deep feature F2 fed back by this feedback mechanism and the shallow feature F1 extracted by the shallow feature extraction unit 201, the related feature Fr is generated.

[0021] The adder 206 adds the related feature Fr generated by the related feature generator 205 to the shallow feature F1 to calculate a combined feature Fc by combining these features. The combined feature Fc calculated by the adder 206 is input to the deep feature extractor 202.

[0022] When the combined feature Fc is input from the adder 206, the deep feature extraction unit 202 performs the same arithmetic processing on the combined feature Fc as on the superficial feature F1, thereby extracting a combined feature F3 that represents the combined feature of the vascular pattern in the area to be authenticated. The combined feature F3 extracted by the deep feature extraction unit 202 is output to the classification unit 207.

[0023] The classification unit 207 classifies the vascular patterns based on the comprehensive feature F3 input from the deep feature extraction unit 202. The classification unit 207 calculates a probability score based on the degree of match with each of the vascular patterns pre-registered for each person to be authenticated, for example, from the comprehensive feature F3. This makes it possible to identify which of the vascular patterns pre-registered for each person to which the vascular pattern of the part to be authenticated in the image acquired by the image acquisition unit 100 belongs. The classification unit 207 is configured using, for example, a trained CNN.

[0024] The authentication unit 208 performs biometric authentication based on the classification result of the blood vessel pattern by the classification unit 207. For example, from among authentication information preset for each person to be authenticated, the authentication unit 208 acquires authentication information corresponding to the person to be authenticated identified by the classification unit 207 and outputs it to an external device as the biometric authentication result. This allows the external device connected to the biometric authentication device 1 to perform processing according to the authentication information of the person to be authenticated. Note that the authentication unit 208 may also perform other processing.

[0025] The learning control unit 300 performs learning of the CNN that constitutes the shallow feature extraction unit 201 and the deep feature extraction unit 202 in the arithmetic processing unit 200. At this time, the learning control unit 300 uses images containing vascular patterns of the body parts to be authenticated of various people as learning images, and causes each of the functional blocks of the arithmetic processing unit 200, namely the shallow feature extraction unit 201, the deep feature extraction unit 202, the memory unit 203, the feature alignment unit 204, the related feature generation unit 205, the adder 206, and the classification unit 207, to execute the above-mentioned processes. Then, the classification results of the vascular patterns obtained for each learning image are reflected in each layer of the CNN model, thereby enabling CNN learning.

[0026] 2 is a diagram showing an example of the hardware configuration of the arithmetic processing unit 200 in the biometric authentication device 1. The arithmetic processing unit 200 is configured using a computer such as a server or a PC, and includes a processor 10, a memory 20, a recording device 30, an input / output interface 40, an input device 50, and an output device 60. In the arithmetic processing unit 200, these are connected to each other via a communication bus.

[0027] When the authentication target part 2 (e.g., a finger) of the person to be authenticated is placed in a predetermined position in the image acquisition unit 100, the image acquisition unit 100 photographs the authentication target part 2 and acquires a captured image including the blood vessel pattern in the authentication target part 2.

[0028] The processor 10 executes a predetermined program to control the operation of the arithmetic processing unit 200 and cause a computer such as a server or PC to function as the arithmetic processing unit 200. The processor 10 is configured using, for example, a central processing unit (CPU), a micro processing unit (MPU), a graphics processing unit (GPU), etc. The memory 20 is a volatile storage medium and is used as a work area when the processor 10 executes a program. The recording device 30 is a recording medium capable of storing various programs and data and is configured using, for example, a hard disk drive (HDD), a solid state drive (SSD), etc. The input / output interface 40 is a part that performs interface processing of data input and output to the arithmetic processing unit 200. Data of a captured image acquired by the image acquisition unit 100 is input to the arithmetic processing unit 200 via the input / output interface 40, recorded in the recording device 30, and used in processing performed by the processor 10. The input device 50 is a device that accepts data input from a user to the arithmetic processing unit 200 and is configured using, for example, a keyboard, a mouse, a touch panel, a microphone, etc. The output device 60 is a device that outputs information to the user, and is configured using, for example, a display, a printer, a speaker, and the like.

[0029] FIG. 3 is a flowchart showing the flow of processing executed by the biometric authentication device according to one embodiment of the present invention.

[0030] In step S100, a learning process is executed. Here, under the control of the learning control unit 300, a large number of learning images are acquired, and each learning image is processed by the functional blocks of the shallow feature extraction unit 201, deep feature extraction unit 202, memory unit 203, feature alignment unit 204, related feature generation unit 205, adder 206, and classification unit 207 of the arithmetic processing unit 200. Then, using the classification results of the vascular patterns of each learning image obtained, learning is performed on the CNNs constituting the shallow feature extraction unit 201 and the deep feature extraction unit 202. Details of the learning process executed in step S100 will be described later with reference to the flowchart of FIG. 4.

[0031] In step S150, it is determined whether or not an authentication request has been made. For example, if it is detected that the person to be authenticated has placed a finger or palm, which is the body part to be authenticated, at a predetermined position on the image acquisition unit 100, it is determined that an authentication request has been made and the process proceeds to step S200; otherwise, it is determined that an authentication request has not been made and the process remains in step S150. Note that the determination in step S150 may be made using other conditions as long as it can be determined that the image acquisition unit 100 is in a state where it can acquire a captured image including the vascular pattern of the body part to be authenticated.

[0032] In step S200, authentication processing is performed using the CNN trained by the learning processing in step S100. Here, a finger or palm of the person to be authenticated is set as the body part to be authenticated, and a captured image including the blood vessel pattern of the body part to be authenticated is acquired by the image acquisition unit 100. Each functional block of the arithmetic processing unit 200 performs its respective processing on this captured image, thereby acquiring a biometric authentication result for the person to be authenticated. Details of the authentication processing performed in step S200 will be described later with reference to the flowchart in FIG. 5.

[0033] In step S300, it is determined whether or not to perform CNN re-learning. Here, for example, if preset re-learning conditions are met or if a user instructs to re-learn, it is determined that CNN re-learning is to be performed, and the process returns to step S100. On the other hand, if these conditions are not met, it is determined that CNN re-learning is not to be performed, and the process returns to step S150.

[0034] FIG. 4 is a flowchart showing the flow of the learning process executed in step S100 of FIG.

[0035] In step S101, a learning image is acquired. Here, for example, the user can acquire the learning image by inputting images including blood vessel patterns of the body parts to be authenticated of various people prepared in advance to the arithmetic processing unit 200. Alternatively, a captured image acquired by the image acquisition unit 100 may be used as the learning image.

[0036] In step S102, the shallow layer feature extraction unit 201 extracts shallow layer features F1 for learning from the learning image acquired in step S101.

[0037] In step S103, the deep feature extraction unit 202 extracts deep features F2 for learning based on the shallow features F1 extracted in step S102.

[0038] In step S104, the deep feature F2 for learning extracted in step S103 is stored in the storage unit 203.

[0039] Through the processing of steps S102 to S104 described above, shallow layer features for learning F1 obtained by inputting the learning image acquired in step S101 into the shallow layer feature extraction unit 201, and deep layer features for learning F2 obtained by inputting the shallow layer features F1 into the deep layer feature extraction unit 202 are obtained, and the deep layer features for learning F2 are stored in the storage unit 203. This series of processing is also called primary CNN arithmetic processing. Note that in the primary CNN arithmetic processing, image information input to the CNN flows forward from the shallow layer feature extraction unit 201 to the deep layer feature extraction unit 202.

[0040] In step S105, the shallow layer features F1 for learning acquired in step S101 are acquired. Here, similar to step S102, the shallow layer features F1 for learning may be acquired by inputting the learning image to the shallow layer feature extraction unit 201. Alternatively, the shallow layer features F1 obtained in step S102 may be stored in the storage unit 203, and the shallow layer features F1 for learning may be acquired by reading them out from the storage unit 203.

[0041] In step S106, the deep feature F2 for learning stored in step S104 is read from the storage unit 203.

[0042] In step S107, the feature alignment unit 204 adjusts the deep feature F2 read out in step S106 to match the shallow feature F1 acquired in step S105. As described above, the number of dimensions (width, height, number of channels) of the information of the deep feature F2 is changed so that they match those of the shallow feature F1, thereby adjusting the deep feature F2 to match the shallow feature F1, and an adjusted deep feature F2′ is acquired.

[0043] In step S108, based on the shallow feature F1 acquired in step S105 and the adjusted deep feature F2' acquired in step S107, the related feature generation unit 205 generates a related feature Fr for learning that indicates the relationship between the shallow feature F1 for learning and the deep feature F2. This process is also called a relationship process. The method for generating the related feature Fr will be described in detail later.

[0044] In step S109, the adder 206 calculates a synthesized feature Fc for learning by adding the related feature Fr generated in step S108 to the shallow layer feature F1 acquired in step S105. This process is also called a synthesis process.

[0045] In the processing of steps S105 to S109 described above, information output from the deep feature extraction unit 202 in the CNN flows in the reverse direction toward the shallow feature extraction unit 201 via a feedback mechanism configured by the storage unit 203, the feature alignment unit 204, and the related feature generation unit 205.

[0046] In step S110, an overall feature F3 for learning is extracted based on the combined feature Fc calculated in step S109 using the deep feature extraction unit 202. Here, similar to step S103, the combined feature Fc is input to the deep feature extraction unit 202 and its output is obtained, thereby making it possible to extract the overall feature F3 for learning based on the combined feature Fc.

[0047] By the processing of step S110 described above, a comprehensive feature for learning F3 is obtained by inputting the combined feature for learning Fc to the deep feature extraction unit 202. This processing is also called secondary CNN calculation processing.

[0048] In step S111, the classification unit 207 classifies the vascular patterns based on the comprehensive feature F3 for learning extracted in step S110. As described above, the classification unit 207 calculates a probability score for each registered vascular pattern of each person to be authenticated based on the comprehensive feature F3 output from the deep feature extraction unit 202 and input to the classification unit 207, thereby classifying the vascular patterns contained in the learning images. This makes it possible to determine which person's vascular pattern the learning image acquired in step S101 corresponds to, along with the likelihood of this.

[0049] In step S112, a classification loss is calculated based on the difference between the classification result obtained in step S111 and the correct classification result. Here, the classification loss can be calculated based on, for example, the probability score for each vascular pattern calculated in step S111 and the correct classification result set in advance for the learning image.

[0050] In step S113, the classification loss value calculated in step S112 is fed back to the CNN that constitutes the shallow layer feature extraction unit 201 and the deep layer feature extraction unit 202 in the calculation processing unit 200, and the gradient value of the weight in each layer of the CNN is calculated.

[0051] In step S114, the weights in each layer of the CNN are updated based on the gradient values ​​calculated in step S113, thereby performing learning of the CNN model.

[0052] Through the processing of steps S112 to S114 described above, each layer of the CNN is trained based on the classification result of the vascular pattern for the training image obtained by inputting the training comprehensive feature F3 to the classification unit 207. As a result, the weight value in each layer of the CNN is updated so that the classification loss value for the classification result becomes smaller.

[0053] In step S115, it is determined whether or not to terminate the learning process. Here, for example, if a preset learning termination condition is met or if a user instructs to terminate learning, it is determined that the learning process should be terminated, and the learning process shown in the flowchart in Fig. 4 is terminated. On the other hand, if these conditions are not met, it is determined that the learning process should be continued without being terminated, and the process returns to step S101, and the processes from step S101 onwards, which have already been described, are repeated.

[0054] FIG. 5 is a flowchart showing the flow of the authentication process executed in step S200 of FIG.

[0055] In step S201, an image for authentication is acquired. As described above, a captured image including the blood vessel pattern of the body part to be authenticated of the person to be authenticated is acquired by the image acquisition unit 100, and this captured image is acquired as the image for authentication.

[0056] In steps S202 to S211, the authentication image acquired in step S201 is subjected to the same processes as steps S102 to S111 in Fig. 4. As a result, the blood vessel patterns contained in the authentication image are classified using the CNN trained by the learning process in step S100 in Fig. 3. If the probability score with all registered blood vessel patterns is below a predetermined value, the blood vessel pattern is classified as an unregistered blood vessel pattern.

[0057] In step S212, the authentication unit 208 compares the classification result obtained in step S211 with authentication information preset for each person to be authenticated, and acquires the authentication information corresponding to that person to be authenticated.

[0058] In step S213, the authentication information acquired in step S212 is output from the authentication unit 208 as the biometric authentication result for the authentication image acquired in step S201.

[0059] After the process of step S213 is performed, the authentication process shown in the flowchart of FIG. 5 ends.

[0060] Next, the adjustment process of the deep feature F2 by the feature alignment unit 204, which is performed in steps S107 and S207 in FIGS. 4 and 5, respectively, will be described with reference to FIG.

[0061] When a deep layer feature F2 consisting of information on a predetermined width, height, and number of channels is input to the feature alignment unit 204, the feature alignment unit 204 adjusts these values ​​by arithmetic processing using a neural network consisting of multiple layers so that they match the width, height, and number of channels in the shallow layer feature F1, respectively. As a result, as shown in Fig. 6, it is possible to acquire, as an adjusted deep layer feature F2', information that has the same number of dimensions (width, height, and number of channels) as the shallow layer feature F1 and that represents the same characteristics of the vascular pattern as the deep layer feature F2.

[0062] In the related feature generation unit 205, by using the adjusted deep feature F2′ obtained by the feature alignment unit 204 in this manner, it becomes possible to efficiently generate the related feature Fr.

[0063] Next, the generation process of the related feature Fr by the related feature generator 205, which is performed in steps S108 and S208 in FIGS. 4 and 5, respectively, will be described with reference to FIGS.

[0064] When the shallow layer feature F1 and the adjusted deep layer feature F2' are input to the related feature generation unit 205, the related feature generation unit 205 generates information reflecting the relationship between these features through calculation processing using a neural network consisting of multiple layers. The information obtained in this manner is output as a related feature Fr based on the shallow layer feature F1 and the adjusted deep layer feature F2'. This makes it possible to generate a related feature Fr that represents the relationship between structural features and local features in the vascular patterns of the training image and the authentication image, as shown in Figure 7.

[0065] Fig. 8 is a diagram showing an example of the functional configuration of the related feature generating unit 205. As shown in Fig. 8, the related feature generating unit 205 is configured by combining neural networks 2051, 2052, 2053, and 2054, for example.

[0066] The adjusted deep feature F2′ input to the related feature generation unit 205 is input to the neural network 2051. The neural network 2051 generates a query 2061 based on the adjusted deep feature F2′.

[0067] The shallow layer feature F1 input to the related feature generation unit 205 is input to neural networks 2052 and 2053. The neural network 2052 generates a key 2062 based on the shallow layer feature F1. The neural network 2053 generates a value 2063 based on the shallow layer feature F1.

[0068] The related feature generator 205 generates a correlation matrix 2064 by calculating the inner product of the query 2061 and the key 2062. The correlation matrix 2064 represents the relationship between the structural features of the vascular pattern represented by the shallow feature F1 and the local features of the vascular pattern represented by the adjusted deep feature F2', and can highlight relatively important areas within the vascular pattern. This correlation matrix 2064 indicates important information for capturing non-local features of the vascular pattern.

[0069] The related feature generation unit 205 calculates an attention matrix 2065 that collectively encodes the information represented by the shallow feature F1 and the adjusted deep feature F2′ by multiplying the correlation matrix 2064 by the value 2063. The calculated attention matrix 2065 is input to the neural network 2054.

[0070] The neural network 2054 adjusts the attention matrix 2065 so that the number of dimensions (width, height, and number of channels) of the attention matrix 2065 matches the shallow layer feature F1, and outputs the adjustment result as the related feature Fr.

[0071] FIG. 9 is a diagram showing an example of deep feature F2 extracted by the deep feature extraction unit 202 in steps S103 and S203 in FIGS. 4 and 5, respectively. As shown in FIG. 9, the deep feature F2 includes a region F2h with a high feature value and a region F2l with a low feature value. Region F2h indicates a region having a notable local feature in a learning image or an authentication image, and is an important region when classifying or matching vascular patterns. On the other hand, region F2l indicates a region with low attention or importance in a learning image or an authentication image. In the deep feature F2, region F2h generally corresponds to the foreground region, and region F2l corresponds to the background region.

[0072] Fig. 10 is a diagram showing an example of shallow layer features F1 extracted by the shallow layer feature extraction unit 201 in steps S102 and S202 in Fig. 4 and Fig. 5, respectively. The shallow layer features F1 are configured to include a region F1h with high feature values, as shown in Fig. 10, for example. The region F1h indicates a region having notable structural features in the learning image or authentication image, and is an important region when classifying and matching vascular patterns.

[0073] 11 is a diagram showing an example of the related feature Fr generated by the related feature generation unit 205 in steps S108 and S208 in FIGS. 4 and 5, respectively. As shown in FIG. 11, the related feature Fr includes a region Frh with a high feature value, a region Frl with a low feature value, and a region Frm with a medium feature value. The regions Frh and Frl indicate regions corresponding to the foreground and background regions, respectively, in the learning image or the authentication image. On the other hand, the region Frm indicates a region where the structural features and local features of the vascular pattern are highly related to each other.

[0074] The related feature Fr shown in Fig. 11 includes both the local feature information of the deep feature F2 shown in Fig. 9 and the structural feature information of the shallow feature F1 shown in Fig. 10. Therefore, by using the related feature Fr, it is possible to more accurately identify the vascular pattern than when only the shallow feature F1 or the deep feature F2 is used. Therefore, it is possible to improve the authentication accuracy when a missing region exists in the authentication image obtained by capturing the body part to be authenticated.

[0075] According to the embodiment of the present invention described above, the following advantageous effects can be obtained.

[0076] (1) The biometric authentication device 1 is a device that performs biometric authentication based on the blood vessel pattern of a predetermined part of the human body to be authenticated. This biometric authentication device 1 includes an image acquisition unit 100 that acquires an image of the area to be authenticated, a shallow feature extraction unit 201 that extracts shallow feature F1 (first feature) representing structural features of the vascular pattern from the image acquired by the image acquisition unit 100, a deep feature extraction unit 202 that extracts deep feature F2 (second feature) representing local features of the vascular pattern based on the shallow feature F1 extracted by the shallow feature extraction unit 201, a feedback mechanism (a memory unit 203, a feature alignment unit 204, and a related feature generation unit 205) that feeds back the deep feature F2 extracted by the deep feature extraction unit 202, and a classification unit 207 that classifies the vascular pattern using a calculation result (an overall feature F3) obtained by inputting information (a composite feature Fc) based on the shallow feature F1 and the deep feature F2 fed back by the feedback mechanism into the deep feature extraction unit 202. Then, the authentication unit 208 performs biometric authentication based on the classification result of the blood vessel pattern by the classification unit 207. As a result, in biometric authentication using the blood vessel pattern of the human body, it is possible to improve the authentication accuracy when a missing region exists in an image of the body part to be authenticated.

[0077] (2) The feedback mechanism includes a related feature generation unit 205 that generates a related feature Fr representing the relationship between structural features and local features in a vascular pattern based on the shallow feature F1 and the fed-back deep feature F2. The deep feature extraction unit 202 extracts a comprehensive feature F3 (third feature) representing the comprehensive feature of the vascular pattern based on a combined feature Fc calculated by the adder 206 by adding the related feature Fr generated by the related feature generation unit 205 to the shallow feature F1. The classification unit 207 classifies the vascular pattern based on the comprehensive feature F3 extracted by the deep feature extraction unit 202. This configuration allows for more accurate classification of vascular patterns by taking into account both the local features and structural features of the vascular pattern.

[0078] (3) The feedback mechanism includes a feature alignment unit 204 that adjusts the deep feature F2 to match the shallow feature F1. The related feature generation unit 205 generates the related feature Fr using the adjusted deep feature F2' that corresponds to the deep feature F2 adjusted by the feature alignment unit 204. This configuration makes it possible to easily and efficiently generate the related feature Fr.

[0079] (4) The shallow feature F1 and the deep feature F2 each have information on a predetermined number of dimensions. The feature alignment unit 204 adjusts the number of dimensions of the deep feature F2 to match the number of dimensions of the shallow feature F1, thereby generating an adjusted deep feature F2'. In this manner, the related feature generation unit 205 can adjust the deep feature F2 in a form appropriate for generating the related feature Fr, thereby generating an adjusted deep feature F2'.

[0080] (5) The shallow feature extraction unit 201 and the deep feature extraction unit 202 are each configured by a predetermined layer in a trained CNN, and the CNN layer corresponding to the shallow feature extraction unit 201 is located lower than the CNN layer corresponding to the deep feature extraction unit 202. This makes it possible to realize the shallow feature extraction unit 201 and the deep feature extraction unit 202 that can easily extract the shallow feature F1 and the deep feature F2 from an image, respectively.

[0081] (6) The biometric authentication device 1 includes a learning control unit 300 that performs CNN learning. In the learning process shown in the flowchart of FIG. 4, the learning control unit 300 performs a primary CNN calculation process (steps S102 to S104) to obtain shallow features for learning F1 obtained by inputting a predetermined learning image including a blood vessel pattern to the shallow feature extraction unit 201 and deep features for learning F2 obtained by inputting the shallow features for learning F1 to the deep feature extraction unit 202; an association process (step S108) to generate related features for learning Fr that represent the association between the shallow features for learning F1 and the deep features F2 obtained by the primary CNN calculation process; and a deep feature extraction process (step S109) to extract the related features for learning Fr generated by the association process from the shallow features for learning F1. The biometric authentication device 1 executes a synthesis process (step S109) to calculate a synthesis feature Fc for learning added to the layer feature F1, a secondary CNN operation process (step S110) to input the synthesis feature Fc for learning calculated by the synthesis process to the deep feature extraction unit 202 to obtain an overall feature F3 for learning, and a learning process (steps S112 to S114) to learn each layer of the CNN corresponding to the shallow feature extraction unit 201 and the deep feature extraction unit 202 based on the classification result of the vascular pattern for the learning image obtained by inputting the overall feature F3 for learning calculated by the secondary CNN operation process to the classification unit 207. In this way, the trained CNN required to realize the shallow feature extraction unit 201 and the deep feature extraction unit 202 can be obtained at any timing by the processing of the biometric authentication device 1 itself.

[0082] (7) In the biometric authentication device 1, the image acquisition unit 100 and the calculation processing unit 200 execute the authentication process shown in the flowchart of Fig. 5. In this authentication process, the image acquisition unit 100 acquires an image of the body part to be authenticated (step S201), the shallow layer feature extraction unit 201 extracts shallow layer feature F1 representing structural features of the vascular pattern from the image (step S202), and the deep layer feature extraction unit 202 extracts deep layer feature F2 representing local features of the vascular pattern based on the extracted shallow layer feature F1 (step S203). The extracted deep layer feature F2 is then fed back by a feedback mechanism consisting of the storage unit 203, feature alignment unit 204, and related feature generation unit 205 (steps S205 to S209). Based on the shallow layer feature F1 and the fed-back deep layer feature F2, the deep layer feature extraction unit 202 extracts a comprehensive feature F3 representing the comprehensive features of the vascular pattern (step S210). The classification unit 207 classifies the vascular pattern based on the comprehensive feature F3 (step S211). Based on the classification results of the vascular pattern thus obtained, the authentication unit 208 performs biometric authentication (steps S212 and S213). This configuration improves authentication accuracy when a missing region exists in an image of the body part to be authenticated during biometric authentication using the vascular pattern of the human body.

[0083] It goes without saying that the present invention is not limited to the above-described embodiments and can be modified in various ways without departing from the spirit of the present invention. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those having all of the described configurations. Furthermore, it is possible to add, delete, or replace part of the configuration of the above-described embodiments with other configurations.

[0084] Furthermore, the above-mentioned configurations, functional units, processing units, processing means, etc. may be partially or entirely implemented in hardware, for example, by designing them as integrated circuits. The above-mentioned configurations, functions, etc. may also be implemented in software, with a processor interpreting and executing a program that implements each function. Information such as the programs, tables, and files that implement each function can be stored in a memory, a hard disk, a recording device such as an SSD, an IC card, an SD card, a DVD, or other recording media.

[0085] In addition, in the above figures, the control lines and information lines shown are those that are considered necessary for explanation, and do not necessarily show all the control lines and information lines that are actually implemented. For example, it can be considered that almost all components are actually connected to each other. [Explanation of symbols]

[0086] 1: Biometric authentication device 2: Part to be authenticated 10: Processor 20: Memory 30: Recording device 40: Input / output interface 50: Input device 60: Output device 100: Image acquisition unit 200: Processing unit 201: Shallow feature extraction unit 202: Deep feature extraction unit 203: Storage section 204: Feature alignment unit 205: Related feature generation unit 206: Addition section 207: Classification Department 208: Authentication section 300: Learning control unit

Claims

1. A device for performing biometric authentication using blood vessel patterns at a predetermined part of a human body to be authenticated, an image acquisition unit that acquires an image of the authentication target body part; a first feature extraction unit that extracts a first feature representing a structural feature of the vascular pattern from the image acquired by the image acquisition unit; a second feature extraction unit that extracts a second feature representing a local feature of the vascular pattern based on the first feature extracted by the first feature extraction unit; a feedback mechanism that feeds back the second feature amount extracted by the second feature amount extraction unit; a classification unit that classifies the vascular pattern using a calculation result obtained by inputting information based on the first feature amount and the second feature amount fed back by the feedback mechanism to the second feature amount extraction unit, A biometric authentication device that performs the biometric authentication based on the classification result of the blood vessel pattern by the classification unit.

2. The biometric authentication device according to claim 1, the feedback mechanism includes a related feature generating unit that generates a related feature representing a relationship between a structural feature and a local feature in the vascular pattern based on the first feature and the second feature that has been fed back; the second feature extraction unit extracts a third feature representing a comprehensive feature of the vascular pattern based on a combined feature obtained by adding the related feature generated by the related feature generation unit to the first feature; The classification unit classifies the vascular pattern based on the third feature extracted by the second feature extraction unit.

3. The biometric authentication device according to claim 2, the feedback mechanism includes a feature alignment unit that adjusts the second feature to match the first feature, The related feature generation unit generates the related feature using the second feature adjusted by the feature alignment unit.

4. The biometric authentication device according to claim 3, the first feature amount and the second feature amount each have information of a predetermined number of dimensions; The feature alignment unit adjusts the number of dimensions of the second feature to match the number of dimensions of the first feature.

5. The biometric authentication device according to any one of claims 1 to 4, the first feature extraction unit and the second feature extraction unit are each configured by a predetermined layer in a trained CNN (Convolutional Neural Network), A biometric authentication device, wherein a layer of the CNN corresponding to the first feature extraction unit is lower than a layer of the CNN corresponding to the second feature extraction unit.

6. The biometric authentication device according to claim 5, A learning control unit that performs learning of the CNN, The learning control unit a first-order CNN calculation process for obtaining first learning features obtained by inputting a predetermined learning image including the vascular pattern to the first feature extraction unit and second learning features obtained by inputting the first learning features to the second feature extraction unit; an association process for generating a training-use associated feature that represents an association between the first training feature and the second training feature obtained by the primary CNN operation process; a synthesis process for calculating synthetic training features by adding the training related features generated by the relevance process to the first training features; a second-order CNN calculation process for inputting the synthetic feature for training calculated by the synthesis process into the second feature extraction unit to obtain a third feature for training; a learning process for learning each layer of the CNN corresponding to the first feature extraction unit and the second feature extraction unit, respectively, based on the classification result of the vascular pattern for the training image obtained by inputting the third training feature obtained by the secondary CNN arithmetic process to the classification unit.

7. A method for biometric authentication based on a blood vessel pattern at a predetermined part of a human body to be authenticated, comprising: Acquire an image of the area to be authenticated; extracting a first feature amount representing a structural feature of the vascular pattern from the image; extracting a second feature amount representing a local feature of the vascular pattern based on the first feature amount; feeding back the second feature amount; extracting a third feature quantity representing a comprehensive feature of the vascular pattern based on the first feature quantity and the fed-back second feature quantity; classifying the vascular pattern based on the third feature amount; A biometric authentication method for performing the biometric authentication based on the classification result of the blood vessel pattern.

Images