Biometric authentication processing program, biometric authentication processing device, and biometric authentication processing method

Abstract

The present invention improves the accuracy of biometric authentication in a case where the posture of a living body has changed.　A biometric authentication processing device (10) is provided with a control unit (11) and an imaging unit (13). The imaging unit (13) captures a biometric image (a1) of a part of the human body of a user. The control unit (11) acquires the biometric image (a1), performs multiple different biometric feature extractions on the biometric image (a1), and generates multiple biometric feature images (b1, b2, b3) having different characteristics. Further, the control unit (11) synthesizes the multiple biometric feature images (b1, b2, b3) to generate a synthesized biometric feature image (c1). Additionally, the control unit (11) generates biometric feature data (d1) from the synthesized biometric feature image (c1) using a predetermined calculation, and performs identity authentication processing on the basis of a degree of similarity between the generated biometric feature data (d1) and registered biometric feature data (d2).

Description

Biometric authentication processing program, biometric authentication processing device, and biometric authentication processing method

[0001] The present invention relates to a biometric authentication processing program, a biometric authentication processing device, and a biometric authentication processing method.

[0002] In recent years, identity verification has been performed in various situations, and biometric authentication using biometric features of parts of the human body, such as fingerprints, palm vein patterns, and faces, has become widespread. Related technologies include, for example, a technology that performs biometric authentication by removing background from biometric images using a spectral image of a captured biometric image and a spectral image of unwanted external light. Another technology has been proposed that performs biometric authentication by transferring a first image of a biometric part captured without activating an illumination source and a second image captured with an illumination source activated to a neural network. Another technology has been proposed that divides a palm image into sub-images and performs biometric authentication based on feature vectors determined from each sub-image. Another technology has been proposed that performs biometric authentication based on feature values ​​of a first biometric image of a user captured over a first period and feature values ​​of second and third biometric images of the user captured over a second period.

[0003] Japanese Patent Application Publication No. 2022-68055 International Publication No. 2021 / 205016 US Patent Application Publication No. 2019 / 0392189 Japanese Patent Application Publication No. 2022-177762

[0004] Conventional biometric authentication has low robustness (stability against external disturbances) and is easily affected by changes in the body's posture. This can lead to false rejection errors, where a person is not identified as the person in question, and the accuracy of biometric authentication is reduced due to changes in the body's posture.

[0005] In one aspect, the present invention aims to provide a biometric authentication processing program, a biometric authentication processing device, and a biometric authentication processing method that improve biometric authentication accuracy when a change in the posture of a living subject occurs.

[0006] To solve the above problems, a biometric authentication processing program is provided. The biometric authentication processing program causes a computer to execute the following processes: extracting a plurality of different biometric features from a biometric image to generate a plurality of biometric feature images having different characteristics, synthesizing the plurality of biometric feature images to generate a composite biometric feature image, generating biometric feature data from the composite biometric feature image based on a predetermined calculation, and performing authentication processing based on the similarity between the biometric feature data and registered biometric feature data.

[0007] In order to solve the above problems, there is also provided a biometric authentication processing device that performs control similar to that of the biometric authentication processing program.Furthermore, in order to solve the above problems, there is provided a biometric authentication processing method in which a computer performs control similar to that of the biometric authentication processing program.

[0008] According to one aspect, it is possible to improve the accuracy of biometric authentication when a change in posture of a living body occurs. The above and other objects, features, and advantages of the present invention will become apparent from the following description taken in conjunction with the accompanying drawings which illustrate preferred embodiments of the present invention by way of example.

[0009] 1 is a diagram for explaining an example of a biometric authentication processing device. FIG. 1 is a diagram for explaining an example of functional blocks of a biometric authentication processing device. FIG. 1 is a diagram for explaining an example of processing from acquisition of a biometric image to generation of a biometric feature vector. FIG. 2 is a flowchart for explaining an example of registration processing of a biometric feature vector. FIG. 3 is a flowchart for explaining an example of matching processing of a biometric feature vector. FIG. 4 is a flowchart for explaining an example of pre-learning operation of a neural network. FIG. 1 is a diagram for explaining an example of generation processing of a weighted composite biometric feature image. FIG. 2 is a diagram for explaining an example of change in combination of operation parameters of a biometric feature extraction filter. FIG. 3 is a diagram for explaining an example of change in combination of operation parameters of a biometric feature extraction filter. FIG. 4 is a flowchart for explaining an example of processing for changing combination of operation parameters based on detection results of posture change. FIG. 5 is a diagram for explaining an example of functional blocks of a biometric authentication processing device. FIG. 6 is a diagram for explaining an example of operation of a biometric feature extraction filter and a biometric feature expansion processing unit. FIG. 7 is a diagram for explaining an example of simulation results.

[0010] Hereinafter, the present embodiment will be described with reference to the drawings. In this specification and the drawings, elements having substantially the same functions are designated by the same reference numerals, and redundant description may be omitted.

[0011] 1 is a diagram illustrating an example of a biometric authentication processing device. The biometric authentication processing device 10 includes a control unit 11 and an image capturing unit 13. The image capturing unit 13 is an image sensor such as a camera that captures a biometric image of a part of a user's body (such as a fingerprint, palm vein pattern, or face). The control unit 11 performs biometric authentication processing based on the captured biometric image. The functions of the control unit 11 are realized, for example, by a processor (not shown) included in the biometric authentication processing device 10 executing a predetermined program.

[0012] [Step S1] The control unit 11 acquires a biometric image a1 captured by the imaging unit 13. In this example, the biometric image a1 is an image of a palm vein pattern. [Step S2] The control unit 11 performs multiple different biometric feature extractions on the biometric image a1 to generate multiple biometric feature images b1, b2, and b3 with different characteristics. For example, as a first biometric feature extraction, the control unit 11 generates a biometric feature image b1 by extracting line segments of the vein pattern from the biometric image a1. As a second biometric feature extraction, the control unit 11 generates a biometric feature image b2 by expanding (thickening) the extracted line segments of the vein pattern. As a third biometric feature extraction, the control unit 11 generates a biometric feature image b3 by further expanding the extracted line segments of the vein pattern than in the second biometric feature extraction.

[0013] [Step S3] The control unit 11 generates a composite biometric characteristic image c1 by combining the multiple biometric characteristic images b1, b2, and b3. [Step S4] The control unit 11 generates biometric characteristic data d1 representing the individual's biometric characteristics from the composite biometric characteristic image c1 based on a predetermined calculation.

[0014] [Step S5] The control unit 11 performs authentication processing (identity determination processing) to determine whether the user is the person in question based on the similarity between the biometric feature data d1 and the registered biometric feature data d2. In this way, the biometric authentication processing device 10 extracts multiple different biometric features from the biometric image to generate multiple biometric feature images with different characteristics, and generates biometric feature data from a composite biometric feature image that combines the multiple biometric feature images with different characteristics.

[0015] This makes it possible to determine the identity of a person using biometric feature data generated based on a composite biometric feature image that contains more information than biometric feature data generated from a single biometric feature image, thereby improving the accuracy of biometric authentication when changes in the body's posture occur.

[0016] 2 is a diagram showing an example of functional blocks of a biometric authentication processing device. The biometric authentication processing device 10 includes a control unit 11, a storage unit 12, and an image capturing unit 13. The control unit 11 includes a biometric image acquisition unit 11a, a biometric feature extraction filter 11b, a biometric feature image storage unit 11c, a composite biometric feature image generation unit 11d, a neural network 11e, and an identity determination processing unit 11f. The storage unit 12 includes at least a biometric feature vector DB (database) 12a and a determination result DB 12b.

[0017] Furthermore, biometric feature extraction filter 11b includes N (N≧2) biometric feature extraction filters with different operation parameters, and biometric feature image storage unit 11c includes N biometric feature image storage units. In the example of Fig. 2, N=3 is shown, and biometric feature extraction filter 11b includes biometric feature extraction filters 11b1, 11b2, and 11b3, and biometric feature image storage unit 11c includes biometric feature image storage units 11c1, 11c2, and 11c3.

[0018] The biometric image acquisition unit 11a acquires a biometric image of a user captured by the imaging unit 13. The biometric feature extraction filter 11b performs filtering processes on the acquired biometric image, such as removing background images, normalizing posture variations of the biometric subject, and extracting regions of interest, to extract a biometric feature image suitable for biometric authentication. Note that the biometric feature extraction filters 11b1, 11b2, and 11b3 perform filtering processes with different operation parameters to generate three biometric feature images g1, g2, and g3, each having different characteristics.

[0019] The biometric characteristic image storage unit 11c1 stores the biometric characteristic image g1, the biometric characteristic image storage unit 11c2 stores the biometric characteristic image g2, and the biometric characteristic image storage unit 11c3 stores the biometric characteristic image g3.

[0020] The composite biometric characteristic image generation unit 11d reads and combines the biometric characteristic images g1, g2, and g3 from the biometric characteristic image storage units 11c1, 11c2, and 11c3 to generate a composite biometric characteristic image gm. The generated composite biometric characteristic image gm is input to the pre-trained neural network 11e.

[0021] The neural network 11e generates a biometric feature vector v1 by converting the biometric features of the user into a numerical sequence from the composite biometric feature image gm, and stores the biometric feature vector v1 in the biometric feature vector DB 12a. The neural network 11e is, for example, a convolutional neural network (CNN).

[0022] The identity determination processing unit 11f calculates the similarity between the generated biometric feature vector v1 and one or more registered biometric feature vectors v2 already registered in the biometric feature vector DB 12a.

[0023] Since the similarity between the biometric feature vectors of the same person and the similarity between the biometric feature vector of the same person and the biometric feature vector of another person is high, the identity determination processing unit 11f determines whether the target user is the same person or another person based on the similarity. The determination result is stored in the determination result DB 12b.

[0024] Here, the identity determination processing unit 11f may calculate the similarity by any method, for example, by calculating the cosine similarity of biometric feature vectors. This method normalizes the lengths of the biometric feature vectors, calculates the cosine value of the angle between the two biometric feature vectors, and determines that the closer the cosine value is to 1, the higher the similarity, and the closer it is to -1, the lower the similarity.

[0025] Alternatively, the similarity may be calculated based on the Hamming distance obtained by normalizing the values ​​of each dimension of the biometric feature vector to a binary value of 0 or 1. For example, after normalizing to a binary value of 0 or 1, with a positive value being 1 and a negative value being 0, the Hamming distance can be obtained depending on whether the values ​​of each dimension are equal, and the magnitude of the Hamming distance can be used as the similarity of the biometric feature vectors. In the case of the Hamming distance, the closer the value is to 0, the higher the similarity, and the closer it is to the number of dimensions of the vector, the lower the similarity. Note that if the larger the value, the higher the similarity, the positive and negative of the Hamming distance may be reversed.

[0026] Furthermore, the Hamming distance calculation does not necessarily require binarization, and may be multi-valued. In this case, the values ​​of each dimension of the biometric feature vector may be expressed as multi-valued values, for example, by converting them into values ​​from 0 to 10, and the similarity of the biometric feature vectors may be determined based on the degree of agreement between those values ​​(the number of matching values) or the difference between those values ​​(the sum of the absolute values ​​of the difference between the values).

[0027] 3 is a diagram showing an example of the hardware configuration of a biometric authentication processing device. The biometric authentication processing device 10 is entirely controlled by a processor 101 having the functions of a control unit 11. A memory 102 and multiple peripheral devices are connected to the processor 101 via a bus 109. The processor 101 may be a multiprocessor. The processor 101 is, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or a DSP (Digital Signal Processor). At least some of the functions realized by the processor 101 executing a program may be realized by an electronic circuit such as an ASIC (Application Specific Integrated Circuit) or a PLD (Programmable Logic Device).

[0028] The memory 102 is used as the main storage device of the biometric authentication processing device 10, and also as the biometric feature image storage unit 11c. The memory 102 temporarily stores at least a portion of the OS (Operating System) programs and application programs executed by the processor 101. The memory 102 also stores various data used in processing by the processor 101. The memory 102 may be a volatile semiconductor storage device such as a RAM (Random Access Memory).

[0029] The peripheral devices connected to the bus 109 include a storage device 103 , a GPU (Graphics Processing Unit) 104 , an input interface 105 , an optical drive device 106 , a device connection interface 107 , and a network interface 108 .

[0030] The storage device 103 has the functions of the memory unit 12 and writes and reads data electrically or magnetically to and from a built-in recording medium. The storage device 103 is used as an auxiliary storage device for the biometric authentication processing device 10. The storage device 103 stores the OS program, application programs, and various data. Note that the storage device 103 may be, for example, a hard disk drive (HDD) or a solid state drive (SSD).

[0031] The GPU 104 is an arithmetic unit that performs image processing and is also called a graphics controller. A display 201 is connected to the GPU 104. The GPU 104 displays an image on the screen of the display 201 in accordance with instructions from the processor 101. The display 201 may be a display device using organic electroluminescence (EL) or a liquid crystal display device.

[0032] A keyboard 202 and a mouse 203 are connected to the input interface 105. The input interface 105 transmits signals sent from the keyboard 202 and the mouse 203 to the processor 101. The mouse 203 is an example of a pointing device, and other pointing devices can also be used. Examples of other pointing devices include a touch panel, a tablet, a touch pad, and a trackball.

[0033] The optical drive device 106 uses a laser beam or the like to read data recorded on an optical disc 204 or write data to the optical disc 204. The optical disc 204 is a portable recording medium on which data is recorded so that it can be read by reflected light. Examples of the optical disc 204 include a DVD (Digital Versatile Disc), a DVD-RAM, a CD-ROM (Compact Disc Read Only Memory), and a CD-R (Recordable) / RW (Re-Writable).

[0034] The device connection interface 107 is a communication interface for connecting peripheral devices to the biometric authentication processing device 10. For example, a memory device 205 or a memory reader / writer 206 can be connected to the device connection interface 107. The memory device 205 is a recording medium equipped with a function for communicating with the device connection interface 107. The memory reader / writer 206 is a device for writing data to the memory card 207 or reading data from the memory card 207. The memory card 207 is a card-type recording medium. In addition, an imaging unit 13 such as a camera for capturing an image of a part of the human body can be connected to the device connection interface 107.

[0035] The network interface 108 is connected to the network 4. The network interface 108 transmits and receives data to and from other computers or communication devices via the network 4. The network interface 108 is a wired communication interface that is connected by a cable to a wired communication device such as a switch or a router. The network interface 108 may also be a wireless communication interface that is communicatively connected by radio waves to a wireless communication device such as an access point.

[0036] The biometric authentication processing device 10 can realize the processing functions of the present invention using the hardware described above. The biometric authentication processing device 10 realizes the processing functions of the present invention by executing a program recorded on, for example, a computer-readable recording medium. The program describing the processing content to be executed by the biometric authentication processing device 10 can be recorded on various recording media.

[0037] For example, a program to be executed by the biometric authentication processing device 10 can be stored in the storage device 103. The processor 101 loads at least a part of the program in the storage device 103 into the memory 102 and executes the program. The program to be executed by the biometric authentication processing device 10 can also be recorded in a portable recording medium such as the optical disk 204, the memory device 205, or the memory card 207. The program stored in the portable recording medium becomes executable after being installed in the storage device 103 under the control of the processor 101, for example. The processor 101 can also read and execute the program directly from the portable recording medium.

[0038] 4 is a diagram illustrating an example of processing from acquiring a biometric image to generating a biometric feature vector. In the following description, biometric authentication is performed using a biometric image of a palm vein pattern. However, other parts of the human body, such as a fingerprint, face, or iris, can also be used as the biometric image.

[0039] [Step S11] The control unit 11 acquires a biometric image g0 captured by the imaging unit 13. [Step S12] The control unit 11 performs filtering on the biometric image g0 by applying biometric feature extraction filters 11b1, 11b2, and 11b3 having different operating parameters to generate multiple biometric feature images g1, g2, and g3 with different characteristics. As an example, it is assumed here that biometric feature images g1, g2, and g3 are generated by applying different filter coefficients to the same type of image filter that extracts line segments from an input image, thereby extracting line segments of different thicknesses.

[0040] The biometric feature extraction filter 11b1 has, for example, a first filter coefficient as an operation parameter, and extracts line segments indicating the running path of the vein pattern in the biometric image g0 by filtering based on the first filter coefficient. As a result, the biometric feature extraction filter 11b1 generates and outputs a biometric feature image g1 having thinning features with the thinnest line segments.

[0041] The biometric feature extraction filter 11b2 also has, for example, a second filter coefficient as an operation parameter for extracting line segments thicker than the first filter coefficient, and extracts line segments indicating the running path of the vein pattern in the biometric image g0 by edge extraction processing based on the second filter coefficient. As a result, the biometric feature extraction filter 11b2 generates and outputs a biometric feature image g2 having expanded features (thin) in which the extracted line segments are thicker than those in the biometric feature image g1.

[0042] Furthermore, the biometric feature extraction filter 11b3 has, for example, a third filter coefficient as an operation parameter for extracting line segments thicker than the second filter coefficient, and extracts line segments indicating the running path of the vein pattern in the biometric image g0 by filtering based on the third filter coefficient. As a result, the biometric feature extraction filter 11b3 generates and outputs a biometric feature image g3 having expanded features (thick) in which the extracted line segments are thicker than those in the biometric feature image g2.

[0043] Each of the multiple biometric feature images g1, g2, and g3, each with different characteristics, is a one-channel image (grayscale image) containing only black-and-white shading information. Furthermore, as the line segments are expanded from thin to thick, the biometric feature images become more resistant to posture changes (vibration fluctuations). In this example, biometric feature image g2 is more resistant to posture changes than biometric feature image g1, and biometric feature image g3 is more resistant to posture changes than biometric feature image g2. Therefore, the thicker the line segments, i.e., in the order of biometric feature images g1, g2, and g3, the lower the false rejection rate when posture changes occur during authentication. On the other hand, the thinner the line segments, i.e., in the order of biometric feature images g3, g2, and g1, the lower the false acceptance rate during authentication.

[0044] [Step S13] The control unit 11 combines the generated biometric feature images g1, g2, and g3 to generate a single combined biometric feature image gm. In this case, the control unit 11 assigns each of the multiple biometric feature images g1, g2, and g3, which are single-channel images, to a different channel of the combined biometric feature image, and copies the contents of the single-channel image to the corresponding channel of the combined biometric feature image. The combined biometric feature image gm generated by combining the images into multiple channels in this way is a multi-channel image with multiple channels. Each pixel of the multi-channel image is assigned pixel information for the corresponding pixel of each of the biometric feature images g1, g2, and g3.

[0045] [Step S14] The control unit 11 inputs the composite biometric feature image gm generated as a multi-channel image to the neural network 11e, and generates, as an output of the neural network 11e, a single biometric feature vector v1 in which the individual's biometric features are converted into a numerical sequence.

[0046] In the biometric authentication process using the neural network 11e, the biometric feature image input to the neural network has conventionally been a one-channel image. In contrast, in the biometric authentication processing device 10, a single composite biometric feature image gm, which is a composite of multiple biometric feature images with different characteristics, is input to the neural network 11e. For this reason, a neural network having a structure compatible with multi-channel images (three-channel images in this embodiment) is used as the neural network 11e.

[0047] The composite biometric feature image gm is a single multi-channel image in which biometric feature images with different characteristics are stored in multiple channels during the generation stage, and the amount of information is greater than that of a single conventional biometric feature image. For example, when three biometric feature images g1, g2, and g3 are combined to generate a three-channel composite biometric feature image gm as shown in Figure 4 and input this to the neural network 11e, the amount of image information processed by the neural network 11e is three times that of the conventional method.

[0048] In this way, applying more information to the neural network enables authentication with higher accuracy than before. That is, authentication is performed using both image information, such as biometric feature image g3, which is resistant to posture changes and has a low false rejection rate, and image information, such as biometric feature image g1, which has a low false acceptance rate. This makes it possible to improve the robustness against posture changes and also improve the accuracy of biometric authentication.

[0049] In this example, in the filtering process of the biometric feature image, the biometric features are represented by line segments, and the differences between the biometric feature extraction filters with different operating parameters are represented by differences in the thickness of the line segments, but the filtering process of the biometric feature extraction filters is not limited to this. For example, a biometric feature extraction filter that extracts biometric features as feature points instead of line segments may be used, or a biometric feature extraction filter that visualizes and extracts frequency components of the biometric image may be used.

[0050] Furthermore, a visible light image simply captured using visible light is also a three-channel image of red, green, and blue, but the configuration of this embodiment differs from the case where this captured image is input directly, or a single image to which one type of biometric feature extraction filter has been applied, into a neural network.

[0051] In this embodiment, a biometric feature extraction filter having a plurality of different operating parameters is applied to a captured biometric image to generate a plurality of biometric feature images with different characteristics. Just before inputting these biometric feature images into a neural network, these multiple biometric feature images are combined into a single composite biometric feature image. However, because the composite biometric feature image contains biometric features with a plurality of different characteristics, the amount of information input to the neural network increases.

[0052] 5 is a flowchart showing an example of a biometric feature vector registration process. [Step S21] The control unit 11 acquires a biometric image captured by the imaging unit 13. [Step S22] The control unit 11 executes a repeated loop on the acquired biometric image, applying n biometric feature extraction filters with different operating parameters to generate n biometric feature images with different characteristics (n=1, 2, 3, ...).

[0053] [Step S23] The control unit 11 synthesizes the generated n biometric feature images to generate a single composite biometric feature image. For example, the control unit 11 associates the n biometric feature images with each channel of the composite biometric feature image, which is a multi-channel image. The control unit 11 then copies the n biometric feature images to each of the n different channels to generate the composite biometric feature image.

[0054] [Step S24] The control unit 11 inputs the generated composite biometric feature image into the neural network 11e to generate a biometric feature vector v1. [Step S25] The control unit 11 stores and registers the generated biometric feature vector v1 in the biometric feature vector DB 12a for use in subsequent identity determination processing. For example, the biometric feature vector v1 is registered in association with a user ID that identifies the user being photographed. Note that the storage location of the biometric feature vector v1 is not limited to the biometric feature vector DB 12a, and it may also be stored, for example, on an IC card owned by each individual.

[0055] 6 is a flowchart showing an example of a biometric feature vector matching process. [Step S31] The control unit 11 acquires a biometric image captured by the imaging unit 13. [Step S32] The control unit 11 executes a repeated loop on the acquired biometric image, applying n biometric feature extraction filters with different operating parameters to generate n biometric feature images with different characteristics (n=1, 2, 3).

[0056] [Step S33] The control unit 11 synthesizes the generated n biometric feature images to generate one synthesized biometric feature image. [Step S34] The control unit 11 inputs the generated synthesized biometric feature image to the neural network 11e to generate a biometric feature vector v1.

[0057] [Step S35] The control unit 11 compares the generated biometric feature vector v1 with a registered biometric feature vector v2 previously registered in the biometric feature vector DB 12a according to the registration flow of Figure 5 to calculate the similarity. For example, in the case of one-to-one authentication, the registered biometric feature vector v2 corresponding to the user ID acquired along with the biometric image is obtained from the biometric feature vector DB 12a, and the similarity with the biometric feature vector v1 is calculated. For example, if the calculated similarity is greater than a preset identity determination threshold, the user is determined to be the identity of the person, and if it is less than the identity determination threshold, the user is determined to be a non-identity. The determination result is stored in the determination result DB 12b, and various services are activated based on the determination result.

[0058] 7 is a flowchart showing an example of the pre-learning operation of the neural network. The captured biometric image DB 12c, the training composite biometric feature image data set DB 12d, the test composite biometric feature image data set DB 12e, the basic trained neural network DB 12f, and the trained neural network weight parameter DB 12g shown in FIG. 7 are included in the storage unit 12 shown in FIG.

[0059] [Step S41] The control unit 11 generates composite biometric feature images for all biometric images in the captured biometric image DB 12c. [Step S42] The control unit 11 divides a group of composite biometric feature images consisting of multiple composite biometric feature images into a training composite biometric feature image dataset and a test composite biometric feature image dataset. The control unit 11 then stores the training composite biometric feature image dataset in the training composite biometric feature image dataset DB 12d and stores the test composite biometric feature image dataset in the test composite biometric feature image dataset DB 12e.

[0060] [Step S43] The control unit 11 reads the basic trained neural network from the basic trained neural network DB 12f. [Step S44] The control unit 11 inputs the training synthetic biometric feature image dataset read from the training synthetic biometric feature image dataset DB 12d into the neural network (the read basic trained neural network).

[0061] [Step S45] The control unit 11 evaluates the output of the neural network and feeds back the evaluation results to the weight parameters of each layer that constitutes the neural network.

[0062] [Step S46] The control unit 11 inputs the test synthetic biometric feature image data set read from the test synthetic biometric feature image data set B12e into the neural network to which the weight parameters have been fed back for verification. A learning loop that repeats steps S44, S45, and S46 is executed until learning is complete. [Step S47] The control unit 11 stores the weight parameters of the new neural network that has completed learning in the trained neural network weight parameter DB 12g.

[0063] As described above, a composite biometric feature image obtained by combining a plurality of biometric feature images is used as input during pre-training of the neural network (training of weight parameters of each layer constituting the neural network).

[0064] For example, the biometric images registered in the captured biometric image DB 12c are images taken of different people, and each registered biometric image is associated with a person ID that identifies that person. In the above learning loop, in step S44, a composite biometric feature image based on the biometric images of a total of m people is input to the neural network. This allows the neural network to learn a classifier that classifies the input images into m people. When the neural network generated in this way is used in biometric authentication processing, multiple numerical values ​​output from a predetermined intermediate layer in the neural network are used as a biometric feature vector.

[0065] Next, an embodiment in which a plurality of biometric feature images having different characteristics are weighted and then combined will be described with reference to FIGS. 8 and 9. FIG. 8 is a diagram showing an example of functional blocks of a biometric authentication processing device. The biometric authentication processing device 10-1 shown in FIG. 8 includes a control unit 11-1, a storage unit 12, and an image capturing unit 13. The control unit 11-1 includes a weighted combined biometric feature image generation unit 11g instead of the combined biometric feature image generation unit 11d shown in FIG. 2, but the other configuration is the same as that shown in FIG. 2.

[0066] The weighted composite biometric feature image generator 11g weights and combines a plurality of biometric feature images having different characteristics to generate a weighted composite biometric feature image gm1. The weighting process will be described using an example in which the number of generated biometric feature images is three, each of which is a one-channel image (grayscale image).

[0067] The weighted composite biometric characteristic image generating unit 11g assigns weight values ​​to each of the biometric characteristic images g1, g2, and g3 when generating the composite biometric characteristic image gm1 by combining the biometric characteristic images g1, g2, and g3.

[0068] If the weighting value is w, which is uniform for all n biometric feature images, then w = 1 / n is set so that when all pixels are at their maximum values, the pixel value of the composite biometric feature image does not exceed the maximum value even when they are added together. This value is set because there are n images to be composited, and they are composited by addition. Therefore, when three biometric feature images g1, g2, and g3 are composited, w = 1 / 3 is set.

[0069] Any value not exceeding 1 / 3 may be used. Although equal weights are applied to all biometric feature images in this example, the weights applied to each image may be set unequal. For example, the weight w1 applied to biometric feature image g1 may be set to w1 = 0.2, the weight w2 applied to biometric feature image g2 may be set to w2 = 0.3, and the weight w3 applied to biometric feature image g3 may be set to w3 = 0.5. In this case, w1 + w2 + w3 = 1.0, and the sum of the weighted and added pixel values ​​will not exceed the maximum value.

[0070] When the weights are set unevenly, for example, the highest weight can be set for the biometric characteristic image g3 with the thickest line segment, and the lowest weight can be set for the biometric characteristic image g1 with the thinnest line segment. By setting the weights in this way, biometric authentication processing can be performed that is more robust against posture changes.

[0071] In addition, although the weight values ​​are given as decimal numbers in this example, integer arithmetic may be used instead of decimal numbers to speed up processing. In this case, by dividing the biometric feature images before composition by the number of images (=3) in advance, the maximum value can be prevented from being exceeded even when additive composition is performed.

[0072] 9 is a diagram for explaining an example of the process of generating a weighted composite biometric feature image. [Step S51] The control unit 11-1 acquires a biometric image g0 captured by the imaging unit 13.

[0073] [Step S52] The control unit 11-1 performs filtering on the biometric image g0 by applying biometric feature extraction filters 11b1, 11b2, and 11b3 having different operation parameters to generate a plurality of biometric feature images g1, g2, and g3 having different characteristics.

[0074] [Step S53] The control unit 11-1 weights and combines the generated biometric feature images g1, g2, and g3 to generate a single weighted composite biometric feature image gm1. For example, a fixed, uniform weight value w (=0.3333...) is applied to each of the biometric feature images g1, g2, and g3. In this case, the weighted composite biometric feature image gm1 is generated in which overlapping areas of the three biometric feature images g1, g2, and g3 are represented by dark lines (lines with large pixel values) and areas with less overlap are represented by light lines (lines with small pixel values). Note that the composite biometric feature image gm1 after additive combination is a grayscale image (one-channel image) with varying shading.

[0075] [Step S54] The control unit 11 inputs the weighted composite biometric feature image gm1 to the neural network 11e1 and obtains a biometric feature vector v1 as the output of the neural network 11e1. Note that the composite biometric feature image gm1 is a one-channel image, which differs from the configuration shown in FIG. 2. A neural network with a structure for one-channel images is used as the neural network 11e1. For example, a neural network with an existing structure that receives a single biometric feature image generated from the original biometric image, rather than a composite biometric feature image, can be used as the neural network 11e1.

[0076] In this way, the control unit 11-1 weights and combines biometric feature images with different characteristics to generate a one-channel weighted composite biometric feature image, and inputs the weighted composite biometric feature image into a neural network to generate a biometric feature vector. This improves robustness against posture changes and improves biometric authentication accuracy.

[0077] Next, an embodiment in which a combination of operation parameters to be set in a biometric feature extraction filter is changed in accordance with a change in posture will be described with reference to Figures 10 to 12. In the case of a change in posture that indicates fluctuations in a living body, for example, in the case of a palm, a physical guide may be provided so that when the palm is held over the imaging unit 13, the palm is always held in the same posture.

[0078] When such a guidance guide is provided, posture fluctuations tend to be small. Conversely, when no physical guidance guide is provided and the user is allowed to freely hold their palm over the image capture unit 13, posture fluctuations tend to be large. Which operation mode is adopted depends on the system.

[0079] In this way, when whether the posture variation is "small" or "large" varies depending on the operating mode, in this embodiment, multiple operating parameters to be set in each of multiple biometric feature extraction filters are changed depending on the magnitude of the posture variation of the living body.

[0080] 10 and 11 are diagrams showing examples of changes in the combination of operating parameters of the biometric feature extraction filter. Fig. 10 shows an operation mode sy1 when the rate of change in posture change is less than a predetermined rate of change.

[0081] In the operating mode sy1, the control unit 11 sets the first operation parameter to the biometric feature extraction filter 11b1, the second operation parameter to the biometric feature extraction filter 11b2, and the third operation parameter to the biometric feature extraction filter 11b3.

[0082] The biometric feature extraction filter 11b1 extracts line segments indicating the running path of the vein pattern in the biometric image g0 based on the first operation parameter, thereby generating and outputting a biometric feature image g11 having the thinnest line segments.

[0083] Furthermore, the biometric feature extraction filter 11b2 extracts line segments indicating the running path of the vein pattern in the biometric image g0 based on the second operation parameter, thereby generating and outputting a biometric feature image g12 in which the extracted line segments are thicker than those in the biometric feature image g11.

[0084] Furthermore, the biometric feature extraction filter 11b3 extracts line segments indicating the running path of the vein pattern in the biometric image g0 based on the third operation parameter, thereby generating and outputting a biometric feature image g13 in which the extracted line segments are thicker than those in the biometric feature image g12.

[0085] 11 shows an operation mode sy2 in which the rate of posture change is equal to or greater than a predetermined rate. In operation mode sy2, the control unit 11 sets the first operation parameter to biometric feature extraction filter 11b1, the third operation parameter to biometric feature extraction filter 11b2, and the fourth operation parameter to biometric feature extraction filter 11b3.

[0086] The biometric feature extraction filter 11b1 generates and outputs a biometric feature image g11 of line segments indicating the running path of the vein pattern of the biometric image g0, as described above, based on the first operation parameter.

[0087] Furthermore, the biometric feature extraction filter 11b2 generates and outputs a biometric feature image g13, whose line segments are thicker than those of the biometric feature image g12, from the biometric image g0 based on the third operation parameter, as described above.

[0088] Furthermore, the biometric feature extraction filter 11b3 extracts line segments indicating the running path of the vein pattern in the biometric image g0 based on the fourth operation parameter, thereby generating and outputting a biometric feature image g14 in which the extracted line segments are thicker than those in the biometric feature image g13.

[0089] Here, in the operating mode sy1 shown in FIG. 10, the rate of change in posture variation is less than a predetermined rate and the posture variation is small, so that generally the fluctuation of the line segments extracted by the biometric feature extraction filter is small.

[0090] Therefore, the control unit 11 selects a combination of operation parameters (first, second, and third operation parameters) that preferentially extracts thin line segments over thick line segments from the biometric image g0. Then, each operation parameter in this combination is applied to each of the biometric feature extraction filters 11b1, 11b2, and 11b3. As a result, the operation mode sy1, which has small posture variation, is less robust against posture variation than the operation mode sy2, but it is possible to perform biometric authentication with a low false acceptance rate.

[0091] On the other hand, in the operating mode sy2 shown in FIG. 11, the rate of change in posture variation is equal to or greater than a predetermined rate, and therefore the posture variation is large, so that the fluctuation of the line segments extracted by the biometric feature extraction filter generally becomes large.

[0092] Therefore, the control unit 11 selects a combination of operation parameters (first, third, and fourth operation parameters) that preferentially extracts thick line segments from the biometric image g0. Then, each operation parameter from this combination is applied to the biometric feature extraction filters 11b1, 11b2, and 11b3, respectively. This allows the operation mode sy2, which has large posture fluctuations, to absorb fluctuations in line segments due to posture fluctuations. Therefore, compared to the operation mode sy1, this mode is more robust against posture fluctuations, enabling biometric authentication with a low false rejection rate when posture fluctuations occur.

[0093] In this way, the control unit 11 changes the combination of operation parameters to be set in the biometric feature extraction filter depending on the operating mode in which posture variation is small (for example, a system in which a guide is installed) or in which posture variation is large (for example, a system in which a guide is not installed). This allows biometric authentication processing to be performed in accordance with posture variation, thereby improving the accuracy of biometric authentication.

[0094] 12 is a flowchart showing an example of a process for changing the combination of movement parameters based on the result of detecting a posture change. [Step S61] The control unit 11 calculates the rate of change of posture change corresponding to the acquired biometric image.

[0095] [Step S62] The control unit 11 compares the calculated fluctuation rate with a predetermined fluctuation rate stored in advance. If it detects that the fluctuation rate is less than the predetermined fluctuation rate, it proceeds to the processing of step S63, and if the fluctuation rate is equal to or greater than the predetermined fluctuation rate, it proceeds to the processing of step S64.

[0096] [Step S63] If the variation rate is less than a predetermined variation rate, it can be said that the posture variation is small. In this case, the control unit 11 sets the first combination of operation parameters (filter characteristics) to multiple biometric feature extraction filters.

[0097] [Step S64] If the variation rate is equal to or greater than a predetermined variation rate, it can be said that the posture variation is large. In this case, the control unit 11 sets the second combination of movement parameters in multiple biometric feature extraction filters.

[0098] For example, the first combination is the first, second, and third motion parameters described above, and the second combination is the first, third, and fourth motion parameters described above. The second combination includes motion parameters with a larger expansion rate for expanding the line segments of the biometric image than the first combination. Therefore, when posture variation is small, multiple biometric feature extraction filters set with the motion parameters of the first combination are applied, thereby increasing the definition of the line segments of the biometric feature image. Furthermore, when posture variation is large, multiple biometric feature extraction filters set with the motion parameters of the second combination are applied, thereby absorbing fluctuations in the line segments due to posture variation.

[0099] In this way, the control unit 11 can autonomously detect posture changes and change the combination of operation parameters based on the posture change detection results. This allows the operation parameters to be set in the biometric feature extraction filter to be variably set adaptively to the detected posture changes, thereby improving operability and biometric authentication accuracy.

[0100] Next, an embodiment will be described in which a predetermined biometric feature extraction filter among a plurality of biometric feature extraction filters is replaced with a biometric feature expansion processor. FIG. 13 is a diagram showing an example of functional blocks of a biometric authentication processing device. The biometric authentication processing device 10-2 includes a control unit 11-2, a storage unit 12, and an imaging unit 13. The control unit 11-2 includes a biometric image acquisition unit 11a, a biometric feature extraction filter 11b1, a biometric feature image storage unit 11c, a biometric expansion processor 11h, a composite biometric feature image generator 11d, a neural network 11e, and an identity determination processor 11f. The biometric expansion processor 11h includes biometric feature expansion processors 11h1 and 11h2.

[0101] The biometric feature extraction filter 11b1 performs, for example, edge extraction filtering on the biometric image acquired by the biometric image acquisition unit 11a to generate and output a biometric feature image g1 of line segments of the vein pattern. The biometric feature image storage unit 11c1 stores the biometric feature image g1.

[0102] The biometric feature expansion processing unit 11h1 reads the biometric feature image g1 from the biometric feature image storage unit 11c1, and generates the biometric feature image g2 by expanding the line segments of the biometric feature image g1 by a first expansion rate. The biometric feature image storage unit 11c2 stores the biometric feature image g2.

[0103] The biometric feature expansion processing unit 11h2 reads the biometric feature image g2 from the biometric feature image storage unit 11c2 and expands the line segments of the biometric feature image g2 by a second expansion rate that is greater than the first expansion rate to generate a biometric feature image g3. The biometric feature image storage unit 11c3 stores the biometric feature image g3. Other operations are the same as those in FIG. 2.

[0104] 14 is a diagram showing an example of the operation of the biometric feature extraction filter and biometric feature expansion processor. The biometric feature extraction filter 11b1 extracts line segments of the vein pattern from the biometric image g0 to generate a biometric feature image with thinned line segments. The biometric feature expansion processors 11h1 and 11h2 then sequentially perform expansion processing based on the biometric feature image with thin line segments to generate a biometric feature image with thick line segments and expanded features.

[0105] Here, biometric feature extraction filters often require a relatively long processing time. On the other hand, biometric feature expansion processing often requires a shorter processing time than biometric feature extraction filters. Therefore, line segment extraction from a biometric image is performed by a biometric feature extraction filter, and the subsequent line segment expansion processing is performed by a biometric feature expansion processor. This shortens the processing time for biometric feature processing, enabling faster biometric authentication processing.

[0106] Fig. 15 shows an example of the simulation results. Table ta1 has items for a data set, a conventional device, and the biometric authentication processing device 10. The conventional device performs biometric authentication based on a biometric feature vector generated from a single biometric feature image. The biometric authentication processing device 10 performs biometric authentication based on a biometric feature vector generated from a composite biometric feature image obtained by multi-channel synthesis of the above-mentioned three channels.

[0107] For a data set with small posture variations, the false rejection error of the conventional device was 0.9%, while the false rejection error of the biometric authentication processing device 10 was 0.6%, which is lower than that of the conventional device and represents a 33% improvement in error rate.

[0108] Furthermore, for a data set with large posture variations, the false rejection error of the conventional device was 2.6%, while the false rejection error of the biometric authentication processing device 10 was 1.2%, which is lower than that of the conventional device and represents a 50% improvement in the error rate.

[0109] In both cases where the posture variation is small and large, a high improvement rate is obtained by the biometric authentication processing device 10. Furthermore, when the posture variation is particularly large, the error rate is improved by 50%, which indicates that robustness against posture variation is improved.

[0110] As described above, the biometric authentication processing device extracts multiple different biometric features to generate multiple biometric feature images with different characteristics, and then performs identity verification using a biometric feature vector obtained from a composite biometric feature image created by combining the biometric feature images. This makes the device more resistant to changes in biometric posture, reduces false rejection errors, and improves biometric authentication accuracy. Furthermore, the functions of the biometric authentication processing device are achieved through software improvements, allowing for low-cost implementation.

[0111] In the above-described biometric feature extraction filter, for example, line segment extraction by morphology processing or line segment extraction by thinning can be applied as the filter processing. Also, a two-dimensional Gabor filter can be applied.

[0112] Morphological processing involves performing contraction processing on a biometric image (e.g., converting a vein pattern into thin line segments). Contraction processing, the opposite of dilation processing, is a process of contracting the binarized result by one pixel, making it possible to connect broken line segments and separate connected parts.

[0113] Thinning involves removing white pixel areas in a binary image from the outside and converting them into line pixels with a width of one pixel. Because an object can be represented by a center line, it becomes possible to recognize the object's unique features. Furthermore, the 2D Gabor filter is a filter process that extracts the direction of line segments contained in an image, making it possible to extract the directional components of line segments (extracting the direction in which line segments are oriented).

[0114] The above different processes can be applied to multiple biometric feature extraction filters. For example, biometric feature extraction filter 11b1 can be set to perform thinning as an operation parameter, biometric feature extraction filter 11b2 can be set to perform morphology as an operation parameter, and biometric feature extraction filter 11b3 can be set to perform two-dimensional Gabor filtering as an operation parameter.

[0115] The processing functions of the biometric authentication processing device of the present invention described above can be realized by a computer. In this case, a program describing the processing contents of the functions that the biometric authentication processing device should have is provided. By executing the program on a computer, the processing functions are realized on the computer.

[0116] The program describing the processing contents can be recorded on a computer-readable recording medium. Examples of computer-readable recording media include magnetic storage units, optical disks, magneto-optical recording media, and semiconductor memories. Examples of magnetic storage units include hard disk drives (HDDs), flexible disks (FDs), and magnetic tapes. Examples of optical disks include CD-ROMs / RWs. Examples of magneto-optical recording media include MOs (Magneto Optical disks).

[0117] When distributing a program, for example, the program may be recorded on a portable recording medium such as a CD-ROM and sold. Alternatively, the program may be stored in the memory of a server computer and transferred from the server computer to other computers via a network.

[0118] A computer that executes a program stores, for example, a program recorded on a portable recording medium or a program transferred from a server computer in its own storage unit. The computer then reads the program from its own storage unit and executes processing in accordance with the program. Note that the computer can also read the program directly from a portable recording medium and execute processing in accordance with the program.

[0119] The computer can also execute the processing according to the received program each time the program is transferred from a server computer connected via a network. At least a part of the processing functions can also be realized by electronic circuits such as DSPs, ASICs, and PLDs.

[0120] The foregoing merely illustrates the principles of the present invention. Further, since numerous modifications and changes will be apparent to those skilled in the art, the present invention is not limited to the exact construction and application shown and described above, and all corresponding modifications and equivalents are deemed to be within the scope of the present invention as defined by the appended claims and their equivalents.

[0121] 10 Biometric authentication processing device 11 Control unit 13 Imaging unit a1 Biometric image b1, b2, b3 Biometric characteristic image c1 Synthesized biometric characteristic image d1 Biometric characteristic data d2 Registered biometric characteristic data

Claims

1. A biometric authentication processing program that causes a computer to execute the following processes: extracting a plurality of different biometric features from a biometric image to generate a plurality of biometric feature images having different characteristics; generating a composite biometric feature image by combining the plurality of biometric feature images; generating biometric feature data from the composite biometric feature image based on a predetermined calculation; and performing authentication processing based on the similarity between the biometric feature data and registered biometric feature data.

2. The biometric authentication processing program according to claim 1, wherein said processing comprises inputting said composite biometric feature image to a neural network that executes said predetermined calculation to generate said biometric feature data including a biometric feature vector.

3. A biometric authentication processing program as described in claim 1, wherein the processing generates each of the plurality of biometric feature images as a one-channel grayscale image, and then converts the plurality of one-channel grayscale images into a multi-channel image and combines them to generate the composite biometric feature image.

4. A biometric authentication processing program as described in claim 1, wherein the processing generates a weighted composite biometric feature image by weighting the pixel values ​​of the multiple biometric feature images based on a weighting value and additively combining them as the composite biometric feature image.

5. A biometric authentication processing program according to claim 4, wherein said processing determines said weighting value with 1 / n as a maximum value, where n is the number of said plurality of biometric characteristic images to be additively synthesized.

6. The biometric authentication processing program according to claim 1, wherein said processing extracts the biometric features using a plurality of biometric feature extraction filters having different operating parameters.

7. The biometric authentication processing program of claim 6, wherein the multiple biometric feature extraction filters include a first biometric feature extraction filter, a second biometric feature extraction filter and a third biometric feature extraction filter, the first biometric feature extraction filter generating a first biometric feature image by extracting a first line segment from the biometric image, the second biometric feature extraction filter generating a second biometric feature image by extracting a second line segment thicker than the first line segment from the biometric image, and the third biometric feature extraction filter generating a third biometric feature image by extracting a third line segment thicker than the second line segment from the biometric image, and the first biometric feature image, the second biometric feature image and the third biometric feature image are combined to generate the combined biometric feature image.

8. A biometric authentication processing program according to claim 6, wherein said processing changes a combination of a plurality of motion parameters to be set in each of said plurality of biometric feature extraction filters in accordance with the magnitude of a change in posture of the biometric subject.

9. A biometric authentication processing program as described in claim 8, wherein, when it is detected that the rate of change of the posture change is less than a predetermined rate of change, the processing sets the multiple operation parameters of a first combination to each of the multiple biometric feature extraction filters, and when it is detected that the rate of change of the posture change is equal to or greater than a predetermined rate of change, the processing sets the multiple operation parameters of a second combination to each of the multiple biometric feature extraction filters, the second combination including operation parameters having a larger expansion rate for expanding the thickness of line segments in the biometric image than the first combination.

10. A biometric authentication processing program as claimed in claim 1, wherein the processing extracts the biometric features using a biometric feature extraction filter having operational parameters for extracting line segments from the biometric image, and a biometric feature expansion process for expanding the line segments.

11. A biometric authentication processing device having: an imaging unit that captures a biometric image; and a control unit that acquires the captured biometric image, extracts a plurality of different biometric features from the biometric image to generate a plurality of biometric feature images having different characteristics, generates a composite biometric feature image by combining the plurality of biometric feature images, generates biometric feature data from the composite biometric feature image based on a predetermined calculation, and performs authentication processing based on the similarity between the biometric feature data and registered biometric feature data.

12. A biometric authentication processing method, in which a computer performs a plurality of different biometric feature extractions on a biometric image to generate a plurality of biometric feature images having different characteristics, generates a composite biometric feature image by combining the plurality of biometric feature images, generates biometric feature data from the composite biometric feature image based on a predetermined calculation, and performs authentication processing based on the similarity between the biometric feature data and registered biometric feature data.

Images