Training program, training method, and training device

Abstract

The present invention generates a neural network that enables biometric authentication unaffected by changes in the posture of a body part.　A processing unit (12) transforms a training image (2-1) for machine learning by a neural network (1) that calculates authentication data for biometric authentication from an inputted biological image into three-dimensional data (3a) using a first transformation based on an internal parameter (13) of an imaging device for biological images, generates three-dimensional data (3b) by changing the posture of a body part (4) included in the three-dimensional data (3a), generates a training image (2-1a) by transforming the three-dimensional data (3b) using a second transformation that is the inverse of the first transformation, and uses at least the training image (2-1) and the training image (2-1a) to train the neural network (1) by machine learning.

Description

Learning program, learning method and learning device

[0001] The present invention relates to a learning program, a learning method, and a learning device.

[0002] In recent years, biometric authentication using various biological parts has been used. For example, fingerprint authentication is often performed by touching a biological part (finger) to a biometric sensor. On the other hand, palm vein authentication and iris authentication are performed without contact.

[0003] Furthermore, neural networks generated by machine learning may be used in biometric authentication processes. For example, neural networks may be used to calculate authentication data for registration and verification from captured biometric images.

[0004] As a related technology for biometric authentication, a biometric authentication system using a local classifier based on a neural network has been proposed, which classifies the palm area as a region of interest. Also, as a related technology for neural networks, a data labeling method has been proposed, which acquires the three-dimensional position coordinates of bone points based on depth images collected by a depth camera and labels the joint information of the bone points based on the three-dimensional position coordinates and the two-dimensional position coordinates on the depth image in order to train a neural network that tracks the hand.

[0005] International Publication No. 2014 / 039732 U.S. Patent Application Publication No. 2022 / 0366717

[0006] In contactless biometric authentication, the posture of a biometric part relative to a biometric sensor may not be constant. Posture fluctuations in a biometric part are a major factor in causing authentication errors. In one aspect, the present invention aims to provide a learning program, a learning method, and a learning device capable of generating a neural network that enables biometric authentication that is robust against posture fluctuations in a biometric part.

[0007] In one proposal, a learning program is provided that causes a computer to execute the following processes: converting a first training image for machine learning a neural network that calculates authentication data for biometric authentication from an input biometric image into first three-dimensional data using a first transformation formula based on internal parameters of an imaging device for the biometric image; generating second three-dimensional data by varying the posture of a biological part included in the first three-dimensional data; generating a second training image by transforming the second three-dimensional data using a second transformation formula that performs an inverse transformation of the transformation by the first transformation formula; and executing machine learning of the neural network using at least the first training image and the second training image.

[0008] In addition, one proposal provides a learning method in which a computer executes a process similar to the process based on the above-mentioned learning program. Furthermore, one proposal provides a learning device that executes a process similar to the process based on the above-mentioned learning program.

[0009] In one aspect, a neural network can be generated that enables biometric authentication that is robust to changes in posture of a body part. The above and other objects, features, and advantages of the present invention will become apparent from the following description taken in conjunction with the accompanying drawings illustrating preferred embodiments of the present invention.

[0010] 1 is a diagram illustrating an example of a configuration and processing of a learning device according to a first embodiment; FIG. 2 is a diagram illustrating an example of a configuration of a biometric authentication system according to a second embodiment; FIG. 3 is a diagram illustrating an example of a hardware configuration of a learning device; FIG. 4 is a diagram illustrating a CNN that generates authentication data; FIG. 5 is a diagram illustrating a normalization process of an input image; FIG. 6 is a diagram illustrating a procedure for generating an augmented image; FIG. 7 is a diagram illustrating an example of a configuration of processing functions provided in a learning device; FIG. 8 is a diagram illustrating an example of data included in a sensor dataset; FIG. 9 is a flowchart illustrating an example of a learning process; FIG. 10 is a flowchart illustrating an example of a learning data augmentation process; FIG. 11 is a diagram illustrating an example of a configuration of processing functions provided in an authentication device; FIG. 12 is a flowchart illustrating an example of a registration process; FIG. 13 is a flowchart illustrating an example of a matching process; FIG. 14 is a diagram illustrating a discrete value selection probability for a translation amount in the z-axis direction; FIG. 15 is a diagram illustrating an example of a configuration of an authentication data generation unit provided in an authentication device according to a third embodiment; FIG. 16 is a diagram illustrating an example of a configuration of processing functions of a learning device according to a third embodiment; FIG. 17 is a flowchart illustrating an example of a learning process; FIG. 18 is a flowchart illustrating an example of a learning data augmentation process; FIG. 19 is a diagram illustrating an example of a configuration of processing functions of an authentication device according to a third embodiment;

[0011] Hereinafter, embodiments of the present invention will be described with reference to the drawings. [First Embodiment] Fig. 1 is a diagram showing an example of the configuration and processing of a learning device according to a first embodiment. The learning device 10 shown in Fig. 1 is a device that generates, by machine learning, a neural network 1 used in biometric authentication processing. As the biometric authentication, contactless biometric authentication, such as palm vein authentication, is applied in which biometric information is read without a biological part coming into contact with a biometric sensor.

[0012] The learning device 10 has a memory unit 11 and a processing unit 12. The memory unit 11 is a storage area allocated in a storage device provided in the learning device 10. The memory unit 11 stores internal parameters 13 of the biosensor. The internal parameters 13 include information such as focal length and image center coordinates, for example. The processing unit 12 is, for example, a processor provided in the learning device 10. The following processing of the processing unit 12 is realized, for example, by the processor executing a program.

[0013] When a biometric image is input, the neural network 1 calculates authentication data (enrollment data, matching data) for biometric authentication. Furthermore, the neural network 1 can learn using a plurality of training images prepared in advance. The plurality of training images are biometric images obtained by capturing images of body parts. In this embodiment, for example, n training images 2-1 to 2-n are prepared in advance. The training images 2-1 to 2-n may be stored in the storage unit 11.

[0014] The processing unit 12 selects one or more training images from the training images 2-1 to 2-n and performs a training data augmentation process to increase the number of training images. Here, as an example, it is assumed that the training image 2-1 is selected. The processing unit 12 converts the selected training image 2-1 into three-dimensional data 3a using a first conversion formula based on the internal parameters 13.

[0015] Next, the processing unit 12 converts the three-dimensional data 3a into three-dimensional data 3b by varying the posture of the body part 4 included in the three-dimensional data 3a. In this process, a three-dimensional posture variation is applied to the body part 4. In the example of Fig. 1, the body part 4 is the palm of a hand, and the posture of the body part 4 is varied so that the surface of the palm changes from being approximately parallel to the xy plane in the three-dimensional coordinate system to being tilted relative to the xy plane.

[0016] Next, the processing unit 12 converts the three-dimensional data 3b into two-dimensional data using a second conversion formula that is an inverse conversion of the conversion using the first conversion formula, thereby generating a training image 2-1a. In this way, a training image 2-1a showing the body part 4 in a state of posture change is generated from the training image 2-1.

[0017] During training of the neural network 1, the processing unit 12 uses at least the training image 2-1 and the training image 2-1a to perform machine learning of the neural network 1. In practice, the machine learning of the neural network 1 is performed using the training images 2-1 to 2-n and multiple training images newly generated from multiple original training images, such as the training image 2-1a.

[0018] This makes it possible to generate a neural network 1 that can calculate authentication data that is robust against posture changes of the body part 4, compared to when training is performed using only training images 2-1 to 2-n. In particular, the above process makes it possible to vary the body part 4 in three-dimensional space while it is converted into three-dimensional data, thereby generating additional training images that accurately reproduce posture changes of the body part 4 that may actually occur. By using these additionally generated training images in addition to the original training images for training, it becomes possible to generate a neural network 1 that can calculate authentication data that is reliably robust against posture changes of the body part 4. By using such a neural network 1 during authentication processing, it becomes possible to perform highly accurate authentication processing that is robust against posture changes of the body part 4.

[0019] Here, the second transformation formula can be a camera matrix that transforms three-dimensional coordinates in three-dimensional space into two-dimensional coordinates. In contactless biometric authentication, there may be an appropriate distance between the biometric sensor and the biometric part when a biometric image is captured. Furthermore, there are many cases in which the biometric sensor and the biometric part are approximately parallel when a biometric image is captured. In such cases, among the parameters of the camera matrix, the parameter indicating the distance of the biometric part in the z-axis direction can be set to a fixed value indicating the appropriate distance, and the parameter indicating rotation can be set to 0. This makes it possible to convert a training image into three-dimensional data by inverse transformation of the camera matrix, or to convert three-dimensional data after posture change into two-dimensional data using the camera matrix.

[0020] Second Embodiment Next, a system in which vein authentication using palm veins is performed as an example of biometric authentication will be described.

[0021] FIG. 2 is a diagram showing an example of the configuration of a biometric authentication system according to a second embodiment. As shown in FIG. 2, the biometric authentication system includes a learning device 100 and an authentication device 200. The authentication device 200 includes a biometric sensor 201 for acquiring biometric information of a user. In this embodiment, palm vein information is used as the biometric information. In this case, the biometric sensor 201 includes, for example, a light-emitting unit that irradiates the palm with near-infrared light and an imaging unit that receives the near-infrared light reflected from the palm to capture a biometric image.

[0022] The authentication device 200 generates authentication data to be used for authentication by performing image processing on the captured biometric image. The authentication device 200 generates authentication data for registration (enrollment data) from the biometric information of the user to be registered and registers the generated enrollment data in a database. The authentication device 200 also generates authentication data for matching (matching data) from the biometric information of the user to be authenticated, matches the generated matching data with the enrollment data registered in the database, and outputs an authentication result.

[0023] In the authentication device 200, authentication data is generated using a convolutional neural network (CNN). The learning device 100 generates the CNN by performing machine learning using biometric images (learning images) for learning that have been captured in advance.

[0024] By learning the CNN, weight coefficients between nodes in the CNN are calculated, and a weight data set including the calculated weight coefficients is generated. The generated weight data set is passed to the authentication device 200, and the authentication device 200 sets the weight coefficients included in the weight data set in the CNN, thereby becoming able to generate authentication data.

[0025] Learning device 100 is connected to authentication device 200, for example, via a network. In this case, learning device 100 can transfer the generated weight data set to authentication device 200 via the network. The generated weight data set may also be transferred to authentication device 200 via a portable recording medium.

[0026] Fig. 3 is a diagram showing an example of the hardware configuration of a learning device. The learning device 100 is realized, for example, as a computer as shown in Fig. 2. As shown in Fig. 2, the learning device 100 includes a processor 101, a random access memory (RAM) 102, a hard disk drive (HDD) 103, a graphics processing unit (GPU) 104, an input interface (I / F) 105, a reader 106, and a communication interface (I / F) 107.

[0027] The processor 101 comprehensively controls the entire learning device 100. The processor 101 is, for example, a central processing unit (CPU), a micro processing unit (MPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), or a programmable logic device (PLD). The processor 101 may also be a combination of two or more elements selected from the group consisting of a CPU, an MPU, a DSP, an ASIC, and a PLD. The processor 101 is an example of the processing unit 12 shown in FIG. 1 .

[0028] RAM 102 is used as the main storage device of learning device 100. RAM 102 temporarily stores at least a portion of the OS (Operating System) program and application programs executed by processor 101. RAM 102 also stores various data necessary for processing by processor 101.

[0029] The HDD 103 is used as a secondary storage device for the learning device 100. The HDD 103 stores an OS program, application programs, and various data. Note that other types of non-volatile storage devices, such as a solid state drive (SSD), can also be used as the secondary storage device.

[0030] A display device 104a is connected to the GPU 104. The GPU 104 displays an image on the display device 104a in accordance with an instruction from the processor 101. The display device 104a may be a liquid crystal display or an organic EL (Electroluminescence) display.

[0031] An input device 105a is connected to the input interface 105. The input interface 105 transmits a signal output from the input device 105a to the processor 101. The input device 105a may be a keyboard or a pointing device. The pointing device may be a mouse, a touch panel, a tablet, a touch pad, a trackball, or the like.

[0032] A portable recording medium 106a is detachably attached to the reading device 106. The reading device 106 reads data recorded on the portable recording medium 106a and transmits the data to the processor 101. The portable recording medium 106a may be an optical disk, a semiconductor memory, or the like.

[0033] The communication interface 107 transmits and receives data via a network to and from other devices such as the authentication device 200. The above hardware configuration can realize the processing functions of the learning device 100. Note that, for example, the authentication device 200 can also be realized as a computer equipped with a processor and memory.

[0034] 4 is a diagram for explaining a CNN that generates authentication data. As described above, the authentication device 200 generates authentication data using the CNN 50. This CNN 50 is trained by the training device 100. The training process of the CNN 50 is executed as follows.

[0035] In the learning device 100, n learning images I are used for learning the CNN 50. １ ~I ｎ A training image I is prepared. １ ~I ｎ are biometric images obtained by capturing images of the palms of different people. １ ~I ｎ Each of the above has an ID as identification information for identifying the person to be imaged. １ ~ID ｎ The learning device 100 associates the learning image I １ ~I ｎ is used as training data, and ID １ ~ID ｎis used as the correct answer data, and the input image is converted into the learning image I １ ~I ｎ The number of categories (number of categories = n) that classify into n people corresponding to each of the IDs is learned. １ ~ID ｎ The probability value P corresponding to each of １ ~P ｎ A CNN 50 is generated that calculates

[0036] The authentication device 200 generates authentication data using the CNN 50 generated in this way. As the authentication data, a feature vector having a plurality of feature quantities as elements is calculated. The CNN 50 includes an input layer, a plurality of intermediate layers (hidden layers), and an output layer. The CNN 50 generates an input image I from the biometric sensor 201. ｉｎ When input, an m-dimensional feature vector (V １ , V ２ , ..., V ｍ ) is output. The output feature vector (V １ , V ２ , ..., V ｍ ) is the input image I ｉｎ This information indicates the characteristics of the

[0037] Note that the feature vector (V １ , V ２ , ..., V ｍ ) may be output from two or more predetermined hidden layers. As an example of the values ​​of n and m, n=10000 and m=512 are applicable.

[0038] 5 is a diagram illustrating the normalization process of an input image. It is desirable that the biometric image used for authentication captures the authentication part (the palm in this embodiment) in a consistent posture. However, in reality, a user does not always position the authentication part at the correct angle relative to the biometric sensor 201. Therefore, captured biometric images often contain variations in the posture of the authentication part. Such posture variations are a major cause of biometric authentication errors, and minimizing posture variations is an important factor in improving authentication accuracy.

[0039] Therefore, as shown in FIG. 5 , the authentication device 200 performs a "normalization process" on the input image obtained from the biometric sensor 201 to correct the palm orientation in the image, and inputs the normalized input image (normalized image) to the CNN 50. In the normalization process, rotation and translation are performed so that the orientation and position of the palm in the image are in a predetermined state. For example, in the rotation process, the palm outline is detected from the input image, a rotation angle is calculated based on the angle of a line extending generally vertically in the outline, and the image is rotated by the calculated rotation angle.

[0040] By inputting the normalized image after such processing into the CNN 50, it becomes possible to calculate a feature vector that is less likely to cause authentication errors. However, even with such normalization processing, it is not always possible to correctly correct the palm posture. For this reason, the CNN 50 that processes the normalized image is also required to be robust against posture changes.

[0041] Therefore, in this embodiment, an augmentation process is performed on the training images used for training the CNN 50. In the augmentation process, new training images (augmented images) are generated by intentionally varying the palm posture in the original training images, and the CNN 50 is trained using both the original training images and the augmented images. For example, by using an augmented image in which the palm is rotated in the original training image for training, a CNN 50 that is resistant to rotation can be generated.

[0042] For example, the following two methods are conceivable for expanding training images in an image processing CNN. The first method uses affine transformation. In this method, the two-dimensional coordinates of the image are transformed by calculation using matrices that perform scaling, rotation, and translation. The second method uses homography transformation (projection transformation). In this method, four pairs of points before and after movement are selected from the image. Since there are eight independent parameters in homography transformation, a transformation matrix (homography matrix) is uniquely determined by selecting four pairs of movement points, and the two-dimensional coordinates of the image are transformed using the determined matrix.

[0043] However, these first and second methods only simulate three-dimensional changes in the subject. Therefore, even if a training image containing a palm is augmented using the above methods, it is not possible to generate an augmented image that accurately represents the three-dimensional pose changes of the palm that actually occur. Therefore, even if a CNN 50 is trained using augmented images generated using the above methods, it is not necessarily possible to generate a CNN 50 that enables high-performance authentication processing that is resistant to palm pose changes.

[0044] Here, the three-dimensional coordinates (X ｗ , Y ｗ , Z ｗ ) into two-dimensional coordinates (u, v) on the image can be performed using the following equation (1) using the camera matrix P:

[0045] In equation (1), s is a variable indicating the magnification ratio. The camera matrix P is ｘ , f ｙ , c ｘ , c ｙ a 3-by-3 internal parameter matrix including １１ , r １２ , ..., r ３２ , r ３３ and t １ , t ２ , t ３ It is expressed as a product with an external parameter matrix of 3 rows and 4 columns including:

[0046] The intrinsic parameter matrix is ​​a matrix that converts three-dimensional coordinates in the camera coordinate system into two-dimensional coordinates on the image, and f ｘ , f ｙ indicate the focal lengths in the x and y directions, respectively, and (c ｘ , c ｙ ) indicates the image center coordinate. ｘ , f ｙ , c ｘ , c ｙ indicates an internal parameter of the camera, and is determined in advance according to the specifications of the image sensor of the biometric sensor 201.

[0047] The extrinsic parameter matrix is ​​a matrix that converts three-dimensional coordinates in the global coordinate system into three-dimensional coordinates in the camera coordinate system. １１ , r １２ , ..., r ３２ , r ３３ The 3-by-3 matrix containing t １ , t ２ , t ３ The 3-row, 1-column matrix including is a translation vector representing a parallel translation.

[0048] In this embodiment, an augmented image including a palm to which three-dimensional pose variations have been accurately added is generated using such a camera matrix P. Fig. 6 is a diagram showing the procedure for generating an augmented image.

[0049] The learning device 100 performs a normalization process on the original learning image (original image) to generate a normalized image. Next, the learning device 100 performs an inverse transformation on the generated normalized image using the camera matrix P, thereby converting the two-dimensional data of the normalized image into three-dimensional data. The learning device 100 then applies a posture variation to the converted three-dimensional data. As a result, a three-dimensional variation is applied to the subject in the original image while the subject is positioned in three-dimensional space. The learning device 100 then converts the three-dimensional data after the variation into two-dimensional data using the camera matrix P. As a result, an augmented image is generated.

[0050] Generally, when converting a two-dimensional image into a three-dimensional image, the height in the z-axis direction is indefinite. １１ , r １２ , ..., r ３２ , r ３３ , t １ , t ２ , t ３ cannot be determined from a single captured image.

[0051] However, in non-contact biometric authentication such as palm vein authentication, it is necessary to determine an appropriate shooting distance (Z ０) is set in advance for each biosensor 201. Therefore, the captured image obtained by the biosensor 201 is captured when the distance from the biosensor 201 is Z ０ and the surface of the biological part to be imaged is approximately parallel to the imaging surface of the biosensor 201. In particular, a normalized image to which normalization processing has been applied has a larger image area when the distance from the biosensor 201 is closer to Z. ０ It can be assumed that the image is close to , and the surface of the biological part is nearly parallel to the imaging plane.

[0052] From this, the height in the z-axis direction is set to the above Z ０ It can be set to a fixed value Z ０ The external parameter matrix t １ , t ２ , t ３ is determined, and the external parameter matrix r is calculated by assuming that the palm surface is parallel to the imaging plane. １１ , r １２ , ..., r ３２ , r ３３ is 0. Therefore, the learning device 100 can convert one learning image into three-dimensional data as shown in FIG. 6, and can vary the palm posture three-dimensionally by applying a matrix for posture variation to the three-dimensional data. The learning device 100 can then convert the three-dimensional data after the variation into two-dimensional data, thereby generating an augmented image including a palm to which three-dimensional posture variation has been accurately added.

[0053] In addition, the above f ｘ , f ｙ , c ｘ , c ｙ , r １１ , r １２ , ..., r ３２ , r ３３ The unit of is mm. Therefore, in order to obtain coordinates in pixel units on an image, the pixel size ([mm]) of the image pickup element in the biometric sensor 201 is actually required.

[0054] 4 to 6, for the sake of simplicity, it is assumed that the biometric image is input as is to the CNN 50 in the learning device 100 and the authentication device 200. However, in reality, a feature extraction process is performed to detect a vein pattern from the normalized biometric image, and the processed feature image is input to the CNN 50.

[0055] Furthermore, the generation of the above-described augmented image is not limited to cases where palm veins are used as biometric information, but can also be applied to other biometric information for which an appropriate distance from a biometric sensor to the biometric authentication site is determined. For example, it can be applied to iris authentication, non-contact fingerprint authentication in which a physical guide or the like is used to maintain a constant distance from the biometric sensor to the finger, and face authentication in which a person's standing position is specified or the size of the face captured by the biometric sensor is guided to be constant.

[0056] Next, details of the processing of the learning device 100 will be described. Fig. 7 is a diagram showing an example of the configuration of processing functions provided in the learning device. The learning device 100 includes a storage unit 110, a learning control unit 121, a normalization processing unit 122, a feature extraction unit 123, a data augmentation processing unit 124, and a CNN learning unit 125.

[0057] The storage unit 110 is a storage area allocated in a storage device included in the learning device 100, such as the RAM 102 or the HDD 103. The storage unit 110 stores a training image database (DB) 111, a sensor data set 112, and a weight data set 113.

[0058] A plurality of training images used for training the CNN 50 are registered in advance in the training image database 111. Identification information (ID) that identifies the person being imaged is added to each training image. Information indicating the specifications of the biometric sensor 201 used in the authentication device 200 is registered in the sensor dataset 112. Weighting coefficients between nodes included in the CNN 50 generated by training are registered in the weight dataset 113.

[0059] The processes of the learning control unit 121, the normalization processing unit 122, the feature extraction unit 123, the data augmentation processing unit 124, and the CNN learning unit 125 are realized, for example, by the processor 101 executing a predetermined application program.

[0060] The learning control unit 121 comprehensively controls the learning process of the CNN 50. The normalization processing unit 122 performs normalization processing on the training images acquired from the training image database 111 and outputs normalized images. The feature extraction unit 123 performs feature extraction processing on the normalized images to extract vein patterns and outputs feature images.

[0061] The data augmentation processing unit 124 performs an augmentation process to augment the image for training (feature image). The data augmentation processing unit 124 generates an augmented image by three-dimensionally varying the posture of the palm depicted in the feature image. The CNN training unit 125 performs a training process for the CNN 50 using the original feature image before the augmentation process and the augmented image, and determines weighting coefficients between nodes within the CNN 50. The CNN 50 stores a data set of the determined weighting coefficients in the storage unit 110 as a weighting data set 113.

[0062] 8 is a diagram showing an example of data included in the sensor data set 112. The sensor data set 112 includes the focal lengths f in the x and y directions of the image sensor of the biosensor 201. ｘ , f ｙ , the image center coordinates of the image sensor (c ｘ , c ｙ ), the cell size elen of the image sensor, and the body height Z ０ is registered. ０ indicates the height (appropriate distance) of the biological part relative to the biosensor 201. These parameters are determined according to the specifications of the biosensor 201. The units of these parameters are all mm.

[0063] Next, the processing of the learning device 100 will be described using a flowchart. Fig. 9 is a flowchart showing an example of the learning processing. [Step S11] The learning control unit 121 reads out the sensor data set 112 from the storage unit 110, and calculates parameters of the camera matrix P based on the sensor data set 112.

[0064] [Step S12] The learning control unit 121 initializes the weighting coefficients of the CNN 50. For example, the initial values ​​of the weighting coefficients are set to random values. [Step S13] The learning control unit 121 extracts a predetermined number of training images (for example, approximately 16 images) from the training image database 111. In this process, the predetermined number of training images may be extracted randomly from the training images registered in the training image database 111, or the training images may be extracted in order, a predetermined number at a time.

[0065] [Step S14] A training data expansion process is executed. In this process, feature images for training the CNN 50 are generated based on each of the predetermined number of extracted training images. [Step S15] The CNN training unit 125 executes a training process for the CNN 50 using each generated feature image, and updates the weight coefficients of the CNN 50. In this process, machine learning is executed using each feature image as training data and the person ID associated with the training image from which each feature image was generated as correct answer data.

[0066] The CNN learning unit 125 calculates the learning loss when each feature image is input to the CNN 50. The learning loss is a value indicating the error between the value calculated by the CNN 50 when each feature image is input and the correct answer. The CNN learning unit 125 updates the weight coefficients of the CNN 50 by performing optimization calculations using an optimization algorithm based on the calculated learning loss. As the optimization algorithm, for example, SGD (Stochastic Gradient Descent) or Adam (Adaptive Moment Estimation) can be used.

[0067] [Step S16] The learning control unit 121 determines whether to end the learning process. For example, it determines that the learning process should be ended when the processes of steps S13 to S15 have been executed a predetermined number of times. If the learning process should be continued, the process proceeds to step S13. If the learning process should be ended, the process proceeds to step S17.

[0068] [Step S17] The CNN training unit 125 stores the weighting coefficients set in the CNN 50 in the storage unit 110 as the weighting data set 113. Fig. 10 is a flowchart showing an example of training data expansion processing. In step S14 of Fig. 9, the processing of Fig. 10 is performed for each of the training images extracted in step S13.

[0069] [Step S21] The normalization processing unit 122 performs normalization processing on the training images. As a result, the posture of the palm in the training images is corrected to a more or less correct posture, and a normalized image is generated. Note that the normalization processing may also include processing to correct the palm from a curled state to an open state.

[0070] [Step S22] The feature extraction unit 123 performs feature extraction processing on the normalized image to extract a vein pattern and generate a feature image. In this feature extraction processing, for example, vein enhancement processing using a band-pass filter or the like is applied to the normalized image, and further noise reduction processing is applied.

[0071] [Step S23] The data expansion processing unit 124 converts the two-dimensional data of the feature image into three-dimensional data using a matrix that performs an inverse transformation of the transformation using the camera matrix P, to which the parameters calculated in step S11 of Fig. 9 are set. As a result, the coordinates of the vein pattern in the feature image are converted into coordinates in three-dimensional space.

[0072] [Step S24] The data augmentation processing unit 124 probabilistically selects a variation parameter for varying the orientation and position of the vein pattern in three-dimensional space. That is, a predetermined probability is set for each of the settable values ​​for one setting item of the variation parameter, and one of the values ​​is selected with the set probability. Here, as an example, it is assumed that the same probability is set for each of the settable values, and one of the values ​​is selected with the same probability (i.e., randomly).

[0073] 10 , it is assumed that not only the orientation of the vein pattern but also the position of the vein pattern can be varied. Rotation angles around the x-axis, y-axis, and z-axis are used as variation parameters for the orientation variation. Parallel translation amounts in the x-axis, y-axis, and z-axis directions are used as variation parameters for the position variation.

[0074] In FIG. 10 , as an example, predetermined discrete values ​​are used as the rotation angle and the translation amount. The discrete values ​​corresponding to each variation parameter include 0, one or more positive values, and one or more negative values. One of the discrete values ​​is randomly selected for each variation parameter. Note that floating-point random values ​​may also be used as the rotation angle and the translation amount.

[0075] If 0 is selected as all the variation parameters, the posture and position do not vary. In this case, in step S26 described later, the feature image generated in step S22 is output as is. That is, in the process of FIG. 10 , the original feature image may be output as is as a feature image for training the CNN 50, or an augmented image in which at least one of the posture and position of the original feature image is varied may be generated and output as a feature image for training.

[0076] In addition, when selecting each variable parameter, the probability of selecting 0 may be set to be greater than the probability of selecting any other discrete value, so that the original feature image is more likely to be output as a learning feature image than the extended image.

[0077] [Step S25] The data extension processing unit 124 converts the three-dimensional data calculated in step S23 so that the fluctuation to which the fluctuation parameters selected in step S25 are applied is added to the vein pattern.

[0078] [Step S26] The data augmentation processing unit 124 converts the three-dimensional data obtained in step S25 into two-dimensional data using the camera matrix P to which the parameters calculated in step S11 of Fig. 9 are set. This generates feature images for training the CNN 50.

[0079] In addition, if 0 is selected as all the variable parameters in step S24, the processing in step S25 and the transformation processing using the camera matrix P in step S26 may be skipped, and the feature image generated in step S22 may be output as is as the feature image for learning.

[0080] According to the process shown in FIG. 10 , variations in posture and position in three-dimensional space are applied to the vein pattern in the feature image, and the feature image including the varied vein pattern is generated as an augmented image. This makes it possible to generate an augmented image that accurately represents the three-dimensional variations of a palm that actually occur. Then, the CNN 50 is trained using such augmented images, and the trained CNN 50 is used during authentication processing, enabling biometric authentication that is robust against variations in posture and position, thereby improving authentication accuracy.

[0081] The training images registered in the training image database 111 may be normalized images. In this case, the normalization processing unit 122 of the training device 100 is unnecessary, and the feature extraction unit 123 executes feature extraction processing on the training images (normalized images) extracted from the training image database 111 by the training control unit 121. This reduces the amount of processing during training and shortens the time required for training the CNN 50.

[0082] Furthermore, the training images registered in the training image database 111 may be feature images that have been subjected to normalization and feature extraction. In this case, the normalization processing unit 122 and feature extraction unit 123 of the training device 100 are unnecessary, and the data augmentation processing unit 124 performs data augmentation processing on the training images (feature images) extracted from the training image database 111.

[0083] Next, the details of the processing of the authentication device 200 will be described. This further reduces the amount of processing during learning, and shortens the time required for learning the CNN 50. Fig. 11 is a diagram showing an example of the configuration of processing functions provided in the authentication device. The authentication device 200 includes a storage unit 210, a normalization processing unit 221, a feature extraction unit 222, an authentication data generation unit 223, a registration processing unit 224, and a matching processing unit 225.

[0084] The storage unit 210 is a storage area secured in a storage device (not shown) included in the authentication device 200. The storage unit 210 stores the weight data set 113 generated by the learning device 100. The storage unit 210 also stores an authentication database (DB) 211. In the authentication database 211, registration data corresponding to each user to be authenticated is registered in association with a user ID that identifies the user.

[0085] The processing of the normalization processing unit 221, the feature extraction unit 222, the authentication data generation unit 223, the registration processing unit 224, and the matching processing unit 225 is realized, for example, by a processor (not shown) provided in the authentication device 200 executing a predetermined firmware program.

[0086] The normalization processing unit 221 performs normalization processing on the biometric image captured by the biometric sensor 201 and outputs the normalized image. The feature extraction unit 222 performs feature extraction processing on the normalized image to extract a vein pattern and outputs a feature image. The authentication data generation unit 223 inputs the feature image to the CNN 50 formed using the weight data set 113 and calculates a feature vector as authentication data.

[0087] During the registration process, the registration processing unit 224 registers the calculated feature vector as registration data in the authentication database 211 together with the user ID. During the authentication process, the matching processing unit 225 matches the calculated feature vector as matching data with the registration data in the authentication database 211 and outputs an authentication result. For example, when one-to-one authentication is performed, the matching processing unit 225 reads out registration data corresponding to a specified user ID from the authentication database 211 and calculates a matching score indicating the similarity between the read registration data and the matching data. If the calculated matching score is equal to or greater than a predetermined threshold, the matching processing unit 225 outputs an authentication result indicating successful authentication.

[0088] Next, the processing of the authentication device 200 will be described using a flowchart. Fig. 12 is a flowchart showing an example of the registration processing. [Step S31] The normalization processing unit 221 acquires a captured image captured by the biometric sensor 201 together with a user ID that identifies the user of the image capture target.

[0089] [Step S32] The normalization processing unit 221 performs normalization processing on the captured image to generate a normalized image using a procedure similar to that of step S21 in Fig. 10. [Step S33] The feature extraction unit 222 performs feature extraction processing on the normalized image to extract a vein pattern using a procedure similar to that of step S22 in Fig. 10, to generate a feature image.

[0090] [Step S34] The authentication data generation unit 223 calculates a feature vector by inputting the feature image to the CNN 50 to which the weighting coefficients of the weight data set 113 have been set. [Step S35] The registration processing unit 224 registers the calculated feature vector as registration data in the authentication database 211 together with the user ID acquired in step S31.

[0091] 13 is a flowchart showing an example of the matching process. In the example shown in FIG. 13, it is assumed that one-to-one authentication is performed. [Step S41] The normalization processing unit 221 acquires an image captured by the biometric sensor 201 together with a user ID.

[0092] [Step S42] The normalization processing unit 221 performs normalization processing on the captured image to generate a normalized image using a procedure similar to that of step S21 in Fig. 10. [Step S43] The feature extraction unit 222 performs feature extraction processing on the normalized image to extract a vein pattern using a procedure similar to that of step S22 in Fig. 10, to generate a feature image.

[0093] [Step S44] The authentication data generation unit 223 calculates a feature vector by inputting the feature image to the CNN 50 to which the weighting coefficients of the weight data set 113 have been set. [Step S45] The matching processing unit 225 reads, from the authentication database 211, the registration data corresponding to the user ID acquired in step S41.

[0094] [Step S46] The matching processing unit 225 calculates a matching score indicating the degree of similarity between the read enrollment data and the feature vector (matching data) calculated in step S44. The matching score may be, for example, a value inversely proportional to the inter-vector distance between the enrollment data and the matching data, or the cosine similarity between the enrollment data and the matching data. The matching processing unit 225 compares the calculated matching score with a predetermined threshold. If the matching score is equal to or greater than the threshold, the matching processing unit 225 outputs an authentication result indicating successful authentication. If the matching score is less than the threshold, the matching processing unit 225 outputs an authentication result indicating unsuccessful authentication.

[0095] 12 and 13, authentication data (feature vectors) are calculated by the CNN 50 trained by the learning device 100 using the augmented images in addition to the original training images. This enables biometric authentication that is robust against variations in the posture and position of the palm relative to the biometric sensor 201, thereby improving authentication accuracy.

[0096] 10, among the discrete values ​​corresponding to each variation parameter, the same selection probability is set for discrete values ​​on the positive side of 0 and discrete values ​​on the negative side of 0. However, among the variation parameters, the selection probabilities for the amount of translation in the z-axis direction may be set asymmetrically.

[0097] FIG. 14 is a diagram for explaining the probability of selecting a discrete value for the amount of translation in the z-axis direction. The value in the z-axis direction represents the distance from the biometric sensor 201 to the palm of the hand, which is a biological part. As described above, the Z ０ In the case where the palm moves parallel to the z-axis, the distance to the palm is Z ０ If it is farther away, Z ０ For example, even if the palm is rotated in the same direction by the same angle, the palm appears to tilt more when the distance in the z-axis direction is smaller.

[0098] Therefore, the data augmentation processing unit 124 may set the probability of setting a discrete value greater than 0 for the amount of parallel translation in the z-axis direction, among the settable discrete values, to be lower than the probability of setting a discrete value less than 0. In this case, by stochastically learning a large number of images that have large visual variations and are difficult to authenticate, it is possible to generate training images for the CNN 50 that can improve authentication accuracy.

[0099] Third Embodiment An STN (Spatial Transformer Network) with a posture correction function may be placed before a CNN for image classification. In this case, it is possible to train the STN simultaneously with the CNN. Therefore, as the third embodiment, an example in which an STN is placed before the CNN 50 in the second embodiment will be described.

[0100] 15 is a diagram showing an example of the configuration of an authentication data generation unit included in an authentication device according to the third embodiment. The authentication device 200a shown in FIG. 15 includes an authentication data generation unit 223a. In the authentication data generation unit 223a, an STN 60 is arranged before a CNN 50.

[0101] The authentication data generating unit 223a receives the input image I ｉｎWhen input, the STN 50 performs a normalization process to correct the posture and generates a normalized image, and the CNN 50 generates authentication data (feature vector) based on the normalized image. That is, the STN 60 realizes the same function as the normalization processing unit 221 of the second embodiment. Furthermore, the learning device can simultaneously train the STN 60 and the CNN 50.

[0102] Fig. 16 is a diagram showing an example of the configuration of processing functions of a learning device according to the third embodiment. In Fig. 16, the same processing functions as those in Fig. 7 are denoted by the same reference numerals. The learning device 100a according to the third embodiment includes a storage unit 110, a learning control unit 121, a feature extraction unit 123, a data augmentation processing unit 124, and an STN / CNN learning unit 125a.

[0103] The storage unit 110 stores a training image database 111, a sensor data set 112, and a weight data set 113a. The weight data set 113a includes weight coefficients included in the trained STN 60 and CNN 50.

[0104] The feature extraction unit 123 performs feature extraction processing to extract vein patterns on the training image acquired from the training image database 111 instead of the normalized image, and outputs the feature image to the data extension processing unit 124.

[0105] The processing of the STN / CNN training unit 125a is realized by a processor included in the authentication device 200a executing a predetermined application program. Under the control of the training control unit 121, the STN / CNN training unit 125a trains both the STN 60 and the CNN 50 using the training feature images obtained by the data augmentation processing unit 124, and generates a weight data set 113a.

[0106] Fig. 17 is a flowchart showing an example of the learning process. In the learning process shown in Fig. 17, steps S12a, S14a, S15a, and S17a are executed instead of steps S12, S14, S15, and S17 in the process of Fig. 9, respectively.

[0107] [Step S12a] The learning control unit 121 initializes the weighting coefficients of the STN 60 and the CNN 50. For example, the initial values ​​of the weighting coefficients are set using random values. [Step S14a] A learning data expansion process is executed. This process generates feature images for learning the STN 60 and the CNN 50 based on each of the predetermined number of extracted learning images. This process also differs from step S14 in FIG. 9 in that an expanded image is generated from a feature image that has not been normalized.

[0108] [Step S15a] The STN / CNN learning unit 125a executes learning processing for the STN 60 and the CNN 50 using each feature image generated in step S14a, and updates the weighting coefficients of the STN 60 and the CNN 50. In this processing, similar to step S15 in Fig. 9 , machine learning is executed using each feature image as training data and the person ID associated with the learning image from which each feature image was generated as ground truth data.

[0109] [Step S17a] The STN / CNN training unit 125a stores the weighting coefficients set for the STN 60 and the CNN 50 as a weighting data set 113a in the storage unit 110. FIG. 18 is a flowchart showing an example of training data expansion processing. In step S14a of FIG. 17, the processing of FIG. 18 is performed for each of the training images extracted in step S13. In addition, in the training data expansion processing shown in FIG. 18, step S22a is performed instead of steps S21 and S22 of FIG. 10.

[0110] [Step S22a] The feature extraction unit 123 performs feature extraction processing for extracting a vein pattern from the training image to generate a feature image. The feature extraction processing may be performed in the same manner as in step S22 of FIG. 10 .

[0111] Through the above process, both the STN 60 and the CNN 50 are trained, and a weight data set 113a is generated as a training result. The weight data set 113a is transferred to the authentication device 200a via, for example, a network or a portable recording medium.

[0112] Fig. 19 is a diagram showing an example of the configuration of processing functions of an authentication device according to the third embodiment. In Fig. 19, processing functions similar to those in Fig. 11 are denoted by the same reference numerals. An authentication device 200a according to the third embodiment includes a storage unit 210, a feature extraction unit 222, an authentication data generation unit 223a, a registration processing unit 224, and a matching processing unit 225.

[0113] The storage unit 210 stores an authentication database 211 and a weight data set 113a generated by the learning device 100a. The feature extraction unit 222 performs feature extraction processing to extract a vein pattern from a biometric image captured by the biometric sensor 201, rather than a normalized image, and outputs the feature image to the authentication data generation unit 223a.

[0114] The processing of the authentication data generation unit 223a is realized, for example, by a processor included in the authentication device 200a executing a predetermined firmware program. The authentication data generation unit 223a inputs a feature image to an STN 60 formed using the weight data set 113a and normalizes the feature image. The authentication data generation unit 223a also inputs the normalized image to a CNN 50 formed using the weight data set 113a and calculates a feature vector as authentication data.

[0115] Fig. 20 is a flowchart showing an example of the registration process. In the registration process shown in Fig. 20, steps S33a, S34a1, and S34a2 are executed instead of steps S32 to S34 in the process of Fig. 12.

[0116] [Step S33a] The feature extraction unit 222 performs feature extraction processing to extract a vein pattern from the captured image acquired in step S31, and generates a feature image. [Step S34a1] The authentication data generation unit 223a inputs the feature image to the STN 60 for which a weighting coefficient for STN 60 included in the weight data set 113a is set, normalizes the feature image, and generates a normalized image.

[0117] [Step S34a2] The authentication data generating unit 223a inputs the normalized image to the CNN 50 to which the weighting coefficients for the CNN 50 included in the weight data set 113a are set, and calculates a feature vector.

[0118] The calculated feature vector is input to the registration processing unit 224 and registered as registration data together with the user ID in the authentication database 211. Although not shown, in the matching process by the authentication device 200a, a feature vector as matching data is calculated by the same processes as steps S31 to S34a2 in Fig. 20. Then, the matching data is matched with the registration data by the process of step S45 in Fig. 13, and the authentication result is output.

[0119] According to the third embodiment described above, not only the CNN 50 but also the STN 60 are trained using training images and augmented images. As a result, the normalization process by the STN 60 is also optimized by the augmented images that have been given three-dimensional variations to the training images, so that the STN 60 can be generated that is capable of performing accurate normalization process for posture variations.

[0120] [Fourth embodiment] Fig. 21 is a diagram showing an example of the configuration of a biometric authentication system according to the fourth embodiment. The biometric authentication system shown in Fig. 21 includes learning device 100b, authentication server 300, and authentication terminals 410 and 420. Of these, at least authentication server 300 and authentication terminals 410 and 420 are connected via a network.

[0121] In this biometric authentication system, the processing functions of the authentication device 200 are realized in a client-server manner by an authentication server 300 and client devices, namely, authentication terminals 410 and 420. The authentication terminal 410 includes a biometric sensor 401, and a biometric image captured by the biometric sensor 401 is transmitted to the authentication server 300, where data registration and data matching are performed. Similarly, the authentication terminal 420 includes a biometric sensor 402, and a biometric image captured by the biometric sensor 402 is transmitted to the authentication server 300, where data registration and data matching are performed.

[0122] The authentication server 300 generates authentication data (feature vectors) using the CNN 50, similar to the above-described authentication device 200. The learning device 100b executes a CNN learning process and passes the obtained weight data set to the authentication server 300.

[0123] Here, the specifications of the biometric sensor 401 of the authentication terminal 410 are assumed to be different from the specifications of the biometric sensor 402 of the authentication terminal 420. In such a case, even if the same biometric part is captured by both the biometric sensors 401 and 402 at the same posture and distance, the appearance of the biometric part in the captured image will be different. Therefore, when the learning device 100b generates an augmented image for training the CNN 50, it is necessary to use a camera matrix P that matches the sensor specifications for the biometric sensor 401 and the biometric sensor 402.

[0124] Therefore, the learning device 100b holds sensor data sets corresponding to the biometric sensors 401 and 402. The learning device 100b generates an augmented image using the sensor data set for the biometric sensor 401, and generates a CNN 50 for the biometric sensor 401 using the original training image and the augmented image. The learning device 100b also generates an augmented image using the sensor data set for the biometric sensor 402, and generates a CNN 50 for the biometric sensor 402 using the original training image and the augmented image.

[0125] Then, the weight data set of the CNN 50 for the biometric sensor 401 and the weight data set of the CNN 50 for the biometric sensor 402 are transferred from the learning device 100b to the authentication server 300. The authentication server 300 stores each weight data set in association with a sensor ID that identifies the sensor type of the biometric sensors 401 and 402.

[0126] The authentication server 300 receives a sensor ID corresponding to the biometric sensor 401 along with a captured image from the authentication terminal 410. The authentication server 300 forms a CNN 50 using weight parameters associated with the received sensor ID, and generates authentication data using the CNN 50. Similarly, the authentication server 300 receives a sensor ID corresponding to the biometric sensor 402 along with a captured image from the authentication terminal 420. The authentication server 300 forms a CNN 50 using weight parameters associated with the received sensor ID, and generates authentication data using the CNN 50.

[0127] This processing enables highly accurate biometric authentication using biometric images captured by biometric sensors 401 and 402 with different specifications, regardless of the differences in the specifications. For example, even if the same three-dimensional variation is applied to a biometric part, if the specifications of the biometric sensor that captures the image are different, the appearance of the biometric part after the variation in the captured image will change. For example, if the focal length of the biometric sensor is different, the size and other aspects of the biometric part will also change in the captured image. The above-described learning process by the learning device 100b makes it possible to reproduce the differences in appearance due to differences in the specifications of such biometric sensors through learning. Therefore, by calculating authentication data using a CNN 50 trained for each biometric sensor specification, highly accurate authentication processing that is resistant to three-dimensional variations in the biometric part can be achieved.

[0128] Figure 22 is a diagram showing an example of the configuration of processing functions provided in a learning device. In Figure 22, processing functions similar to those in Figure 7 are denoted by the same reference numerals. The learning device 100b includes a storage unit 110, a learning control unit 121b, a normalization processing unit 122, a feature extraction unit 123, a data augmentation processing unit 124, and a CNN learning unit 125. The processing of the learning control unit 121b is realized by a processor provided in the learning device 100b executing a predetermined application program.

[0129] The storage unit 110 stores a training image database 111, sensor data sets 112b1 and 112b2, and weight data sets 113b1 and 113b2. The sensor data set 112b1 is associated with the sensor ID of the biometric sensor 401, and camera parameters corresponding to the sensor type of the biometric sensor 401 are registered in advance. The sensor data set 112b2 is associated with the sensor ID of the biometric sensor 402, and camera parameters corresponding to the sensor type of the biometric sensor 402 are registered in advance. The weight data set 113b1 is associated with the sensor ID of the biometric sensor 401, and the weight data set 113b2 is associated with the sensor ID of the biometric sensor 402.

[0130] In the learning device 100b, under the control of the learning control unit 121b, the normalization processing unit 122, the feature extraction unit 123, the data augmentation processing unit 124, and the CNN learning unit 125 generate a weight dataset to be set in the CNN in a procedure basically similar to that of the second embodiment. However, the learning control unit 121b specifies a sensor ID to the data augmentation processing unit 124, and the data augmentation processing unit 124 generates a camera matrix P based on the sensor dataset corresponding to the specified sensor ID and generates an augmented image using the camera matrix P. When the CNN weight dataset is generated by the learning process, the CNN learning unit 125 associates the sensor ID with the weight dataset and stores it in the storage unit 110.

[0131] FIG. 23 is a diagram illustrating an example of the configuration of processing functions provided in the authentication server and authentication client. First, the authentication terminal 410 includes a storage unit 430a and a communication unit 440a. The storage unit 430a is a storage area allocated in a storage device provided in the authentication terminal 410. The storage unit 430a stores sensor ID data 431a, in which the sensor ID of the biometric sensor 401 is described. The processing of the communication unit 440a is realized, for example, by a processor provided in the authentication terminal 410 executing a predetermined firmware program. During data registration and authentication, the communication unit 440a transmits to the authentication server 300 a biometric image captured by the biometric sensor 401, the user ID, and the sensor ID described in the sensor ID data 431a. During authentication, the communication unit 440a receives an authentication result from the authentication server 300 and outputs the authentication result via a display device or the like.

[0132] Similarly, the authentication terminal 420 includes a storage unit 430b and a communication unit 440b. The storage unit 430b stores sensor ID data 431b describing the sensor ID of the biometric sensor 402. At the time of data registration and authentication, the communication unit 440b transmits to the authentication server 300 a biometric image captured by the biometric sensor 402, the user ID, and the sensor ID described in the sensor ID data 431b. At the time of authentication, the communication unit 440b also receives an authentication result from the authentication server 300 and outputs the authentication result via a display device or the like.

[0133] The authentication server 300 includes a storage unit 310, a communication unit 311, a normalization processing unit 312, a feature extraction unit 313, an authentication data generation unit 314, a registration processing unit 315, and a matching processing unit 316. The storage unit 310 is a storage area allocated in a storage device included in the authentication server 300. The storage unit 310 stores weight data sets 113b1 and 113b2 generated by the learning device 100b, and authentication databases 311a and 311b. The sensor ID of the biometric sensor 401 is associated with the authentication database 311a, and the sensor ID of the biometric sensor 402 is associated with the authentication database 311b.

[0134] The processing of the communication unit 311, normalization processing unit 312, feature extraction unit 313, authentication data generation unit 314, registration processing unit 315, and matching processing unit 316 is realized, for example, by a processor provided in the authentication server 300 executing a predetermined application program.

[0135] During data registration or authentication, the communication unit 311 receives a biometric image, a user ID, and a sensor ID from the authentication terminals 410 and 420. During authentication, the communication unit 311 also transmits the authentication result to the authentication terminals 410 and 420.

[0136] The normalization processing unit 312, feature extraction unit 313, authentication data generation unit 314, registration processing unit 315, and matching processing unit 316 basically perform the same processing as the normalization processing unit 221, feature extraction unit 222, authentication data generation unit 223, registration processing unit 224, and matching processing unit 225 in FIG. 11 , respectively. However, the authentication data generation unit 314 forms a CNN 50 using a weight data set corresponding to the sensor ID received by the communication unit 311, and calculates authentication data (feature vectors) using the CNN 50. The registration processing unit 315 registers the registration data in an authentication database corresponding to the sensor ID. The matching processing unit 316 extracts the registration data from the authentication database corresponding to the sensor ID, matches it with the matching data, and transmits the authentication result to the communication unit 311.

[0137] The normalization processing unit 312 may be installed in the authentication terminals 410 and 420. In this case, the normalized image is transmitted from the authentication terminals 410 and 420 to the authentication server 300 together with the user ID and the sensor ID. The normalization processing unit 312 and the feature extraction unit 313 may be installed in the authentication terminals 410 and 420. In this case, the feature image is transmitted from the authentication terminals 410 and 420 to the authentication server 300 together with the user ID and the sensor ID.

[0138] Furthermore, the normalization processing unit 312, the feature extraction unit 313, and the authentication data generation unit 314 may be installed in the authentication terminals 410 and 420. In this case, the weight data set 113b1 is stored in the storage unit 430a of the authentication terminal 410, and the authentication data generation unit 314 of the authentication terminal 410 calculates authentication data (feature vectors) using the CNN 50 formed using the weight data set 113b1. The communication unit 440a of the authentication terminal 410 transmits the calculated authentication data to the authentication server 300 together with the user ID and the sensor ID.

[0139] Furthermore, the weight data set 113b2 is stored in the storage unit 430b of the authentication terminal 420, and the authentication data generation unit 314 of the authentication terminal 420 calculates authentication data (feature vector) using the CNN 50 formed using the weight data set 113b2. The communication unit 440b of the authentication terminal 420 transmits the calculated authentication data to the authentication server 300 together with the user ID and the sensor ID.

[0140] The processing functions of the devices shown in each of the above embodiments (e.g., learning devices 10, 100, 100a, 100b, authentication devices 200, 200a, authentication server 300, authentication terminals 410, 420) can be realized by a computer. In this case, a program describing the processing content of the functions that each device should have is provided, and the processing functions are realized on the computer by executing the program. The program describing the processing content can be recorded on a computer-readable recording medium. Examples of computer-readable recording media include magnetic storage devices, optical disks, and semiconductor memories. Examples of magnetic storage devices include hard disk drives (HDDs) and magnetic tapes. Examples of optical disks include CDs (Compact Discs), DVDs (Digital Versatile Discs), and Blu-ray Discs (BD, registered trademark).

[0141] When distributing a program, for example, the program is recorded on a portable recording medium such as a DVD or CD and sold. Alternatively, the program can be stored in a storage device of a server computer and transferred from the server computer to other computers via a network.

[0142] A computer that executes a program stores, for example, a program recorded on a portable recording medium or a program transferred from a server computer in its own storage device. The computer then reads the program from its own storage device and executes processing in accordance with the program. Note that the computer can also read the program directly from a portable recording medium and execute processing in accordance with that program. The computer can also execute processing in accordance with the program received each time a program is transferred from a server computer connected via a network.

[0143] The foregoing merely illustrates the principles of the present invention. Further, since numerous modifications and changes will be apparent to those skilled in the art, the present invention is not limited to the exact construction and application shown and described above, and all corresponding modifications and equivalents are deemed to be within the scope of the present invention as defined by the appended claims and their equivalents.

[0144] REFERENCE SIGNS LIST 1 Neural network 2-1 to 2-n, 2-1a Learning images 3a, 3b Three-dimensional data 4 Body part 10 Learning device 11 Storage unit 12 Processing unit 13 Internal parameters

Claims

1. A learning program that causes a computer to execute the following processes: converting a first learning image for machine learning a neural network that calculates authentication data for biometric authentication from an input biometric image into first three-dimensional data using a first transformation formula based on internal parameters of an imaging device for the biometric image; generating second three-dimensional data by varying the posture of a biological part included in the first three-dimensional data; generating a second learning image by transforming the second three-dimensional data using a second transformation formula that performs an inverse transformation of the transformation using the first transformation formula; and executing machine learning of the neural network using at least the first learning image and the second learning image.

2. The learning program according to claim 1, wherein the first transformation formula and the second transformation formula are generated based on the internal parameters and a predetermined distance from the imaging device to a biological part to be imaged.

3. The learning program according to claim 1, wherein the amount of variation in posture of the body part included in the first three-dimensional data is determined probabilistically.

4. A learning program as described in claim 1, wherein in generating the second three-dimensional data, at least one of the posture or position of the biological part contained in the first three-dimensional data is varied by probabilistically setting a rotation angle and a parallel translation amount in three-dimensional space.

5. The learning program according to claim 4, wherein in generating the second three-dimensional data, the setting probability of the amount of parallel translation in the z-axis direction is set to a value such that the probability of moving in the negative direction is higher than the probability of moving in the positive direction.

6. A learning program as described in claim 1, wherein in the conversion to the first three-dimensional data, a normalization process is performed on the first learning image to correct the posture of the body part on the first learning image to a predetermined posture, thereby generating a normalized image, and the normalized image is converted to the first three-dimensional data by the first conversion formula.

7. The learning program according to claim 1, wherein in the machine learning of the neural network, a Spatial Transformer Network (STN) that is placed in front of the neural network and corrects the posture of the body parts on the inputted body image is machine-learned together with the neural network.

8. The learning program of claim 1, wherein each of a plurality of training images including the first training image is associated with an identification number that identifies the person being photographed, and in the machine learning of the neural network, a classifier that classifies the person is generated by machine learning using the plurality of training images and the second training image as teacher data and the identification number as correct answer data, and the identification number that is the same as that of the first training image is used as the identification number corresponding to the second training image, and during biometric authentication processing, when the neural network receives the biometric image as input, it calculates a feature vector output from an intermediate layer included in the neural network as the authentication data.

9. A learning method, in which a computer converts a first learning image for machine learning a neural network that calculates authentication data for biometric authentication from an input biometric image into first three-dimensional data using a first transformation formula based on internal parameters of an imaging device for the biometric image, generates second three-dimensional data by varying the posture of a biological part included in the first three-dimensional data, generates a second learning image by transforming the second three-dimensional data using a second transformation formula that performs an inverse transformation of the transformation using the first transformation formula, and performs machine learning of the neural network using at least the first learning image and the second learning image.

10. A learning device having a processing unit that converts a first learning image for machine learning a neural network that calculates authentication data for biometric authentication from an input biometric image into first three-dimensional data using a first transformation formula based on internal parameters of an imaging device for the biometric image, generates second three-dimensional data by varying the posture of a biological part included in the first three-dimensional data, generates a second learning image by transforming the second three-dimensional data using a second transformation formula that performs an inverse transformation of the transformation using the first transformation formula, and performs machine learning of the neural network using at least the first learning image and the second learning image.

Images