Personal authentication method, personal authentication device, and program

Abstract

In order to provide a personal authentication method that does not require image alignment each time a target is measured and also does not require template matching, this personal authentication method is configured so that a computer: acquires hyperspectral imaging data of a target; sets a region of interest for the acquired hyperspectral imaging data; extracts spectral information in the region of interest; and uses the extracted spectral information to evaluate similarities to pre-registered data.

Description

Personal authentication method, personal authentication device, and program

[0001] The present invention relates to a personal authentication method, a personal authentication device, and a program.

[0002] As biometric authentication using palm images, for example, fingerprints, veins, and palm prints are used.

[0003] For example, Patent Document 1 (JP 2009-544108 A) discloses a method for performing a biometric authentication function, which includes illuminating a skin site on an individual, receiving light scattered from the skin site under multispectral conditions, deriving multiple biometric authentication modalities from the received light, fusing the multiple biometric authentication modalities into a combined biometric authentication modality, and analyzing the combined biometric authentication modality to perform the biometric authentication function.

[0004] Special Publication No. 2009-544108

[0005] Conventional palm-based authentication methods require image alignment for each palm measurement. Furthermore, conventional authentication methods require template matching. Therefore, conventional authentication methods still have issues with authentication accuracy and processing speed.

[0006] The present invention provides a method for personal authentication that does not require image alignment for each measurement of the subject, nor does it require template matching.

[0007] That is, a first aspect of the present invention provides a personal authentication method, characterized in that a computer executes the steps of: acquiring hyperspectral imaging data of a target; setting a region of interest for the acquired hyperspectral imaging data; extracting spectral information in the region of interest; and evaluating the similarity of the extracted spectral information with pre-registered data. Here, the target is a biometric feature that indicates a biometric feature, such as a palm, a sole, or a skin surface including a face. The region of interest includes at least a portion of the biometric feature.

[0008] In the personal authentication method of the present invention, it is preferable to detect landmarks in the acquired hyperspectral imaging data, and set the region of interest using the landmarks.

[0009] In addition, in the personal authentication method of the present invention, it is preferable to generate a feature vector using the extracted spectral information, and evaluate the similarity using the generated feature vector. Here, the feature vector is a vector composed of feature quantities. The feature vector can be generated, for example, by describing local features of the image that constitutes the spectral information.

[0010] In addition, in the personal authentication method of the present invention, it is preferable to generate a pseudo-color image from the acquired hyperspectral imaging data, and detect the landmarks in the generated pseudo-color image. Here, the pseudo-color image is obtained by pseudo-coloring the grayscale image by assigning an appropriate color to each density value.

[0011] A second aspect of the present invention provides a personal authentication device comprising: an acquisition unit that acquires hyperspectral imaging data of a target; a setting unit that sets a region of interest for the acquired hyperspectral imaging data; a feature extraction unit that extracts spectral information in the region of interest; and an evaluation unit that uses the extracted spectral information to evaluate the similarity with pre-registered data.

[0012] A third aspect of the present invention provides a program for causing a computer to execute the steps of: acquiring hyperspectral imaging data of a target; setting a region of interest for the acquired hyperspectral imaging data; extracting spectral information in the region of interest; and evaluating similarity with pre-registered data using the extracted spectral information.

[0013] The personal authentication method, personal authentication device, and program of the present invention eliminate the need for image alignment for each measurement of the subject, and also eliminate the need for template matching, which is expected to improve authentication accuracy and processing speed.

[0014] 1 is a flowchart showing the steps of a personal authentication method according to an embodiment of the present invention. It is a diagram showing an overview of acquiring hyperspectral imaging data P of a target T, setting landmarks L, and setting a region of interest. It is an explanatory diagram of landmarks L set on a palm. It is a diagram showing an overview of extracting feature images and generating feature vectors. It is an explanatory diagram of deep metric learning as an example of a personal authentication method. It is a block diagram showing the software configuration of a personal authentication device 1. It is a schematic diagram of a hyperspectral camera 3 and a palm rest 5. It is a diagram showing an example of a cross-sectional hyperspectral image obtained from subject 1 in an experimental example. It is a diagram showing a cross-sectional hyperspectral image of the hand of subject 1. It is a diagram showing a cross-sectional hyperspectral image of the hand of subject 2-10. It is a diagram showing a feature histogram of subject 1. It is a diagram showing a feature histogram of subjects 2-10. It is a comparison of different dimensionality reduction algorithms for subject data. It is a diagram showing a probability distribution histogram showing the relationship between Euclidean distance and probability density within and between subjects. It is a diagram showing significant differences in Euclidean distance within and between subjects. It is a diagram showing changes in false acceptance rate (FAR) and false rejection rate (FRR) under different thresholds. 1A-1C show receiver operating characteristic (ROC) curves of different dimensionality reduction algorithms, 1B show clustering results for data of subject A and other subjects acquired under various tilt conditions in additional experiments, 1C show clustering results for data of subject A and other subjects acquired under various lighting conditions in additional experiments, and 1D show clustering results for data of subject A and other subjects acquired under various hover conditions in additional experiments.

[0015] Representative embodiments of the personal authentication method, personal authentication device, and program according to the present invention will be described in detail below with reference to the drawings. However, the present invention is not limited to these drawings. Furthermore, since the drawings are intended to conceptually explain the present invention, dimensions, ratios, and numbers may be exaggerated or simplified as necessary to facilitate understanding.

[0016] 1. Personal Authentication Method The personal authentication method according to this embodiment will be described mainly with reference to FIGS.

[0017] The personal authentication method includes the execution of the following processes by a computer (see FIG. 1): Step 1: Acquiring hyperspectral imaging (HSI) data P of a target T; Step 2: Setting a region of interest for the acquired HSI data P; Step 3: Extracting spectral information in the region of interest; Step 4: Using the extracted spectral information, evaluating the similarity with pre-registered data. The above processes will be explained in order below.

[0018] (1) Step 1 In step 1, HSI data P of the target T is acquired (see the upper diagram in Figure 2). The HSI data P is image data obtained by photographing the target T with the hyperspectral camera 3. The target T is a biometric feature that indicates the characteristics of a living body. In this embodiment, the palm is used as an example of the target T, but the present invention is not limited to the palm and may be any skin surface, such as the sole of the foot or the face. Furthermore, there are no restrictions on the orientation or position of the target T in the HSI data P. In other words, this method is robust against translational movement and rotation of the target T.

[0019] The hyperspectral camera 3 can capture images as tens to hundreds of pieces of image data in wavelength widths of several nanometers across a wide wavelength range from visible light to infrared. In this embodiment, a camera that is compatible with a wavelength range of, for example, 400 to 1000 nm, that is, light from visible light to near-infrared, can be suitably used, but is not limited to this. The image data obtained from the hyperspectral camera 3 is also called cube data because it is managed in a three-dimensional (3D) structure in which images for each wavelength are superimposed.

[0020] Image data captured by the hyperspectral camera 3, i.e., HSI data P, may be transmitted via a wired or wireless communication network to a computer constituting the personal authentication device 1. Preprocessing such as noise reduction may be performed on the HSI data P in the computer. Known image processing such as 3D Gaussian can be used for noise reduction.

[0021] (2) Step 2 In step 2, a region of interest (ROI) is set for the acquired HSI data P. For example, landmarks L may be detected in the acquired HSI data P and the ROI may be set using the landmarks L (see the middle diagram in Figure 2). The landmarks L are marks that indicate pre-set positions on the target T. In the example of the palm shown in Figure 2, landmarks are set mainly at the positions of the joints. The ROI is a region or portion in the image to which image processing should be applied, and includes at least a portion of the biometric features.

[0022] The landmarks L are determined by image analysis and can be realized by known software. Examples of image analysis software that can be used include, but are not limited to, MediaPipe Hands, an open-source machine learning library provided by Google, Inc. MediaPipe Hands can set 21 landmarks L on a palm image (see FIG. 3).

[0023] Here, the acquired HSI data P is usually expressed in black and white. Therefore, to facilitate detection of the landmarks L, a pseudo-color image may be generated from the acquired HSI data P. The pseudo-color image is obtained by pseudo-coloring the grayscale image by assigning an appropriate color to each density value. In this case, the landmarks L are set in the generated pseudo-color image. The pseudo-color image may be, for example, an RGB image. The pseudo-color image can be generated using known image processing software such as Adobe Photoshop (registered trademark).

[0024] Returning to the setting of the ROI, the position of the ROI to be set can be anywhere as long as it clearly shows the individual's characteristics. In the example shown in the lower diagram of Figure 2, the ROI is set as a straight line connecting the midpoint x of #9 (base of the middle finger) and #13 (base of the ring finger) with #0 (wrist).

[0025] (3) Step 3 In step 3, spectral information in the ROI is extracted. For example, a feature image is created by stacking data on the ROI. In the example shown in the upper part of Figure 4, a two-dimensional image is obtained as the feature image, with the position on the ROI (straight line 0-x) on the horizontal axis and the wavelength on the vertical axis. If the ROI has width, the feature image becomes a three-dimensional image.

[0026] The feature image contains unique patterns depending on the palm lines and blood vessel distribution. In other words, the feature image is thought to reflect different features in the short wavelength band (upper side in the upper diagram of Figure 4) and the long wavelength band (lower side in the upper diagram of Figure 4). Therefore, the feature image reflects individual characteristics such as mottle features, capillaries, and veins, and is thought to be unique, which contributes to improving identification accuracy.

[0027] The feature image may be resized for evaluation processing. There is no limit to the size of the resized data, but the data size may be, for example, 100 x 100 pixels. The resizing processing can be performed by known image processing software, and for example, the open source software ImageJ is available.

[0028] (4) Step 4 In step 4, the target T is evaluated using the extracted spectral information. For example, a feature vector may be generated using the extracted spectral information (e.g., a feature image) (see the lower diagram in FIG. 4), and the generated feature vector may be used to evaluate the similarity with pre-registered data. Here, the feature vector is a vector composed of feature quantities. The feature vector may be generated, for example, by describing local features of the image that constitutes the spectral information.

[0029] More specifically, feature vectors can be calculated using known image feature extraction techniques, such as local binary patterns (LBPs) and their derivatives. LBPs represent the density pattern of neighboring pixels relative to a pixel of interest. For example, a binarization process is performed on each of the pixels (e.g., eight pixels) adjacent to the pixel of interest, assigning a "1" if the neighboring pixel's value is greater than the pixel of interest's value and a "0" if the neighboring pixel's value is less than the pixel of interest's value. The resulting "1s" and "0s" are then arranged (e.g., clockwise) according to the arrangement of the neighboring pixels to create an eight-digit number string. The resulting number string is then converted into an 8-bit binary number to obtain the LBP value of the pixel of interest. Performing this process on the pixels of the entire image generates an LBP image, each pixel of which has an 8-bit LBP value. The bottom diagram in Figure 4 shows an example of a feature vector obtained by LBP. Using LBP reduces computational costs while providing robustness to brightness changes and high discrimination performance. The above process can be performed using known image processing software, or a development environment compatible with the process can be used. An example of the latter is the function IMAQ Extract LBP Feature Vector VI in the development environment LabVIEW (LabVIEW 2020, National Instruments Corporation, USA).

[0030] The obtained feature vector is evaluated for similarity with pre-registered data to perform personal authentication. Deep metric learning, for example, can be used to evaluate the similarity.

[0031] Deep metric learning is a technique for learning the distance and similarity between data using deep learning. As shown in Figure 5, metric learning calculates the distance and similarity between data ((B) in the figure) from the original data ((A) in the figure). By updating the weights ((C) in the figure) so that data from the same class is closer and data from different classes is farther apart, an embedding space that effectively represents the data is learned ((D) in the figure). As a result, in the embedding space, data belonging to the same class are placed close to each other, and data belonging to different classes are placed far apart ((E) in the figure). The distance between data in this space reflects their similarity. Therefore, high-dimensional data can be mapped into a low-dimensional feature space using deep learning. Newly input data is mapped onto the created feature space, and the distance or similarity between the data is calculated using metric learning. In other words, data belonging to the same class are placed close to each other, and data belonging to different classes are placed far apart, making it possible to evaluate similarity. This enables individual identification. The above process can be performed using known machine learning software; for example, the open-source machine learning library for Python, scikit-learn, is available.

[0032] 2. Personal authentication device 1 and program The personal authentication device 1 will be described with reference to Fig. 6. (1) Configuration of personal authentication device 1 As shown in the figure, the personal authentication device 1 includes an image acquisition unit 11, a setting unit 12, a feature extraction unit 13, and an evaluation unit 14.

[0033] The image acquisition unit 11 is configured to acquire and store HSI data P of the target T. The image acquisition unit 11 may be an interface that acquires the HSI data P from the hyperspectral camera 3, or may be the hyperspectral camera 3 itself. The image acquisition unit 11 may perform preprocessing such as noise reduction on the acquired HSI data P.

[0034] The setting unit 12 is configured to set an ROI for the acquired HSI data P. Alternatively, the setting unit 12 may cause an external computer to set the ROI, for example, through an application programming interface (API). The setting unit 12 may perform pseudo-coloring processing on the HSI data P prior to setting the ROI.

[0035] The feature extraction unit 13 is configured to extract spectral information in the ROI. For example, the feature extraction unit 13 can create a feature image (a two-dimensional or three-dimensional image) as the spectral information by stacking data on the ROI. The feature extraction unit 13 may resize the created feature image to a predetermined size.

[0036] The evaluation unit 14 is configured to use the extracted spectral information to evaluate the target T. For example, the evaluation unit 14 can generate a feature vector from the spectral information and use this feature vector to evaluate the similarity with pre-registered data. Note that the generation of the feature vector may be the role of the feature extraction unit 13.

[0037] (2) Hardware Configuration of the Computer Constituting the Personal Authentication Device 1 The following describes the hardware configuration of a computer that can execute the personal authentication method, i.e., the personal authentication device 1. The computer includes a calculation unit, a storage unit, and a communication interface (I / F), and may further include an input device and an output device. The computer may be composed of one computer or multiple computers.

[0038] The arithmetic unit realizes the above-mentioned various functions (including the image acquisition unit 11, the setting unit 12, the feature extraction unit 13, and the evaluation unit 14) by reading various programs and data into the storage device and executing them. The arithmetic unit may be configured with semiconductor integrated circuits such as a central processing unit (CPU), a graphics processing unit (GPU), and a microprocessor.

[0039] The storage device includes a random access memory (RAM) and a read-only memory (ROM) that store the various data and programs described above. The storage device may include, for example, a hard disk drive, a solid-state drive, and a flash memory.

[0040] The input device is used to input various data and is, for example, a keyboard, a mouse, a touch panel, a button, a microphone, etc. The input device may include the hyperspectral camera 3. The output device is used to output various data and is, for example, a display, a printer, a speaker, etc.

[0041] The communication interface is an interface for connecting to wired and wireless communication networks, such as an adapter for connecting to Ethernet (registered trademark), a modem for connecting to a public telephone network, a wireless communication device for wireless communication, a USB (Universal Serial Bus) connector for serial communication, an RS232C connector, etc. The communication interface can be used for communication with the hyperspectral camera 3 or for communication with a required API.

[0042] (3) Security System Using Personal Authentication Device 1 The personal authentication device 1 can be combined with other machines and tools to form a security system. One example of such other machines and tools is a hyperspectral camera 3. The hyperspectral camera 3 separates light into wavelengths and captures images, so it can obtain more information than an RGB camera. The wavelength range available in the hyperspectral camera 3 has already been described. The hyperspectral camera 3 may be of the push-broom (line scan) type or the snapshot type. Any hyperspectral camera 3 may be used as long as it supports a sufficient wavelength range; for example, a camera manufactured by Eva Japan Co., Ltd. may be used.

[0043] Another example of a component of the security system is a palm rest 5 (see FIG. 7 ). The rest 5 has a transparent plate 51 on its upper surface. The transparent plate 51 is the portion on which the palm serving as the target T is placed and may be composed of, for example, a glass plate. The transparent plate 51 may have dimensions of, for example, approximately 200 mm x 200 mm to 300 mm x 300 mm. The transparent plate 51 may be approximately parallel to the installation surface of the rest 5 or may be inclined toward the user at, for example, an angle of 20° to 40°. The rest 5 may also have a light source 52. The light source 52 irradiates light onto the underside of the transparent plate 51 and may be, for example, a light-emitting diode. While there is no limit to the light intensity of the light source 52, increasing the light intensity and the number of light sources can shorten the exposure time, i.e., the imaging time, on the camera side. The hyperspectral camera 3 may be disposed on the underside of the transparent plate 51 of the rest 5.

[0044] A representative embodiment of the present invention has been described above. In this embodiment, by using full-wavelength information obtained by HSI, two-dimensional (or three-dimensional) images of spatial (positional) information and wavelength information can be obtained non-contact and non-invasively. By utilizing such biometric imaging data, it is possible to evaluate various characteristics according to the absorption of light in biological (skin) tissue and the propagation depth of each wavelength. In other words, this two-dimensional image information is considered to reflect a wide variety of information, such as palm prints, mottled patterns, capillary blood vessel patterns, and vein patterns. As such, this two-dimensional information contains a wealth of information for biometric authentication, and is expected to provide improved authentication accuracy compared to conventional personal authentication methods using RGB images.

[0045] In this embodiment, the region of interest for measurement is extracted using artificial intelligence (AI) image analysis, eliminating the need for image alignment for each measurement and enabling extraction of approximately the same position for each measurement. This means that the system is robust against translation and rotation of the target T. Furthermore, it is expected that personal authentication can be performed even when the palm is partially or completely lifted, or when the light direction is different. Furthermore, personal identification is performed by evaluating the similarity or distance between the extracted feature vector and registered data, eliminating the need for template matching as in the past. These processes are expected to further improve identification accuracy and processing speed.

[0046] In other embodiments, the palm may be photographed using a camera other than a hyperspectral camera to determine the ROI. Here, the non-hyperspectral camera may be, for example, an RGB camera. There are no particular limitations on the performance of the RGB camera, and any camera capable of capturing an image sufficient for setting the ROI described below may be used. AI image analysis (e.g., MediaPipe) is applied to the captured image to recognize the hand and set the ROI. Details of hand recognition and ROI setting have been described in this embodiment. Only the portion of the hand corresponding to the set ROI is captured by the hyperspectral camera. The specifications of the hyperspectral camera have been described in this embodiment. However, the hyperspectral camera is adjusted to roughly match the angle of view of the RGB camera. This results in a hyperspectral image with a limited imaging range. The obtained hyperspectral image is subjected to the analysis described in this embodiment to extract feature images (e.g., spectral information), followed by clustering and personal authentication. This configuration and processing are expected to significantly reduce the time from palm photography to personal authentication.

[0047] In yet another embodiment, the intensity of light reflected from the palm of the hand can be recorded with a video camera during HSI measurement to evaluate heart rate variability. This allows us to determine whether the measurement target is a living organism, and can be used to determine whether or not an image is being used to identify someone else.

[0048] In another embodiment, a person's health can be monitored by continuously acquiring HSI data from the same person. For example, a health management method and system can be configured to continuously evaluate heart rate variability extracted from HSI data to understand the person's health. The method and system may issue an alert if it detects heart rate variability that differs from normal heart rate variability.

[0049] Experimental Example The following is an experimental example. The purpose of this experimental example is to demonstrate that local cross-sectional hyperspectral images of the palm can accurately identify individuals.

[0050] The experimental setup is outlined below. The experimental equipment included a hyperspectral camera, a palm rest, a broadband illumination source, and a computer. Hyperspectral images of the hand were captured using a hyperspectral camera (NH-A-S, Eva Japan Co., Ltd., Japan) equipped with a single-focus lens (f = 12 mm, M118FM12, Tamron Co., Ltd., Japan) that provided a spectral resolution of 5 nm across the entire 400-1000 nm range. The rest had a scan area of ​​240 mm x 240 mm, was installed 900 mm above the floor, and was tilted at an angle of approximately 30 degrees relative to the horizontal. A 5 mm-thick, highly transparent glass plate (OOKABE GLASS Co., Ltd., Japan) was inserted to transmit visible to near-infrared light. The subjects placed their palms on the glass plate, and hyperspectral images were acquired while maintaining a certain distance between the lens and the subject. The placement of the palm on the rest was unrestricted, except that the fingers were facing forward. Hyperspectral images of the palm were acquired through glass using a low-angle shot from a hyperspectral camera. The tip of the hyperspectral camera lens was positioned approximately 500 mm behind the platform, approximately 450 mm above the floor, and tilted at an angle of approximately 40° relative to the horizontal. A 500W halogen lamp (CTW-1550, Sankyo Corporation, Japan) was installed below the scanning section and illuminated the subject's palm from behind the glass. Software provided by the camera manufacturer (NH Capture, Eva Japan Co., Ltd., Japan) was used to control the camera and acquire data. Spectral reflectance was captured with the hyperspectral camera, and hyperspectral cube data (640 × 480 pixels, 121 bands) was saved to a computer hard disk. The scan speed was set to 20 lines / s, and the camera exposure time was set to 0.05 s. The total scan time was approximately 24 s.

[0051] Landmarks were detected from palm images using MediaPipe Hands (version 0.10.1), an open-source image processing machine learning library. To improve the recognition accuracy of MediaPipe Hands, pseudo-RGB images were generated from the hyperspectral images. The RGB images were generated using image processing software developed in the LabVIEW development environment. Next, a pre-trained MediaPipe Hand landmark model was applied to the RGB palm image to automatically generate 20 landmarks (see Figure 3 ). Using these landmarks, a line was drawn as an ROI passing through landmark #0, and the midpoints of landmarks #9 and #13 were determined using ImageJ (version 1.53t). Before tracing the ROI on the hyperspectral image, appropriate filters were applied in both the spatial and spectral directions to reduce image noise.

[0052] The hyperspectral image was resliced ​​along the ROI using the "Reslice" function in ImageJ to generate a 2D spatiospectral image. The resliced ​​image was resized to 100 × 100 pixels using bilinear interpolation. Features were extracted from the resized 2D spatiospectral image using local binary patterns (LBP). The resized image was divided into 25 non-overlapping square subregions of equal size (20 × 20). Features were extracted from each subregion, and a histogram was created for each subregion. These histograms were concatenated into a long vector, which served as the feature vector for the hyperspectral image of the hand. Image processing was performed using the IMAQ Extract LBP Feature Vector VI function in the development environment LabVIEW.

[0053] Feature values ​​extracted from biometric data were analyzed using k-means algorithms along with principal component analysis (PCA), t-distributed stochastic neighbor embedding (t-SNE), and uniform manifold approximation and projection (UMAP). The above process was performed using the Python machine learning library scikit-learn. These dimensionality reduction techniques convert feature vectors from high-dimensional space to low-dimensional space (from 225 dimensions to 2 dimensions in this experiment) while preserving meaningful characteristics of the original data. The similarity between different feature vectors after dimensionality reduction was evaluated using Euclidean distance. Small Euclidean distances are expected within subjects, while larger distances between subjects. Furthermore, the statistical significance of the difference in mean Euclidean distance between two groups (intra- and inter-subject) was evaluated using an unpaired Welch t-test. Kaleidagraph 5.0 (Hulinks Co., Ltd., Japan) was used for this process. To define the reference threshold, the false acceptance rate (FAR) and false rejection rate (FRR) were calculated. FAR is defined as the number of incorrectly accepted individuals divided by the total number of incorrect matches, and FRR is defined as the number of incorrectly rejected individuals divided by the total number of correct matches. The total intra-class matches was 900, and the total inter-class matches was 9,000. FAR and FRR were calculated at each Euclidean distance threshold and increased stepwise. Furthermore, recognition performance was evaluated using the equal error rate (EER), the point at which FAR and FRR are equal. Clustering performance for user identification was evaluated using receiver operating characteristic (ROC) curves, where the true acceptance rate (TAR), defined as 1-FRR, was plotted as a function of FAR. To quantitatively evaluate performance based on the ROC curve, the area under the curve (AUC) was calculated.

[0054] Ten healthy adults (seven men and three women) participated in this example. Participants' ages ranged from 24 to 47 years old. Each subject had their palm scanned ten times using the experimental device.

[0055] The experimental results are presented below. Figure 8 shows an example of a hyperspectral image and a cross-sectional image along a line of interest. The cross-sectional image consists of a series of spectra ranging from 400 to 1000 nm with 5 nm intervals. The sequence of spectra depicted a texture pattern. Furthermore, shadow lines perpendicular to the cutting line were observed in the image. These shadow lines corresponded to palm surface morphologies such as interphalangeal joint lines, palmar lines, palm prints, and hand wrinkles.

[0056] Figure 9 shows a cross-sectional image of one subject and Figure 10 shows cross-sectional images of the remaining subjects. Overall, the cross-sectional hyperspectral images exhibit a hierarchical structure based on a brightness gradient. Brightness was dark in the short wavelength range and bright in the mid- to long-wavelength range, with a clear brightness distribution observed in each layer. Vertical shadow lines were also present in the cross-sectional images. Images from the same subject tended to exhibit similar patterns, while images from different subjects tended to exhibit different patterns. Feature vectors extracted using LBP histograms from the data in Figures 9 and 10 are shown in Figures 11 and 12, respectively. While similar trends were observed in the feature vectors from the same subject, no similarity was observed in the histogram patterns from different subjects.

[0057] Figures 13(a)-(c) show the performance of PCA, t-SNE, and K-means with UMAP255 clustering of feature vectors extracted from cross-sectional images using LBP. The figures visually demonstrate that the data clusters are well separated. In particular, UMAP shows the best clustering, t-SNE shows better clustering than PCA, and PCA also shows good clustering.

[0058] To determine the performance of biological clustering using hyperspectral images, we calculated the distribution of Euclidean distance-based discriminant functions. The Euclidean distances for inter- and intra-subject matching were analyzed (Fig. 14(a)-(c)). In all cases, the distribution curves based on Gaussian approximations showed a clear bimodal distribution. The results also showed that inter-subject distances were broadly distributed, while intra-subject distances were significantly more peaked. Furthermore, in all cases, inter-subject distances were significantly larger than intra-subject distances (P<0.0001) (Fig. 15(a)-(c)).

[0059] Table 1 below summarizes the average values, standard deviations, maximum values, and minimum values ​​obtained from FIGS.

[0060] FAR and FRR were used to evaluate the authentication accuracy. Figures 16(a)-(c) show the changes in FAR and FRR under different Euclidean distances in each dimensionality-reduced space using PCA, t-SNE, and UMAP. The horizontal axis of these graphs represents the threshold for the normalized Euclidean distance in each dimensionality-reduced space, ranging from 0 to 1. The closer the normalized Euclidean distance between two data points in this space is to 0, the more likely they are from the same subject. Conversely, the closer the distance is to 1, the more likely they are from different subjects. By setting the threshold within a range, distances below the threshold are considered to be from the same subject, and distances above the threshold are considered to be from different subjects. Therefore, as the threshold increases, FAR increases and FRR decreases. Similarly, as the threshold decreases, FRR increases and FAR decreases. The threshold is located at the intersection of the FAR and FRR plots. The value at this intersection represents the EER. The lowest threshold was observed for UMAP, followed by t-SNE and PCA.

[0061] Figure 17 shows the calculation results of the receiver operating characteristic (ROC) curve. The area under the curve (AUC) is used as the optimization target because it adequately represents the ROC performance. The validation results of EER and AUC are shown in Table 2. From Table 2, it can be seen that UMAP outperforms other methods.

[0062]

[0063] Additional Experiments Additional experiments were conducted to investigate the effects of hand placement and light source position and intensity on image formation and person identification results.

[0064] In additional experiments, the hand was not only in contact with the glass plate but also tilted laterally and anteriorly at approximately 5, 10, 20, and 30 degrees, and five images were taken for each condition. For each palm tilt condition, data from subject A obtained from the additional experiment and data from randomly selected subjects 4, 7, and 10 from the main experiment (obtained at the baseline position) were clustered using UMAP. The results are shown in Figures 18(a)-(d). Subject A's data is enclosed in a dashed box in the graph. Subject A's data is clustered, although there is some variability due to differences in tilt angle. In other words, even under various palm tilt conditions, the ROI was appropriately set using AI image analysis, and hyperspectral tomographic images were obtained. As a result, individuals were appropriately clustered.

[0065] In experiments on light source position and intensity, the light source was illuminated on the right side (default), left side, front side, back side, and two left and right sides of the palm rest. The results of subject A using four different light source positions and dual light sources were clustered using UMAP along with the data of subjects 4, 7, and 10. The results are shown in Figure 19. The data of subject A is enclosed in a dashed box in the graph. Although there is some variability due to differences in the light source position, it can be seen that the data of subject A is sufficiently clustered. It was also found that the hyperspectral image processing can be shortened by using two light sources.

[0066] Additionally, images were taken under palm hover and supported hover conditions. In the palm hover condition, the palm was raised approximately 10 mm from the glass plate and held in that position during the scan. In the supported hover condition, the wrist was pressed against the glass plate and the palm was raised approximately 10 mm from the pressed position. Data from subject A (whether the palm touched the glass or not) and data from subjects 4, 7, and 10 (acquired at the baseline position) were clustered using UMAP. The results are shown in Figure 20. Subject A's data is enclosed in a dashed box in the graph. While there is some variability due to differences in palm contact and non-contact and light source, subject A's data is well clustered. In other words, even under various palm hover conditions, AI image analysis properly set the ROI and acquired hyperspectral tomographic images. As a result, individuals were properly clustered.

[0067] Summary of Experimental Example: This experimental example demonstrated an effective biometric authentication technique using cross-sectional hyperspectral images of the palm. Cross-sectional hyperspectral images reveal patterns unique to each individual. The developed system uses a machine learning library to automatically set the ROI without complex image registration. Despite the small ROI compared to the entire palm, optimal performance achieved an area under the ROC curve of 0.98, with an EER of 0.04% at best performance. This technology was shown to help access several services more easily, quickly, and securely.

[0068] Personal authentication and identification using palm images has attracted considerable attention. Palm vein patterns, which are the dense network of veins that spread across the palm, are also used for authentication. However, scanning a large area of ​​the palm increases the time required for data acquisition and analysis.

[0069] In this example, the scanned area of ​​the palm was reduced to a single line in the hyperspectral image. Although spatial information about the distribution of veins in the palm was lost, the cross-sectional hyperspectral image of the palm was highly informative in the wavelength direction. The hyperspectral image of the palm provided distinct features at each wavelength. These features were generated by the skin's absorption distribution and the different tissue penetration depths of light wavelengths. The cross-sectional hyperspectral image exhibited overlapping gradient patterns from short to long wavelengths. The wavelength gradient pattern corresponded to the intensity change in the hyperspectral image across all spectral bands. The low-intensity layer, corresponding to approximately 400 to 600 nm, exhibited a speckled pattern in the spectral image, appearing as dark regions (Figures 9 and 10). Furthermore, vein-like patterns were also observed in some of the longer wavelength bands. The high-intensity layer above 600 nm exhibited a different pattern, with the blood vessel-like pattern also appearing as dark regions in the spectral image (Figures 9 and 10).

[0070] The low brightness at short wavelengths in hyperspectral images is primarily due to two factors. First, the sensor's sensitivity is low at the extremes of the spectrum. Second, short wavelengths can be absorbed by the numerous capillaries near the skin surface. In contrast, the high brightness at mid- to long-wavelengths is due to the deeper penetration of light, which is due to the fewer and larger blood vessels compared to capillaries. For the second reason, hyperspectral images show a speckled pattern at short wavelengths that disappears as the wavelength increases, revealing vein-like patterns. The distribution of capillaries and veins in the skin varies from person to person. Personal identification using vascular patterns typically uses a two-dimensional distribution, but blood vessels are distributed in three dimensions. Because hyperspectral images use light of different wavelengths to penetrate the skin at different depths, cross-sectional hyperspectral images reflect the distribution of blood vessels in the depth direction. As a result, cross-sectional images can provide unique patterns for individual identification.

[0071] Furthermore, the palm print matched the striped shadows in the cross-sectional hyperspectral image (Figure 8). Cross-sectional hyperspectral images contain information not only about the inside of the body but also about the surface. Multi-biometrics, including palm prints and knuckle prints, palm prints and veins on the back of the hand, and palm prints and veins on the palm, have been successfully used to efficiently improve accuracy. This shows that cross-sectional images, even in limited areas, may contain enough features to identify individuals. Cross-sectional images of the same location contain individual-specific patterns, enabling authentication.

[0072] Extracting an ROI is a critical step in the palm recognition process, as the location of the ROI significantly impacts feature extraction within the palm image. Most ROI extraction algorithms utilize keypoints between the fingers to establish a coordinate system. It is important to note that in contactless images, palm images have numerous translational and rotational variations. In this test example, an AI-based approach was used to extract an ROI from the palm image. The feature vector extracted using this ROI can classify each individual. Therefore, the AI-based approach successfully selected a nearly identical ROI position each time. The scan area in this example was not as large as the palm, allowing the hand to be placed anywhere. In fact, for one subject, some of the hand placements were intentionally changed across 10 scans. For example, the fingertips were positioned at angles ranging from approximately 0° to 90°. However, these features from the subject using ROI detection were nearly identical (Figure 9). Therefore, the AI-based ROI placement technique is robust to palm misalignment and rotation. Additionally, additional experimental results show that the technology is robust to palm tilt and hover.

[0073] Cross-sectional images extracted from the ROIs were converted into feature vectors using LBP. To visualize data relationships, we converted the high-dimensional feature vectors into two-dimensional vectors using dimensionality reduction algorithms such as PCA, t-SNE, and UMAP (Figure 13). These results show 10 clusters of the 10-subject dataset based on similarity. As a result, the texture patterns of the cross-sectional hyperspectral images (Figures 9 and 10) and feature vectors (Figures 11 and 12) contained distinct features. Clustering results for the UMAP dimensionality-reduced data revealed high aggregation of data for each label (Figure 13(c)). UMAP is a method in which similar data in the original feature space are closely plotted after dimensionality reduction. Comparing the performance of dimensionality reduction methods in clustering, UMAP produced better results than the other two algorithms. Furthermore, UMAP has a rigorous mathematical foundation yet is easy to use with a scikit-learn-compatible API. UMAP is also one of the fastest manifold learning implementations available, significantly faster than most t-SNE implementations, so UMAP is likely to offer advantages not only in clustering accuracy but also in processing speed.

[0074] In this experimental example, we demonstrate biometric authentication using hyperspectral palm images spanning visible to near-infrared wavelengths at a resolution of 5 nm. First, a machine learning-based method for ROI detection is introduced. Next, feature vectors are extracted from longitudinal cross-sectional hyperspectral images to capture spectral binding and skin surface morphology. Finally, a dimensionality reduction technique is used to evaluate the effectiveness of the proposed biometric authentication method. The evaluation results demonstrate that hyperspectral personal identification can achieve excellent performance.

[0075] The present invention can be used for personal biometric authentication, security systems, health management, and the like.

[0076] Although typical embodiments of the present invention have been described above, the present invention is not limited to these, and various design modifications are possible, which are also included in the present invention.

[0077] REFERENCE SIGNS LIST 1 personal authentication device 11 image acquisition unit 12 setting unit 13 feature extraction unit 14 evaluation unit 3 hyperspectral camera 5 mounting base

Claims

1. A personal authentication method characterized in that a computer executes the steps of: acquiring hyperspectral imaging data of a target; setting a region of interest for the acquired hyperspectral imaging data; extracting spectral information in the region of interest; and evaluating the similarity of the extracted spectral information with pre-registered data.

2. The personal authentication method according to claim 1, further comprising the steps of: detecting landmarks in the acquired hyperspectral imaging data; and setting the region of interest using the landmarks.

3. The method of claim 1, further comprising the steps of: generating a feature vector using the extracted spectral information; and evaluating the similarity using the generated feature vector.

4. The personal authentication method according to claim 2, further comprising: generating a pseudo-color image from the acquired hyperspectral imaging data; and detecting the landmarks in the generated pseudo-color image.

5. A personal authentication device comprising: an acquisition unit that acquires hyperspectral imaging data of a target; a setting unit that sets a region of interest for the acquired hyperspectral imaging data; a feature extraction unit that extracts spectral information in the region of interest; and an evaluation unit that uses the extracted spectral information to evaluate the similarity with pre-registered data.

6. A program for causing a computer to execute the steps of: acquiring hyperspectral imaging data of a target; setting a region of interest for the acquired hyperspectral imaging data; extracting spectral information in the region of interest; and evaluating the similarity of the extracted spectral information with pre-registered data.

Images