Biometric authentication processing program, biometric authentication processing method, and information processing device

Abstract

The present invention improves determination accuracy of a spoofing attack on biometric authentication.　An information processing device (10) determines the brightness of image data (13) from a plurality of pixel values included in the image data (13). The information processing device (10) generates, from the image data (13), frequency component data (14) indicating a frequency component of a frequency section corresponding to the brightness so as to indicate a frequency component of a frequency section (16a) when the brightness is at a level (15a), and to indicate a frequency component of a frequency section (16b) including the frequency higher than that of the frequency section (16a) when the brightness is at a level (15b) that is higher than the level (15a). The information processing device (10) inputs the image data (13) and the frequency component data (14) into a machine learning model (17), thereby determining whether the image data (13) indicates a spoofing attack on biometric authentication.

Description

Biometric authentication processing program, biometric authentication processing method, and information processing device

[0001] The present invention relates to a biometric authentication processing program, a biometric authentication processing method, and an information processing device.

[0002] Information processing systems sometimes use personal authentication technologies to determine whether a person is a registered person. Personal authentication technologies include biometric authentication technologies that use a person's physical or behavioral characteristics to authenticate the person. Information processing systems sometimes capture images of the person to be authenticated to obtain biometric images such as a face image or a palm image, and then perform biometric authentication using the biometric images.

[0003] Information processing systems that perform biometric authentication may be subject to spoofing attacks in which an attacker attempts to successfully authenticate a user by impersonating a registered user. The act of presenting false biometric information to an information processing system is sometimes called a presentation attack. For example, an attacker may display a false biometric image prepared in advance on a mobile device and present the mobile device to the information processing system. With the widespread use of high-resolution imaging devices and display devices, there is a risk that information processing systems may mistakenly authenticate an attacker based on a false biometric image.

[0004] An image processing device has been proposed that generates a corrected image from an input image in which shadows caused by lighting have been reduced. This image processing device generates a high-frequency component image having frequency components higher than the reference frequency while changing the reference frequency. The image processing device generates the corrected image by weighting and combining multiple high-frequency component images with different reference frequencies.

[0005] In addition, a face recognition method has been proposed for detecting impersonation attacks in a face recognition personal authentication system by presenting a face image of another person displayed on a display device. Another fraud detection method has been proposed for detecting impersonation attacks by extracting noise features specific to fraudulent face images from acquired face images. This fraud detection method separates a face image into multiple color channel images and inputs the differences between the multiple color channel images into a convolutional neural network.

[0006] An image detection device has also been proposed that determines whether a captured facial image is forged. This image detection device extracts spatial features from the facial image, extracts frequency features from the frequency components of the facial image, and determines whether the image is forged using the spatial features and frequency features. A method has also been proposed to improve the spoofing resistance of neural networks for image recognition. Another spoofing detection method has been proposed that combines Fourier transforms, multiple classification methods, and multiple feature extraction methods to determine whether a facial image corresponds to a spoofing attack.

[0007] Japanese Patent Publication No. 2014-16912 U.S. Patent No. 10,061,996 Chinese Patent Application Publication No. 111,460,931 International Publication No. 2022 / 161286 U.S. Patent No. 1,148,1615

[0008] Rafael Henrique Vareto, Matheus A. Diniz and William Robson Schwartz, "Face Spoofing Detection on Low-Power Devices using Embeddings with Spatial and Frequency-based Descriptors", Proc. of the 24th Iberoamerican Congress on Pattern Recognition (CIARP 2019), pp. 187-197, October 2019

[0009] As described above, an information processing system may determine whether an input image is an image for a spoofing attack. However, the information processing system may capture an image of a person to be authenticated in an environment where the ambient lighting conditions change, and may accept images under various lighting conditions. The lighting conditions affect the brightness and color distribution of an image, and therefore are related to the image features used to determine a spoofing attack. Therefore, in an environment where the lighting conditions change, the accuracy of determining a spoofing attack may decrease. Therefore, in one aspect, the present invention aims to improve the accuracy of determining a spoofing attack against biometric authentication.

[0010] In one aspect, a biometric authentication processing program is provided that causes a computer to perform the following processes: Determine the brightness of the image data from a plurality of pixel values ​​included in the image data; Generate frequency component data from the image data that indicates frequency components in a frequency range corresponding to the brightness, such that when the brightness is at a first level, the frequency components are in a first frequency range, and when the brightness is at a second level higher than the first level, the frequency components are in a second frequency range that includes frequencies higher than the first frequency range; Input the image data and the frequency component data into a machine learning model to determine whether the image data indicates a spoofing attack against biometric authentication.

[0011] According to another aspect, there is provided a biometric authentication processing method executed by a computer, and an information processing device having a storage unit and a processing unit.

[0012] In one aspect, the accuracy of determining a spoofing attack on biometric authentication is improved. The above and other objects, features and advantages of the present invention will become apparent from the following description taken in conjunction with the accompanying drawings illustrating preferred embodiments of the present invention.

[0013] FIG. 1 is a diagram for explaining an information processing device according to a first embodiment; FIG. 2 is a diagram illustrating an example of hardware of an information processing device according to a second embodiment; FIG. 3 is a diagram illustrating an example of a presentation attack against biometric authentication; FIG. 4 is a diagram illustrating an example of image data with different illumination light levels; FIG. 5 is a diagram illustrating an example of conversion from image data to frequency component data; FIG. 6 is a diagram illustrating an example of detection of reflected light from image data; FIG. 7 is a diagram illustrating an example of the structure of a machine learning model; FIG. 8 is a diagram illustrating an example of a basic score used for channel attention; FIG. 9 is a block diagram illustrating an example of functions of an information processing device according to a second embodiment; and FIG. 10 is a flowchart illustrating an example of a procedure for determining a presentation attack.

[0014] The present embodiment will be described below with reference to the drawings. Note that a plurality of embodiments may be combined and implemented. [First Embodiment] An information processing device 10 of the first embodiment receives image data used for biometric authentication and determines whether the received image data indicates a spoofing attack against the biometric authentication. Biometric authentication authenticates an individual to be authenticated based on the physical characteristics or behavioral characteristics of the person to be authenticated. Biometric authentication includes face authentication using a face image and palm authentication using a palm image.

[0015] In a spoofing attack, an attacker presents fraudulently prepared biometric information of another person instead of presenting the biometric information of the attacker, who is the person to be authenticated. This type of spoofing attack is sometimes called a presentation attack. For example, an attacker displays a biometric image of another person's face or palm on a mobile device and holds the mobile device over a camera in an authentication device. The image data thus captured by the authentication device is not a genuine biometric image obtained by directly capturing an image of the person to be authenticated with the camera of the authentication device, but a fake biometric image obtained by capturing an image of a non-human object. While fake biometric images often have different characteristics from genuine biometric images, they may not be easy to distinguish from genuine biometric images.

[0016] Therefore, the information processing device 10 uses a machine learning model to improve the accuracy of determining whether an impersonation attack has occurred. Following the determination of whether an impersonation attack has occurred, the information processing device 10 may use the same image data to perform biometric authentication to identify the individual to be authenticated. Furthermore, another information processing device may perform biometric authentication based on the determination result of the information processing device 10. The information processing device 10 may be a client device or a server device. The information processing device 10 may be called a computer, a biometric authentication processing device, or a machine learning device.

[0017] 1 is a diagram illustrating an information processing apparatus according to a first embodiment. The information processing apparatus 10 includes a storage unit 11 and a processing unit 12. The storage unit 11 may be a volatile semiconductor memory such as a random access memory (RAM). Alternatively, the storage unit 11 may be a non-volatile storage such as a hard disk drive (HDD) or a flash memory.

[0018] The processing unit 12 is, for example, a processor such as a central processing unit (CPU), a graphics processing unit (GPU), or a digital signal processor (DSP). However, the processing unit 12 may also include an electronic circuit such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA). The processor executes a program stored in a memory such as a RAM (which may be the storage unit 11). A set of processors may be called a multiprocessor or simply a "processor."

[0019] The storage unit 11 stores image data 13. The image data 13 represents a biometric image used for biometric authentication. For example, the image data 13 is a face image or a palm image. The image data 13 may be a monochrome image including only one color channel, or a color image including multiple color channels such as an RGB (Red Green Blue) image. The image data 13 has a certain size defined by a height and a width. The image data 13 includes pixel values ​​of multiple pixels arranged in a grid pattern and whose positions are specified by two-dimensional coordinates. The pixel values ​​may be referred to as pixel intensity or luminance.

[0020] The storage unit 11 also stores a trained machine learning model 17. The machine learning model 17 may be a neural network such as a convolutional neural network (CNN). The information processing device 10 may train the machine learning model 17, or another information processing device may train the machine learning model 17.

[0021] The machine learning model 17 receives image data 13 and frequency component data 14 (described later), and determines whether the image data 13 is an image of an impersonation attack. The machine learning model 17, for example, extracts feature amounts of the image data 13 and feature amounts of the frequency component data 14, and combines these two types of feature amounts to determine whether an attack has occurred. The machine learning model 17 may process the image data 13 and the frequency component data 14 using different convolution blocks. The machine learning model 17 is trained using training data that includes, for example, input data corresponding to the image data 13 and the frequency component data 14, and a label indicating whether an attack has occurred.

[0022] The processing unit 12 acquires image data 13. For example, the processing unit 12 receives the image data 13 from an imaging device for biometric authentication. The processing unit 12 determines the brightness of the image data 13 from the pixel values ​​of multiple pixels included in the image data 13. This brightness indicates the overall brightness of the image data 13. For example, the processing unit 12 calculates a representative value such as the average value, median value, or mode value for the pixel values ​​of the multiple pixels.

[0023] The processing unit 12 may classify the image data 13 into one of two or more levels using the calculated representative value and one or more thresholds. For example, the processing unit 12 classifies the image data 13 into one of levels 15a, 15b, and 15c. Level 15a indicates weak brightness, level 15b indicates medium brightness, and level 15c indicates strong brightness. The processing unit 12 may determine the brightness to be level 15a when the representative value is less than a first threshold. Alternatively, the processing unit 12 may determine the brightness to be level 15b when the representative value is equal to or greater than the first threshold and equal to or less than a second threshold. Alternatively, the processing unit 12 may determine the brightness to be level 15c when the representative value exceeds the second threshold.

[0024] The processing unit 12 generates frequency component data 14 from the image data 13 by frequency decomposition. The frequency component data 14 is, for example, two-dimensional data indicating vertical and horizontal frequency components of the image data 13. The frequency component data 14 has a fixed size. The frequency component data 14 indicates frequency components in a partial frequency range depending on brightness. For example, the processing unit 12 generates the frequency component data 14 by an inverse discrete cosine transform (IDCT). The inverse discrete cosine transform converts a spatial signal into a frequency domain signal. The processing unit 12 may first select a frequency range and extract only the frequency components in the selected frequency range from the image data 13. Alternatively, the processing unit 12 may first extract all frequency components from the image data 13 and then generate the frequency component data 14 by extracting frequency components in a partial frequency range depending on brightness.

[0025] The frequency component data 14 preferably includes higher frequency components as the image data 13 becomes brighter. For example, the processing unit 12 divides the entire frequency range into two or more frequency intervals using one or more division points. Each frequency interval indicates a range of continuous frequencies. Two adjacent frequency intervals may not include overlapping frequencies, or may include partially overlapping frequencies. For example, the processing unit 12 defines frequency intervals 16a, 16b, and 16c.

[0026] Frequency interval 16a includes low frequencies, frequency interval 16b includes medium frequencies, and frequency interval 16c includes high frequencies. Frequency interval 16b is a higher frequency interval than frequency interval 16a, and frequency interval 16c is a higher frequency interval than frequency interval 16b. For example, when the brightness is at level 15a, processing unit 12 selects frequency interval 16a, when the brightness is at level 15b, processing unit 12 selects frequency interval 16b, and when the brightness is at level 15c, processing unit 12 selects frequency interval 16c.

[0027] The processing unit 12 determines whether the image data 13 indicates a spoofing attack against biometric authentication by inputting the image data 13 and the frequency component data 14 to the machine learning model 17. The machine learning model 17 may output a flag indicating whether or not the image data 13 indicates a spoofing attack, or may output the probability of a spoofing attack or the probability of not a spoofing attack.

[0028] If the processing unit 12 determines that there is an impersonation attack, it may refuse to perform personal identification using the image data 13. On the other hand, if the processing unit 12 determines that there is no impersonation attack, it may allow personal identification using the image data 13. Furthermore, the processing unit 12 may output the determination result by the machine learning model 17. The processing unit 12 may transmit the determination result to a non-volatile storage, may display it on a display device, or may transmit it to another information processing device.

[0029] As described above, the information processing device 10 according to the first embodiment determines the brightness of the image data 13 from a plurality of pixel values ​​included in the image data 13. The information processing device 10 generates, from the image data 13, frequency component data 14 indicating frequency components in a frequency interval corresponding to the brightness. When the brightness is at level 15a, the frequency component data 14 indicates frequency components in frequency interval 16a, and when the brightness is at level 15b, which is higher than level 15a, the frequency component data 14 indicates frequency components in frequency interval 16b, which includes frequencies higher than frequency interval 16a. The information processing device 10 inputs the image data 13 and the frequency component data 14 into the machine learning model 17, thereby determining whether the image data 13 indicates a spoofing attack against biometric authentication.

[0030] This allows the information processing device 10 to detect spoofing attacks that present false biometric information, thereby reducing the risk of erroneously authenticating an attacker as a registered person, thereby improving the security level of biometric authentication.

[0031] Furthermore, the information processing device 10 inputs the frequency component data 14 in addition to the image data 13 to the machine learning model 17, thereby emphasizing frequency components in different frequency intervals depending on the brightness of the image data 13. This improves the determination accuracy of the machine learning model 17 even when the lighting conditions at the time of capturing the image data 13 are unstable. For example, even when a camera for generating the image data 13 may be used under various lighting conditions, such as during the day, at night, indoors, or outdoors, the determination accuracy is stably improved.

[0032] Note that the information processing device 10 may select frequency interval 16a when the brightness is level 15a, and may select frequency interval 16b when the brightness is level 15b. Furthermore, the information processing device 10 may select frequency interval 16c including a higher frequency than frequency interval 16b when the brightness is level 15c, which is higher than level 15b. This stably improves the determination accuracy even when the lighting conditions are unstable.

[0033] For example, when the brightness of the image data 13 is low, the visibility of dark image areas is improved by emphasizing low-frequency components, thereby improving the determination accuracy of the machine learning model 17. When the brightness of the image data 13 is high, the detailed shape of the living body is clarified by emphasizing high-frequency components, thereby improving the determination accuracy of the machine learning model 17. When the brightness of the image data 13 is medium, various features are emphasized in a balanced manner by emphasizing mid-frequency components.

[0034] Furthermore, the information processing device 10 may determine the presence or absence of reflected light from the gradient of pixel values ​​between adjacent pixels included in the image data 13. Furthermore, the information processing device 10 may generate frequency component data 14 so as to indicate frequency components in a frequency range corresponding to brightness and the presence or absence of reflected light. This allows the information processing device 10 to extract appropriate features useful for determining whether or not a spoofing attack has occurred, even if the color or pattern of a part of the image region included in the image data 13 deviates from the original color or pattern of the biological surface due to strong reflected light.

[0035] Furthermore, the information processing device 10 may select frequency interval 16a when the brightness is level 15a. Furthermore, the information processing device 10 may select frequency interval 16b when the brightness is level 15b or when the brightness is level 15c and there is no reflected light. Furthermore, the information processing device 10 may select frequency interval 16c when the brightness is level 15c and there is reflected light. This allows the frequency interval to be selected taking into consideration the presence or absence of reflected light in addition to the brightness, thereby improving the determination accuracy of the machine learning model 17.

[0036] Furthermore, the information processing device 10 may determine the brightness of the image data 13 based on the average of multiple pixel values ​​included in the image data 13 and one or more threshold values. This allows the intensity of illumination when the image data 13 is captured to be appropriately classified.

[0037] Second Embodiment An information processing device 100 according to a second embodiment performs biometric authentication using a machine learning model. The information processing device 100 corresponds to the information processing device 10 according to the first embodiment. The information processing device 100 may be a client device or a server device. The information processing device 100 may also be called a computer, a biometric authentication device, or a machine learning device. Note that training of the machine learning model and biometric authentication using the machine learning model may be performed in different information processing devices.

[0038] 2 is a diagram illustrating an example of hardware of an information processing apparatus according to the second embodiment. The information processing apparatus 100 includes a CPU 101, a RAM 102, a nonvolatile storage 103, a GPU 104, an input interface 105, a medium reader 106, and a communication interface 107, all connected via a bus. The CPU 101 corresponds to the processing unit 12 in the first embodiment. The RAM 102 and the nonvolatile storage 103 correspond to the storage unit 11 in the first embodiment.

[0039] The CPU 101 is a processor that executes program instructions. The CPU 101 loads programs and data stored in the non-volatile storage 103 into the RAM 102 and executes the programs. The information processing apparatus 100 may have multiple processors.

[0040] The RAM 102 is a volatile semiconductor memory that temporarily stores programs executed by the CPU 101 and data used in calculations by the CPU 101. The information processing device 100 may also include a type of volatile memory other than RAM.

[0041] The nonvolatile storage 103 stores software programs such as an operating system (OS), middleware, and application software, as well as data. The nonvolatile storage 103 is, for example, an HDD, a solid state drive (SSD), or a flash memory.

[0042] The GPU 104 performs image processing in cooperation with the CPU 101 and outputs an image to a display device 111 connected to the information processing device 100. The display device 111 is, for example, a CRT (Cathode Ray Tube) display, a liquid crystal display, an organic EL (Electro Luminescence) display, or a projector. Other types of output devices, such as a printer, may also be connected to the information processing device 100.

[0043] The GPU 104 may also be used as a general purpose computing on graphics processing unit (GPGPU). The GPU 104 may execute a program in response to an instruction from the CPU 101. The information processing apparatus 100 may include a volatile semiconductor memory other than the RAM 102 as a GPU memory.

[0044] The input interface 105 receives an input signal from an input device 112 connected to the information processing apparatus 100. The input device 112 is, for example, a mouse, a touch panel, or a keyboard. A plurality of input devices may be connected to the information processing apparatus 100.

[0045] The medium reader 106 is a reading device that reads programs and data recorded on the recording medium 113. The recording medium 113 is, for example, a magnetic disk, an optical disk, or a semiconductor memory. Magnetic disks include flexible disks (FDs) and HDDs. Optical disks include compact discs (CDs) and digital versatile discs (DVDs). The medium reader 106 copies the programs and data read from the recording medium 113 to the RAM 102 or the non-volatile storage 103. The read programs may be executed by the CPU 101.

[0046] The recording medium 113 may be a portable recording medium. The recording medium 113 may be used to distribute programs and data. The recording medium 113 and the non-volatile storage 103 may also be referred to as computer-readable recording media.

[0047] The communication interface 107 communicates with the image capture device 115 via the network 114. The communication interface 107 may be a wired communication interface that communicates using a wired cable, or may be a wireless communication interface that communicates using a wireless link.

[0048] The communication interface 107 receives image data from the imaging device 115 and stores it in the RAM 102. The image data represents a biometric image according to a biometric authentication method. For example, in the case of face authentication, the person to be authenticated places their face in front of the imaging device 115, and the imaging device 115 outputs face image data. In the case of palm authentication, the person to be authenticated holds their palm over the imaging device 115, and the imaging device 115 outputs palm image data. Note that the imaging device 115 may be incorporated into the information processing device 100. For example, the information processing device 100 may be a smartphone or tablet terminal that uses biometric authentication for user authentication.

[0049] Next, we will explain biometric authentication using biometric images and presentation attacks that present false biometric information. Biometric authentication has a variety of uses, including the following. Biometric authentication is sometimes used for access control, which determines whether to allow entry to a specific building or room. Biometric authentication is also sometimes used to identify the parties involved in financial transactions and prevent fraudulent financial transactions. Biometric authentication is also sometimes used to identify patients in medical institutions.

[0050] Biometric authentication may also be used for data access control, which determines whether access to data is permitted. Biometric authentication may also be used for device access control, which determines whether use of a device is permitted. Biometric authentication may also be used in labor management, which manages workers' working hours, work locations, and work content. Biometric authentication may also be used in criminal investigations, which identify criminal suspects.

[0051] On the other hand, biometric authentication systems may be subject to spoofing attacks in which a third party attempts to break through biometric authentication by impersonating a registered user. In a presentation attack, instead of directly presenting the physical or behavioral characteristics of the person to be authenticated to the image capture device 115, an object that exhibits the physical or behavioral characteristics of another person is presented to the image capture device 115.

[0052] FIG. 3 is a diagram showing an example of a presentation attack against biometric authentication. In the following description of the second embodiment, palm authentication using palm image data is assumed. Typically, a person to be authenticated presents their actual hand 151. The information processing device 100 directly captures an image of the hand 151 using the imaging device 115 and extracts features from this image data. The information processing device 100 compares the extracted features with the registered information to determine whether the person to be authenticated is the registered person.

[0053] On the other hand, an attacker who carries out a presentation attack may typically perform the following fraudulent acts. The attacker uses the smartphone 152 to capture an image of another person's hands in advance and saves the image data of the other person in the smartphone 152. The attacker displays and presents this image data on the smartphone 152. The information processing device 100 captures an image of the screen of the smartphone 152 using the imaging device 115. As a result, the information processing device 100 obtains fake image data that is not a direct image of a real hand.

[0054] In biometric authentication, the features of the biometric image at the time of registration do not completely match those at the time of authentication, so the information processing device 100 authenticates the person to be authenticated as the registrant if the similarity between the features of the received image data and the registration information is equal to or greater than a certain level. Therefore, there is a risk that the information processing device 100 will mistakenly authenticate an attacker as the registrant based on fake image data.

[0055] However, there are often differences in features related to realism, such as texture, gloss, and color distribution, between genuine image data of a real hand 151 and fake image data of the smartphone 152 screen. Feature quantities suitable for determining realism often differ from feature quantities suitable for identifying individuals. Therefore, before comparing the image data with the registered information, the information processing device 100 separately determines whether the image data corresponds to a presentation attack.

[0056] The information processing device 100 uses a trained machine learning model to determine whether or not a presentation attack exists from image data. If it is determined that the image data does not correspond to a presentation attack, the information processing device 100 compares the image data with registered information. On the other hand, if it is determined that the image data corresponds to a presentation attack, the information processing device 100 outputs a warning without comparing the image data with registered information.

[0057] However, the information processing device 100 may be used under various lighting conditions. Here, lighting includes both natural lighting and artificial lighting. Differences in lighting conditions include differences in time of day, such as daytime and nighttime, and differences in location, such as outdoors and indoors. Lighting conditions include the number of light sources, the intensity of the light sources, and the positional relationship between the image capture device 115 and the light sources. The information processing device 100 may acquire image data captured under various lighting conditions. For example, if the information processing device 100 is a mobile device equipped with an image capture device 115, such as a smartphone or tablet terminal, the lighting conditions may change for each biometric authentication.

[0058] In this case, differences in lighting conditions affect the features that indicate the realism of the living body in the image data. Therefore, if the image data is input directly into a machine learning model to detect presentation attacks, the detection accuracy may not be sufficiently high.

[0059] 4 is a diagram showing examples of image data with different illumination light levels. Image data 161 to 165 are image data captured with different illumination light levels. Image data 161 has the weakest illumination light. In image data 161, most of the image area showing the surface of the living body is dark. Image data 162 has illumination light that is slightly stronger than that of image data 161. In image data 162, most of the image area showing the surface of the living body is slightly brighter than that of image data 161.

[0060] Image data 163 has an even stronger illumination light than image data 162. In image data 163, the living body was imaged under moderate illumination light, and the pattern and color of the living body surface are appropriately represented. Image data 164 has an even stronger illumination light than image data 163. In image data 164, the living body was imaged under strong illumination light, and the color of the living body surface has shifted toward whiter than it actually is. Image data 165 has an even stronger illumination light than image data 164. In image data 165, the living body was imaged under very strong illumination light, and the image shows light that is strongly reflected by the illumination light on the living body surface. The image area where the reflected light is captured is a very bright white compared to the surrounding area, and does not appropriately represent the color of the living body surface.

[0061] In this way, the information processing device 100 may receive image data with different brightness levels. While it is useful to use the frequency components of image data to detect presentation attacks, the particularly useful frequency components differ depending on the brightness level.

[0062] Therefore, the information processing device 100 extracts frequency components in different frequency ranges from the image data according to the brightness, and inputs the frequency component data to the machine learning model in addition to the original image data. This enhances frequency components in specific frequency ranges. A system that extracts frequency components in different frequency ranges according to the brightness may be called an adaptive frequency decomposition system (AFDS).

[0063] 5 is a diagram showing an example of conversion from image data to frequency component data. The information processing device 100 receives image data 171. The information processing device 100 analyzes the pixel intensities of multiple pixels included in the image data 171 to determine the illumination light level and the presence or absence of reflected light. The pixel intensities are sometimes called pixel values ​​or luminance. The information processing device 100 extracts one of frequency component data 172, 173, and 174 from the image data 171 according to the illumination light level and the presence or absence of reflected light.

[0064] In the second embodiment, an inverse discrete cosine transform (IDCT) is used to transform image data 171 into frequency component data 172, 173, and 174. However, other transformation algorithms for transforming spatial data into frequency domain data may be used. The frequency component data 172, 173, and 174 are, for example, two-dimensional data representing the vertical and horizontal frequency components of image data 171. The entire frequency is divided into three frequency intervals. The frequencies of the division points are specified in advance by the user when training the machine learning model.

[0065] Frequency component data 172 corresponds to the lowest frequency interval of the three frequency intervals. Therefore, frequency component data 172 includes low frequency components. Frequency component data 173 corresponds to the middle frequency interval of the three frequency intervals. Therefore, frequency component data 173 includes mid frequency components. Frequency component data 174 corresponds to the highest frequency interval of the three frequency intervals. Therefore, frequency component data 174 includes high frequency components.

[0066] The information processing device 100 selects one of the low-frequency range, the mid-frequency range, and the high-frequency range depending on the illumination light level and the presence or absence of reflected light. When the illumination light is weak, the information processing device 100 selects the low-frequency range. When the illumination light is weak, the visibility of dark image areas is improved by emphasizing the low-frequency components. Therefore, the information processing device 100 inputs the image data 171 and the frequency component data 172 into a machine learning model.

[0067] When the illumination light is strong and there is reflected light, the information processing device 100 selects the high-frequency range. When the illumination light is strong and there is reflected light, emphasizing the high-frequency components makes it easier to grasp the detailed shape of the living body. Therefore, the information processing device 100 inputs the image data 171 and the frequency component data 174 into a machine learning model.

[0068] Otherwise, the information processing device 100 selects the mid-frequency range. The selection of the mid-frequency range includes cases where the illumination light is moderate and cases where the illumination light is strong but there is no reflected light. When the illumination light is in a moderate range, emphasizing the mid-frequency components extracts well-balanced features of the shape and color of the living body. Therefore, the information processing device 100 inputs the image data 171 and the frequency component data 173 into a machine learning model.

[0069] Note that the information processing device 100 may first select a frequency interval and extract only the frequency components of the selected frequency interval from the image data 171. Alternatively, the information processing device 100 may first convert the image data 171 into frequency component data 172, 173, and 174 and select one of the frequency component data 172, 173, and 174.

[0070] Next, the determination of the illumination light level and the presence or absence of reflected light will be described. The information processing device 100 calculates the average intensity μ shown in Equation (1) for the received image data. In Equation (1), M is the number of pixels in the horizontal direction, N is the number of pixels in the vertical direction, and (i, j) are the position coordinates of the pixel. R ｉ，ｊ is the pixel intensity of the red channel, G ｉ，ｊ is the pixel intensity of the green channel, B ｉ，ｊ denotes the pixel intensity of the blue channel. This mean intensity μ is the average of all pixel intensities in the three color channels, and represents the overall brightness of the image data.

[0071]

[0072] The average intensity μ is a value between 0 and 255. The information processing device 100 determines the average intensity μ by using a threshold T ｂｒｉｇｈｔ and threshold T ｄｉｍ The threshold T ｂｒｉｇｈｔ is the threshold T ｄｉｍ is greater than the threshold T ｂｒｉｇｈｔ , T ｄｉｍ is specified by the user in advance when training the machine learning model. ｂｒｉｇｈｔ If the average intensity μ exceeds the threshold T ｄｉｍ If the average intensity μ is less than the threshold T ｄｉｍ greater than or equal to threshold T ｂｒｉｇｈｔ If it is equal to or less than this, the brightness of the image data is determined to be normal and the illumination light level is determined to be medium.

[0073] Note that some image data may contain both bright and dark image regions. Therefore, the information processing device 100 uses the variance σ shown in Equation (2) in addition to the average intensity μ. ２ In equation (2), I ｉ，ｊ is the pixel intensity of one pixel, e.g., the average of the pixel intensities of the three color channels. ２ indicates the breadth of the pixel intensity distribution. ２ When is large, bright and dark image regions coexist in the image data.

[0074]

[0075] variance σ ２ If σ is very large, frequency components in more than one frequency interval may be useful for detecting a presentation attack. ２ exceeds the threshold, the information processing device 100 may generate frequency component data by a method different from that shown in Fig. 5. For example, the information processing device 100 combines frequency component data 172, 173, and 174 corresponding to three frequency intervals and inputs the combined frequency component data to a machine learning model. The combination may be a concatenation of the frequency component data 172, 173, and 174, or a summation of low-frequency components, mid-frequency components, and high-frequency components.

[0076] Next, the information processing device 100 determines whether or not there is reflected light from the image data. The pixel intensity of an image area where reflected light is present is significantly different from the pixel intensity of the surrounding area. Therefore, the information processing device 100 detects edges from the image data.

[0077] The information processing device 100 uses a Sobel filter S ｘ The gradient of the pixel intensity in the horizontal direction is calculated for each pixel using the Sobel filter S ｘ is a matrix of 3x3 size. The horizontal gradient is calculated from the pixel intensity of the pixel of interest and the pixel intensities of the eight surrounding pixels around the pixel. Similarly, the information processing device 100 calculates the Sobel filter S ｙ The gradient of pixel intensity in the vertical direction is calculated for each pixel using the Sobel filter S ｙ is a matrix of size 3 × 3. The vertical gradient is calculated from the pixel intensity of the pixel of interest and the pixel intensities of the eight surrounding pixels around that pixel.

[0078] The information processing device 100 calculates the gradient magnitude G shown in Equation (3) for each pixel. In Equation (3), I is image region data of 3x3 size centered on the pixel of interest. The pixel intensities of the nine pixels included in this image region data are, for example, the average of the pixel intensities of the three color channels. If the image data contains a pixel whose G exceeds a threshold, the information processing device 100 determines that reflected light is present. On the other hand, if the image data does not contain a pixel whose G exceeds a threshold, the information processing device 100 determines that reflected light is not present.

[0079]

[0080] 6 is a diagram showing an example of detection of reflected light from image data. When a Sobel filter is applied to image data 175, image data 176 is generated from the image data 175. The image data 176 shows edges where pixel intensity changes significantly. The presence or absence of reflected light is determined using this image data 176.

[0081] The information processing device 100 converts image data into frequency component data using a function F shown in Equation (4). Equation (4) represents an inverse discrete cosine transform. In Equation (4), (x, y) are the position coordinates of a pixel included in the image data, and (u, v) are the position coordinates of a frequency component included in the frequency component data. f(x, y) is the pixel intensity at position (x, y), and F(u, v) is the frequency component at position (u, v).

[0082]

[0083] Next, the machine learning model will be described. The information processing device 100 uses a convolutional neural network as a machine learning model for detecting presentation attacks. FIG. 7 is a diagram showing an example of the structure of the machine learning model. The machine learning model includes CNN blocks 131 to 138, spatial attention units 141 and 142, a channel attention unit 143, and a fully connected layer 144. Each of the CNN blocks 131 to 138 includes one or more convolutional layers. The convolutional layers perform convolutional integration on tensor-format input data using a matrix-format kernel to generate a tensor-format feature map. The multiple coefficients included in the kernel are parameters that are optimized through machine learning.

[0084] CNN block 131 accepts image data. CNN block 131 generates one or more feature maps using one or more kernels. CNN block 132 obtains the feature maps from CNN block 131. CNN block 132 generates one or more new feature maps using one or more kernels. CNN block 133 obtains the feature maps from CNN block 132. CNN block 133 generates one or more new feature maps using one or more kernels. CNN block 134 obtains the feature maps from CNN block 133. CNN block 134 generates one or more new feature maps using one or more kernels.

[0085] CNN block 135 receives frequency component data. CNN block 135 generates one or more feature maps using one or more kernels. CNN block 136 obtains the feature maps from CNN block 135. CNN block 136 generates one or more new feature maps using one or more kernels. CNN block 137 obtains the feature maps from CNN block 136. CNN block 137 generates one or more new feature maps using one or more kernels. CNN block 138 obtains the feature maps from CNN block 137. CNN block 138 generates one or more new feature maps using one or more kernels.

[0086] The fully connected layer 144 acquires feature maps from the CNN blocks 134 and 138. The fully connected layer 144 uses the numerical values ​​of all elements included in these feature maps to calculate a score, which is a scalar value between 0 and 1. Thus, the score is calculated based on both the feature amounts extracted from the image data and the feature amounts extracted from the frequency component data. The weights applied to the elements included in the feature map are parameters optimized through machine learning. Based on the score, the machine learning model determines whether the input image data corresponds to a presentation attack. For example, if the score exceeds a threshold, the machine learning model determines that it is a presentation attack, and if the score does not exceed the threshold, the machine learning model determines that it is not a presentation attack.

[0087] The spatial attention unit 141 acquires a feature map from the CNN blocks 131 and 135. The spatial attention unit 141 uses the acquired feature map to calculate weights for each of multiple image regions included in the image data. The weights for the image regions are calculated based on both features extracted from the image data and features extracted from the frequency component data. These weights may be referred to as attention. Typically, image regions that include a living body have a large weight, and image regions that do not include a living body have a small weight. The spatial attention unit 141 is trained to narrow down image regions that include a living body through training of a machine learning model.

[0088] The CNN blocks 132 and 136 acquire weight data indicating the weights of image regions from the spatial attention unit 141. The weight data may be a feature map in tensor format. The CNN block 132 uses the weight data acquired from the spatial attention unit 141 in addition to the feature map acquired from the CNN block 131. Similarly, the CNN block 136 uses the weight data acquired from the spatial attention unit 141 in addition to the feature map acquired from the CNN block 135.

[0089] The spatial attention unit 142 acquires feature maps from the CNN blocks 132 and 136. Similar to the spatial attention unit 141, the spatial attention unit 142 uses the acquired feature maps to calculate weights for each of multiple image regions included in the image data. The CNN blocks 133 and 137 acquire weight data indicating the weights of the image regions from the spatial attention unit 142. The CNN block 133 uses the weight data acquired from the spatial attention unit 142 in addition to the feature map acquired from the CNN block 132. Similarly, the CNN block 137 uses the weight data acquired from the spatial attention unit 142 in addition to the feature map acquired from the CNN block 136.

[0090] The channel attention unit 143 acquires feature maps from the CNN blocks 133 and 137. The channel attention unit 143 uses the acquired feature maps to calculate weights for each of multiple channels. The channels handled by the channel attention unit 143 include three color channels: a red channel, a green channel, and a blue channel. The channels also include three frequency channels: a low-frequency channel, a mid-frequency channel, and a high-frequency channel. The weights for these six channels are calculated based on both features extracted from image data and features extracted from frequency component data. These weights may be referred to as attention.

[0091] The CNN blocks 134 and 138 obtain weight data indicating channel weights from the channel attention unit 143. The weight data may be a feature map in tensor format. The CNN block 134 uses the weight data obtained from the channel attention unit 143 in addition to the feature map obtained from the CNN block 133. Similarly, the CNN block 138 uses the weight data obtained from the channel attention unit 143 in addition to the feature map obtained from the CNN block 137.

[0092] In the task of detecting presentation attacks, it is often found that there are differences in the importance of these six channels. Therefore, the channel attention unit 143 has basic scores that are applied to the six channels to calculate weights that reflect this knowledge. The channel attention unit 143 calculates the channel weights by taking into account the feature maps of the CNN blocks 133 and 137 in addition to the basic scores.

[0093] 8 shows an example of basic scores used for channel attention. Table 145 associates channels with basic scores. For example, the basic score for the red channel is 0.7, the basic score for the green channel is 0.8, the basic score for the blue channel is 0.5, the basic score for the low-frequency channel is 0.3, the basic score for the mid-frequency channel is 0.5, and the basic score for the high-frequency channel is 0.9.

[0094] Generally, in detecting presentation attacks, the green channel is highly relevant to skin texture and has relatively high importance. On the other hand, the blue channel is relatively less relevant to attacks and has relatively low importance. Therefore, the basic score of the green channel is high and the basic score of the blue channel is low. Also, low-frequency channels are often relatively less relevant to attacks and have relatively low importance. On the other hand, high-frequency channels are highly relevant to edges and noise specific to attacks and have relatively high importance. Therefore, the basic score of the low-frequency channel is low and the basic score of the high-frequency channel is high.

[0095] By calculating the weights of image regions in the first half of the process, the machine learning model can narrow down the image regions of interest early on, making it easier to extract biometric features. On the other hand, by calculating the channel weights in the second half of the process, the machine learning model can improve the accuracy of calculating the probability that input image data corresponds to a presentation attack.

[0096] Next, the functions and processing procedures of the information processing device 100 will be described. FIG. 9 is a block diagram showing an example of functions of the information processing device according to the second embodiment. The information processing device 100 includes a training data storage unit 121, a model storage unit 122, a registrant storage unit 123, a machine learning unit 124, a user registration unit 125, an attack detection unit 126, and a user authentication unit 127. The training data storage unit 121, the model storage unit 122, and the registrant storage unit 123 are implemented using, for example, the RAM 102 or the non-volatile storage 103. The machine learning unit 124, the user registration unit 125, the attack detection unit 126, and the user authentication unit 127 are implemented using the CPU 101 or the GPU 104 and a program.

[0097] The training data storage unit 121 stores training data used to train the machine learning model. The training data associates image data corresponding to a biometric image with a ground truth label indicating whether or not the image is a presentation attack. The training data includes image data representing genuine biometric images and image data representing fake biometric images, such as images of a smartphone screen. The training data also includes image data captured under various lighting conditions.

[0098] The model storage unit 122 stores a machine learning model such as that shown in Fig. 7. The machine learning model stored in the model storage unit 122 is a trained machine learning model including optimized parameter values. The registrant storage unit 123 stores user information of authorized users registered in the information processing device 100. The user information associates an identifier assigned to a user with feature amounts of biometric information possessed by the user, such as feature amounts of a palm image.

[0099] The machine learning unit 124 uses the training data stored in the training data storage unit 121 to train a machine learning model for detecting presentation attacks, and stores the trained machine learning model in the model storage unit 122. For example, the machine learning unit 124 determines the illumination light level and the presence or absence of reflected light for each piece of image data included in the training data, and generates frequency component data in a frequency range according to the illumination light level and the presence or absence of reflected light.

[0100] The machine learning unit 124 inputs image data and frequency component data into a machine learning model to estimate the presence or absence of a presentation attack. The machine learning unit 124 compares the estimation result with the correct label included in the training data to calculate an error. The machine learning unit 124 updates the parameter values ​​included in the machine learning model using an error backpropagation method to reduce the error. The machine learning unit 124 repeatedly calculates the error and updates the parameter values.

[0101] The user registration unit 125 registers a new user in the information processing device 100. At this time, the user registration unit 125 acquires image data from the imaging device 115. The user registration unit 125 extracts features for personal authentication from the image data. The user registration unit 125 also assigns an identifier to the user. However, the identifier may be input by the user. The user registration unit 125 associates the identifier with the features and stores them in the registrant storage unit 123.

[0102] During biometric authentication, the attack detection unit 126 acquires image data from the imaging device 115. The attack detection unit 126 uses a machine learning model stored in the model storage unit 122 to determine whether the image data corresponds to a presentation attack. If the attack detection unit 126 determines that the image data corresponds to a presentation attack, the attack detection unit 126 stops the biometric authentication and outputs a warning message. On the other hand, if the attack detection unit 126 determines that the image data does not correspond to a presentation attack, the attack detection unit 126 allows the user authentication unit 127 to perform biometric authentication processing.

[0103] To detect a presentation attack, the attack detection unit 126 determines the illumination light level of the image data and the presence or absence of reflected light, and generates frequency component data in a frequency range corresponding to the illumination light level and the presence or absence of reflected light. The machine learning unit 124 inputs the image data and frequency component data into a machine learning model to determine the presence or absence of a presentation attack.

[0104] If the attack detection unit 126 does not detect a presentation attack, the user authentication unit 127 extracts features for personal authentication from the image data. The user authentication unit 127 compares the extracted features with the features stored in the registrant storage unit 123. If there is a registrant whose feature similarity exceeds a threshold, the user authentication unit 127 determines that the biometric authentication is successful. On the other hand, if there is no registrant whose feature similarity exceeds the threshold, the user authentication unit 127 determines that the biometric authentication is unsuccessful. The user authentication unit 127 outputs an authentication result. The authentication result may include the identifier of the corresponding registrant.

[0105] The user authentication unit 127 may also accept an identifier of the person to be authenticated. In this case, the user authentication unit 127 may read out a feature corresponding to the accepted identifier from the registrant storage unit 123 and compare only the feature extracted from the image data with the read-out feature. The feature may also be expressed as a numerical vector. In this case, the user authentication unit 127 may determine whether the distance between the numerical vectors is less than a threshold.

[0106] 10 is a flowchart showing an example of a procedure for determining a presentation attack. In step S10, the attack detection unit 126 acquires image data from the imaging device 115. In step S11, the attack detection unit 126 calculates the average intensity of multiple pixels included in the image data. The attack detection unit 126 compares the average intensity with two thresholds to determine whether the illumination light level is weak, medium, or strong.

[0107] In step S12, the attack detection unit 126 calculates the magnitude of the pixel intensity gradient for each of the multiple pixels included in the image data. The attack detection unit 126 determines whether or not reflected light is present based on the magnitude of the gradient. If there is a pixel whose gradient magnitude exceeds a threshold, it is determined that reflected light is present. If there is no pixel whose gradient magnitude exceeds the threshold, it is determined that reflected light is not present.

[0108] In step S13, the attack detection unit 126 determines whether the illumination light level is weak. If the illumination light level is weak, the process proceeds to step S14. If the illumination light level is not weak, the process proceeds to step S15. In step S14, the attack detection unit 126 selects a low-frequency interval from the three frequency intervals. The attack detection unit 126 uses an inverse discrete cosine transform to extract low-frequency components included in the selected frequency interval from the image data. Then, the process proceeds to step S20.

[0109] In step S15, the attack detection unit 126 determines whether the illumination light level is medium. If the illumination light level is medium, the process proceeds to step S16. If the illumination light level is not medium, the process proceeds to step S17. In step S16, the attack detection unit 126 selects a medium frequency interval from the three frequency intervals. The attack detection unit 126 uses an inverse discrete cosine transform to extract medium frequency components included in the selected frequency interval from the image data. Then, the process proceeds to step S20.

[0110] In step S17, the attack detection unit 126 determines whether the illumination light level is strong and reflected light is present. If the illumination light level is strong and reflected light is present, processing proceeds to step S18. If the illumination light level is strong and reflected light is absent, processing proceeds to step S19. In step S18, the attack detection unit 126 selects a high-frequency interval from the three frequency intervals. The attack detection unit 126 uses an inverse discrete cosine transform to extract high-frequency components of frequencies included in the selected frequency interval from the image data. Then, processing proceeds to step S20.

[0111] In step S19, the attack detection unit 126 selects a mid-frequency interval from the three frequency intervals. The attack detection unit 126 extracts mid-frequency components included in the selected frequency interval from the image data using an inverse discrete cosine transform. In step S20, the attack detection unit 126 inputs the image data and frequency component data indicating the frequency components extracted in any of steps S14, S16, S18, and S19 into a machine learning model.

[0112] In step S21, the attack detection unit 126 determines whether the image data corresponds to a presentation attack according to the output of the machine learning model. The attack detection unit 126 outputs the determination result. The attack detection unit 126 may display the determination result on the display device 111, store it in the non-volatile storage 103, or transmit it to another information processing device.

[0113] As described above, the information processing apparatus 100 according to the second embodiment extracts features from image data using a machine learning model and detects presentation attacks against biometric authentication from the extracted features. This reduces the risk of erroneously authenticating an attacker as a registered user, thereby improving the security level of biometric authentication.

[0114] The information processing device 100 also determines the illumination light level of the image data and the presence or absence of reflected light. The information processing device 100 extracts low-frequency components, mid-frequency components, or high-frequency components from the image data depending on the illumination light level and the presence or absence of reflected light. The information processing device 100 inputs frequency component data indicating the extracted frequency components into the machine learning model in addition to the image data itself. This emphasizes frequency components useful for detecting presentation attacks, taking into account differences in lighting conditions. Therefore, the accuracy of detecting presentation attacks is stably improved even when the imaging device 115 may be used under various lighting conditions.

[0115] The foregoing merely illustrates the principles of the present invention. Further, since numerous modifications and changes will be apparent to those skilled in the art, the present invention is not limited to the exact construction and application shown and described above, and all corresponding modifications and equivalents are deemed to be within the scope of the present invention as defined by the appended claims and their equivalents.

[0116] REFERENCE SIGNS LIST 10 Information processing device 11 Storage unit 12 Processing unit 13 Image data 14 Frequency component data 15a, 15b, 15c Levels 16a, 16b, 16c Frequency intervals 17 Machine learning model

Claims

1. A biometric authentication processing program that causes a computer to execute the following process: determine the brightness of image data from multiple pixel values ​​contained in the image data; generate frequency component data from the image data that indicates frequency components in a frequency range corresponding to the brightness, such that if the brightness is at a first level, the data indicates frequency components in a first frequency range, and if the brightness is at a second level higher than the first level, the data indicates frequency components in a second frequency range that includes frequencies higher than the first frequency range; and input the image data and the frequency component data into a machine learning model to determine whether the image data indicates a spoofing attack against biometric authentication.

2. A biometric authentication processing program as described in claim 1, wherein the generating process includes a process of selecting the first frequency interval if the brightness is at the first level, selecting the second frequency interval if the brightness is at the second level, and selecting a third frequency interval including a higher frequency than the second frequency interval if the brightness is at a third level higher than the second level.

3. A biometric authentication processing program as described in claim 1, wherein the process of determining the brightness includes a process of determining the presence or absence of reflected light from a gradient between adjacent pixel values ​​among the plurality of pixel values, and the process of generating includes a process of generating the frequency component data so as to indicate frequency components in a frequency range corresponding to the brightness and the presence or absence of reflected light.

4. A biometric authentication processing program as described in claim 3, wherein the generating process includes a process of selecting the first frequency interval when the brightness is at the first level, selecting the second frequency interval when the brightness is at the second level and when the brightness is at a third level higher than the second level and the reflected light is not present, and selecting a third frequency interval including a higher frequency than the second frequency interval when the brightness is at the third level and the reflected light is present.

5. The biometric authentication processing program according to claim 1, wherein the brightness is determined from an average of the plurality of pixel values ​​and one or more thresholds.

6. A biometric authentication processing method in which a computer performs the following processes: determining the brightness of image data from multiple pixel values ​​contained in the image data; generating frequency component data from the image data indicating frequency components in a frequency range corresponding to the brightness, such that if the brightness is at a first level, the frequency components in a first frequency range are indicated, and if the brightness is at a second level higher than the first level, the frequency components in a second frequency range including frequencies higher than the first frequency range are indicated; and inputting the image data and the frequency component data into a machine learning model to determine whether the image data indicates a spoofing attack against biometric authentication.

7. An information processing device having: a memory unit that stores image data and a machine learning model; and a processing unit that determines the brightness of the image data from a plurality of pixel values ​​included in the image data, generates frequency component data from the image data that indicates frequency components in a frequency range corresponding to the brightness, so that if the brightness is at a first level, the data indicates frequency components in a first frequency range, and if the brightness is at a second level higher than the first level, the data indicates frequency components in a second frequency range that includes frequencies higher than the first frequency range, and inputs the image data and the frequency component data into the machine learning model, thereby determining whether the image data indicates a spoofing attack against biometric authentication.

Images