Face authentication device, face authentication system and face authentication method

Abstract

To perform face authentication by efficiently determining whether a face image is a face image obtained by imaging a face of a living body.SOLUTION: A face authentication device 10 segments a partial image of an eye on the basis of an acquired face image, and calculates an opening / closing degree score of the eye and an eye-likeness degree score on the basis of the partial image. Then, the face authentication device discriminates whether blinking is performed from data of a plurality of opening / closing degree scores, and discriminates that the face image is not one obtained by imaging a living body when the blinking is not detected. If the blinking is detected, the face authentication device also detects a section when the blinking is performed and calculates the ratio of the number of frame images included during the blinking section to the number of frame images in which the eye-likeness score is a prescribed threshold or less in the blinking section, discriminates the face image obtained by imaging the living body when the ratio is smaller than the prescribed threshold, and performs face authentication in the case of being the face image obtained by imaging the living body.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a face recognition device, a face recognition system, and a face recognition method for efficiently determining whether a face image is a face image of a living body and performing face recognition. [Background technology]

[0002] Conventionally, there is known a face authentication system that authenticates a user by comparing a face image captured by an imaging device with a pre-registered face image. In such a face authentication system, a fraudulent act of impersonating a legitimate person may be committed by capturing a face photo of the legitimate person by an imaging device.

[0003] For this reason, a technique is known for determining that a face image captured by an imaging device is not a facial photograph but is a facial image of a person's face. For example, Patent Document 1 discloses a technique for determining whether or not there is a blink based on the amount of change in the number of pixels forming the eye area of ​​a face image, thereby identifying a living body. Patent Document 2 discloses a technique for detecting that a face image is a photograph of a person (living body) when the eye area included in the face image performs an eye closing action. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2006-330936 [Patent Document 2] Patent Publication No. 2015-041307 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the above-mentioned patent documents 1 and 2 have a problem in that if the face photograph captured by the imaging device is tilted, or if the eye part of the face photograph is covered by a finger or the like, the number of pixels forming the eye area of ​​the face photograph changes, leading to an erroneous determination that a blink has occurred.

[0006] The present invention has been made to solve the problems (issues) associated with the above-mentioned conventional technology, and aims to provide a face recognition device, a face recognition system, and a face recognition method that can efficiently distinguish whether a face image is an image of a living body or not, and perform face recognition. [Means for solving the problem]

[0007] In order to solve the above-mentioned problems and achieve the object, the present invention provides a face recognition device that performs face recognition of a person to be authenticated by comparing a face image of the person to be authenticated with a registered face image that has been registered in advance, and includes an acquisition unit that acquires a plurality of face images obtained by successively capturing an image of the face of the person to be authenticated, an opening and closing degree score calculation unit that calculates an opening and closing degree score indicating the degree of opening and closing of eyes included in the plurality of face images, an eye-likeness score calculation unit that calculates an eye-likeness score indicating the degree of eye-likeness of an eye area included in the plurality of face images, a blinking segment detection unit that detects blinking segments based on a time series change in the opening and closing degree score, a biometric discrimination unit that determines whether the face image is a face image captured of a living organism based on the time series change in the eye-likeness score and the blinking segment, and a face recognition processing unit that performs face recognition of the person to be authenticated, provided that the face image is a face image captured of a living organism.

[0008] In the above-mentioned invention, the present invention further comprises a notification unit that notifies the user that the face image is not a face image of a living body when a blink segment is not detected by the blink segment detection unit.

[0009] In addition, in the above invention, the blink interval detection unit detects the blink interval from the frame marking the first change point to the frame marking the second change point, which is obtained by differentiating the broken line connecting the opening and closing degree scores for each frame.

[0010] In addition, in the present invention, in the above-mentioned invention, the biometric discrimination unit determines that the face image is a face image captured of a living body if the proportion of frames among the multiple frames included in the blinking period, in which the eye-likeness score does not meet a predetermined threshold value, is equal to or less than a certain percentage.

[0011] Further, in the present invention, in the above-mentioned invention, the eye-likeness score calculation unit calculates the eye-likeness score using a trained model that has been machine-learned using teacher data including a plurality of training images of eye regions and eye-likeness scores as ground truth data.

[0012] In addition, in the above invention, the present invention provides a method for calculating the eye opening / closing degree score using a trained model that has been machine-learned using teacher data including training images of multiple eye areas and eye opening / closing degree scores as correct answer data.

[0013] In addition, in the above-mentioned invention, the present invention further includes a face angle calculation unit that calculates the angle of a face contained in the multiple face images, and the biometric discrimination unit determines whether or not the face image is a face image captured of a living body based on a time series change in the eye-likeness score, the blinking period, and the face angle.

[0014] In addition, in the above-mentioned invention, the present invention further includes a face blur calculation unit that calculates the blur of faces included in the plurality of face images, and the biometric discrimination unit determines whether or not the face image is a face image captured of a living organism based on a time series change in the eye-likeness score, the blinking interval, and the face blur degree.

[0015] In addition, in the above-mentioned invention, the present invention further includes a face score calculation unit that calculates a face score indicating the degree of face-likeness of a face part included in the multiple face images, and the biometric discrimination unit determines whether or not the face image is a face image captured of a living body based on a time series change in the eye-likeness score, the blinking interval, and the face score.

[0016] The present invention also provides a biometric discrimination device that determines whether or not a facial image is a facial image obtained by capturing an image of a person, and includes an acquisition unit that acquires a plurality of facial images obtained by successively capturing an image of a person's face, an opening and closing score calculation unit that calculates an opening and closing score indicating the degree of opening and closing of eyes included in the plurality of facial images, an eye-likeness score calculation unit that calculates an eye-likeness score indicating the degree of eye-likeness of an eye area included in the plurality of facial images, a blinking segment detection unit that detects blinking segments based on a time series change in the opening and closing score, and a biometric discrimination unit that determines whether or not the facial image is a facial image obtained by capturing an image of a living organism based on the time series change in the eye-likeness score and the blinking segment.

[0017] The present invention also provides a face recognition system having an imaging device that captures a face image of a person to be authenticated, and a face recognition device that performs face authentication of the person to be authenticated by comparing the face image with a pre-registered registered face image, wherein the face recognition device includes an acquisition unit that acquires a plurality of face images obtained by successively capturing images of the face of the person to be authenticated, an opening and closing degree score calculation unit that calculates an opening and closing degree score indicating the degree of opening and closing of the eyes included in the plurality of face images, an eye-likeness score calculation unit that calculates an eye-likeness score indicating the degree of eye-likeness of an eye area included in the plurality of face images, a blinking segment detection unit that detects blinking segments based on a time series change in the opening and closing degree score, a biometric discrimination unit that determines whether the face image is a face image captured of a living organism based on the time series change in the eye-likeness score and the blinking segment, and a face recognition processing unit that performs face authentication of the person to be authenticated, provided that the face image is a face image captured of a living organism.

[0018] The present invention also provides a face authentication method in a face authentication device that performs face authentication of a person to be authenticated by comparing a face image of the person to be authenticated with a registered face image that has been registered in advance, the method including: an acquisition step of acquiring a plurality of face images by continuously capturing images of the face of the person to be authenticated; an opening and closing degree score calculation step of calculating an opening and closing degree score indicating the degree of opening and closing of the eyes included in the plurality of face images; an eye-likeness score calculation step of calculating an eye-likeness score indicating the degree of eye-likeness of an eye area included in the plurality of face images; a blinking segment detection step of detecting a blinking segment based on a time series change in the opening and closing degree score; a biometric discrimination step of discriminating whether the face image is a face image captured of a living organism based on the time series change in the eye-likeness score and the blinking segment; and a face authentication processing step of performing face authentication of the person to be authenticated, provided that the face image is a face image captured of a living organism. Effect of the Invention

[0019] According to the present invention, it is possible to efficiently determine whether or not a face image is a face image obtained by capturing an image of the face of a living body, and perform face authentication. [Brief description of the drawings]

[0020] [Figure 1] FIG. 1 is a diagram showing an overview of a face recognition device according to a first embodiment. [Diagram 2] FIG. 2 is a diagram showing the system configuration of the face authentication system according to the first embodiment. [Diagram 3] FIG. 3 is a functional block diagram showing the configuration of the face authentication device shown in FIG. [Figure 4] FIG. 4 is an explanatory diagram for explaining an overview of supervised learning of the trained model shown in FIG. [Diagram 5] FIG. 5 is an explanatory diagram for explaining an overview of the opening / closing degree score calculation unit shown in FIG. [Figure 6] FIG. 6 is an explanatory diagram for explaining an overview of the eye-likeness score calculation unit shown in FIG. [Figure 7] FIG. 7 is an explanatory diagram for explaining an overview of blink detection. [Figure 8]FIG. 8 is an explanatory diagram for explaining an overview of the blink period detection unit. [Figure 9] FIG. 9 is an explanatory diagram for explaining an overview of the deletion rate calculation unit. [Figure 10] FIG. 10 is a flowchart (part 1) showing the processing procedure of the face recognition device shown in FIG. [Figure 11] FIG. 11 is a flowchart (part 2) showing the processing procedure of the face recognition device shown in FIG. [Figure 12] FIG. 12 is a diagram illustrating an overview of a face recognition device according to the second embodiment. [Figure 13] FIG. 13 is a diagram showing an overview of a face authentication device according to the first modification. [Figure 14] FIG. 14 is a diagram showing an overview of a face recognition device according to the second modification. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0021] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of a face recognition device, a face recognition system, and a face recognition method according to the present invention will be described in detail with reference to the drawings.

[0022] [Embodiment 1] An outline of the face recognition device according to the first embodiment will be described. FIG. 1 is an explanatory diagram for explaining an outline of the face recognition device 10 according to the first embodiment. As shown in FIG. 1, the face recognition device 10 inputs a face image composed of a plurality of frame images (still images of each frame constituting a moving image), and generates a partial image by cutting out pixels of an eye portion from one of the input frame images. The face recognition device 10 uses the partial image to calculate the degree of opening and closing of the eyes as an opening and closing degree score, and calculates an eye-likeness score of the partial image. Here, the eye-likeness score is a score that indicates the likelihood that the acquired image is an eye. Then, the face recognition device 10 judges whether the acquired image is of a living body or not based on the eye opening and closing degree data and the eye-likeness score of the plurality of frame images. After that, the face recognition device 10 performs face recognition processing.

[0023] The face recognition device 10 includes a cutout processing unit 15b, an open / closed score calculation unit 15c, an eye-likeness score calculation unit 15d, a biometrics determination unit 15e, and a face recognition processing unit 15h. The cutout processing unit 15b cuts out images of the eye portion from a plurality of frame images acquired by the face recognition device 10. In order to cut out the image of the eye portion from the face image, for example, reference data of the feature points of the eye can be prepared and cut out by matching the feature points with the face image.

[0024] The open / closed degree score calculation unit 15c, as will be described later in detail, prepares a large number of teacher images of eyes and correct answer data, performs supervised learning by deep learning to create a learned model, and inputs the eye image into the learned model to calculate the open / closed degree score of the eye. The eye-likeness score calculation unit 15d, as will be described later in detail, prepares a large number of teacher images of eyes and correct answer data, performs supervised learning by deep learning to create a learned model, and inputs the eye image into the learned model to calculate an eye-likeness score that indicates the likelihood that the input image is an eye.

[0025] The living body discrimination unit 15e discriminates whether the acquired face image is a face image of a living body based on the open / closed degree score and the eye-likeness score. Although details will be described later, it discriminates whether blinking has occurred based on data of a plurality of open / closed degree scores, and if blinking is not detected, it discriminates that the face image is not a face image of a living body. If blinking is detected, it detects a section in which blinking has occurred, and further calculates the number of frame images included in the blinking section and the ratio of the number of frame images in the blinking section whose eye-likeness score is equal to or less than a predetermined threshold, and if the ratio is smaller than the predetermined threshold, it discriminates that the face image is a face image of a living body. If the ratio is equal to or greater than the predetermined threshold, it discriminates that the face image is not a face image of a living body.

[0026] When the biometrics discrimination unit 15e determines that the face image is a facial image of a living body, the face recognition processing unit 15h performs face recognition processing using the facial image. Specifically, authentication data that stores feature points of multiple people is prepared in advance, and the acquired face image is compared with the authentication data to identify (authenticate) the person.

[0027] In this way, the face recognition device 10 according to the first embodiment cuts out a partial image of the eyes based on the acquired face image, calculates an eye open / closed degree score and an eye-likeness score based on the partial image, and determines whether the acquired face image is a face image captured of a living body based on the open / closed degree score and the eye-likeness score. If the acquired face image is a face image captured of a living body, face recognition is performed.

[0028] Next, a face authentication system will be described. Fig. 2 is a diagram showing the system configuration of the face authentication system according to embodiment 1. As shown in Fig. 2, the face authentication system has a face authentication device 10 and imaging devices 20a, 20b, and 20c (hereinafter, may be collectively referred to as "imaging devices 20"). The face authentication device 10 and the imaging devices 20 are connected to a network N.

[0029] The face recognition device 10 performs a process of acquiring a moving image of a face image transmitted from the imaging device 20, a process of cutting out a partial image of an eye from the face image, a process of calculating an eye open / closed degree score from the partial image of the eyes, a process of calculating an eye-likeness score from the partial image of the eyes, a process of calculating a blinking period from a time-series change in the open / closed degree score, a process of determining whether or not the face image is a face image of a living body based on the eye-likeness score and the blinking period, and a face recognition process from the face image. The imaging device 20 performs a process of capturing a moving image of a face image.

[0030] Next, a description will be given of the configuration of the face authentication device 10 shown in Fig. 2. Fig. 3 is a functional block diagram showing the configuration of the face authentication device 10 shown in Fig. 2. As shown in Fig. 3, the face authentication device 10 has a display unit 11, an input unit 12, a communication I / F unit 13, a storage unit 14, and a control unit 15.

[0031] The display unit 11 is a display device such as a liquid crystal panel or a display device, and the input unit 12 is an input device such as a keyboard or a mouse. The communication I / F unit 13 is an interface unit for communicating with other devices such as the imaging device 20.

[0032] The memory unit 14 is a storage device such as a hard disk drive or a non-volatile memory, and stores an opening / closing degree learned model 14a, an eye-likeliness learned model 14b, opening / closing degree score data 14c, eye-likeliness score data 14d, and face authentication data 14e, etc.

[0033] The open / close degree trained model 14a is a model obtained as a result of performing supervised learning by deep learning using a large number of teacher images A with different eye open / close degrees and correct answer data R. Specifically, as shown in FIG. 4, the teacher images A are input to a CNN (Convolutional Neural Network) B, and backpropagation is performed based on the correct answer data R, for example, to repeat a supervised learning process for determining the weight of each path, thereby generating a trained model M. Note that the correct answer data R, i.e., the probability that it is a known phenomenon, can be assigned by a person in charge of the supervised learning. The trained model M is generated by a device such as a server or cloud.

[0034] The eye-likeness trained model 14b is a model obtained as a result of supervised learning by deep learning using a large number of teacher images A of eye images and correct answer data R. The eye-likeness trained model 14b is generated using a server, a cloud, or the like, similar to the open / close degree trained model 14a.

[0035] The open / close score data 14c is data on the time series change of the open / close score obtained by inputting a partial image of the eye into the open / close learned model 14a. The eye-likeness score data 14d is data on the time series change of the eye-likeness score obtained by inputting a partial image of the eye into the eye-likeness learned model 14b. The face authentication data 14e is facial feature data of multiple people to be authenticated.

[0036] The control unit 15 is a control unit that performs overall control of the face recognition device 10, and includes an image acquisition processing unit 15a, a clipping processing unit 15b, an open / close score calculation unit 15c, an eye-likeness score calculation unit 15d, a biometrics discrimination unit 15e, a face recognition processing unit 15h, and a display control unit 15i. In practice, these programs are loaded into a CPU and executed, causing the image acquisition processing unit 15a, the clipping processing unit 15b, the open / close score calculation unit 15c, the eye-likeness score calculation unit 15d, the biometrics discrimination unit 15e, the face recognition processing unit 15h, and the display control unit 15i to execute processes corresponding thereto.

[0037] The image acquisition processing unit 15a performs a process of acquiring a face image transmitted from the imaging device 20. The cut-out processing unit 15b performs a process of cutting out an eye partial image from the acquired face image. Specifically, the face image is compared with eye feature data (not shown) to cut out an image of a part having eye features. Note that a learned model using machine learning or deep learning may be used for the cut-out process.

[0038] The open / close score calculation unit 15c is a processing unit that calculates the open / close score of the partial image of the eyes using the open / close learned model 14a. Specifically, as shown in Fig. 5, a cut-out image that is a partial image of the eyes of a face image is input to the open / close learned model 14a, and an open / close score of 0 to 25 is calculated from the output from the open / close learned model 14a.

[0039] Eye-likeness score calculation unit 15d inputs an excised image, which is a partial image of an eye, into eye-likeness trained model 14b using eye-likeness trained model 14b, and calculates an eye-likeness score between 0 and 1 from the output from eye-likeness trained model 14b as a score of the likelihood that the excised image is an eye.

[0040] The living body discrimination unit 15e is a processing unit that discriminates whether or not the face image is a face image obtained by capturing a living body, based on the open / close degree score data 14c and the eye-likeness score data 14d. The living body discrimination unit 15e has a blinking period detection unit 15f and a deletion rate calculation unit 15g.

[0041] The blinking section detection unit 15f is a processing unit that detects a blinking section based on the open / closed score data 14c and the eye-likeness score data 14d, or based on the open / closed score data 14c. When the blinking section detection unit 15f detects a blinking section based on the open / closed score data 14c and the eye-likeness score data 14d, specifically, as shown in Fig. 7, the open / closed score data 14c (see Fig. 7(a)) is compared with the eye-likeness score data 14d (Fig. 7(d)) ​​for each frame ID, and when the eye-likeness score is smaller than a predetermined threshold T, the open / closed score of the frame ID is deleted (see Fig. 7(b)).

[0042] Then, the second derivative of the time-series change data of the open / close score is calculated from which data with an eye-likeness score smaller than a predetermined threshold T has been deleted (see FIG. 7(c)). After that, blink periods are detected from the time-series change of the second derivative.

[0043] Here, the blinking period is detected when there is a time series change in the second derivative that is convex downward and then convex upward, and when there is a time series change in the second derivative that is convex upward and then convex downward, it is determined that there was a blink, and the number of frames between them is the blinking period. Note that in the case of Figure 7, since there is only a change in the second derivative that is convex downward and then convex upward, it is determined that there was no blink, and the blinking period is 0 frames.

[0044] Furthermore, when the blink section detection unit 15f detects a blink section based on the opening / closing score data 14c, specifically, as shown in Fig. 8, it calculates (see Fig. 8(b)) the second derivative of the opening / closing score data 14c (see Fig. 8(a)). Then, when there is a time series change in the second derivative that is convex downward and then convex upward, and when there is a time series change in the second derivative that is convex upward and then convex downward, the blink section detection unit 15f determines that a blink has occurred, and regards the number of frames therebetween as the blink section.

[0045] In addition, as shown in FIG. 8(b), in the section from frame ID 13 to 19 and in the frame IDs where the second derivative changes downward and then changes upward over time, the initial time series change in the second derivative is determined to be invalid.

[0046] The deletion rate calculation unit 15g is a processing unit that calculates the proportion of frames to be deleted from a blink segment as a deletion rate based on the blink segment and the eye-likeness score. Specifically, as shown in Fig. 9(a), the deletion rate is calculated as the number of frames whose eye-likeness scores are smaller than a predetermined threshold value T (see Fig. 9(b)) to be deleted from the calculated number of frames in the blink segment. In this example, the blink segment is 28 frames from frame ID 21 to frame ID 48. The number of frames to be deleted from the blink segment is 27 excluding frame ID 44, so the deletion rate is 96%.

[0047] If the deletion rate is equal to or less than a predetermined threshold (e.g., 30%), the biometric discrimination unit 15e determines that the acquired face image is a face image of a living body, and if the deletion rate is greater than the predetermined threshold, determines that the face image is not a face image of a living body. Note that if the deletion rate is less than the predetermined threshold, the biometric discrimination unit 15e may determine that the acquired face image is a face image of a living body, and if the deletion rate is equal to or greater than the predetermined threshold, determine that the face image is not a face image of a living body.

[0048] The face recognition processing unit 15h performs a face image authentication process based on the registered face recognition data 14e. Specifically, the acquired face image is compared with the registered face recognition data 14e, and if the comparison is successful, the authentication is determined to be successful, and if the comparison is not successful, the authentication is determined to be unsuccessful. A pattern matching method is used for authenticating the face image. The authentication process may be performed by inputting the face image to a trained model that has been machine-learned using a large number of face images for face recognition as teacher images and correct answer data.

[0049] The display control unit 15i is a processing unit that controls the display of the authentication result on the display unit 11 based on the presence or absence of blinking and the result of face authentication. Specifically, when the result of the face authentication processing unit 15h is successful authentication, the display control unit 15i controls the display of "authentication successful" on the display unit 11. On the other hand, when there is no blinking and the result of the face authentication processing unit 15h is unsuccessful authentication, the display control unit 15i controls the display of "authentication unsuccessful" on the display unit 11.

[0050] Next, the processing procedure of the face authentication device 10 will be described. Fig. 10 and Fig. 11 are flowcharts showing the processing procedure of the face authentication device 10 shown in Fig. 3. As shown in Fig. 10, the face authentication device 10 acquires a face image from the imaging device 20 via the network N (step S101). Then, the face authentication device 10 cuts out a partial image of the eyes from the face image (step S102) and calculates an open / closed degree score (step S103). After that, the face authentication device 10 calculates an eye-likeness score based on the partial image of the eyes (step S104).

[0051] Then, the face authentication device 10 deletes data whose eye-likeness score is equal to or smaller than a predetermined threshold from the time-varying data of the open / close degree score (step S105), and calculates a second derivative based on the remaining open / close degree score data from which the data has been deleted (step S106). After that, the face authentication device 10 detects the rise and fall of the second derivative based on the second derivative (step S107).

[0052] Then, the face authentication device 10 determines whether or not a blink period is detected (step S108). If the face authentication device 10 does not detect a blink period (step S108: No), it controls the display of "authentication failed" (step S115).

[0053] On the other hand, if the face recognition device 10 detects a blinking period (step S108: Yes), Based on the open / close score data, the second derivative coefficient is calculated (step S109). Then, the face authentication device 10 counts the number of data that meets the deletion condition within the detected blinking period (step S110). After that, the face authentication device 10 determines whether the ratio of the deleted data to the number of data within the blinking period is equal to or less than a predetermined threshold (step S111).

[0054] If the ratio of deleted data is not equal to or less than the predetermined threshold (step S111: No), the face authentication device 10 controls the display unit 11 to display "authentication failed" (step S115). On the other hand, if the ratio of deleted data is equal to or less than the predetermined threshold (step S111: Yes), the face authentication device 10 performs face authentication processing (step S112).

[0055] Then, the face authentication device 10 determines whether or not the face authentication is successful (step S113). If the face authentication is not successful (step S113: No), the face authentication device 10 controls the display unit 11 to display "Authentication Failed" (step S115). On the other hand, if the face authentication is successful (step S113: Yes), the face authentication device 10 controls the display unit 11 to display "Authentication Successful" (step S114), and ends the series of processes.

[0056] As described above, in the first embodiment, the face recognition device 10 acquires a face image and cuts out a partial image of the eyes from the face image. The face recognition device 10 then calculates an eye open / closed score and an eye-likeness score based on the partial image of the eyes, and detects a blinking section based on the open / closed score and the eye-likeness score. The face recognition device 10 then calculates the ratio of the number of frames in the blinking section and the number of frames in the blinking section that have an eye-likeness score equal to or less than a predetermined threshold as deleted frames, and performs face recognition when the ratio is equal to or less than the predetermined threshold.

[0057] [Embodiment 2] In the above embodiment 1, a case where biometric determination is performed based on an eye opening / closing score and an eye-likeness score is described. In embodiment 2, a case where biometric determination is performed based on an eye opening / closing score and an eye-likeness score, and a face angle is described.

[0058] Fig. 12 is a diagram showing an overview of a face authentication device 30 according to embodiment 2. Note that the same components as those of the face authentication device 10 shown in Fig. 1 are denoted by the same reference numerals, and detailed description thereof will be omitted. As shown in Fig. 12, the face authentication device 30 has a cutout processing unit 15b, an opening / closing degree score calculation unit 15c, an eye-likeness score calculation unit 15d, a face authentication processing unit 15h, a face angle calculation unit 35a, and a biometrics discrimination unit 35b.

[0059] The face angle calculation unit 35a is a processing unit that calculates the face angle of the acquired face image. Specifically, a face angle learned model is generated by performing machine learning based on a large number of teacher images of face images with different face angles and correct answer data, the face image is input to the face angle learned model, and the face angle of the acquired face image is calculated.

[0060] The biometric discrimination unit 35b is a processing unit that discriminates whether the acquired face image is a face image of a living body based on the data of the time series change of the open / close score, the data of the time series change of the eye-likeness score, and the data of the time series change of the face angle. Specifically, the biometric discrimination unit 35b calculates the second derivative of the time series change data of the open / close score, and calculates the blinking period based on the time series change of the second derivative. Then, the biometric discrimination unit 35b calculates the number of frame ID data whose eye-likeness score is equal to or less than a predetermined threshold and whose face angle is equal to or more than a predetermined threshold among the data of the blinking period, and calculates the deletion rate. After that, if the deletion rate is equal to or less than the predetermined threshold, the biometric discrimination unit 35b discriminates that the acquired face image is a face image of a living body, and if the deletion rate is greater than the predetermined threshold, discriminates that the face image is not a face image of a living body.

[0061] As described above, in the second embodiment, the face authentication device 30 acquires a face image and cuts out a partial image of the eyes from the face image. Then, the face authentication device 10 calculates an eye opening / closing score and an eye-likeness score based on the partial image of the eyes. The face authentication device 30 also calculates a face angle from the acquired face image. Then, the face authentication device 30 detects a blinking section based on the opening / closing score and the eye-likeness score. After that, the face authentication device 10 calculates the number of frames in the blinking section and the ratio of deleted frames to the number of frames in the blinking section, with frames of the blinking section having an eye-likeness score equal to or less than a predetermined threshold or frames of the blinking section having a face angle equal to or greater than a predetermined threshold being deleted frames, and if the ratio is equal to or less than the predetermined threshold, the face image is determined to be a face image of a living body, and face authentication is performed.

[0062] In the above embodiment 2, the biometric discrimination unit 35b calculates the number of frame ID data whose eye-likeness score is below a predetermined threshold and the number of frame ID data whose face angle is above a predetermined threshold from the data of the blinking section, and calculates the deletion rate. However, the present invention is not limited to this, and the biometric discrimination unit 35b may calculate the number of frame ID data whose eye-likeness score is below a predetermined threshold and the number of frame ID data whose face angle is above a predetermined threshold from the data of the blinking section, and calculate the deletion rate.

[0063] <Variation 1> In the above-mentioned embodiment 2, the face angle is calculated from the acquired face image, and whether or not the face image is a face image of a living body is determined based on the open / closed score, the eye-likeness score, and the face angle. In the modified example 1, however, the face blur degree is calculated from the acquired face image, and whether or not the face image is a face image of a living body is determined. Here, the face blur degree is the degree to which the acquired face image is out of focus and blurred.

[0064] Fig. 13 is a diagram showing an overview of a face authentication device 40 according to Modification 1. Note that the same components as those of the face authentication device 30 shown in Fig. 12 are given the same reference numerals, and detailed description thereof will be omitted. As shown in Fig. 13, the face authentication device 40 has a cutout processing unit 15b, an opening / closing degree score calculation unit 15c, an eye-likeness score calculation unit 15d, a face authentication processing unit 15h, a face blur degree calculation unit 45a, and a biometric determination unit 45b.

[0065] The face blur calculation unit 45a is a processing unit that calculates the face blur of the acquired face image. Specifically, a face blur trained model is generated by performing machine learning based on a large number of teacher images of face images with different face blur levels and correct answer data, and the face image is input to the face blur trained model to calculate the face blur of the acquired face image. Note that the face blur level is larger as the face image has a larger blur level.

[0066] The biometric discrimination unit 45b is a processing unit that discriminates whether the acquired face image is a face image of a living body based on the data of the time series change of the open / close score, the data of the time series change of the eye-likeness score, and the data of the time series change of the face blur. Specifically, the face authentication device calculates a second derivative of the time series change data of the open / close score, and calculates a blinking period based on the time series change of the second derivative. Then, the biometric discrimination unit 45b calculates the number of frame ID data whose eye-likeness score is equal to or less than a predetermined threshold and whose face blur is equal to or more than a predetermined threshold among the data of the blinking period, and calculates a deletion rate. After that, if the deletion rate is equal to or less than the predetermined threshold, the biometric discrimination unit 35b discriminates that the acquired face image is a face image of a living body, and if the deletion rate is greater than the predetermined threshold, discriminates that the face image is not a face image of a living body.

[0067] <Variation 2> In the above-mentioned embodiment 2, the face angle is calculated from the acquired face image, and whether or not the face image is a face image of a living body is determined based on the open / closed degree score, the eye-likeness score, and the face angle. In the modified example 2, however, the face score is calculated from the acquired face image, and whether or not the face image is a face image of a living body is determined. Here, the face score is the degree of face-likeness of the acquired face image.

[0068] Fig. 14 is a diagram showing an overview of a face authentication device 50 according to Modification 2. Note that the same reference numerals are used to designate the same parts as those of the face authentication device 30 shown in Fig. 12, and detailed descriptions thereof will be omitted. As shown in Fig. 14, the face authentication device 50 has a cutout processing unit 15b, an opening / closing degree score calculation unit 15c, an eye-likeness score calculation unit 15d, a face authentication processing unit 15h, a face score calculation unit 55a, and a biometric discrimination unit 55b.

[0069] The face score calculation unit 55a is a processing unit that calculates a face score, which is the face-likeness of an acquired face image. Specifically, a face score trained model is generated by performing machine learning based on a large number of teacher images of face images and correct answer data, and the face image is input to the face score trained model to calculate the face score of the acquired face image. Note that the face score has a large value when the face image is face-like.

[0070] The biometric discrimination unit 55b is a processing unit that discriminates whether the acquired face image is a face image of a living body based on the data of the time series change of the open / close score, the data of the time series change of the eye-likeness score, and the data of the time series change of the face score. Specifically, the face authentication device calculates a second derivative of the time series change data of the open / close score, and calculates a blinking period based on the time series change of the second derivative. Then, the biometric discrimination unit 45b calculates the number of frame ID data whose eye-likeness score is equal to or less than a predetermined threshold and the number of frame ID data whose face score is equal to or less than a predetermined threshold among the data of the blinking period, and calculates a deletion rate. After that, if the deletion rate is equal to or less than the predetermined threshold, the biometric discrimination unit 35b discriminates that the acquired face image is a face image of a living body, and if the deletion rate is greater than the predetermined threshold, discriminates that the face image is not a face image of a living body.

[0071] In the above-mentioned first and second embodiments, it has been described that the presence or absence of a blink is detected based on the eye opening / closing score and the eye-likeness score, and the blinking period is detected based on the eye opening / closing score, but the present invention is not limited to this, and the presence or absence of a blink and the blinking period may be detected based on the eye opening / closing score.

[0072] In addition, in the above embodiments 1 and 2, the second derivative of the open / close score is calculated to detect the presence or absence of a blink and the blink period. However, the present invention is not limited to this, and the first derivative of the open / close score may be calculated to detect the presence or absence of a blink and the blink period.

[0073] In the above-mentioned first and second embodiments, a trained model using CNN is used to calculate the eye open / closed degree score and the eye-likeness score, but a trained model generated by other machine learning such as an RNN (Recurrent Neural Network) or a Random Forest may also be used.

[0074] In addition, in the above-mentioned embodiments 1 and 2, the detection of spontaneous blinking of the person being imaged has been described, but the imaging device 20 may be provided with a light-emitting unit such as a flash, and reflex blinking may be detected when the flash is emitted.

[0075] The configurations illustrated in the above embodiments are merely functional schematics, and are not necessarily physically configured as illustrated. In other words, the form of distribution and integration of each device is not limited to that illustrated, and all or part of the devices can be functionally or physically distributed and integrated in any unit according to various loads, usage conditions, etc. [Industrial Applicability]

[0076] INDUSTRIAL APPLICABILITY The face recognition device, face recognition system, and face recognition method according to the present invention are suitable for efficiently determining whether or not a face image is a face image obtained by capturing the face of a living body, and for performing face recognition. [Explanation of symbols]

[0077] 10. Face Recognition Device 11 Display section 12 Input section 13 Communication I / F section 14 Storage section 14a Opening and closing degree trained model 14b Eye-likeness trained model 14c Opening and closing score data 14d Eye-likeness score data 14e Face recognition data 15 Control section 15a Image acquisition processing section 15b Cutting processing section 15c Opening score calculation section 15d Eye-likeness score calculation section 15e Biometrics 15f Blink period detection section 15g Removal rate calculation section 15h Face recognition processing section 16i Display control unit 20 Imaging device 30 Face Recognition Device 35a Face angle calculation unit 35b Biometrics 40 Face Recognition Device 40a Face blur calculation unit 40b Biometric discrimination unit 50 Face Recognition Device 50a Face score calculation section 50b Biometric discrimination unit

Claims

1. A face authentication device that performs face authentication of a person to be authenticated by comparing a face image of the person to be authenticated with a registered face image that has been registered in advance, an acquisition unit that acquires a plurality of facial images by successively capturing images of the face of the person to be authenticated; an eye closure score calculation unit that calculates an eye closure score indicating a degree of eye closure included in the plurality of face images; an eye-likeness score calculation unit that calculates an eye-likeness score indicating a degree of eye-likeness of an eye region included in the plurality of face images; a blinking period detection unit that detects a blinking period based on a time series change in the opening and closing degree score; a biometrics discrimination unit that discriminates whether or not the face image is a face image obtained by capturing a living body based on a time-series change in the eye-likeness score and the blinking period; a face authentication processing unit that performs face authentication of the person to be authenticated on the condition that the face image is a face image captured of a living body; A face recognition device comprising:

2. 2. The face recognition device according to claim 1, further comprising a notification unit that notifies the user that the face image is not a face image of a living body when the blink period detection unit does not detect a blink period.

3. The blink period detection unit is The face recognition device according to claim 1, characterized in that a blink period is detected from a frame that forms a first change point to a frame that forms a second change point, which is obtained by differentiating a broken line connecting the opening and closing degree scores for each frame.

4. The biometric determination unit is 2. The face recognition device according to claim 1, wherein when a ratio of a number of frames in which the eye-likeness score does not satisfy a predetermined threshold value among a plurality of frames included in the blinking period is equal to or less than a certain value, the face image is determined to be a face image captured of a living body.

5. The eye-likeness score calculation unit The face recognition device according to claim 1, wherein the eye-likeness score is calculated using a trained model that has been machine-learned using training data including a plurality of training images of eye regions and eye-likeness scores as ground truth data.

6. The opening / closing score calculation unit is The face recognition device according to claim 1, characterized in that the eye opening / closing score is calculated using a trained model machine-learned using teacher data including a plurality of learning images of the eye area and eye opening / closing scores as correct answer data.

7. A face angle calculation unit is further provided for calculating an angle of a face included in the plurality of face images, The biometric determination unit is Based on the time series change in the eye-likeness score, the blinking period, and the face angle, it is determined whether or not the face image is a face image obtained by capturing an image of a living body. The face authentication device according to claim 1 .

8. a face blur calculation unit that calculates a blur degree of a face included in the plurality of face images, The biometric determination unit is Based on the time series change in the eye-likeness score, the blinking period, and the facial blur level, it is determined whether or not the face image is a face image obtained by capturing a living body. The face authentication device according to claim 1 .

9. a face score calculation unit that calculates a face score indicating a degree of face-likeness of a face part included in the plurality of face images, The biometric determination unit is Based on the time series change in the eye-likeness score, the blinking period, and the face score, it is determined whether or not the face image is a face image obtained by capturing an image of a living body. The face authentication device according to claim 1 .

10. A biometric discrimination device that discriminates whether a face image is a face image of a person, An acquisition unit that acquires a plurality of face images by successively capturing images of a person's face; an eye closure score calculation unit that calculates an eye closure score indicating a degree of eye closure included in the plurality of face images; an eye-likeness score calculation unit that calculates an eye-likeness score indicating a degree of eye-likeness of an eye region included in the plurality of face images; a blinking period detection unit that detects a blinking period based on a time series change in the opening and closing degree score; a biometrics discrimination unit that discriminates whether or not the face image is a face image obtained by capturing a living body based on a time-series change in the eye-likeness score and the blinking period; A biometric discrimination device comprising:

11. A face authentication system having an imaging device that captures a face image of a person to be authenticated, and a face authentication device that performs face authentication of the person to be authenticated by matching the face image with a pre-registered face image, The face recognition device includes: an acquisition unit that acquires a plurality of facial images by successively capturing images of the face of the person to be authenticated; an eye closure score calculation unit that calculates an eye closure score indicating a degree of eye closure included in the plurality of face images; an eye-likeness score calculation unit that calculates an eye-likeness score indicating a degree of eye-likeness of an eye region included in the plurality of face images; a blinking period detection unit that detects a blinking period based on a time series change in the opening and closing degree score; a biometrics discrimination unit that discriminates whether or not the face image is a face image obtained by capturing a living body based on a time-series change in the eye-likeness score and the blinking period; a face authentication processing unit that performs face authentication of the person to be authenticated on the condition that the face image is a face image captured of a living body; A face recognition system comprising:

12. A face authentication method for a face authentication device that performs face authentication of a person to be authenticated by comparing a face image of the person to be authenticated with a registered face image, comprising: an acquisition step of acquiring a plurality of facial images by successively capturing images of the face of the person to be authenticated; a calculation step of calculating an eye closure score indicating a degree of eye closure included in the plurality of face images; an eye-likeness score calculation step of calculating an eye-likeness score indicating a degree of eye-likeness of an eye region included in the plurality of face images; A blinking period detection step of detecting a blinking period based on a time series change in the opening and closing degree score; a biometrics determination step of determining whether or not the face image is a face image obtained by capturing a living body based on a time series change in the eye-likeness score and the blinking period; a face authentication processing step of performing face authentication of the person to be authenticated on the condition that the face image is a face image captured of a living body; A face recognition method comprising:

Images