Counterfeit detection system and counterfeit detection program

Abstract

Problem: high-precision detection of impersonation of a person using a planar object such as a photograph. Solution: imaging means 11 that images a plurality of frames of the verification target at different angles; feature detection means 14 that detects the aspect ratio of the face of the verification target, i.e., the ratio of the vertical length to the horizontal length, from each of the plurality of frames of images imaged by the imaging means 11; and impersonation detection means 14 that detects impersonation of a person using a planar object based on the aspect ratio of the face of the verification target imaged at different angles.

Description

TECHNICAL FIELD

[0001] The present application relates to a forgery detection system and a forgery detection program capable of preventing forgery using a planar object such as a photograph. The detection program of the present application is software executable on a computer, and can be stored in a memory medium readable by a computer such as a CD-ROM, a DVD, a USB memory, an HDD, an SSD (Solid State Drive). BACKGROUND

[0002] Conventionally, there is known a technology in which verification information (for example, a person's eyes or mouth, etc.) is memorized in the form of being included in an information code, and the verification information included in the information code and verification information obtained by photographing a subject are used to verify a person (for example, Patent Literature 1). Also, there is known a technology for preventing forgery using a photograph or the like when verifying a person (for example, Patent Literature 2).

[0003] Here, an information code such as a QR code (registered trademark) has a limited memory capacity, and it is difficult to include information of an entire face or the like as verification information. Therefore, as described in Patent Literature 1, a technology is proposed in which feature points such as eyes or a mouth that are part of a face are included in an information code, and the feature points are compared to thereby verify a person. However, in Patent Literature 1, there is a case in which a malicious third party causes a camera to photograph a photograph or the like obtained by photographing a face of a subject, and thereby performs verification by impersonating the subject.

[0004] In this regard, in Patent Literature 2, verification of a person is performed by comparing feature points, and in order to prevent forgery using a photograph, images photographed at different angles are compared to determine forgery, that is, whether it is a photograph or a real object. Specifically, in Patent Literature 2, a first image and a second image are obtained by photographing a check target at different angles, coordinates of second feature points detected from the second image are converted to a projection to the plane of the first image, and when the error between the converted coordinates and coordinates of first feature points detected from the first image becomes equal to or less than a fixed value, it is determined that impersonation is attempted.

[0005] [Patent Literature]

[0006] [Patent Literature]

[0007] Patent Literature 1: Japanese Patent Application Publication No. 2021-168047

[0008] Patent Literature 2: International Publication No. 2010 / 050206 SUMMARY

[0009] [Problems to be Solved by the Invention]

[0010] Thus, although a configuration for determining whether or not an inspection target is a three-dimensional object by comparing images obtained by photographing the inspection target at different angles is disclosed in Patent Literature 2, as security awareness has increased in recent years, a higher counterfeit determination accuracy is expected for a counterfeit detection system and a counterfeit detection program.

[0011] [Technical means for solving the problem]

[0012] To solve this problem, the inventors have conducted intensive research and have obtained the insight that a higher counterfeit determination accuracy can be achieved by verifying whether or not a subject is a planar object. Specifically, the inventors have obtained the insight that, by determining whether or not a subject is a planar object such as a photograph based on the ratio (aspect ratio) of the longitudinal length to the lateral length of a face in a photograph image obtained by photographing the face of the subject, the data capacity for detecting counterfeits can be reduced and counterfeits using a planar object such as a photograph can be detected with high accuracy.

[0013] [1] The present application is based on the following (1) to (6) counterfeit detection system.

[0014] (1) A counterfeit detection system that detects whether or not a subject is a person who actually exists or a person who appears on a planar object, and has: photographing means that photographs a plurality of frames of a face of a subject at different angles; feature detection means that detects the aspect ratio of the face of the subject from each of the plurality of frame images photographed by the photographing means; and counterfeit detection means that detects a counterfeit of a person using a planar object based on the aspect ratios of the face of the subject photographed at different angles.

[0015] It can be configured as follows. (2) The counterfeit detection system according to (1) described above, wherein the counterfeit detection means calculates, for each of the frame images, a change rate or a change amount of the aspect ratio of the face of the subject with respect to a reference aspect ratio as a planar similarity, and detects a counterfeit based on the plurality of planar similarities calculated.

[0016] (3) The counterfeit detection system according to (2) described above, wherein the counterfeit detection means acquires, as the reference aspect ratio, the aspect ratio of a face of an individual who is registered in advance or the aspect ratio of the face of the subject in a frame image that is photographed first among the plurality of frame images.

[0017] (4) The forgery detection system according to (2) or (3) above, wherein the forgery detection means extracts at least three feature points among the plurality of feature points in each frame image that are arranged apart in a width direction of the face, calculates a ratio of a distance between the feature points on the right and left sides of the face among the extracted feature points as a stereoscopic similarity, and detects forgery based on the plurality of the plane similarities and the plurality of the stereoscopic similarities calculated for each of the frame images.

[0018] (5) The forgery detection system according to (4) above, wherein the forgery detection means, for each of the frame images, increments a plane score when the plane similarity is equal to or greater than a predetermined value, and increments a stereoscopic score when the stereoscopic similarity is equal to or greater than a predetermined value; detects that forgery exists when the plane score is equal to or greater than a predetermined difference D from the stereoscopic score, and detects that forgery does not exist when the stereoscopic score is equal to or greater than the predetermined difference D from the plane score.

[0019] (6) The forgery detection system according to any one of (1) to (5) above, wherein the feature detection means detects the positions of the feature points of the face of the subject for each of the frame images taken at different angles, and the forgery detection means, for each of the frame images, further performs processing of calculating a difference in the positions of the feature points from other frame images taken consecutively, and determines that forgery using both a plane object and a stereoscopic object exists when the difference is equal to or greater than a threshold value TL.

[0020] Further, the present application has as its main object a forgery detection program according to (7) below.

[0021] (7) A forgery detection program that detects whether a verification subject is a person who actually exists or a person appearing on a plane object, and causes a computer to execute: an acquisition function that acquires a plurality of frame images taken at different angles of the verification subject; a feature detection function that detects an aspect ratio of a face of the verification subject from each of the plurality of frame images; and a forgery detection function that detects forgery of a person using a plane object based on the aspect ratio of the face of the verification subject taken at different angles.

[0022] [2] Another aspect of the present application can also have as its main object a forgery detection system according to (8) to (14) below.

[0023] (8) A forgery detection system that detects whether a verification target person is a real person or a person appearing on a flat object, and has: a photographing means that photographs a plurality of frames of a face of the verification target person at different angles; a feature detection means that detects an aspect ratio of a face of the verification target person from each of the plurality of frames of images photographed by the photographing means; and a forgery detection means that detects forgery of a person by a flat object based on the aspect ratios of the face of the verification target person photographed at different angles.

[0024] (9) The forgery detection system according to (8) above, wherein the forgery detection means calculates, for each of the frames of images, a rate of change or an amount of change of the aspect ratio of the face of the verification target person with respect to a reference aspect ratio as a flat similarity, and detects forgery based on the plurality of flat similarities calculated.

[0025] (10) The forgery detection system according to (9) above, wherein the forgery detection means acquires an aspect ratio of a face of the person registered in advance, or an aspect ratio of the face of the verification target person in a frame of image photographed first among the plurality of frames of images, as the reference aspect ratio.

[0026] (11) The forgery detection system according to (8) above, wherein the forgery detection means extracts at least three feature points among a plurality of feature points in each frame of image that are arranged separately in a width direction of the face, calculates a ratio of a distance between the feature points on the right and left sides of the face among the extracted feature points as a stereo similarity, and detects forgery based on the plurality of stereo similarities calculated for each of the frames of images.

[0027] (12) The forgery detection system according to (9) or (10) above, wherein the forgery detection means extracts at least three feature points among a plurality of feature points in each frame of image that are arranged separately in a width direction of the face, calculates a ratio of a distance between the feature points on the right and left sides of the face among the extracted feature points as a stereo similarity, and detects forgery based on the plurality of flat similarities and the plurality of stereo similarities calculated for each of the frames of images.

[0028] (13) The forgery detection system according to (12) above, wherein the forgery detection means performs the following processing for each of the frames of images, that is, when the flat similarity is equal to or greater than a predetermined value, a flat score is incremented, and when the stereo similarity is equal to or greater than a predetermined value, a stereo score is incremented; when the flat score with respect to the stereo score becomes equal to or greater than a predetermined difference D, it is detected that there is forgery, and when the stereo score with respect to the flat score becomes equal to or greater than the difference D, it is detected that there is no forgery.

[0029] (14) The forgery detection system according to any one of (8) to (13) above, wherein the feature detection means detects the positions of the feature points of the face of the subject for each of the frame images taken at different angles, and the forgery detection means further performs, for each of the frame images, processing of calculating the difference in the positions of the feature points from other frame images taken consecutively, and determining that there is a forgery using both the flat object and the three-dimensional object when the difference is equal to or greater than the threshold value TL.

[0030] Further, another aspect of the present application is a recording medium of a forgery detection program as described in (15) to (21) below.

[0031] (15) A recording medium of a forgery detection program for detecting whether a verification subject is a real person or a person appearing on a flat object, the program having: a function of acquiring a plurality of frame images taken at different angles of a verification subject; a function of detecting, from each of the plurality of frame images, the aspect ratio of the face of the verification subject, which is the ratio of the vertical length to the horizontal length; and a function of detecting a forgery of a person using a flat object based on the aspect ratios of the face of the verification subject taken at different angles.

[0032] (16) The recording medium according to (15) above, wherein the function of detecting a forgery calculates, for each of the frame images, the rate of change or the amount of change of the aspect ratio of the face of the verification subject with respect to a reference aspect ratio as a flat similarity, and detects a forgery based on the plurality of flat similarities calculated.

[0033] (17) The recording medium according to (16) above, wherein the function of detecting a forgery acquires, as the reference aspect ratio, the aspect ratio of the face of the person registered in advance or the aspect ratio of the face of the verification subject in a frame image taken first among the plurality of frame images.

[0034] (18) The recording medium according to (15) above, wherein the function of detecting a forgery extracts at least three feature points among a plurality of feature points in each frame image, which are arranged apart in the width direction of the face, calculates the ratio of the distance between the feature points on the right and left sides of the face among the extracted feature points as a three-dimensional similarity, and detects a forgery based on the plurality of three-dimensional similarities calculated for each of the frame images.

[0035] (19) The recording medium according to (16) or (17) above, wherein the forgery detection function extracts at least three feature points of a plurality of feature points in each frame image, which are arranged apart in a width direction of a face, calculates a ratio of a distance between the feature points on the right and left sides of the face as a stereoscopic similarity, and detects forgery based on the plurality of planar similarities and the plurality of stereoscopic similarities calculated for each frame image.

[0036] (20) The recording medium according to (19) above, wherein the forgery detection function, for each frame image, increments a planar score when the planar similarity is equal to or greater than a predetermined value, and increments a stereoscopic score when the stereoscopic similarity is equal to or greater than a predetermined value; detects forgery when the planar score is equal to or greater than a predetermined difference D from the stereoscopic score, and detects no forgery when the stereoscopic score is equal to or greater than the predetermined difference D from the planar score.

[0037] (21) The recording medium according to any one of (15) to (20) above, wherein the feature detection function detects the positions of feature points of a face of the subject for each frame image taken at different angles, and the forgery detection function, for each frame image, further calculates a difference in the positions of the feature points from other frame images taken consecutively, and determines that there is forgery using both a planar object and a stereoscopic object when the difference is equal to or greater than a threshold value TL.

[0038] [Effects of the Invention]

[0039] According to the present application, forgery using a planar object such as a photograph can be detected with high accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0040] [ Figure 1 ] is a configuration diagram of the forgery detection system of the present embodiment.

[0041] [ Figure 2 ] is a diagram showing an example of a login screen that represents login information.

[0042] [ Figure 3 ] is a diagram showing an example of an information code of the present embodiment.

[0043] [ Figure 4 ] is a flowchart showing an information login process.

[0044] [ Figure 5 ] is a flowchart showing a verification process.

[0045] [ Figure 6 ] is a diagram showing an example of a method of acquiring frame images.

[0046] [ Figure 7 ] is a flowchart showing the continuity determination process of step S205.

[0047] [ Figure 8 ] is a flowchart showing the plane determination process of step S207.

[0048] [ Figure 9 ] is a graph for explaining the aspect ratio when the verification target person is actually present and when the person is a photo present on a plane object.

[0049] [ Figure 10 ] is a flowchart showing the three-dimensional determination process of step S208. DETAILED DESCRIPTION

[0050] An embodiment of the counterfeit detection system of the present application will be described based on the drawings. Figure 1 is a block diagram showing the configuration of the counterfeit detection system 1. As shown in Figure 1 , the counterfeit detection system 1 of the present embodiment can be configured only by the information processing device 10. Also, although not shown, it can be configured such that a server that can communicate with the information processing device 10 is provided, and a part of the functions of the information processing device 10 is executed by the server. Hereinafter, a scene in which the information processing device 10 verifies a target person (hereinafter, referred to as a target person) that becomes a verification target will be described.

[0051] The information processing device 10 is a device such as a smartphone, a tablet, a notebook computer, a desktop computer, and the like, and as shown in Figure 1 , has a camera 11, a display 12, a storage 13, and a calculation unit 14. The camera 11 captures an image of the face of a target person or the like. The display 12 displays a verification result or the like. The storage 13 stores, in addition to a counterfeit detection program of the present embodiment, an information code including verification-use registration information that is registered in advance. The calculation unit 14 executes a program stored in the storage 13, and thereby has an information registration function of registering registration information for verifying a target person and a verification function of verifying a target person using the registration information. Further, the information processing device 10 can download a counterfeit detection program from a server, and can install and execute the downloaded program. Hereinafter, each function of the information processing device 10 will be described.

[0052] The information registration function of the information processing device 10 is to register registration information for verification. The information registration function, for example, as shown in Figure 2As shown, the image of the face of the subject is displayed on the screen of the display 12, and the subject is caused to take the image of the face, and the taken image of the face is acquired. Then, the information registration function extracts feature points such as the eyes, nose, mouth, and face contour of the subject from the acquired image of the face of the subject. Further, the extraction method of the feature points can be performed by a known method. Furthermore, the information registration function calculates the position coordinates of the extracted eyes, nose, and mouth, or the ratio of the vertical length of the face to the horizontal length of the face, i.e., the aspect ratio, as the registration information. Further, the vertical length of the face can be appropriately set, for example, to the distance from the tip of the chin to the center of the eyebrows, the distance from the height of the mouth to the height of the eyebrows, the distance from the tip of the chin to the height of the eyes, or the like. Also, the horizontal length of the face can be appropriately set, for example, to the distance from the right temple to the left temple. Also, the information registration function can detect the face contour, and detect the ratio of the longest distance in the horizontal direction (the arrangement direction of the eyes) to the longest distance in the vertical direction (the direction perpendicular to the arrangement direction of the eyes) of the detected face contour, as the ratio of the vertical length of the face to the horizontal length of the face, i.e., the aspect ratio. Further, in the present embodiment, the straight line passing through the two eyes is set as the horizontal straight line of the face, and the length of the line segment from the right temple to the left temple in the horizontal straight line is calculated as the horizontal length of the face. However, for example, the configuration can be such that the left-right direction of the image is set as the horizontal direction, or the configuration can be such that the horizontal direction is set as the horizontal direction by obtaining the horizontal direction from a gyro sensor or the like. Also, in the present embodiment, the straight line orthogonal to the set horizontal straight line of the face can be set as the vertical straight line of the face, and the length of the line segment from the tip of the chin to the center of the eyebrows, the line segment from the mouth to the eyebrows, or the line segment from the tip of the chin to the eyes in the vertical straight line is calculated as the vertical length of the face. Also, the longer the horizontal length and the vertical length of the face, the higher the precision, and therefore, when the vertical line segment or the horizontal line segment is set, it is preferable to set the line segment reaching the end of the face. Then, the information registration function generates the information code 2 including the registration information, and stores the generated information code 2 in the storage 13.

[0053] Here, Figure 3 is a diagram showing an example of the information code 2 of the present embodiment. As shown in Figure 3 the information code 2 of the present embodiment is a square two-dimensional code having an information code area 21, a guide code area 23, and a position detection pattern 25, the information code area 21 is composed of information cells 22 having two or more colors as a unit of display information, and the guide code area 23 has guide cells 24 having the same colors as the information cells 22 and the same number of colors as the information cells 22 in order to identify the colors of the information cells 22.

[0054] The information code region 21 is composed of information units 22 of two or more colors arranged in a lattice shape. The color of the information unit 22 is not particularly limited as long as it is two or more colors, and can be, for example, two colors of white and black, or three or more colors other than white and black. In the present embodiment, the information code 2 has a plurality of guide units 24 in the guide code region 23 in order to improve the recognition accuracy of the color of the information unit 22. The color of the guide unit 24 is the same as the color of the information unit 22, and by comparing the color of the guide unit 24 with the color of the information unit 22, the color of the information unit 22 can be recognized with high accuracy. In particular, in a color QR code, the more the number of colors, the closer the hues of the respective colors, and thus, sometimes, it is difficult to accurately recognize the color of the information unit 22 due to the lighting environment, the printing environment of the information code 2, the display environment of the information code 2 on a display or the like, the fading of the printed information code 2 over time, or the like. Even in such a case, by referring to the guide unit 24, the color of the information unit 22 can be recognized, and the information of the information unit 22 can be accurately grasped. For example, in a case where the color information of the information unit 22 is slightly purple, and it is difficult to recognize whether it is similar to red or similar to blue, if the color information is closer to red than the color information of the purple guide unit 24, it can be determined that the color of the information unit 22 is red. Further, the color information can be quantified as RGB values or CMY values.

[0055] The information code region 21 can record specific information composed of character information and / or binary information of a fixed capacity by changing the display pattern of the information unit 22 as an information unit. The content of the specific information recorded in the information code region 21 is not particularly limited, but in the present embodiment, personal information including biometric authentication information for biometric authentication can be memorized. As the personal information, in addition to the biometric authentication information, information such as a personal number, a passport number, an account number, a license number, a name, a social security number, a date of birth, a place of birth, a maiden name, and the like can be included. Further, as the biometric authentication information, face image information, fingerprint information, iris information, palm shape, retina, blood vessels, voice, ear shape, and the like can be included. In the present embodiment, an example in which, as the biometric authentication information, information based on the image of a face, particularly, the image, coordinates, or aspect ratio of a feature point such as an eye, a nose, and a mouth is memorized is described.

[0056] Further, in the information code 2 of the present embodiment, the information code area 21 in which the specific information is recorded is composed of information units 22 of three or more colors, and thus, as compared with the information code composed of only two colors of white and black, the amount of information that can be recorded is larger. However, the data capacity that can be memorized by the information code 2 is smaller than that of other recording media, and thus, in the case where the entire face image is recorded at a resolution at which face authentication can be performed, even if the information code area 21 is composed of information units 22 of four colors, there is a risk that the memory capacity will be insufficient. Therefore, in the information code 2 of the present embodiment, only a part of the face of the user, such as the eyes, nose, or mouth, which becomes a feature point in face authentication, is memorized, and not the entire face of the user, as a face image for face authentication, and thus, the face image for face authentication can be recorded in the information code 2.

[0057] The authentication function of the information processing apparatus 10 is to determine whether the subject who desires to receive authentication using the information processing apparatus 10 is the person who has logged in the login information, and thus, to authenticate the subject. In particular, in the present embodiment, the authentication function is characterized in that, in order to prevent impersonation using a flat object such as a photograph, it is determined whether the image of the subject photographed by the camera 11 is a flat object such as a photograph or a three-dimensional object, and authentication is performed. Further, the authentication method of the present embodiment will be described later.

[0058] Next, the information login processing of the present embodiment will be described. Figure 4 is a flowchart showing the information login processing of the present embodiment. Further, in the following description, the information login processing shown in Figure 4 is executed by the information login function of the information processing apparatus 10. Further, in the following description, the information login processing shown in Figure 4 is executed by the information login function of the information processing apparatus 10. Further, in the following description, the information login processing shown in

[0059] In step S101, the face image of the subject to be logged in who is photographed by the camera 11 is acquired by the information login function. For example, the information login function can cause the subject to be logged in to photograph the face image by displaying a screen that prompts the subject to be logged in to photograph the face image on the display 12, as shown in Figure 2 In step S101, the face image of the subject to be logged in who is photographed by the camera 11 is acquired by the information login function. For example, the information login function can cause the subject to be logged in to photograph the face image by displaying a screen that prompts the subject to be logged in to photograph the face image on the display 12, as shown in

[0060] In step S102, the information registration function detects feature points of the applicant's facial image. Specifically, the information registration function uses a known method to detect feature points such as the applicant's eyes, nose, mouth, and facial contours from the captured image obtained in step S101. In the subsequent step S103, the information registration function standardizes the feature points detected in step S102. For example, the information registration function can be configured as follows: using a known method, the position of the feature points is changed so that the eyes, nose, or mouth are located at a predetermined reference position. Furthermore, in the subsequent step S104, the information registration function generates the applicant's registration information based on the standardized feature points from step S103. For example, the information registration function can generate standardized coordinates of the eyes, nose, and mouth as registration information. Additionally, the information registration function can generate the aspect ratio (horizontal to vertical length) of the applicant's face as registration information.

[0061] In step S105, the information login function generates a verification information code 2 using the login information generated in step S104. For example, the information login function can use a known method to generate information code 2 containing the login information used in step S104. Subsequently, in the following step S106, the information login function stores the information code 2 generated in step S105 in the storage unit 13.

[0062] Next, the verification process of this embodiment will be described. The verification process of this embodiment is as follows: it determines whether the person to be verified (hereinafter referred to as the verification target) is a login target (hereinafter referred to as the login user) who has logged in during the information login process, and then performs verification. In particular, in the verification process of this embodiment, the objective is to accurately verify the verification target (accurately detect impersonation) even if a malicious verification target uses a photo showing the login user's face, a mask, or a photo distorted according to the shape of the face to impersonate the user. Figure 5 This is a flowchart illustrating the verification process of this embodiment. Furthermore, Figure 5 The verification process shown is performed through the verification function of the information processing device 10.

[0063] In step S201, a personal verification process is performed through the verification function. This personal verification process is used to determine whether the user who has previously logged in using login information is the same person as the user to be verified. The verification function first obtains the personal information of the user. Figure 4The illustrated information registration processing registers the login information. In the present embodiment, the storage section 13 of the information processing apparatus 10 stores information of the feature points of the face of the login person (position information of the eyes, nose, mouth, and face contour, or the aspect ratio of the vertical length and the horizontal length of the face) as the login information in the form of the information code 2, and the authentication function acquires the information code 2 including the login information from the storage section 13 and extracts the login information from the acquired information code 2. Also, the authentication function acquires the face image of the authentication target person. For example, the authentication function can cause the camera 11 to capture the face of the authentication target person from the front by displaying a screen on which a comment such as "Please capture the face from the front." is superimposed on the display 12, and acquire the face image of the authentication target person captured from the front. Then, the authentication function detects the feature points from the acquired face image of the authentication target person. Then, the authentication function compares the feature points of the face of the login person registered in the form of the login information with the feature points of the face of the authentication target person and calculates the similarity. For example, the authentication function can calculate the distances from the nose to the eyes, the distances from the nose to the mouth, the distances between the left and right of the face contour, the distances from the forehead to the chin, and the like in the face image of the login person and the face image of the authentication target person, compare the distances, and thereby calculate the comparison result as the similarity.

[0064] In step S202, the authentication function determines whether the authentication target person is the login person himself / herself on the basis of the similarity calculated in step S201. For example, the authentication function can determine that the authentication target person is the login person himself / herself when the distances from the nose to the eyes, the distances from the nose to the mouth, the distances between the left and right of the face contour, and / or the distances from the forehead to the chin of the login person and the authentication target person calculated in step S201 are equal to or smaller than predetermined values. Then, the authentication function proceeds to step S203 when it is determined that the authentication target person is the login person himself / herself, and proceeds to step S212 when it is determined that the authentication target person is not the login person himself / herself, and displays a message indicating that the authentication has failed.

[0065] Furthermore, even when the registrant and the verification target are the same person, the positions of the feature points may shift depending on factors such as the orientation of the face. Therefore, in steps S201 and S202, the verification function can be configured to determine whether the registrant and the verification target are the same person in the following way: feature points tend to be distributed in high density on parts such as the eyes, nose, mouth, and facial contours. Therefore, it can be configured as follows: when the distribution density of the feature points of the registrant registered in the registration information is greater than or equal to a predetermined value d within a radius r1, and the distribution density of the feature points of the verification target is greater than or equal to a predetermined value d within a radius r2, the feature points specifically corresponding to the same part (eyes, nose, mouth, facial contours, etc.) are considered to be the same person when the feature points of each part partially overlap. Furthermore, the verification function can be configured as follows: by using known methods, the position of feature points extracted from the facial image of the verification target is corrected so that it is consistent with the orientation or size of the face of the registrant obtained as login information, and then it is determined whether they are the same person.

[0066] In step S203, in order to determine whether someone is an imposter by using multiple facial images of the subject taken from different angles, the verification function instructs the subject to move their head. For example, the verification function can display a screen on display 12 with annotations such as "Please move your head left or right." Figure 6 As shown in (A) to (C), the camera 11 captures images of the subject's face from different angles. Furthermore, hereinafter, each of the multiple facial images captured from different angles will be referred to as a frame image.

[0067] In step S204, a reference aspect ratio is obtained through a verification function. This reference aspect ratio is used in the planar determination process in step S207 below. In this embodiment, starting from... Figure 5 The verification process shown initially captures a facial image (frame image) of the verification subject. The ratio of the vertical length to the horizontal length of the verification subject's face is calculated as the aspect ratio, and this calculated aspect ratio is set as the reference aspect ratio. Alternatively, in this embodiment, the configuration may be as follows: information about the registrant's feature points (positional information of eyes, nose, mouth, facial contours, etc., or the aspect ratio of the vertical length to the horizontal length of the face) is stored as registration information in the storage unit 13, and the aspect ratio of the registrant's face stored as registration information is acquired as the reference aspect ratio. Furthermore, the configuration may be as follows: the aspect ratio of the verification subject's face detected from frame images is acquired as the reference aspect ratio; these frame images are frames captured at predetermined points, such as when the verification subject is facing forward, among multiple frame images.

[0068] In steps S205 to S209, for each frame image, the following continuity determination processing, plane determination processing, and stereoscopic determination processing are performed. Hereinafter, the frame image that becomes the processing target will be referred to as an object frame image.

[0069] In step S205, the continuity determination processing is performed by the authentication function. The continuity determination processing refers to processing to determine whether or not the authentication target is using the authentication target's actual face (a stereoscopic object) and a plane object such as a photograph or a mask that presents the face of the registered person, both of which are used to perform impersonation. That is, the following case is assumed: a malicious authentication target who knows that authentication cannot be performed in a case where only a plane object such as a photograph or a mask is used performs impersonation by causing the camera 11 to capture a plane object such as a photograph or a mask and the authentication target's actual face. The purpose of the continuity determination processing is to accurately detect impersonation of the authentication target even in such a case. Here, Figure 7 is a flowchart that shows the continuity determination processing of step S205. Hereinafter, the continuity determination processing of step S205 will be described based on Figure 7 , the continuity determination processing of step S205 will be described.

[0070] First, in step S301, the authentication function detects feature points from the frame image obtained by capturing the face of the authentication target. For example, in the present embodiment, the authentication function extracts feature points from the object frame image using information on feature points included in the registration information. The method of extracting feature points is not particularly limited and a publicly known method can be used.

[0071] In step S302, the authentication function performs comparison of the positions of the feature points detected from the object frame image with the positions of the feature points detected from the frame image obtained immediately before the object frame image (hereinafter, the last frame image). Then, in subsequent step S303, the authentication function determines whether or not the distance of the corresponding feature points is equal to or greater than a threshold value TL using the object frame image and the last frame image compared in step S302. For example, the authentication function specifies, for each element, a feature point for the same element (for example, an eye, a nose, a mouth, a face contour, or the like) as the corresponding feature point between consecutive object frame images and the last frame image, and calculates the distance (difference in position) of the corresponding feature points for each element. Then, the authentication function determines whether or not the total value or the average value of the distances of the corresponding feature points is equal to or greater than a predetermined threshold value TL. Here, the threshold value TL is not particularly limited, but a distance that can determine that the authentication target is using the authentication target's actual face and a photograph image of the registered person both of which are used to perform impersonation can be set as the threshold value TL based on a simulation experiment or the like.

[0072] When the distance between the corresponding feature points between the frame images in succession in step S303 does not reach the threshold TL, the process proceeds to step S304. In step S304, the verification function determines that the frame images taken in succession have continuity and are not being counterfeited using both the actual face of the verification target and the photograph image of the registered person. Also, when the distance between the corresponding feature points is determined to be the threshold TL or more in step S303, the process proceeds to step S305, and the verification function determines that the frame images taken in succession do not have continuity and are being counterfeited using both the actual face of the verification target and the photograph image of the registered person. Then, the process returns to step S301. Figure 5 , the process proceeds to step S206.

[0073] In step S206, the verification function determines whether or not continuity was determined in the continuity determination process of step S205. When continuity was determined, the process proceeds to step S207. On the other hand, when no continuity was determined, the process proceeds to step S212, and the verification function displays a verification error because the verification target and the registered person are not the same person. Further, when the process proceeds to step S212, the process returns to step S201. Figure 7 In the continuity determination process shown in FIG. 6, the configuration can be such that continuity is determined for all frame images, or the configuration can be such that continuity is determined only for frame images from the time when the verification target started nodding, for example, for 1 second, and further, the configuration can be such that continuity is determined for randomly selected frame images. Also, the configuration can be such that when it is determined that there is only one frame image that does not have continuity, it is determined that counterfeiting exists, or the configuration can be such that when it is determined that a predetermined number or more of frame images do not have continuity, it is determined that counterfeiting exists.

[0074] In the subsequent step S207, the verification function performs a plane determination process. The plane determination process is a process for determining whether or not the verification target is not being counterfeited using a plane object such as a photograph. Here, Figure 8 is a flowchart showing the plane determination process of step S207. Hereinafter, the plane determination process of step S207 will be described based on Figure 8 First, in step S401, the verification function calculates the aspect ratio of the face of the verification target based on the positions of the feature points of the frame image, such as the face contour, or the left and right temples, and the chin and the brow center. Also, in step S402, the verification function calculates the aspect ratio R2 of the face of the registered person based on the positions of the feature points of the photograph image, such as the face contour, or the left and right temples, and the chin and the brow center. Then, in step S403, the verification function determines whether or not the aspect ratio R2 of the face of the registered person is within a predetermined range of the aspect ratio Rl of the face of the verification target. When the aspect ratio R2 of the face of the registered person is within the predetermined range of the aspect ratio Rl of the face of the verification target, the process proceeds to step S404. On the other hand, when the aspect ratio R2 of the face of the registered person is not within the predetermined range of the aspect ratio Rl of the face of the verification target, the process proceeds to step S412, and the verification function displays a verification error because the verification target and the registered person are not the same person. Further, when the process proceeds to step S412, the process returns to step S201.

[0075] First, in step S401, the verification function calculates the aspect ratio of the face of the verification target based on the positions of the feature points of the frame image, such as the face contour, or the left and right temples, and the chin and the brow center. Also, in step S402, the verification function calculates the aspect ratio R2 of the face of the registered person based on the positions of the feature points of the photograph image, such as the face contour, or the left and right temples, and the chin and the brow center. Then, in step S403, the verification function determines whether or not the aspect ratio R2 of the face of the registered person is within a predetermined range of the aspect ratio Rl of the face of the verification target. When the aspect ratio R2 of the face of the registered person is within the predetermined range of the aspect ratio Rl of the face of the verification target, the process proceeds to step S404. On the other hand, when the aspect ratio R2 of the face of the registered person is not within the predetermined range of the aspect ratio Rl of the face of the verification target, the process proceeds to step S412, and the verification function displays a verification error because the verification target and the registered person are not the same person. Further, when the process proceeds to step S412, the process returns to step S201. Figure 5The aspect ratio R2 acquired in the step S204 shown is used to calculate the planar similarity. Specifically, the verification function calculates the rate of change of the aspect ratio Rl of the face of the verification target person with respect to the reference aspect ratio R2 as the planar similarity. For example, the verification function can calculate the absolute value of (the reference aspect ratio R2 - the aspect ratio Rl of the face of the verification target person) / the reference aspect ratio R2 | (R2 - Rl) / R2 | as the planar similarity.

[0076] Here, Figure 9 is a diagram for explaining the aspect ratio when the verification target person is actually present and when the face image is present on a planar object such as a photograph. (A) is a frame image captured in a state in which the verification target person is facing straight ahead, (B) is a frame image obtained by tilting a photograph in a diagonal direction and capturing the photograph, and (C) is a frame image captured in a state in which the verification target person is facing in a diagonal direction. Also, in (A) to (C) of Figure 9 , the vertical length and the horizontal length of the face of each frame image are shown by solid arrows. Also, in (B) and (C) of Figure 9 , the horizontal length H0 of the face in (A) of Figure 9 is shown by a broken line. Further, in (B) and (C) of Figure 9 , the horizontal length of the face in (A) is shown by a broken line. Figure 9 , and the horizontal length of the face in (B) and (C) is shown by a solid line. Figure 9 , the difference between the horizontal length of the face in (A) and the horizontal length of the face in (B) and (C) is shown as Wl and W2.

[0077] In the example shown in Figure 9 , the distance between the verification target person and the photograph and the camera 11 is adjusted so that the vertical length V0 to V2 of the face in (A) to (C) becomes the same length. Here, as shown in (B) of Figure 9 , in the case in which the photograph is tilted in a diagonal direction (in the case in which it is rotated in the roll direction), the horizontal length Hl of the face of the subject person present on the photograph becomes shorter in accordance with the tilt angle of the photograph. On the other hand, as shown in (C) of Figure 9 , in the case in which the verification target person is actually present, even if the verification target person turns the face in a diagonal direction, the horizontal length H2 of the face of the verification target person does not greatly change compared with the horizontal length H0 of the face in (A) in which the verification target person is facing straight ahead. Therefore, as shown in (C) of Figure 9 , in the case in which the verification target person is actually present, the difference W2 between the horizontal length Hl of the face in the case in which the verification target person is facing in a diagonal direction and the length H0 of the face in the case in which the verification target person is facing straight ahead is small, and as shown in (B) of Figure 9the face image presented on the planar object, the difference Wl between the horizontal length Hl of the face of the verification target person when the planar object is tilted toward the oblique direction and the length H0 of the face when the verification target person is facing straight ahead is larger. As a result, when the actual face of the verification target person is tilted toward the oblique direction, the aspect ratio of the face of the verification target person becomes a value close to the reference aspect ratio, and the rate of change from the reference aspect ratio is small, so the planar similarity tends to decrease. On the other hand, regarding the aspect ratio of the face of the verification target person when the verification target person is facing the oblique direction, the more the planar object is tilted toward the oblique direction, the more the aspect ratio deviates from the reference aspect ratio, and the rate of change from the reference aspect ratio is large, so the planar similarity tends to increase.

[0078] Therefore, when the verification target person whose actual face is to be verified rotates the actual face by indicating a left-right turning based on the turning, the number of frame images in which the planar similarity is equal to or greater than the predetermined value TS decreases. On the other hand, when a third party who attempts to perform impersonation rotates the face image presented on the planar object to perform a left-right turning, the number of frame images in which the planar similarity is equal to or greater than the predetermined value TS increases. Therefore, in the present embodiment, the number of frame images in which the planar similarity is equal to or greater than the predetermined value TS is counted as the planar score Sl, and based on the planar score Sl, impersonation using a photograph or the like as a planar object is detected.

[0079] That is, in step S403, the planar similarity calculated in step S402 is judged by the verification function as to whether or not it is equal to or greater than the predetermined value TS. When the planar similarity is equal to or greater than the predetermined value TS, step S404 is entered, and the planar score Sl is incremented by the verification function. On the other hand, when the planar similarity does not reach the predetermined value TS, step S404 is not entered, and the planar determination processing shown in FIG. 7 is ended. Figure 8 In the present embodiment, the planar similarity is calculated for each frame image, and the planar score Sl is incremented each time the planar similarity is judged to be equal to or greater than the predetermined value TS in each frame image. Therefore, the planar score Sl is an index indicating the likelihood that the face of the verification target person is a face image presented on a planar object, and it can be judged that the higher the planar score Sl, the higher the likelihood that the face of the verification target person is a face image presented on a planar object.

[0080] In step S208, the stereoscopic determination processing is performed by the verification function. The stereoscopic determination processing is processing for judging whether or not the frame image is an image of a stereoscopic object such as a person actually present. Figure 10 is a flowchart of the stereoscopic determination processing of step S208. Hereinafter, the stereoscopic determination processing of step S208 will be described based on the flowchart of FIG. 8. Figure 10 The stereoscopic determination processing of step S208 will be described.

[0081] like Figure 10 As shown, firstly, in step S501, the verification function extracts at least three feature points from multiple feature points of the target frame image that are separated along the width direction of the face, and calculates the distance between these feature points (the distance between the right and left sides of the face). These feature points are preferably located in positions that are as far apart as possible; examples include feature points corresponding to the nose and left and right cheeks, the nose and left and right temples, the nose and left and right contours, and the nose and left and right ears. For example, the verification function can be configured to calculate the distance from the left cheek to the nose of the verification target as L1 and the distance from the right cheek to the nose as L2; ​​or it can be configured to calculate the distance from the left temple to the nose of the verification target as L1 and the distance from the right temple to the nose as L2; ​​or it can be configured to calculate the distance from the left ear to the nose as L1 and the distance from the right ear to the nose as L2. Furthermore, in this embodiment, the verification function calculates the distance L1 from the left cheek to the nose and the distance L2 from the right cheek to the nose of the verification target.

[0082] Furthermore, in this embodiment, it is preferable to configure the calculation of distances L1 and L2 to include the concavity and convexity of the face. For example, the verification function can be configured such that the calculated distance (height of the concavity and convexity) in the direction of the concavity and convexity of the face is greater than the actual distance, thereby taking the concavity and convexity of the face into account to determine whether it is a three-dimensional object. Moreover, the height of the concavity and convexity of the face can be calculated using known methods, and can be configured to be obtained from the changes in feature points between consecutive frame images, or the information processing device 10 can be configured to have a built-in sensor such as LiDAR, and height information can also be obtained through LiDAR or the like.

[0083] In step S502, the verification function compares the distances L1 and L2 of the feature points calculated in step S501, and calculates the comparison result as the stereo similarity. In this embodiment, the verification function calculates the ratio (L1 / L2) of the distance L1 from the left cheek to the nose and the distance L2 from the right cheek to the nose of the verification subject as the stereo similarity.

[0084] Here, in the case of the face image appearing on a planar object such as a photograph, even when the orientation of the planar object is changed, the distance Ll from the left cheek to the nose and the distance L2 from the right cheek to the nose of the verification target person hardly change, and thus the frequency of the stereoscopic similarity being similar to the already calculated stereoscopic similarity based on another frame image increases. In contrast, in the case of the face of the verification target person being a real face (a stereoscopic object), when the orientation of the face is changed, the ratio (Ll / L2) of the distance Ll from the left cheek to the nose to the distance L2 from the right cheek to the nose of the verification target person tends to greatly change, and thus the frequency of the stereoscopic similarity being similar to the already calculated stereoscopic similarity based on another frame image decreases. Therefore, in the present embodiment, the number of frame images in which the stereoscopic similarity is not similar to the already calculated stereoscopic similarity is counted as the stereoscopic score S2, and based on the stereoscopic score S2, it is determined whether the verification target person is a real person (a stereoscopic object) or not.

[0085] That is, in step S503, the verification function determines whether the stereoscopic similarity calculated in step S502 and the already calculated stereoscopic similarity based on another frame image are similar or not by determining whether the difference between the stereoscopic similarity and the already calculated stereoscopic similarity is within a fixed range or not. In the case where the difference between the stereoscopic similarity and the already calculated stereoscopic similarity is within a range exceeding the fixed range, and the stereoscopic similarity and the already calculated stereoscopic similarity are not similar, the process proceeds to step S504, and the verification function increments the count of the stereoscopic score S2. Further, in subsequent step S505, the verification function decrements the count of the planar score Sl. On the other hand, in the case where the difference between the stereoscopic similarity and the already calculated stereoscopic similarity is within the fixed range, and the stereoscopic similarity and the already calculated stereoscopic similarity are similar, the process ends Figure 10 The stereoscopic determination process shown in FIG. 7 proceeds to step S209 shown in FIG. 8. Figure 5 In the present embodiment, the stereoscopic similarity is calculated for each frame image, and the stereoscopic score S2 is incremented each time the stereoscopic similarity and the already calculated stereoscopic similarity are similar in each frame image. Therefore, the stereoscopic score S2 is an index indicating the likelihood that the verification target person is a real person (a stereoscopic object), and it can be said that the higher the stereoscopic score S2 is, the higher the likelihood that the verification target person is a real person. Further, the size of the fixed range that is the criterion for whether the stereoscopic similarity and the already calculated stereoscopic similarity are similar or not is not particularly limited, and can be appropriately set by simulation experiments or the like.

[0086] In step S209, the verification function determines whether the absolute value of the difference between the planar score S1 and the stereo score S2 is equal to or greater than a predetermined difference D. Here, the difference D is preferably set to a threshold value such that a large difference exists between the planar score S1 and the stereo score S2, and it is possible to determine that the verification target is a real person or a planar object such as a photograph. An appropriate difference D can be set through simulation experiments or the like. In the case where the absolute value of the difference between the planar score S1 and the stereo score S2 does not reach the difference D, it is determined that the verification is insufficient, and the process returns to step S205 in order to calculate the planar score S1 and the stereo score S2 based on another frame image. Specifically, the process of steps S205 to S208 is performed using another frame image that is acquired after the target frame image. In contrast, in the case where the absolute value of the difference between the planar score S1 and the stereo score S2 is equal to or greater than the difference D, the process proceeds to step S210.

[0087] In step S210, the verification function determines whether the stereo score S2 is greater than the planar score S1. In the case where the stereo score S2 is greater than the planar score S1 by a large difference (a difference equal to or greater than the difference D), the process proceeds to step S211, and it is determined that the verification target is the actual face of the verification target person. Information indicating that the verification is completed (verification success) is output to the display 12. On the other hand, in the case where the planar score S1 is greater than the stereo score S2 by a large difference (a difference equal to or greater than the difference D), the process proceeds to step S212, and information indicating that the verification is not possible (verification error) is output to the display 12.

[0088] As described above, in the present embodiment, for each of the plurality of frame images captured by the camera 11, the ratio of the longitudinal length to the lateral length of the face of the verification target person is detected as the aspect ratio, the rate of change of the aspect ratio of the verification target person with respect to the reference aspect ratio is calculated as the plane similarity for each frame image, and based on the plane similarity, it is possible to determine whether the target person is a person appearing on a plane object such as a photograph, and thus it is possible to detect forgery using a plane object such as a photograph with high accuracy. In particular, in the present embodiment, the plane object determination processing for determining a plane object is performed in conjunction with the three-dimensional object determination processing for determining a three-dimensional object, and thus it is possible to perform detection with even higher accuracy. That is, in the present embodiment, even when it is determined in the plane object determination processing that it is not a plane object, in the case where it is determined in the three-dimensional object determination processing that it is not a three-dimensional object, it is regarded as a verification error, and thus it is possible to perform verification using a plane object such as a photograph with even higher accuracy. Also, in the present embodiment, the plane determination processing is performed for each frame image, the number of frame images determined to be plane objects is scored as the plane score S1, and the number of frame images determined to be three-dimensional objects is scored as the three-dimensional score S2. Then, in the case where the difference between the plane score S1 and the three-dimensional score S2 is a large difference, that is, the difference amount D or more, when the plane score S1 is higher than the three-dimensional score S2, it is determined to be a plane object, and when the three-dimensional score S2 is higher than the plane score S1, it is determined to be a three-dimensional object, and thus it is possible to comprehensively determine whether the verification target person is a plane object using a plurality of frame images, and thus it is possible to improve the determination accuracy of forgery. Also, in the present embodiment, the aspect ratio, which has a smaller memory capacity, is used, and thus even in the case where the memory capacity of the login information is limited, it is possible to accurately perform forgery determination.

[0089] The above describes preferred embodiments of the present application, but the technical scope of the present application is not limited to the above-described embodiments. Various changes and modifications can be added to the above-described embodiments, and the forms added with such changes or modifications are also included in the technical scope of the present application.

[0090] For example, in the above-described embodiments, a configuration in which the login information is memorized in a form included in the information code 2 is exemplified, but the configuration is not limited to this, and a configuration in which the login information is directly memorized can be provided.

[0091] Further, in the above-described embodiment, the absolute value of the change rate of the aspect ratio Rl of the face of the verification target person with respect to the reference aspect ratio R2, |(R2-Rl) / R2|, is exemplified as the configuration of the planar similarity, but the configuration is not limited to this, and for example, the absolute value of the change amount (difference) of the aspect ratio Rl of the face of the verification target person with respect to the reference aspect ratio R2, |Rl-R2|, can be set as the configuration of the planar similarity. In this case, the configuration can be set such that the planar score S l is incremented when the planar similarity is equal to or greater than a predetermined value TS', and the planar score S l is not incremented when the planar similarity is less than the predetermined value TS'.

[0092] Further, in the above-described embodiment, the ratio (Ll / L2) of the distance Ll from the left cheek to the nose of the verification target person to the distance L2 from the right cheek to the nose is exemplified as the configuration of the stereoscopic similarity, but the configuration is not limited to this, and for example, the difference (Ll-L2) of the distance Ll from the left cheek to the nose of the verification target person to the distance L2 from the right cheek to the nose can be set as the configuration of the stereoscopic similarity. In this case, the configuration can also be set such that the stereoscopic score S2 is incremented when the stereoscopic similarity is similar to the existing stereoscopic similarity.

[0093] Further, in the above-described embodiment, the ratio (Ll / L2) of the distance Ll from the left cheek to the nose of the verification target person to the distance L2 from the right cheek to the nose is exemplified as the configuration of the stereoscopic similarity in the stereoscopic determination processing, but for example, the configuration can be set such that the up-and-down direction head movement is also considered to determine whether it is a stereoscopic object, and for this, further, the chin and the forehead are detected as the feature points, the distance L3 from the chin to the nose and the distance L4 from the nose to the forehead are used, and the ratio (L3 / L4) thereof is also calculated as the stereoscopic similarity. In this case, the configuration can be set such that the stereoscopic similarity indicated by Ll / L2 and the existing stereoscopic similarity are similar, and / or the stereoscopic similarity indicated by L3 / L4 and the existing stereoscopic similarity are similar, and it is determined to be a planar object.

[0094] [SYMBOL DESCRIPTION]

[0095] 1: counterfeit detection system

[0096] 10: information processing apparatus

[0097] 11: camera

[0098] 12: display

[0099] 13: storage

[0100] 14: arithmetic unit

[0101] 2: information code

[0102] 21: information code area

[0103] 22: information unit

[0104] 23: guide code area

[0105] 24: guide unit

[0106] 25: positioning pattern

Claims

1. A forgery detection system that detects whether a verification target person is a real person or a person appearing on a flat object, and has: image capturing means that captures a plurality of frames of a face of the verification target person at different angles; feature detection means that detects an aspect ratio of a face of the verification target person from each of the plurality of frames captured by the image capturing means; and forgery detection means that detects forgery of a person using a flat object based on the aspect ratio of the face of the verification target person captured at different angles; the forgery detection means calculates, for each of the frames, a rate of change or an amount of change of the aspect ratio of the face of the verification target person with respect to a reference aspect ratio as a flat similarity, the reference aspect ratio being more different from the aspect ratio of the face of the verification target person, the higher the calculated flat similarities, and determines that there is forgery.

2. The forgery detection system according to claim 1, wherein the forgery detection means extracts at least three feature points of a plurality of feature points in each frame, the three feature points being arranged apart in a width direction of the face, calculates a ratio of a distance between a right feature point and a left feature point of the face among the extracted feature points as a stereo similarity, and detects forgery based on the stereo similarities calculated for each of the frames.

3. The forgery detection system according to claim 2, wherein the forgery detection means, for each of the frames, increments a flat score when the flat similarity is equal to or greater than a predetermined value, and increments a stereo score when the stereo similarity is equal to or greater than a predetermined value, and determines that there is forgery when the flat score is equal to or greater than a predetermined difference from the stereo score, and determines that there is no forgery when the stereo score is equal to or greater than the predetermined difference from the flat score.

4. The forgery detection system according to claim 1, wherein the feature detection means detects positions of feature points of a face of the verification target person for each of the frames captured at different angles, and the forgery detection means, for each of the frames, further increments a flat score when the flat similarity is equal to or greater than a predetermined value, and increments a stereo score when the stereo similarity is equal to or greater than a predetermined value, and determines that there is forgery using both a flat object and a stereo object when a difference between the positions of the feature points of the other frames captured consecutively is equal to or greater than a threshold value.

5. A computer program product that detects forgery of a verification target person being a real person or a person appearing on a flat object, and causes a computer to execute: an acquisition function that acquires a plurality of frames of the verification target person captured at different angles; a feature detection function that detects an aspect ratio of a face of the verification target person from each of the plurality of frames; and a forgery detection function that detects forgery of a person using a flat object based on the aspect ratio of the face of the verification target person captured at different angles. ​ ​ ​ ​ ​ ​ ​ ​ ​ The above forgery detection function calculates, for each of the above frame images, a rate of change or an amount of change of the aspect ratio of the face of the above verification target person with respect to the reference aspect ratio as a plane similarity, and the higher the calculated plurality of the above plane similarities, the more the above reference aspect ratio and the aspect ratio of the face of the above verification target person diverge, and it is determined that there is forgery.

6. The computer program product according to claim 5, wherein the above feature detection function detects, for each frame image taken at a different angle, a position of a feature point of the face of the target person, The above forgery detection function further performs, for each of the above frame images, a process of calculating a difference in the position of the feature point from other frame images taken consecutively, and in the case where the above difference is equal to or greater than a threshold value, it is determined that there is forgery using both a planar object and a three-dimensional object.

Images