A method for detecting fraudulent user authentication, a server that performs user authentication, and a program executed on the server that performs user authentication.

Abstract

This invention provides a method for detecting fraudulent identity verification using a typical mobile communication terminal, a server for performing identity verification, and a program executed on the authentication server, all of which do not impose a burden on the user. [Solution] A fraud detection method for which a mobile communication terminal 10 is used to take a picture of the person's face or an image of their identification document and transmit it to a server 30, and the server 30 performs authentication of the person, comprising: a camera information determination step of acquiring and determining information from the camera 11 of the mobile communication terminal 10; a touch panel output determination step of acquiring and determining the time-series change in the contact area of ​​the touch panel 12 when the touch panel is clicked; an acceleration sensor output determination step of acquiring and determining the output of the acceleration sensor 13 when the face image is taken; and a fraud detection step of determining whether the authentication of the person is fraudulent based on the determination results of the camera information determination step, the touch panel output determination step, and the acceleration sensor output determination step.

Description

Technical Field

[0001] The present invention relates to a method for determining fraud in personal authentication via the Internet, a server for performing personal authentication, and a program executed on the server for performing personal authentication.

[0002] Regarding methods for determining fraud in personal authentication, many patents have been filed in recent years. For example, Patent Document 1 (Japanese Patent Application Laid-Open No. 2023-545565) discloses an image detection method capable of detecting that a face image is forged even if it is a forged face image using sophisticated face processing technology. The face image detection method described in Patent Document 1 is an image detection method executed by a computer device, and includes steps of acquiring a target face image, and based on the target face image, obtaining a frequency domain image of the target face image and spatial domain features of the target face image, where the frequency domain image is an image obtained by performing frequency domain conversion on the target face image, and the spatial domain features are features obtained by performing feature extraction on the target face image; a step of performing feature extraction based on the frequency domain image to obtain frequency domain features of the target face image; a step of fusing the spatial domain features and the frequency domain features to obtain fusion features of the target face image; and a step of obtaining a detection result of the target face image based on the fusion features of the target face image, where the detection result is used to indicate whether the target face image is a forged face image.

[0003] Patent Document 2 (Re-Published Japanese Patent Application No. 2019 / 163066) discloses a forgery detection device capable of performing forgery detection without being affected by color dependency during face authentication. The impersonation detection device described in Patent Document 2 includes: a face image acquisition unit that acquires a first image frame from an imaging device that includes the face of the subject when light is emitted by a light-emitting device attached to the imaging device, and a second image frame that includes the face of the subject when the light-emitting device is turned off; a face information extraction unit that extracts information identifying the part of the subject's face from the first image frame as first face information and extracts information identifying the part of the subject's face from the second image frame as second face information; a feature quantity calculation unit that uses the first face information to determine the brightness value of the face part in the first image frame and uses the second face information to determine the brightness value of the face part in the second image frame, and calculates a feature quantity that reflects the three-dimensional shape of the subject's face and does not depend on the color of the surface of the subject's face based on the obtained brightness values; and an impersonation determination unit that determines the authenticity of the subject captured by the imaging device based on the feature quantity.

[0004] Patent Document 3 (Chinese Patent Publication No. 111274928) discloses a biodetection method that can effectively counter biodetection attack means such as video injection, improve the effectiveness of biodetection, and further improve the accuracy and safety of identity verification. The biological detection method described in Patent Document 3 involves: upon receiving a biological detection request, generating a detection interface which includes a portrait region and a ray emission region, the positions of the portrait region and the ray emission region changing randomly over time; projecting a ray onto the target to be detected via the ray emission region and generating reflected light on the surface of the target; collecting image features formed on the surface of the target of detection based on the reflected light and tracking the line of sight generated on the surface of the target; determining that the target is a living organism if, based on the image features, the change in reflected light matches the change in the position of the ray emission region, and the tracked line of sight information matches the change in the position of the human image region.

[0005] Patent Document 4 (Chinese Patent Publication No. 109460697) discloses an auxiliary filtering device that can prevent impersonation using photographs and / or videos without being combined with fingerprint authentication or the like, which would lead to increased product costs. The auxiliary filtering device of Patent Document 4 comprises a first camera configured to capture a first image, a second camera configured to capture a second image, a memory configured to pre-store the relative relationship between the position and size of an object in an image, and a processor. The processor includes the steps of: calculating the position and size of a face in the first image and the second image, respectively; comparing the calculated results with the relative relationship in the memory and excluding unsuitable objects; excluding unsuitable objects according to a plurality of first images captured by the first camera at different times, or according to a plurality of second images captured by the second camera at different times; and identifying that the current face in the first image or the second image is an unsuitable object when there is no difference in facial expression between the first images or between the second images.

[0006] Patent Document 5 (Chinese Patent Publication No. 111178340) discloses an image recognition method that improves the security of face recognition when pre-prepared attack images with faces are directly introduced into a face recognition device, completely replacing the data collected by the camera of the face recognition device. The image recognition method described in Patent Document 5 first acquires a target face image. Next, it performs wavelet analysis on the target face image with a predetermined number of layers to obtain a target wavelet human face image with a predetermined number of layers. Subsequently, the target wavelet human face image with a predetermined number of layers is input into a predetermined image recognition model to obtain a predicted value corresponding to the target face image. Here, the image recognition model generates wavelet human face training images with a predetermined number of layers corresponding to multiple face training images, and each face training image is trained with a corresponding label value. The multiple face training images include attack images and non-attack images. Finally, based on the predicted value, the attack image recognition result corresponding to the target face image is determined.

[0007] Patent document 6 (Chinese Patent Publication No. 117275099) discloses a detection method that loads a pre-acquired facial video file using a virtual camera to detect impersonation attacks that circumvent facial recognition authentication or biometric detection. The detection method described in Patent Document 6 involves receiving video data collected during the process of an operating device executing a challenge task. The challenge task is triggered when the operating device calls a camera to perform an identification task. The challenge task is used to influence the camera's acquisition operation at the physical layer. The video data is identified, and imaging feature information of the video data is obtained. It is determined whether the imaging feature information and the video feature information corresponding to the challenge task match, and if they do not match, it is determined that the detection of the identification task has failed. (The challenge task involves adjusting optical parameters or vibrating the imaging device.)

[0008] Patent document 7 (Chinese Patent Publication No. 117079317) discloses a method for effectively detecting attack behavior in injected video. The method for identifying video injection attacks described in Patent Document 7 involves acquiring a video stream to be detected during the face recognition process. The video stream to be detected includes a series of facial images. Pose data is acquired when the target terminal captures a facial video during the current face recognition process. The video stream to be detected and the pose data are input into a pre-trained feature comparison model to obtain a match score between the video stream to be detected and the pose data. Based on the match score, it is determined whether or not the video stream to be detected is an injection attack video stream.

[0009] Patent document 8 (Chinese Patent Publication No. 116682181) discloses a method, apparatus, and storage medium for detecting a biological organism based on vibration signals. The biological detection method described in Patent Document 8 is a biological detection method based on vibration signals, and its features include the following steps: after a terminal device detects a face, it randomly generates a vibration control signal S; the terminal device controls a vibration motor to vibrate based on the vibration control signal S; at the same time the vibration motor vibrates, the camera of the terminal device records a biological video of the user, and the sensor of the terminal device collects a vibration signal D due to the vibration of the terminal device; and it includes the steps of analyzing and comparing the vibration signal D and the vibration control signal S to determine whether or not the biological video is actually a biological video that was recorded.

[0010] Patent document 9 (Chinese Patent Publication No. 116318700) discloses a facial recognition method that improves the security of the facial recognition process. The facial recognition method described in Patent Document 9 includes the following steps: After the first stage of facial recognition, the shooting parameters of the image acquisition device are adjusted based on the adjustment command received from the server. The current image is acquired from the image acquisition device after the shooting parameters have been adjusted. The current image is sent to the server, whereupon the server determines the facial recognition result based on the adjustment command and the current shooting parameter values ​​characterized by the current image.

[0011] Patent document 10 (Chinese Patent Publication No. 116129533) discloses a biodetection method and apparatus that can better detect the aggressiveness of injection attacks, such as injecting pre-prepared video material via an application program interface (API). The biodetection method described in Patent Document 10 is applied to a terminal device, and the method includes the following: launching a face application program; detecting that the terminal device has launched the biodetection process in the face application program; and acquiring internal operating environment information of the terminal device formed when the terminal device executes the biodetection process. Here, the operating environment information is as follows: the terminal device executes information that is not necessary to use when executing the biodetection process, but the terminal device executes operating information of the terminal device that affects the biodetection process. The acquired operating environment information is sent to a server, which then determines whether or not an attack has occurred. [Prior art documents] [Patent Documents]

[0012] [Patent Document 1] Special Publication No. 2023-545565 [Patent Document 2] Re-tabled publication 2019 / 163066 [Patent Document 3] Chinese Patent Publication No. 111274928 [Patent Document 4] Chinese Patent Publication No. 109460697 [Patent Document 5] Chinese Patent Publication No. 111178340 [Patent Document 6] Chinese Patent Publication No. 117275099 [Patent Document 7] Chinese Patent Publication No. 117079317 [Patent Document 8] Chinese Patent Publication No. 116682181 [Patent Document 9] Chinese Patent Publication No. 116318700 [Patent Document 10] Chinese Patent Publication No. 116129533 [Overview of the Initiative] [Problems that the invention aims to solve]

[0013] In recent years, eKYC (electronic Know Your Customer) and other identity verification methods have become widespread, where users capture facial images with the cameras of mobile communication devices such as smartphones, send them to a server via the internet, and the server performs identity verification. However, recently, injection attacks have been observed that attempt to bypass this authentication method by replacing the camera input with an arbitrary image or video using some means, making countermeasures against these injection attacks crucial.

[0014] There are two main ways to replace the camera input section. The first method is to use a software camera (virtual camera) to input an image or video in a personal computer as camera footage. FIG. 5 is a schematic diagram showing an example of the configuration of an illegal identity authentication system 110 when image / video data, a virtual camera (software camera), and an eKYC Frontend are configured on a personal computer. In a normal identity authentication system 100, the server 30 is connected to a mobile communication terminal 10 such as a smartphone via the Internet 20 (see FIG. 1), and a face image captured by the mobile communication terminal 10 is sent to the server 30. However, in the case of FIG. 5, by disguising the personal computer 40 as the mobile communication terminal 10, while displaying the eKYC Frontend 43 for the mobile communication terminal 10 on the personal computer 40, it is possible to input an arbitrary image or / and video via the virtual camera 42. By doing this, it becomes possible to perform identity verification using a face image of another person and to perform identity verification using a face image or video created using generative AI including Deep Fake. In the case of a face image of another person, since it is a real photo, it cannot be distinguished by conventional authenticity determination.

[0015] The second method is to physically modify the mobile communication terminal 10 so that an image and / or video can be directly streamed from another personal computer or the like instead of the image of the camera. FIG. 6 is a schematic diagram showing an example of the configuration of an illegal identity authentication system 130 in which the camera output terminal 11a is taken out from the modified mobile communication terminal 60 and the image / video data 61 and the terminal / data conversion circuit 62 for inputting to the camera output terminal 11a are connected. However, since this method also requires hardware knowledge, it is considered that it cannot be easily implemented.

[0016] The image detection method described in Patent Document 1 can be used to determine the application of face processing technologies such as Deep Fake, but it is difficult to detect by this method when using a face image of another person who has not been processed for an injection attack. The forgery detection device described in Patent Document 2 causes the light emitting device to emit light or turn off during face authentication, so there is a possibility that the user may feel uncomfortable. The biometric detection method described in Patent Document 3 may also make the user feel uncomfortable because the position of the display target changes during face authentication. The auxiliary filtering device described in Patent Document 4 requires two in-cameras (cameras on the display side). Since it is rare for ordinary mobile terminals to have two in-cameras, it cannot be used for fraud determination in eKYC using ordinary mobile terminals. The image identification method described in Patent Document 5 assumes that the attack image is a compressed image and the non-attack image is an uncompressed image. When using a face image of someone else who has not been processed for an injection attack, it is difficult to detect using this method.

[0017] The detection method described in Patent Document 6 determines whether it is an injection attack by comparing the image when the optical parameters are adjusted or the image when the operating device is driven to vibrate with a normal image. When the operating device is driven to vibrate, the user may feel uncomfortable. The method for identifying an injection video attack described in Patent Document 7 determines whether it is an injection attack based on the matching score between the video stream of the detection target and the posture data collected from the inertial sensor. Although it seems that the user does not feel uncomfortable, when the change in the posture data of the mobile terminal is small, the accuracy of determining whether it is an injection video decreases. The method for detecting a living body described in Patent Document 8 determines by whether the frequency of the vibration motor of the mobile terminal device matches the frequencies of the collected video and audio. Since the operating device is driven to vibrate, the user may feel uncomfortable.

[0018] The face authentication method described in Patent Document 9 determines whether it is an injection attack based on the images before and after adjusting the shooting parameters of the image collection device and the content of the shooting parameter adjustment. Although it seems that the user's discomfort is small, in the case of a virtual camera, if the image can be corrected according to the shooting parameters, this method may not be able to determine that an injection attack is an injection attack. The biodetection method described in Patent Document 10 determines whether a terminal device is performing an injection attack by acquiring operating environment information such as memory usage when collecting facial information from the terminal device. However, it is necessary to install a program on the mobile device to acquire the operating environment information. Furthermore, since the operating environment information differs depending on the mobile device, this method may not be able to determine the presence or absence of an injection attack regardless of the type of mobile device.

[0019] The objective of the present invention is, (a) Using only universally installed devices, it can be executed regardless of the type of mobile device, (b) The user performing facial recognition does not need to take any special action, such as the mobile device's light-emitting device turning on / off or the mobile device vibrating, so that the user does not feel uncomfortable (without degrading the UX), (c) No special UI (user interface) is required to determine whether an injection attack has occurred, or the amount of UI added is kept to a minimum. The objective is to provide a method for detecting fraudulent user authentication, a server for performing user authentication, and a program executed on the server for performing user authentication. [Means for solving the problem]

[0020] (1) The fraud detection method, which follows a single phase, is a fraud detection method for a case where a mobile communication terminal equipped with a camera, a touch panel, and an accelerometer captures an image of the user's face and / or an image of their identification document and sends it to a server, and the server performs user authentication. The method comprises: a camera information determination step, which acquires information from the camera equipped in the mobile communication terminal and determines that the authentication is fraudulent if the camera information is unnatural; a touch panel output determination step, which acquires the contact area and the time-series change of the contact area when the touch panel is clicked and determines that the authentication is fraudulent if the time-series change of the contact area to the touch panel is less than or equal to a predetermined ratio, or if the contact area is less than or equal to a predetermined value; an accelerometer output determination step, which acquires the output of the accelerometer when the face image is captured at predetermined time intervals and calculates the tilt of the mobile communication terminal from the output of the accelerometer, and determines that the authentication is fraudulent if the time-series change of the output of the accelerometer or the time-series change of the tilt is less than or equal to a predetermined value; and a fraud detection step, which determines whether user authentication is fraudulent based on the determination results of one or more of the camera information determination step, the touch panel output determination step, and the accelerometer output determination step. The predetermined time interval is, for example, 200ms. Examples of identification documents include My Number Card, driver's license, and health insurance card.

[0021] Various methods have been proposed to counter injection attacks that attempt to bypass eKYC (electronic Know Your Customer) authentication, which involves taking facial images and other images with the camera of a mobile communication device such as a smartphone, sending them to a server via the internet, and having the server perform identity verification. These attacks involve replacing the camera input with an arbitrary image and / or video using some means. However, methods that require special devices, such as the two front-facing cameras described in Patent Document 4, are limited to the types of smartphones that can implement them. Furthermore, if the user needs to direct their gaze to the display target position on the screen, as in Patent Document 3; if the light-emitting device lights up or turns off during facial recognition, as in Patent Document 2; or if the smartphone vibrates, as in Patent Documents 6 or 8, the user undergoing facial recognition may feel uncomfortable. Furthermore, it is difficult to equip all smartphones with special interfaces, such as those described in Patent Document 10, that acquire operating environment information, including the memory usage of the terminal device.

[0022] The one-step fraud detection method provides a method for detecting fraud that can perform identity verification using a mobile communication terminal that does not have a special device or interface, and without the user having to perform any special operations or feeling any discomfort. Specifically, the system acquires information from the camera on the mobile communication terminal, the contact area and its time-series change when the touch panel is touched, and the time-series change in the output and tilt of the mobile communication terminal's accelerometer to determine whether the user authentication is fraudulent.

[0023] Regarding the information obtained from the camera of a mobile communication terminal, this is based on the fact that the camera information acquired is unnatural in attacks where an eKYC Frontend for smartphones is displayed on a PC and arbitrary images and / or videos are input via a virtual camera. Examples of unnatural camera information include cases where the camera name included in the camera information is blank, where the camera name is the software name used for the virtual camera, where the model name and the number of cameras installed on the iPhone do not correspond, and cases where the device ID, kind (types such as video input, video output, microphone input, microphone output, etc.), group ID, label, etc. are not generic. The contact area and its change over time when touching a touch panel are based on the fact that clicking a mobile device with a finger typically results in a larger contact area compared to clicking a computer screen with a mouse, and that the contact area changes over time. In other words, if the contact area is small, or if there is no change in the contact area, it is judged to be an injection attack.

[0024] The time-series changes in the output or tilt of the accelerometer of a mobile communication terminal are based on the fact that when a facial image for user authentication is taken with the front camera of the mobile communication terminal, the mobile communication terminal is held in the hand when taking the picture, so the tilt of the mobile communication terminal changes over time. In other words, if the output or tilt of the accelerometer of the mobile communication terminal does not change at all over time, it will be judged as an injection attack. On the other hand, each of these fraud detection methods may fail to detect fraud depending on the efforts of the injection attacker. Therefore, a fraud detection method that follows a single phase includes a fraud detection step that comprehensively determines whether the authentication is fraudulent based on one or more of these fraud detection results. Alternatively, it may include a fraud detection step that comprehensively determines whether the authentication is fraudulent based on three fraud detection results.

[0025] (2) The fraud detection method according to the second invention is a fraud detection method that follows one phase, in which the fraud detection step may quantify the results of the camera information determination step, the touch panel output determination step, and the acceleration sensor output determination step, and determine whether the user authentication is fraudulent by averaging these numerical values.

[0026] For example, in the touch panel output detection step, if the contact area is very small or does not change at all, it can be clearly determined to be an injection attack. However, if the values ​​are intermediate, it becomes difficult to make a judgment. In such cases, it is desirable to make a comprehensive judgment in conjunction with other detection steps.

[0027] In the fraud detection step of the fraud detection method according to the second invention, the determination results of the camera information determination step, the touch panel output determination step, and the acceleration sensor output determination step are each assigned a numerical value, for example, 0 if it is clearly fraudulent, 1 if it is not fraudulent, and between 0 and 1 if it is intermediate. The numerical values ​​of the determination results of the camera information determination step, the touch panel output determination step, and the acceleration sensor output determination step are averaged, and if the average value is less than or equal to a predetermined value, it is determined to be fraudulent. In this way, it is possible to comprehensively determine whether or not the user authentication is fraudulent.

[0028] (3) The fraud detection method according to the third invention may, in addition to the determination by averaging numerical values, determine fraud if one of the determination results of the camera information determination step, the touch panel output determination step, and the acceleration sensor output determination step is clearly determined to be fraudulent, as described in the fraud detection method according to the second invention.

[0029] In this case, if any of the camera information determination step, touch panel output determination step, or acceleration sensor output determination step clearly determines that something is fraudulent, then the fraud determination step will also definitely determine that something is fraudulent, thus ensuring that fraud is reliably detected. Specifically, for example, in the camera information determination step, if the camera name is the software name used for the virtual camera, it may be determined to be invalid regardless of the other determination steps. Alternatively, in the touch panel output determination step, if there is no time-series change in the contact area to the touch panel (meaning a mouse is being used), or in the acceleration sensor output determination step, if there is no change in the tilt of the mobile communication terminal 10, it may be determined to be invalid regardless of the other determination steps.

[0030] (4) The fraud detection method according to the fourth invention is a fraud detection method that follows one aspect, further comprising a hash value determination step in which multiple facial images are acquired at predetermined time intervals, hash values ​​of the multiple facial images are calculated, and if the hash values ​​of the multiple facial images do not change, it is determined that the authentication is fraudulent. In the fraud detection step, it may be determined whether the authentication is fraudulent based on the determination results of the camera information determination step, the touch panel output determination step, the acceleration sensor output determination step, and the hash value determination step. The predetermined time interval is, for example, 200ms.

[0031] One method of injection attack involves sending pre-prepared facial image data instead of capturing a real image of the user. In this case, since there is only one type of facial image data, if multiple facial images are acquired at predetermined time intervals, the same facial image will be sent. If the data for multiple facial images is the same, then naturally the hash values ​​of the data for all of them will also be the same. On the other hand, if a real image of the user's face is acquired at predetermined time intervals, even if the facial images appear the same, subtle changes in facial expression and / or dark current noise will cause the hash values ​​of the data for multiple facial images to differ. Therefore, if the hash values ​​of multiple facial images do not change, it can be determined that the attack is malicious. In the fraud detection method according to the fourth invention, the accuracy of fraud detection can be improved by adding this hash value determination step to the camera information determination step, touch panel output determination step, and acceleration sensor output determination step.

[0032] (5) The fraud detection method according to the fifth invention is a fraud detection method that follows one aspect, further comprising an image acceleration sensor linkage determination step in which multiple facial images are acquired at predetermined time intervals, the change in the tilt of the mobile communication terminal is calculated from the output of the acceleration sensor, and if the change in the multiple facial images does not correspond to the change in the tilt of the mobile communication terminal, it is determined that the authentication is fraudulent. In the fraud detection step, it may be determined whether the user authentication is fraudulent based on the determination results of the camera information determination step, the touch panel output determination step, the acceleration sensor output determination step, and the image acceleration sensor linkage determination step. The predetermined time interval is, for example, 200ms.

[0033] When a user holds a mobile communication device in their hand and takes multiple images of their face at predetermined time intervals, the tilt of the mobile communication device changes between each capture, and the captured face images change in accordance with this change in tilt. Therefore, by checking whether the changes in the multiple face images correspond to the changes in the tilt of the mobile communication device—specifically, whether, for example, when the face image moves slightly upward, the tilt of the mobile communication device also changes slightly upward—it is possible to determine whether the received face images were taken by a user holding the mobile communication device in their hand. In the fraud detection method according to the fifth invention, the accuracy of fraud detection can be improved by adding this image acceleration sensor linkage detection step to the camera information detection step, touch panel output detection step, and acceleration sensor output detection step.

[0034] (6) The fraud detection method according to the sixth invention is a fraud detection method that follows one aspect, further comprising a black border detection step which determines that the facial image transmitted from a mobile communication terminal is fraudulent if there is a black border around the periphery, and in the fraud detection step, it may be determined whether the user authentication is fraudulent based on the determination results of the camera information detection step, the touch panel output detection step, the acceleration sensor output detection step, and the black border detection step.

[0035] When using a virtual camera to transmit images and / or videos from a computer as camera footage, if the handling of the virtual camera is not sophisticated, a black border may appear around the image. Therefore, if there is a black border around the periphery of a facial image, it can be definitively determined to be an injection attack. In the fraud detection method according to the sixth invention, the accuracy of fraud detection can be improved by adding this black border detection step to the camera information detection step, touch panel output detection step, and acceleration sensor output detection step.

[0036] (7) The fraud detection method according to the seventh invention is a fraud detection method that follows one aspect, further comprising a battery temperature determination step in which the mobile communication terminal is equipped with a temperature sensor for detecting the battery temperature, and the fraud detection method determines that it is fraudulent if the temperature difference between the start of user authentication and a predetermined time after the start of user authentication is less than or equal to a predetermined value, and in the fraud detection step, it may be determined whether user authentication is fraudulent based on the determination results of the camera information determination step, the touch panel output determination step, the acceleration sensor output determination step, and the battery temperature determination step.

[0037] Mobile communication devices running eKYC consume a considerable amount of power due to their camera and CPU, causing the battery to overheat. Image processing, in particular, is a heavy process and contributes to the rise in battery temperature. Therefore, by measuring the change in battery temperature from the start of user authentication, if the change in battery temperature is below a predetermined value, there is a high probability that it is an injection attack using a virtual camera on a PC. In the fraud detection method according to the seventh invention, the accuracy of fraud detection can be improved by adding this battery temperature detection step to the camera information detection step, touch panel output detection step, and acceleration sensor output detection step.

[0038] (8) The fraud detection method according to the eighth invention is a fraud detection method that follows one aspect, further comprising an image shutter speed linkage determination step in which the camera of the mobile communication terminal is equipped with a shutter speed change function, multiple facial images are taken with different shutter speeds, and the fraud detection method determines that it is fraudulent if the change in brightness of the multiple facial images taken does not correspond to the change in shutter speed at the time of shooting, and in the fraud detection step, it may determine whether the user authentication is fraudulent based on the determination results of the camera information determination step, the touch panel output determination step, the acceleration sensor output determination step, and the image shutter speed linkage determination step.

[0039] In the case of a virtual camera, it is possible to change the brightness of the image, but it is difficult to change the brightness in sync with changes in shutter speed. Therefore, for example, if the shutter speed set in automatic exposure control is (1 / 100)S, the camera can switch the shutter speed to (1 / 100)S, (1 / 10)S, and (1 / 1000)S in a short time (less than 1 second), capture and transmit facial images, and then the server can compare the timing of the shutter speed changes with the timing of the image brightness changes to determine whether the transmitted facial image was taken with a mobile communication terminal or a virtual camera. In the example above, three different shutter speeds are used, but it is also possible to switch between two different shutter speeds, or any number of different shutter speeds. In the fraud detection method according to the eighth invention, the accuracy of fraud detection can be improved by adding this image shutter speed linkage detection step to the camera information detection step, touch panel output detection step, and acceleration sensor output detection step.

[0040] (9) The fraud detection method according to the ninth invention is a fraud detection method that follows one aspect, further comprising a touch panel acceleration sensor linkage determination step which determines that fraud is occurring if the time-series change in the contact area of ​​the touch panel is greater than or equal to a predetermined value and the time-series change in the tilt of the mobile communication terminal is less than or equal to a predetermined value, wherein the fraud detection step may determine whether the user authentication is fraudulent based on the determination results of the camera information determination step, the touch panel output determination step, the acceleration sensor output determination step, and the touch panel acceleration sensor linkage determination step.

[0041] When you click on a mobile device such as a smartphone with your finger, the contact area of ​​the touch panel increases, and at the same time, the device's tilt changes slightly upward because it is being pressed by your finger. On the other hand, in the case of an injection attack using a virtual camera, there is no tilt at all when clicking with a mouse. Also, when you click on the touch panel of a tablet, the tablet tilts slightly, but the change in tilt of the tablet when clicking is small compared to a mobile device. Therefore, if the change in the tilt of the mobile communication terminal is less than or equal to a predetermined value around the time of a click when the time-series change in the contact area of ​​the touch panel exceeds a predetermined value, it can be determined that this is an injection attack using a virtual camera. In the fraud detection method according to the ninth invention, the accuracy of fraud detection can be improved by adding this image shutter speed linkage detection step to the camera information detection step, touch panel output detection step, and acceleration sensor output detection step.

[0042] (10) The server that follows the other phase is a server that performs identity authentication using a facial image of the person and / or an image of an identity verification document transmitted from a mobile communication terminal equipped with a camera, a touch panel, and an accelerometer, and comprises: a facial recognition unit that compares the transmitted facial image with a reference facial image to determine whether it is the same person; a camera information determination unit that acquires information from the camera equipped in the mobile communication terminal and determines that it is fraudulent if the camera information is unnatural; a touch panel output determination unit that acquires the contact area and the time-series change of the contact area when the touch panel is clicked and determines that it is fraudulent if the time-series change of the contact area to the touch panel is less than or equal to a predetermined ratio, or if the contact area is less than or equal to a predetermined value; an accelerometer output determination unit that acquires the output of the accelerometer when the facial image is captured at predetermined time intervals and calculates the tilt of the mobile communication terminal from the output of the accelerometer, and determines that it is fraudulent if the time-series change of the output of the accelerometer or the time-series change of the tilt is less than or equal to a predetermined value; and a fraud determination unit that, when the facial recognition unit determines that it is the same person, determines whether the identity authentication is fraudulent based on the determination results of the camera information determination unit, the touch panel output determination unit, and the accelerometer output determination unit. Furthermore, if the user's facial image is not used for authentication, the facial recognition component may be omitted.

[0043] In eKYC and similar identity verification methods, the server's facial recognition unit compares the user's facial image sent from a mobile communication terminal with a reference facial image read from an IC card such as a My Number Card, or one previously stored on the server, to verify their identity. However, in recent years, injection attacks have been observed in which attackers attempt to bypass identity verification by replacing the camera input with an arbitrary image and / or video using some means. In such cases, it is not possible to determine whether there is fraud in the identity verification process based solely on the comparison of facial images. The server, which follows other phases, includes a camera information determination unit, a touch panel output determination unit, an acceleration sensor output determination unit, and an acceleration sensor output determination unit, and the fraud determination unit determines whether the user authentication is fraudulent based on the determination results of these units.

[0044] (11) Furthermore, a program that follows other phases is a program executed on a server that performs identity authentication using a facial image of the person and / or an image of an identity verification document transmitted from a mobile communication terminal equipped with a camera, a touch panel, and an accelerometer, and when the server compares the transmitted facial image with a reference facial image and determines that it is the same person, it performs a camera information determination process that acquires information from the camera equipped on the mobile communication terminal and determines that it is fraudulent if the camera information is unnatural; a touch panel output determination process that acquires the contact area and the time-series change of the contact area when the touch panel is clicked and determines that it is fraudulent if the time-series change of the contact area to the touch panel is less than or equal to a predetermined ratio, or if the contact area is less than or equal to a predetermined value; an accelerometer output determination process that acquires the output of the accelerometer at predetermined time intervals, calculates the tilt of the mobile communication terminal from the output of the accelerometer, and determines that it is fraudulent if the time-series change of the output of the accelerometer or the time-series change of the tilt is less than or equal to a predetermined value; and a fraud determination process that determines whether identity authentication is fraudulent based on one or more determination results from the camera information determination process, the touch panel output determination process, and the accelerometer output determination process.

[0045] Furthermore, the program that follows other phases is a program used to determine whether the authentication is fraudulent, using the CPU of the server that performs the authentication according to the other phases. Even when comparing the transmitted face image with a reference face image, some processing may be included in the above program and performed by the CPU, rather than using dedicated hardware. [Brief explanation of the drawing]

[0046] [Figure 1] This is a schematic diagram illustrating an example of a normal user authentication system configuration. [Figure 2] This is a schematic diagram showing an example of the server configuration used in an identity verification system. [Figure 3] This is a schematic flowchart showing the first flow of fraud detection in the fraud detection unit. [Figure 4] This is a schematic flowchart showing the second flow of fraud detection in the fraud detection unit. [Figure 5] This is a schematic diagram illustrating an example of the configuration of a fraudulent identity verification system. [Figure 6] This is a schematic diagram illustrating another example of a fraudulent identity verification system configuration. [Modes for carrying out the invention]

[0047] Embodiments of the present invention will be described below with reference to the drawings. In the following description, identical parts are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed descriptions of them will not be repeated.

[0048] [Embodiment] (Configuration of a normal identity verification system) Figure 1 shows an example of the configuration of a normal user authentication system 100, and Figure 2 shows an example of the configuration of a server 30 used in the user authentication system in this embodiment. In Figure 1, a normal user authentication system 100 is configured such that a mobile communication terminal 10, such as a smartphone, and a server 30 are connected via the internet 20. The mobile communication terminal 10 is equipped with a camera 11, a touch panel 12 with display functionality, an accelerometer 13, a temperature sensor 14, and an eKYC Frontend 15. Camera 11 captures an image of the user's face, and the captured image is sent to server 30 via eKYC Frontend. The touch panel 12 is used when the user clicks with their finger, and the contact area with the touch panel 12 is sent to the server 30. The acceleration sensor 13 detects the magnitude and direction of acceleration, including gravitational acceleration, applied to the mobile communication terminal 10, thereby measuring the tilt and change in tilt of the mobile communication terminal 10, and the measurement results are sent to the server 30. The temperature sensor 14 is located near the battery of the mobile communication terminal 10, measures the temperature near the battery, and sends the measurement result to the server 30. The data sent to server 30 is then sent to eKYC Backend 32 via eKYC API 31.

[0049] The eKYC Backend32 has various functions for user authentication, but this section mainly describes the functions related to fraud detection when the server 30 performs user authentication. In Figure 2, the facial recognition unit 321 compares the transmitted facial image with a reference facial image to determine if it is the same person. In this case, the reference facial image is, for example, the facial image of the person recorded on the IC chip of the My Number Card by the mobile communication terminal 10, or the facial image of the person registered in advance on the server 30. Note that if the mobile communication terminal 10 only takes a picture of the identity verification document and sends it to the server 30, the facial recognition unit 321 is not necessary.

[0050] The camera information determination unit 322 acquires information from the camera 11 of the mobile communication terminal 10 and determines whether the user authentication is fraudulent. This is based on the fact that in an attack where an eKYC Frontend 43 for smartphones is displayed on the PC 40 (see Figure 5) and arbitrary images and / or videos are input via the virtual camera 42, the acquired camera information 11 is unnatural. Unnatural camera information includes, for example, cases where the camera name included in the camera information is blank, the camera name is the software name used by the virtual camera 42, in the case of an iPhone the model name and the number of cameras installed do not correspond, or other cases where the device ID, kind (type such as video input, video output, microphone input, microphone output), group ID, label, etc. are not general-purpose.

[0051] The touch panel output determination unit 323 acquires the contact area when the touch panel 12 is clicked and the time-series change of the contact area to determine whether the user authentication is fraudulent. Normally, when the mobile communication terminal 10 is clicked with a finger, the contact area is larger than when the PC screen 40 is clicked with a mouse, and the contact area changes over time. Based on this, if the contact area is very small, or if the contact area does not change over time, it is determined that the PC 40 is connected, and the user authentication is determined to be fraudulent. Furthermore, in order for the touch panel output determination unit 323 to determine whether the action is fraudulent, the user must click the touch panel 12 during eKYC. For this reason, a tutorial screen may be provided before shooting begins, displaying a "next" button on the tutorial screen or prompting the user to swipe to advance through the tutorial pages. The acceleration sensor output determination unit 324 acquires the output of the acceleration sensor 13 at predetermined time intervals when a face image is captured and determines whether the authentication is fraudulent. Normally, when a face image for authentication is captured by the camera 11 of the mobile communication terminal 10, the mobile communication terminal 10 is held in the hand when taking the picture, so the mobile communication terminal 10 is tilted vertically or diagonally, and this tilt changes over time. Therefore, if the tilt of the mobile communication terminal 10 when the face image is captured does not change over time, it is determined that the camera that captured the face image is not the camera 11 of the mobile communication terminal 10, and the authentication is determined to be fraudulent.

[0052] The hash value determination unit 325 acquires multiple facial images at predetermined time intervals, calculates the hash values ​​of the multiple facial images, and determines that the user authentication is fraudulent if the hash values ​​of the multiple facial images do not change. One method of injection attack involves sending pre-prepared facial image data instead of capturing a real image of the user. In this case, since there is only one type of facial image data, if multiple facial images are acquired at predetermined time intervals, the same facial image will be sent. If the data for multiple facial images is the same, then naturally the hash values ​​of the data for all of them will also be the same. On the other hand, if a real image of the user's face is acquired at predetermined time intervals, even if the facial images appear the same, subtle changes in facial expression and / or dark current noise will cause the hash values ​​of the data for multiple facial images to differ. Therefore, if the hash values ​​of multiple facial images do not change, it can be determined that the attack is malicious.

[0053] The image acceleration sensor linkage determination unit 326 acquires multiple face images and the output of the acceleration sensor 13 at predetermined time intervals, calculates the change in the tilt of the mobile communication terminal 10 from the output of the acceleration sensor 13, and determines that it is fraudulent if the changes in the multiple face images do not correspond to the change in the tilt of the mobile communication terminal 10. This is based on the following principle: When a user holds the mobile communication terminal 10 in their hand and takes multiple face images of themselves at predetermined time intervals, the tilt of the mobile communication terminal 10 changes between each shooting timing, and the captured face images also change in accordance with that change in tilt. Therefore, by checking whether the changes in the multiple face images correspond to the change in the tilt of the mobile communication terminal 10, specifically, for example, whether the tilt of the mobile communication terminal 10 also changes slightly upward when the face image moves slightly upward, it is possible to determine whether the face images being sent were taken by a user holding the mobile communication terminal 10 in their hand. The predetermined time interval is, for example, 200ms. The black border detection unit 327 determines that a face image transmitted from the mobile communication terminal 10 is malicious if it has a black border around its periphery. When transmitting images and / or videos from the personal computer 40 as camera footage using the virtual camera 42, a black border may appear around the image if the handling of the virtual camera 42 is not sophisticated. Therefore, if there is a black border around the periphery of the face image, it can be reliably determined that it is an injection attack.

[0054] The battery temperature determination unit 328 continuously measures the temperature of the battery temperature sensor 14 provided in the mobile communication terminal 10, and determines that the user authentication is fraudulent if the temperature difference between the start of user authentication and a predetermined time after the start of user authentication is below a predetermined value. This is based on the following principle: The mobile communication terminal 10 running eKYC consumes a considerable amount of electricity due to the camera 11 and CPU, causing the battery to heat up. Image processing in particular is a heavy process and contributes to the rise in battery temperature. Therefore, by measuring the change in battery temperature from the start of user authentication, if the change in battery temperature is below a predetermined value, there is a high possibility that it is an injection attack using the virtual camera 42 of the PC 40. The image shutter speed synchronization determination unit 329 determines that an image is fraudulent if, when the camera 11 of the mobile communication terminal 10 has a shutter speed change function, it takes multiple facial images at different shutter speeds and the change in brightness of the multiple facial images taken does not correspond to the change in shutter speed at the time of shooting. This is based on the following principle: In the case of the virtual camera 42, it is possible to change the brightness of the image, but it is difficult to change the brightness in sync with the change in shutter speed. Therefore, for example, if the shutter speed set by automatic exposure is (1 / 100)S, the shutter speed can be switched to (1 / 100)S, (1 / 10)S, and (1 / 1000)S in a short time, such as less than 1 second, to take and transmit facial images, and the server 30 can compare the timing of the changes in shutter speed and the timing of the changes in image brightness to determine whether the transmitted facial images were taken by the mobile communication terminal 10 or by the virtual camera 42. In the example above, three different shutter speeds are used, but you can also switch between two different shutter speeds, or any number of shutter speeds you like.

[0055] The touch panel acceleration sensor linkage determination unit 330 determines that an attack is fraudulent if the time-series change in the contact area of ​​the touch panel 12 is greater than or equal to a predetermined value, and the time-series change in the tilt of the mobile communication terminal 10 is less than or equal to a predetermined value. This is based on the following principle: When a mobile communication terminal 10 such as a smartphone is clicked with a finger, the contact area of ​​the touch panel 12 increases, and at the same time, the tilt of the mobile communication terminal 10 changes slightly upward because it is pressed by the finger. On the other hand, in the case of an injection attack using a virtual camera 42, there is no tilt at all when clicking with a mouse, and even when clicking the touch panel 12 of a tablet, the change in the tilt of the tablet at the time of clicking is small. Therefore, if the change in the tilt of the mobile communication terminal 10 is less than or equal to a predetermined value before and after the click when the time-series change in the contact area of ​​the touch panel 12 is greater than or equal to a predetermined value, it can be determined that it is an injection attack using a virtual camera 42.

[0056] (Flowchart of the fraud detection unit 331) Injection attacks against user authentication can take many forms, including the first and second methods described herein, and further new methods may be developed in the future. For this reason, in addition to the more basic camera information determination step, touch panel output determination step, and acceleration sensor output determination step, the present invention includes a hash value determination step, an image acceleration sensor linkage determination step, a black border determination step, a battery temperature determination step, an image shutter speed linkage determination step, and a touch panel acceleration sensor linkage determination step.

[0057] When using these multiple judgment steps to determine whether or not user authentication is fraudulent, it is desirable to make an overall determination of fraud based on the results of each judgment step. Therefore, in the first fraud determination flow of the fraud determination unit 331, the result of each judgment step is numerically assigned to a range of 0 to 1, where 0 is assigned if it is clearly fraudulent and 1 is assigned if it is not fraudulent. The average value obtained by averaging these values ​​is compared with a predetermined value (for example, 0.4), and if the average value is less than or equal to the predetermined value, it is determined to be fraudulent. However, even if the average value is greater than a predetermined value, there are cases where it should be determined to be fraudulent. For example, in the camera information determination step, if the camera name is the software name used for the virtual camera 42, it is almost certainly considered an injection attack using the personal computer 40. In such cases, it should be determined to be fraudulent regardless of the average value. Therefore, in the second flow of fraud determination by the fraud determination unit 331, if it is determined to be clearly fraudulent in any of the determination steps, it is determined to be fraudulent regardless of the determination values ​​in the other steps, and if the results of each determination step cannot be said to be clearly fraudulent, the average value is used to determine whether it is fraudulent or not. Figure 3 shows the first flow of fraud detection by the fraud detection unit 331, and Figure 4 shows the second flow of fraud detection. When the fraud detection unit 331 determines that the face image transmitted by the face recognition unit 321 is the same person as the reference face image, it determines whether the authentication is fraudulent based on the determination results of each determination unit. Figures 3 and 4 show the flow when fraud detection is performed based on the determination results of the camera information determination unit 322, the touch panel output determination unit 323, and the acceleration sensor output determination unit 324.

[0058] (First flow of the fraud detection method) First, following Figure 3, we will explain the flowchart of the first flow of the fraud detection method step by step. (Steps S01-S02) Information from camera 11 is acquired and the determination of whether or not there is fraud is quantified (V1) (corresponding to the camera information determination step). Specifically, information from the built-in camera is obtained from an API provided by the internet browser (the MediaDevices:enumerateDevices method as of 2024). The acquired camera information includes information such as ID and label. If the label is blank or is the software name of virtual camera 42, it is likely an injection attack using PC 40. In addition, if the kind (type such as video input, video output, microphone input, microphone output), group ID, etc. are not general-purpose, it may also be an injection attack using PC 40. In addition, in the case of iPhones, the number of cameras 11 is determined by the model, so if the correspondence between the model name and the number of cameras 11 differs from the standard, there is a high possibility of an injection attack using a personal computer 40. The camera information determination unit 322 quantifies the presence or absence of malicious activity based on this information. For example, if the label is blank or the software name of the virtual camera 42, V1=0; if there is nothing particularly unusual in the information of camera 11, V1=1; and if there is something unusual in some of the information, an appropriate value between 0 and 1 is set according to the content. (Steps S03-S04) The contact area of ​​the touch panel 12 and the time-series change of the contact area when the touch panel 12 is clicked are obtained and the determination of whether or not fraud is present is quantified (V2) (corresponding to the touch panel output determination step). Specifically, if the contact area of ​​the touch panel 12 is very small or there is no time-series change of the contact area, it is considered that the click was made with a mouse rather than a finger, so it is judged to be an injection attack and V2 is quantified as V2=0. If the contact area of ​​the touch panel 12 and the time-series change of the contact area are close to normal, V2 is quantified as V2=1. In intermediate states, for example, if the contact area is considered small for a finger, an appropriate value between 0 and 1 is set according to the content.

[0059] (Steps S05-S06) The time-series change in the output of the acceleration sensor 13 when the face image is captured is obtained and the determination of whether or not there is fraud is quantified (V3) (corresponding to the acceleration sensor output determination step). Specifically, if there is no time-series change in the output of the acceleration sensor 13 of the mobile communication terminal 10 when the face image is captured, or no time-series change in the tilt of the mobile communication terminal 10 calculated from the output of the acceleration sensor 13, it is considered that the face image was not captured by the mobile communication terminal 10, so it is judged to be an injection attack and V3 is quantified as 0. If the time-series change in the output of the acceleration sensor 13 of the mobile communication terminal 10 and the time-series change in the tilt when the face image is captured are normal, V3 is quantified as 1. In intermediate states, for example, if the time-series change in the tilt is smaller than normal, an appropriate value between 0 and 1 is set according to the content.

[0060] (Step S07) The numerical value V1 of the judgment result based on camera information, the numerical value V2 of the judgment result based on touch panel output, and the numerical value V3 of the judgment result based on acceleration sensor output are averaged. (Steps S08-S10) If the average value (V1+V2+V3) / 3 is less than or equal to a predetermined value, the authentication is determined to be fraudulent; if it is greater than the predetermined value, it is determined to be normal. Specifically, for example, if V1, V2, and V3 are each between 0 and 1, the average value will be between 0 and 1, so the predetermined value may be 0.4 (corresponding to the fraud determination step).

[0061] The flowchart in Figure 3 shows the case where it is determined whether user authentication is fraudulent based on the results of the camera information determination step, the touch panel output determination step, and the acceleration sensor output determination step. However, if one or more determination steps are added from the hash value determination step, image acceleration sensor linkage determination step, black border determination step, battery temperature determination step, image shutter speed linkage determination step, and touch panel acceleration sensor linkage determination step, the steps for acquiring information and quantifying the determination results of those steps are added in parallel with steps S03-S04 and S05-S06 in Figure 3, and the values ​​from those steps are averaged in step S07, thereby creating a flowchart for fraud detection that includes the additional determination steps. Furthermore, while the flowchart in Figure 3 determines whether user authentication is fraudulent based on the results of the camera information determination step, the touch panel output determination step, and the acceleration sensor output determination step, it is also possible to determine whether user authentication is fraudulent based on the results of one or more of the output determination steps among the camera information determination step, the touch panel output determination step, and the acceleration sensor output determination step.

[0062] (Second flow of the fraud detection method) Next, following Figure 4, we will explain the flowchart of the second flow of the fraud detection method step by step. The following explanation will primarily focus on the differences between the second flow and the first flow.

[0063] (Steps S01-S02) are identical to the first flow. (Step S11) If the numerical value V1 from the camera information determination step is less than or equal to the second predetermined value, regardless of the results of the touch panel output determination step and the acceleration sensor output determination step, it is concluded that the user authentication is fraudulent. This is because, in the camera information determination step, if it is clearly considered to be an injection attack using the personal computer 40, for example, when label is the software name of the virtual camera 42, V1=0. In this case, if the second predetermined value is set to 0.1, it is possible to determine that the user authentication is fraudulent regardless of the determination results of the touch panel output determination step and the acceleration sensor output determination step.

[0064] (Steps S03-S04) This is the same as the first flow. (Step S12) If the numerical value V2 in the touch panel output determination step is less than or equal to a second predetermined value (e.g., 0.1), similar to step S11, it can be determined that the user authentication is fraudulent, regardless of the determination results of the camera information determination step and the acceleration sensor output determination step. (Steps S05-S06) This is the same as the first flow. (Step S13) If the numerical value V3 from the acceleration sensor output determination step is less than or equal to a second predetermined value (e.g., 0.1), similar to step S11, it can be determined that the user authentication is fraudulent, regardless of the determination results of the camera information determination step and the touch panel output determination step. (Steps S07-S10) These steps are basically the same as the first flow, but differ in that if an error is detected in step S11, S12, or S13, steps S07-S08 and S10 are not executed.

[0065] The flowchart in Figure 4 shows the case where it is determined whether user authentication is fraudulent based on the results of the camera information determination step, the touch panel output determination step, and the acceleration sensor output determination step. Furthermore, if one or more determination steps from the hash value determination step, image acceleration sensor linkage determination step, black border determination step, battery temperature determination step, image shutter speed linkage determination step, and touch panel acceleration sensor linkage determination step are added, the steps of acquiring information for those steps, quantifying the determination results, and comparing them with a second predetermined value are added in parallel with steps S03-S04, S12, S05-S06, and S13 in Figure 4, and by averaging the values ​​of those steps in step S07, a flowchart for fraud detection including the additional determination steps can be created.

[0066] Furthermore, although the above embodiment has been described, the system is not limited thereto. For at least one or more, or all, of the camera information determination step, touch panel output determination step, and acceleration sensor output determination step, the system may also select at least one or more, or all of the hash value determination step, image acceleration sensor linkage determination step, black border determination step, battery temperature determination step, image shutter speed linkage determination step, and touch panel acceleration sensor linkage determination step at any time, and have the fraud determination unit 331 perform the determination. In addition, weighting may be arbitrarily set for each determination step.

[0067] In this embodiment, camera 11 corresponds to "camera", touch panel 12 corresponds to "touch panel", acceleration sensor 13 corresponds to "accelerometer", mobile communication terminal 10 corresponds to "mobile communication terminal", server 30 corresponds to "server", temperature sensor 14 corresponds to "temperature sensor", face recognition unit 321 corresponds to "face recognition unit", camera information determination unit 322 corresponds to "camera information determination unit", touch panel output determination unit 323 corresponds to "touch panel output determination unit", acceleration sensor output determination unit 324 corresponds to "accelerometer output determination unit", and fraud detection unit 331 corresponds to "fraud detection unit".

[0068] While the above describes a preferred embodiment of the present invention, the invention is not limited thereto. It will be understood that various other embodiments can be made without departing from the spirit and scope of the invention. Furthermore, although the operation and effects of the configuration of the present invention are described in this embodiment, these operations and effects are examples and do not limit the invention. [Explanation of Symbols]

[0069] 10 Mobile communication terminals 11 Cameras 12 Touch panel 13. Accelerometer 14. Temperature sensor 30 servers 321 Facial Recognition Section 322 Camera Information Determination Unit 323 Touch panel output determination unit 324 Acceleration sensor output determination unit 331 Fraud Judgment Department

Claims

1. A method for determining fraud in a case where a mobile communication terminal equipped with a camera, a touch panel, and an accelerometer captures an image of the person's face and / or an image of their identification document and transmits it to a server, and the server performs identity verification, A camera information determination step which acquires information from the camera equipped in the mobile communication terminal and determines that the information from the camera is fraudulent if it is unnatural, A touch panel output determination step that acquires the contact area and the time-series change of the contact area when the touch panel is clicked, and determines that the operation is invalid if the time-series change of the contact area to the touch panel is less than or equal to a predetermined ratio, or if the contact area is less than or equal to a predetermined value. An acceleration sensor output determination step is performed, in which the output of the acceleration sensor during the capture of the face image is acquired at predetermined time intervals, the tilt of the mobile communication terminal is calculated from the output of the acceleration sensor, and if the time-series change of the output of the acceleration sensor or the time-series change of the tilt is less than or equal to a predetermined value, it is determined that the device is invalid. A fraud detection method comprising: a fraud detection step that determines whether user authentication is fraudulent based on the determination results of one or more of the camera information determination step, the touch panel output determination step, and the acceleration sensor output determination step.

2. The fraud detection method according to claim 1, wherein the fraud detection step involves quantifying the results of the camera information determination step, the touch panel output determination step, and the acceleration sensor output determination step, and determining whether the user authentication is fraudulent based on the average of these numerical values.

3. The fraud detection method according to claim 2, wherein in the fraud detection step, in addition to the determination by the average of the numerical values, fraud is determined if one of the determination results of the camera information determination step, the touch panel output determination step, and the acceleration sensor output determination step is clearly determined to be fraudulent.

4. Furthermore, the system includes a hash value determination step in which multiple facial images are acquired at predetermined time intervals, the hash values ​​of the multiple facial images are calculated, and if the hash values ​​of the multiple facial images do not change, it is determined that the system is fraudulent. The fraud detection method according to claim 1, wherein the fraud detection step determines whether the user authentication is fraudulent based on the determination results of the camera information determination step, the touch panel output determination step, the acceleration sensor output determination step, and the hash value determination step.

5. Furthermore, the system includes an image acceleration sensor linkage determination step which acquires multiple facial images at predetermined time intervals, calculates the change in the tilt of the mobile communication terminal from the output of the acceleration sensor, and determines that the system is fraudulent if the changes in the multiple facial images do not correspond to the change in the tilt of the mobile communication terminal. The fraud detection method according to claim 1, wherein the fraud detection step determines whether the user authentication is fraudulent based on the determination results of the camera information determination step, the touch panel output determination step, the acceleration sensor output determination step, and the image acceleration sensor linkage determination step.

6. Furthermore, the system includes a black border detection step that determines if a black border is present around the periphery of a facial image transmitted from the mobile communication terminal, The fraud detection method according to claim 1, wherein the fraud detection step determines whether the user authentication is fraudulent based on the determination results of the camera information determination step, the touch panel output determination step, the acceleration sensor output determination step, and the black border determination step.

7. Furthermore, the mobile communication terminal is equipped with a temperature sensor that detects the battery temperature. The fraud detection method includes a battery temperature determination step that determines fraud is occurring if the temperature difference between the start of user authentication and a predetermined time after the start of user authentication is below a predetermined value. The fraud detection method according to claim 1, wherein the fraud detection step determines whether the user authentication is fraudulent based on the determination results of the camera information determination step, the touch panel output determination step, the acceleration sensor output determination step, and the battery temperature determination step.

8. Furthermore, the camera of the mobile communication terminal is equipped with a shutter speed change function, and multiple facial images are taken with different shutter speeds. The fraud detection method includes an image shutter speed linkage detection step that determines fraud is occurring if the change in brightness of multiple captured face images does not correspond to the change in shutter speed at the time of shooting. The fraud detection method according to claim 1, wherein the fraud detection step determines whether the user authentication is fraudulent based on the determination results of the camera information determination step, the touch panel output determination step, the acceleration sensor output determination step, and the image shutter speed linkage determination step.

9. Furthermore, the system includes a touch panel acceleration sensor linkage determination step that determines if the time-series change in the contact area of ​​the touch panel is greater than or equal to a predetermined value, and the time-series change in the tilt of the mobile communication terminal is less than or equal to a predetermined value, thereby determining that the system is fraudulent. The fraud detection method according to claim 1, wherein the fraud detection step determines whether the user authentication is fraudulent based on the determination results of the camera information determination step, the touch panel output determination step, the acceleration sensor output determination step, and the touch panel acceleration sensor linkage determination step.

10. A server that performs identity authentication using an image of the person's face and / or an image of their identification document transmitted from a mobile communication terminal equipped with a camera, a touch panel, and an accelerometer, A facial recognition unit that compares the transmitted facial image with a reference facial image to determine whether it is the same person, A camera information determination unit acquires information from the camera of the mobile communication terminal and determines that the information from the camera is fraudulent if it is unnatural. A touch panel output determination unit acquires the contact area and the time-series change of the contact area when the touch panel is clicked, and determines that the operation is invalid if the time-series change of the contact area to the touch panel is less than or equal to a predetermined ratio, or if the contact area is less than or equal to a predetermined value. An acceleration sensor output determination unit acquires the output of the acceleration sensor at predetermined time intervals when the face image is captured, calculates the tilt of the mobile communication terminal from the output of the acceleration sensor, and determines that it is invalid if the time-series change of the output of the acceleration sensor or the time-series change of the tilt is less than or equal to a predetermined value. A server comprising: a facial recognition unit that determines whether the person authentication is fraudulent based on the determination results of one or more of the camera information determination unit, the touch panel output determination unit, and the acceleration sensor output determination unit when the facial recognition unit determines that the person is the same person.

11. A program that runs on a server that performs identity authentication using a facial image and / or image of an identity document transmitted from a mobile communication terminal equipped with a camera, a touch panel, and an accelerometer, When the server determines that the transmitted face image is of the same person by comparing it with a reference face image, A camera information determination process that acquires information from the camera equipped in the mobile communication terminal and determines that the information from the camera is fraudulent if it is unnatural, A touch panel output determination process that acquires the contact area and the time-series change of the contact area when the touch panel is clicked, and determines that the operation is invalid if the time-series change of the contact area to the touch panel is less than or equal to a predetermined ratio, or if the contact area is less than or equal to a predetermined value. An acceleration sensor output determination process that acquires the output of the acceleration sensor at predetermined time intervals, calculates the tilt of the mobile communication terminal from the output of the acceleration sensor, and determines that the device is invalid if the time-series change of the acceleration sensor output or the time-series change of the tilt is less than or equal to a predetermined value, A program that performs a fraud detection process that determines whether the user authentication is fraudulent based on one or more of the determination results from the camera information determination process, the touch panel output determination process, and the acceleration sensor output determination process.

Images