Method and System for Contactless Spoof Fingerprint Detection Using an Unsupervised Learning-Based Artificial Neural Network

Abstract

The present invention relates to a non-contact method for detecting forged fingerprints using an unsupervised learning-based artificial neural network. The present invention comprises the steps of: receiving a fingerprint image captured in a non-contact manner by a user terminal; detecting whether the fingerprint image is forged in real time simultaneously with the capture using a first artificial intelligence neural network operating in an application of the user terminal; transmitting a fingerprint image that is not detected as forged by the first artificial intelligence neural network to a forgery detection server; receiving the fingerprint image through an input unit at the forgery detection server; performing image preprocessing on the fingerprint image, including normalization, resizing, and region of interest extraction, in a preprocessing unit of the forgery detection server; inputting the preprocessed fingerprint image into a second artificial intelligence neural network including an autoencoder structure to make a second determination of whether it is forged; and, if it is determined to be forged by the second artificial intelligence neural network, transmitting information indicating that the fingerprint image is forged to the user terminal.

Description

Technology Field

[0001] The present invention relates to the field of biometric recognition and security authentication technology, and to a non-contact forged fingerprint detection method and system using an unsupervised learning-based artificial neural network, wherein a first artificial intelligence neural network at a user terminal is used to make a first determination in real time regarding the possibility of forgery of a fingerprint image, and a fingerprint image not determined to be forged is transmitted to a forgery detection server to perform a second precise determination based on restoration characteristics through an unsupervised learning-based second artificial intelligence neural network with an autoencoder structure. Background Technology

[0003] Fingerprint recognition is a representative biometric technology widely used in various authentication services, such as financial transactions, access control, and mobile device unlocking. Recently, contactless fingerprint recognition methods, which acquire fingerprint images using capture devices such as cameras without touching a finger to a sensor, are becoming more widespread due to reasons such as improved user convenience and hygiene concerns.

[0004] However, contactless fingerprint recognition suffers from significant variations in image quality due to fluctuations in the shooting environment, such as lighting conditions, shooting distance, finger angle, focus, and shaking, which can make it vulnerable to forged fingerprint attacks. For instance, presentation attacks using printed materials, video playback on display screens, dummy fingerprints made of artificial materials like silicone or gelatin, or other replication media can be attempted in various forms, posing a problem where it is difficult to reliably block such attacks using only simple image quality-based rules or traditional feature-based classification.

[0005] Furthermore, when building a counterfeit fingerprint detection model using a supervised learning approach, it is necessary to secure a large volume of labeled datasets covering various types of counterfeiting (material, manufacturing method, shooting environment, etc.), and there is the burden of updating the training data and model whenever the type of counterfeiting changes. On the other hand, while unsupervised learning-based autoencoder models trained solely on legitimate fingerprints can be expected to offer relatively better generalization performance regarding changes in counterfeiting types, problems such as increased latency, concentrated server load, and a rise in false positives and false negatives may occur during actual service implementation if the real-time processing in the terminal environment, precise judgment on the server, and the interoperability structure between the two are not systematically designed.

[0006] Furthermore, conventional non-contact forged fingerprint detection technology determined forgery primarily by relying on pattern or contrast information of the captured fingerprint image, and had limitations in that it could not sufficiently account for differences in focus and changes in frequency characteristics caused by the depth of field resulting from optical imaging characteristics.

[0007] In particular, in actual service environments, real-time forgery detection capable of immediate blocking upon capture is required to avoid compromising the user experience; however, considering the computational resources and battery constraints of the terminal, there are limitations in performing high-precision judgments on the terminal alone.

[0008] Prior art patent is Registered Patent No. 10-2840845 (Contactless fingerprint authentication method including a forged fingerprint detection function using AI), but it only provides a technology for determining whether a captured fingerprint is forged by simultaneously employing a noise pattern method, a histogram method, and an outline method. The problem to be solved

[0010] The present invention aims to provide a method and system capable of reliably detecting forged fingerprints even under various types of forgery and changes in the shooting environment by solving the above-mentioned problems by performing a first forgery detection in real-time on a fingerprint image captured in a non-contact manner at a user terminal and performing a second precise forgery judgment based on server-side unsupervised learning. means of solving the problem

[0012] To solve the above-mentioned problem, the non-contact forged fingerprint detection method using an unsupervised learning-based artificial neural network according to the present invention comprises: a step of receiving a fingerprint image captured in a non-contact manner by a user terminal; a step of first detecting whether the fingerprint image is forged in real time simultaneously with the capture by using a first artificial intelligence neural network operating in an application of the user terminal; a step of transmitting a fingerprint image not detected as forged by the first artificial intelligence neural network to a forged detection server; a step of receiving the fingerprint image through an input unit at the forged detection server; a step of performing image preprocessing including normalization, resizing, and region of interest extraction on the fingerprint image in a preprocessing unit of the forged detection server; a step of inputting the preprocessed fingerprint image into a second artificial intelligence neural network including an autoencoder structure to secondarily determine whether it is forged; and a step of transmitting information indicating that the fingerprint image is forged to the user terminal when it is determined to be forged by the second artificial intelligence neural network.

[0013] The non-contact forged fingerprint detection system using an unsupervised learning-based artificial neural network according to the present invention comprises a user terminal that captures a fingerprint image in a non-contact manner and a forgery detection server that receives a fingerprint image transmitted from the user terminal and determines whether the fingerprint image is forged. The user terminal comprises an image acquisition unit that captures a fingerprint image in a non-contact manner, a real-time forgery detection unit that takes the captured fingerprint image as input and determines whether it is forged in real-time through a first artificial intelligence neural network simultaneously with the capture to primarily detect the possibility of forgery of the fingerprint image, and a transmission control unit that controls the transmission of a fingerprint image not detected as forged by the real-time forgery detection unit to the forgery detection server. The forgery detection server comprises an input unit that receives a fingerprint image transmitted from the user terminal, a preprocessing unit that performs image preprocessing including normalization, resizing, and region of interest extraction on the fingerprint image, a forgery determination unit that determines whether the preprocessed fingerprint image is forged using an autoencoder structure, and notifies the user terminal of the forgery status when it is determined to be forged by the forgery determination unit, and when it is not determined to be forged In the case of a forgery determination unit, the forgery determination unit includes a judgment result processing unit that controls to allow subsequent authentication processing for the fingerprint image, and the forgery determination unit includes an input module that receives a preprocessed fingerprint image as input, an encoder module that encodes the features of the fingerprint image into a latent space through a feature extraction flow that is learned in an unsupervised learning manner using only normal fingerprint images and includes channel concentration, center frequency analysis, spatial concentration, and edge frequency concentration, a decoder module that reconstructs the fingerprint image by performing edge frequency restoration, spatial restoration, center frequency restoration, channel restoration, and feature restoration from the latent space, and a restoration module that calculates whether it is forged based on the restoration error and PAD score between the input fingerprint image and the reconstructed fingerprint image. Effects of the invention

[0015] The present invention provides a dual forgery detection structure that combines real-time primary forgery detection at a user terminal and secondary precise forgery judgment based on an autoencoder at a server for fingerprint images captured in a non-contact manner, thereby blocking obvious forgery fingerprints immediately upon capture while minimizing the possibility of misrecognition for actual fingerprints.

[0016] In addition, since it utilizes an autoencoder structure trained using an unsupervised learning method based solely on normal fingerprint images, it is possible to detect forgery without the need for prior collection or labeling of various types of forged fingerprint data.

[0017] In addition, by reflecting the differences in focus and frequency characteristics between the center and periphery based on the depth of field, particularly optical characteristics occurring in a non-contact shooting environment, in the forgery determination, it is possible to effectively distinguish subtle structural and statistical differences between real fingerprints and forged fingerprints.

[0018] In addition, the features of the fingerprint image are compressed into a latent space through an encoding structure that includes channel concentration, spatial concentration, and central and corner frequency analysis, and the restoration error occurring during the restoration process is utilized as a key indicator for forgery detection, thereby inducing a clear restoration failure when a forged fingerprint is input. Brief explanation of the drawing

[0019] FIG. 1 is a configuration diagram illustrating a non-contact forged fingerprint detection system using an unsupervised learning-based artificial neural network according to an embodiment of the present invention. FIG. 2 is a flowchart illustrating a non-contact forged fingerprint detection method using an unsupervised learning-based artificial neural network according to an embodiment of the present invention. FIG. 3 is a conceptual diagram illustrating the operation method of a forgery determination unit according to an embodiment of the present invention. FIG. 4 is an example diagram illustrating a photograph of an actual fingerprint and a restoration rate according to an embodiment of the present invention. FIG. 5 is an illustrative diagram explaining the photograph of a forged fingerprint and the restoration rate according to an embodiment of the present invention. FIGS. 6 and FIGS. 7 are exemplary diagrams illustrating the difference in concentration between a real fingerprint and a forged fingerprint according to an embodiment of the present invention. FIG. 8 is a diagram showing the performance evaluation results of a forged fingerprint detection model according to one embodiment of the present invention. Specific details for implementing the invention

[0020] Specific structural or functional descriptions of embodiments according to the concept of the present invention disclosed herein are provided merely for the purpose of explaining embodiments according to the concept of the present invention, and embodiments according to the concept of the present invention may be implemented in various forms and are not limited to the embodiments described herein.

[0021] Embodiments according to the concept of the present invention may be subject to various modifications and may take various forms; therefore, embodiments are illustrated in the drawings and described in detail in this specification. However, this is not intended to limit the embodiments according to the concept of the present invention to specific disclosed forms, and includes all modifications, equivalents, or substitutions that fall within the spirit and scope of the present invention.

[0022] The terms used herein are used merely to describe specific embodiments and are not intended to limit the invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this specification, terms such as "comprising" or "having" are intended to indicate the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described herein, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0023] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings attached to this specification.

[0025] FIG. 1 is a configuration diagram illustrating a non-contact forged fingerprint detection system using an unsupervised learning-based artificial neural network according to an embodiment of the present invention.

[0026] Referring to FIG. 1, a non-contact forged fingerprint detection system (10) using an unsupervised learning-based artificial neural network is composed of a forged detection server (100) that receives a fingerprint image transmitted from a user terminal and determines whether the fingerprint image is forged, and a user terminal (200) that captures a fingerprint image in a non-contact manner.

[0027] The forgery detection server (100) is composed of an input unit (110), a preprocessing unit (120), a forgery judgment unit (130), a judgment result processing unit (140), a communication unit (150), and a control unit (160).

[0028] The input unit (110) receives a fingerprint image transmitted from a user terminal. The input unit (110) may receive, for example, fingerprint image data (frame or image), image metadata (shooting time, resolution, frame ID, shooting distance information, etc.), terminal identification information, and request type (forgery determination request / subsequent authentication request, etc.) transmitted via a network, and may be configured to buffer or queue the received information in a form suitable for internal processing and transmit it to a subsequent module. Additionally, the input unit (110) may further include a reception verification function that verifies errors during transmission, data corruption, and the appropriateness of the format.

[0029] The preprocessing unit (120) performs image preprocessing on the fingerprint image, including normalization, resizing, and region of interest extraction. The preprocessing unit (120) may include brightness / contrast correction, normalization, resizing, geometric alignment (rotation / translation correction), and region of interest (ROI) extraction. Additionally, the preprocessing unit (120) performs quality inspections such as background removal, noise reduction (filtering), and judgment of blur or overexposure, and may generate a signal to induce re-photography for inputs below a certain level, or control the reliability of the input transmitted to the forgery judgment unit (130). The output of the preprocessing unit (120) may be a preprocessed fingerprint image (or preprocessed feature map) provided as input to the forgery judgment unit (130).

[0030] The forgery determination unit (130) secondarily determines whether the preprocessed fingerprint image is forged by using a second artificial intelligence neural network including an autoencoder structure. The forgery determination unit (130) can utilize the characteristic that restoration failure or an increase in restoration error occurs for a forged fingerprint input having statistical characteristics different from a normal fingerprint by using an autoencoder structure learned in an unsupervised learning manner using only normal fingerprint images.

[0031] Specifically, the forgery determination unit (130) inputs a preprocessed fingerprint image (a single captured image) into a second artificial intelligence neural network with an autoencoder structure to generate a restored image and can calculate a similarity or restoration score between the input image and the restored image. At this time, the restoration score can be set to have a higher value as the input image is a normal fingerprint image and a lower value as the input image is a forged fingerprint image. For example, if the restoration score is less than or equal to a preset threshold value (e.g., 20), it can be determined as a forged fingerprint, and if the restoration score exceeds the threshold value, it can be determined as a real fingerprint. The restoration score may be a value normalized to a range of 0 to 100 by converting the restoration error between the input image and the restored image using a predetermined function.

[0032] Accordingly, the forgery judgment unit (130) can precisely determine whether the input fingerprint image belongs to a normal distribution based on the restoration characteristics of an unsupervised autoencoder using only normal fingerprint images, and performs a second precise judgment after the first real-time forgery detection performed at the user terminal.

[0033] The forgery determination unit (130) may include an input module (131), an encoder module (133), a decoder module (135), and a restoration module (137). The input module (131) receives a preprocessed fingerprint image and provides it to the encoder module (133). The encoder module (133) encodes the features of the input fingerprint image into a feature map (z) of the latent space through a feature extraction flow including channel concentration, center frequency analysis, spatial concentration, and edge frequency concentration. The decoder module (135) reconstructs the fingerprint image using the feature map (z) of the latent space, and the restoration module (137) calculates a restoration error between the input fingerprint image and the reconstructed image, and can determine whether forgery has occurred based on whether the calculated restoration error exceeds a preset threshold. Additionally, the restoration module (137) may be configured to calculate a forgery probability (spoof score) by combining the restoration error with the PAD score calculated by sharing the latent space feature map (z) as an input feature of the PAD classifier.

[0034] The judgment result processing unit (140) controls subsequent operations according to the judgment result of the forgery judgment unit (130). If it is determined to be forgery by the forgery judgment unit, it notifies the user terminal of whether it is forgery, and if it is not determined to be forgery, it controls the subsequent authentication processing for the fingerprint image to be allowed. Specifically, if the forgery judgment unit (130) determines that it is forgery (FAKE), the judgment result processing unit (140) can control the generation of result information indicating that it is forgery (e.g., FAKE flag, forgery probability, restoration error value relative to threshold, etc.) and transmit it to the user terminal (200) through the communication unit (150). Conversely, if the forgery judgment unit (130) determines that it is normal (REAL), the judgment result processing unit (140) can control the generation of a control signal allowing subsequent authentication processing or notify the user terminal (200) or an external authentication system of "normal determination" so that the next step (e.g., fingerprint feature extraction and matching, user authentication approval procedure, etc.) is performed. In addition, the judgment result processing unit (140) can perform retry requests, notification of insufficient reliability (quality degradation), and generate events for log storage.

[0035] The communication unit (150) is configured to perform data transmission and reception with a user terminal (200), and performs the transmission and reception of messages such as fingerprint image data, judgment results, subsequent authentication permission signals, and re-shooting requests. The communication unit (150) can perform communication through a wired or wireless network (e.g., mobile communication network, Wi-Fi, Ethernet) and may further include security / reliability functions such as encryption of transmitted data, authentication token-based session management, and retransmission control.

[0036] The control unit (160) comprehensively controls the operation of each component (110 to 150) of the forgery detection server (100). The control unit (160) can schedule the processing order of input fingerprint images and manage operational parameters such as preprocessing parameters, threshold values, and model versions. Additionally, the control unit (160) can adjust the threshold values ​​to suit the operating environment (e.g., threshold values ​​by security level) or perform version management based on model updates / distribution, depending on the characteristic that the model of the forgery judgment unit (130) is a “model unsupervised learning model based only on normal fingerprint images.” Furthermore, the control unit (160) can be configured to log the results of each processing step (restoration error, PAD score, final spoof score, etc.) to be used for performance monitoring or post-analysis.

[0037] The user terminal (200) includes an image acquisition unit (210) that captures a fingerprint image in a non-contact manner, a real-time forgery detection unit (220) that takes the captured fingerprint image as input and determines whether it is forged in real time through a first artificial intelligence neural network simultaneously with the capture to primarily detect the possibility of forgery of the fingerprint image, and a transmission control unit (230) that controls the transmission of a fingerprint image not detected as forged by the real-time forgery detection unit to the forgery detection server.

[0038] According to one embodiment of the present invention, the first artificial intelligence neural network calculates a judgment score indicating the possibility of forgery for an input fingerprint image, and the judgment score may have a value in the range of 0 to 100. In this case, the higher the value of the judgment score, the higher the probability that the fingerprint image is a forged fingerprint. For example, the real-time forgery detection unit (220) may determine that the fingerprint image is a forgery (Spoof) if the judgment score is greater than or equal to a preset threshold value (e.g., 15 or higher), and determine that it is highly likely to be a real fingerprint if it is less than the threshold value. Accordingly, the user terminal (200) can block obvious forged fingerprints in real time simultaneously with the capture, and by transmitting only fingerprint images that are not determined to be forged to the forgery detection server (100), the server load can be reduced and the processing efficiency of the entire system can be improved.

[0040] FIG. 2 is a flowchart illustrating a non-contact forged fingerprint detection method using an unsupervised learning-based artificial neural network according to an embodiment of the present invention.

[0041] Referring to FIG. 2, a non-contact forged fingerprint detection method using an unsupervised learning-based artificial neural network first receives a fingerprint image captured in a non-contact manner from a user terminal (200) (S201). At this time, the fingerprint image can be acquired through a smartphone camera, tablet, kiosk, or dedicated shooting device without the finger touching the sensor, and can be input in the form of a continuous frame or a single image.

[0042] Using a first artificial intelligence neural network operating in an application of a user terminal (200), the forgery status of the fingerprint image is detected in real time at the same time as the image is captured (S203). At this stage, the first artificial intelligence neural network can be implemented with a lightweight structure and configured to enable fast inference without a capture delay, and performs the role of immediately blocking obvious forged fingerprints (e.g., screen playback images, printed materials, artificial materials, etc.).

[0043] The user terminal (200) transmits a fingerprint image that is not detected as forged by the first artificial intelligence neural network to a forgery detection server (S205). Accordingly, inputs with a high probability of forgery are preemptively blocked at the terminal stage, and only fingerprint images with relatively high reliability are transmitted to the server, thereby reducing server load and unnecessary computation.

[0044] The forgery detection server (100) receives the fingerprint image through the input unit (110) (S207). At this time, the input unit (110) can receive metadata related to shooting and request information along with the fingerprint image data, and can verify the integrity and format conformity of the received data.

[0045] In the preprocessing unit (120) of the forgery detection server (100), image preprocessing including normalization, resizing, and region of interest extraction is performed on the fingerprint image (S209). This step serves to correct for variations caused by the shooting environment, lighting conditions, or differences in the user's hand position, thereby ensuring that a stable judgment is made in the subsequent forgery judgment step.

[0046] The forgery judgment unit (130) of the forgery detection server (100) inputs the preprocessed fingerprint image into a second artificial intelligence neural network including an autoencoder structure to make a second determination of whether it is forged (S211). At this time, the second artificial intelligence neural network is a model trained in an unsupervised learning manner using only normal fingerprint images, generates a restored image for the input fingerprint image, and calculates a PAD score based on the restoration error and latent spatial features between the input image and the restored image.

[0047] When the forgery judgment unit (130) of the forgery detection server (100) determines that the fingerprint image is forged by the second artificial intelligence neural network, it transmits information indicating that the fingerprint image is forged to the user terminal (S213). Accordingly, for the fingerprint image determined to be forged, subsequent authentication procedures may be blocked or additional security measures may be performed.

[0048] If the forgery judgment unit (130) of the forgery detection server (100) is not determined to be forgery by the second artificial intelligence neural network, it extracts fingerprint features from the fingerprint image and matches the extracted fingerprint features with previously registered fingerprint information (S215). For example, fingerprint feature extraction and matching with previously registered fingerprint information may be performed, or the result of the normal determination may be transmitted to an external authentication system.

[0049] The second artificial intelligence neural network described above includes an autoencoder structure trained using an unsupervised learning method with only normal fingerprint images, generates a reconstructed image using the preprocessed fingerprint image as an input image, calculates a reconstruction error between the input image and the reconstructed image, and determines whether it is forged based on whether the reconstruction error exceeds a preset threshold.

[0050] The above-described autoencoder structure includes an encoding step that encodes the features of a fingerprint image into a latent space by sequentially performing channel concentration, central frequency analysis, spatial concentration, and edge frequency concentration after extracting features from a fingerprint image, and a decoder step that restores a fingerprint image by sequentially performing edge frequency restoration, spatial restoration, central frequency restoration, channel restoration, and feature restoration from the latent space.

[0051] The encoder step is configured to remove noise and background from the fingerprint image by applying CBAM including spatial concentration and channel concentration, the latent space is fixed as a latent map of size 4×4×2048, and the restoration error is combined with the PAD score to calculate the forgery probability, thereby simultaneously performing restoration-based judgment and PAD judgment.

[0053] FIG. 3 is a conceptual diagram illustrating the operation method of a forgery determination unit according to an embodiment of the present invention. Referring to FIG. 3, the forgery determination unit (130) includes an input module (131) that receives a fingerprint image preprocessed in a preprocessing unit (120) as input, an encoder module (133) that generates latent space features from the fingerprint image, a decoder module (135) that reconstructs the fingerprint image from the latent space features, and a restoration module (137) that calculates a restoration error between the input image and the reconstructed image.

[0054] The input module (131) provides the preprocessed fingerprint image to the encoder module (133).

[0055] The encoder module (133) includes a feature extraction step for extracting features from a fingerprint image, a channel focus step for emphasizing significant features in the channel direction, a central frequency analysis step for analyzing frequency components in the central region of the fingerprint image, a spatial focus step for emphasizing spatially important regions, and a corner frequency focus step for emphasizing frequency components in the corner region of the fingerprint image, thereby encoding the features of the input fingerprint image into a feature map (z) of the latent space. At this time, the encoder module (133) may be configured to suppress background and noise components by applying spatial and channel attention, and the feature map (z) of the latent space may be fixed to a predetermined size (e.g., 4×4×2048) for real-time processing. In particular, the encoder module (133) may be configured to reflect optical characteristics according to the depth of field occurring in a non-contact shooting environment. In the case of an actual fingerprint image, the core area of ​​the fingerprint is relatively clearly focused when captured, and as it moves toward the outer or corner areas, it contains natural blur components, and accordingly, the frequency distribution between the center and the periphery changes gradually. On the other hand, since a forged fingerprint image is generated using printed materials, display playback images, or artificial materials, it lacks natural frequency changes due to such depth of field, or tends to exhibit abnormal frequency concentration or abrupt pattern changes in the center and corner areas. The encoder module (133) of the present invention extracts these differences as features through a central frequency analysis step and a corner frequency concentration step, and by encoding the extracted frequency-based features into a latent space, it induces a relatively large restoration error for the forged fingerprint during the restoration process by the decoder module (135).

[0056] The decoder module (135) receives a latent spatial feature map (z) generated by the encoder module (133) and reconstructs the input fingerprint image through a reconstruction flow that includes a corner frequency reconstruction step, a spatial reconstruction step, a center frequency reconstruction step, a channel reconstruction step, and a feature reconstruction step. The decoder module (135) may be configured to reduce resolution loss by using a structure that combines upsampling and convolution during the reconstruction process.

[0057] The restoration module (137) calculates a restoration error (restoration loss) based on the difference between the fingerprint image input through the input module (131) and the fingerprint image reconstructed by the decoder module (135), and can determine whether the image is forged based on whether the calculated restoration error exceeds a preset threshold. Additionally, the restoration module (137) can be configured to calculate a forgery probability (spoof score) by combining the restoration error with the PAD score calculated by sharing the latent spatial feature map (z) as an input feature of the PAD classifier. Accordingly, when the input fingerprint image is abnormally restored (increase in restoration error) in light of the “normal restoration characteristics” of an autoencoder trained using only normal fingerprint images in an unsupervised learning method, it is determined to be forged, while the accuracy of forgery detection can be improved through combination with the PAD score.

[0059] FIG. 4 is an example diagram illustrating a photograph of an actual fingerprint and a restoration rate according to an embodiment of the present invention.

[0060] Referring to FIG. 4, actual fingerprint images acquired in a non-contact manner by a user terminal (200) or a shooting device are exemplified at the top, and restored images in which each actual fingerprint image is restored through a forgery judgment unit (130) of a forgery detection server (100) are exemplified at the bottom. Each example image may also display a restoration rate calculated based on the difference between the corresponding input image and the restored image.

[0061] In an embodiment of the present invention, the forgery determination unit (130) includes an autoencoder structure learned in an unsupervised learning manner using only normal fingerprint images. Accordingly, since the actual fingerprint image has characteristics similar to the normal distribution learned by the autoencoder, the reconstructed image reconstructed by the decoder module (135) can be output in a form that relatively well maintains the main features of the input image (e.g., ridge pattern and directionality). As a result, the reconstruction error for the actual fingerprint image tends to be calculated as a relatively low value.

[0062] For example, in the case of actual fingerprint images illustrated in FIG. 4, the restoration rate may be calculated within a predetermined range (e.g., a low value in the 0.001 range), which may be determined to be below a threshold value set by the forgery judgment unit (130). Accordingly, the forgery judgment unit (130) determines that the input fingerprint image is likely to be genuine rather than forged, and the judgment result processing unit (140) can control the process to allow subsequent authentication processing. That is, FIG. 4 provides an example in which the autoencoder-based forgery judgment method of the present invention stably passes genuine fingerprints by calculating a low restoration error for genuine fingerprint inputs.

[0063] In addition, the examples in FIG. 4 visually support the flow in which a second judgment by the forgery detection server (100) determines that an actual fingerprint image transmitted to the forgery detection server (100) is normal according to the first detection result of the real-time forgery detection unit (220) performed at the user terminal (200). Through this, the present invention can improve forgery detection performance while reducing misrecognition of the actual fingerprint by combining the real-time first detection on the terminal side and the precise second judgment on the server side.

[0065] FIG. 5 is an illustrative diagram explaining the photograph of a forged fingerprint and the restoration rate according to an embodiment of the present invention.

[0066] Referring to FIG. 5, FIG. 5 is a diagram exemplarily illustrating a captured image of a forged fingerprint (FAKE) according to an embodiment of the present invention, an autoencoder-based restoration result of a forged judgment unit (130), and a restoration rate. Referring to FIG. 5, at the top, input fingerprint images captured in a non-contact manner are exemplified, such as forged fingerprints generated using silicone, gelatin, printed material, artificial material, or image playback medium, and at the bottom, restored images are exemplified, such as each forged fingerprint image restored through the forged judgment unit (130) of the forged detection server (100). In each example image, a restoration rate calculated based on the difference between the input image and the restored image may be displayed together.

[0067] In an embodiment of the present invention, the forgery determination unit (130) includes an autoencoder structure learned in an unsupervised learning manner using only normal fingerprint images, and thus has the characteristic of not being able to accurately reconstruct the input image for a forged fingerprint image having a texture, frequency characteristics, or spatial pattern different from a normal fingerprint. Accordingly, during the process of restoring the forged fingerprint image by the decoder module (135), the fingerprint ridge structure tends to become blurred or distorted, and local shape information tends to be lost.

[0068] As a result, in the case of the forged fingerprint images shown in FIG. 5, a large difference is observed between the input image and the restored image, and accordingly, the restoration rate can be calculated as a relatively large value (e.g., 0.003 or higher, or a value in the 0.01 range). This restoration rate may be determined to exceed a threshold value preset in the forgery judgment unit (130), and accordingly, the forgery judgment unit (130) determines that the fingerprint image is highly likely to be forged (FAKE). The judgment result processing unit (140) can notify the user terminal (200) of the result determined to be forged, thereby controlling the blocking of subsequent authentication procedures or inducing additional security procedures.

[0069] In addition, the examples in FIG. 5 visually demonstrate the characteristic that the restoration error for forged fingerprints increases significantly compared to actual fingerprints (Fig. 4), which have a relatively small restoration error. Through this, it can be confirmed that the autoencoder-based forgery detection method of the present invention can effectively capture abnormal patterns of forged fingerprints as restoration failures. Furthermore, by calculating the probability of forgery by combining the restoration error and the PAD score, stable forgery detection performance can be provided for various types of forged fingerprints.

[0071] FIGS. 6 and FIGS. 7 are exemplary diagrams illustrating the difference in concentration between a real fingerprint and a forged fingerprint according to an embodiment of the present invention.

[0072] Referring to FIGS. 6 and FIGS. 7, an original fingerprint image captured in a non-contact manner is shown at the top, and at the bottom, a concentration map (e.g., attention map, activation map, or heat map) generated by the encoder module (133) of the forgery judgment unit (130) is visualized in correspondence with the original fingerprint image.

[0073] FIG. 6 is an example of an actual fingerprint, showing that the encoder module (133) forms a relatively uniform and continuous concentration centered on the fingerprint ridge structure and core area. In the case of an actual fingerprint, since the periodicity, directionality, and spatial continuity of the fingerprint ridges are naturally maintained, meaningful features tend to be concentrated in the fingerprint center area and the main ridge distribution area during the channel concentration and spatial concentration stages. Accordingly, the concentration map is naturally distributed throughout the fingerprint area, and relatively low concentration appears in the background area or boundary area.

[0074] On the other hand, FIG. 7 is an example of a forged fingerprint, showing that the concentration map of the encoder module (133) is scattered around the outer edge, boundary, or irregular area of ​​the fingerprint rather than the center of the fingerprint, or is excessively concentrated in specific local areas. This is because the forged fingerprint contains frequency characteristics, surface reflection characteristics, or discontinuous texture patterns that differ from the actual fingerprint as it is generated from printed materials, artificial materials, video playback media, etc. As a result, the encoder module (133) forms an unstable attention distribution for areas that do not match the features learned from the normal fingerprint.

[0075] As such, the difference in the concentration maps illustrated in FIGS. 6 and FIGS. 7 visually reveals the structural and statistical characteristics of the actual fingerprint and the forged fingerprint, and explains that the forgery judgment unit (130) of the present invention can effectively distinguish the forged fingerprint through an encoding structure that combines channel concentration, central frequency analysis, spatial concentration, and corner frequency concentration. In particular, while concentration is stably formed in meaningful feature regions in the actual fingerprint, concentration tends to be dispersed or distorted in the forged fingerprint; therefore, this difference in concentration leads to an increase in restoration error and an increase in the PAD score, thereby providing a technical basis for improving the reliability of forgery judgment.

[0077] FIG. 8 is a diagram showing the performance evaluation results of a forged fingerprint detection model according to one embodiment of the present invention.

[0078] Referring to FIG. 8, performance indicators according to the international standards ISO / IEC 30107-3 (PAD performance criteria) and NIST criteria are shown for the forged fingerprint detection model of the present invention.

[0079] The top of Figure 8 shows the level-specific criteria for PAD performance evaluation in the form of a table, and the acceptance criteria for Attack Presentation Classification Error Rate (APCER) and Bona-fide Presentation Classification Error Rate (BPCER) are presented for each level (Level 1 to Level 3). For example, Level 3 requires a high level of security, where the APCER must be 1% or less and the BPCER must be 5% or less.

[0080] The bottom of FIG. 8 shows the results of applying the forged fingerprint detection model of the present invention to multiple datasets in the form of a bar graph. In the graph, APCER and BPCER are displayed for comparison for each dataset, and it can be confirmed that the model according to the embodiment of the present invention exhibits stable forged fingerprint detection performance for various shooting environments (e.g., monitor playback video, mobile phone shooting video, printed-based forged fingerprint, etc.) and different types of forged fingerprints.

[0081] In particular, as illustrated in FIG. 8, the autoencoder-based forgery judgment unit (130) of the present invention, despite being trained using only normal fingerprint images in an unsupervised learning manner, shows a result of maintaining a low APCER for forged fingerprints while achieving a low BPCER for normal fingerprints. This means that a structure combining forgery judgment using the restoration error between the input fingerprint image and the restored image, and a PAD score using latent spatial features, effectively distinguishes between forged fingerprints and real fingerprints.

[0082] Accordingly, FIG. 8 provides experimental evidence demonstrating that the unsupervised learning-based contactless counterfeit fingerprint detection method and system of the present invention can meet or exceed international standard-level PAD performance requirements, and visually illustrates that it is a highly reliable counterfeit detection technology applicable in various real-world environments.

[0084] Meanwhile, according to one embodiment of the present disclosure, the various embodiments described above may be implemented as software comprising instructions stored on a machine-readable storage medium (e.g., a computer). The machine may include an electronic device according to the disclosed embodiments, which is a device capable of calling instructions stored from the storage medium and operating according to the called instructions. When instructions are executed by a processor, the processor may perform a function corresponding to the instructions directly or by using other components under the control of the processor. Instructions may include code generated or executed by a compiler or an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, "non-transitory" means only that the storage medium does not contain a signal and is tangible, and does not distinguish whether data is stored semi-permanently or temporarily in the storage medium.

[0085] Additionally, according to one embodiment of the present disclosure, the method according to the various embodiments described above may be provided as included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or online through an application store (e.g., Play Store™). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created in a storage medium such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0086] Additionally, according to one embodiment of the present disclosure, the various embodiments described above may be implemented in a recording medium readable by a computer or a similar device using software, hardware, or a combination thereof. In some cases, the embodiments described herein may be implemented as the processor itself. According to a software implementation, embodiments such as the procedures and functions described herein may be implemented as separate software modules. Each of the software modules may perform one or more functions and operations described herein.

[0087] Meanwhile, computer instructions for performing processing operations of the device according to the various embodiments described above may be stored in a non-transitory computer-readable medium. When computer instructions stored in such a non-transitory computer-readable medium are executed by the processor of a specific device, they cause the specific device to perform processing operations in the device according to the various embodiments described above. A non-transitory computer-readable medium refers to a medium that stores data semi-permanently and is readable by a device, rather than a medium that stores data for a short moment, such as a register, cache, or memory. Specific examples of a non-transitory computer-readable medium may include CDs, DVDs, hard disks, Blu-ray discs, USBs, memory cards, ROMs, etc.

[0088] Additionally, each component (e.g., module or program) according to the various embodiments described above may be composed of a single or multiple entities, and some of the aforementioned sub-components may be omitted, or other sub-components may be additionally included in the various embodiments. Generally or additionally, some components (e.g., module or program) may be integrated into a single entity to perform the same or similar functions as those performed by each of the respective components prior to integration. The operations performed by the module, program, or other components according to the various embodiments may be executed sequentially, in parallel, iteratively, or heuristically, or at least some operations may be executed in a different order, omitted, or other operations added.

[0089] The invention has been described with reference to embodiments illustrated in the drawings, but this is merely illustrative, and those skilled in the art will understand that various modifications and equivalent alternative embodiments are possible therefrom. Accordingly, the true technical scope of protection of the invention should be determined by the technical spirit of the appended claims. Explanation of the symbols

[0091] 10: Contactless counterfeit fingerprint detection system 100: Counterfeit detection server 110: Input unit 120: Preprocessing Unit 130: Forgery Detection Unit 131: Input Module 133: Encoder Module 135: Decoder Module 137: Restoration Module 140: Judgment result processing unit 150: Communication unit 160: Control unit 200: User terminal 210: Image acquisition unit 220: Real-time forgery detection unit 230: Transmission control unit

Claims

Claim 1 A non-contact forged fingerprint detection method using an unsupervised learning-based artificial neural network comprises: a step of receiving a fingerprint image captured in a non-contact manner by a user terminal; a step of first detecting whether the fingerprint image is forged in real time simultaneously with the capture using a first artificial intelligence neural network operating in an application of the user terminal; a step of transmitting a fingerprint image not detected as forged by the first artificial intelligence neural network to a forged detection server; a step of receiving the fingerprint image through an input unit at the forged detection server; a step of performing image preprocessing on the fingerprint image, including normalization, resizing, and region of interest extraction, in a preprocessing unit of the forged detection server; a step of inputting the preprocessed fingerprint image into a second artificial intelligence neural network including an autoencoder structure in a forged determination unit of the forged detection server to secondarily determine whether it is forged; and a step of transmitting information indicating that the fingerprint image is forged to the user terminal if it is determined to be forged by the second artificial intelligence neural network. And if it is not determined to be forged by the second artificial intelligence neural network, the method further includes the step of extracting fingerprint features from the fingerprint image and matching the extracted fingerprint features with previously registered fingerprint information, wherein the second artificial intelligence neural network includes an autoencoder structure trained in an unsupervised learning manner using only normal fingerprint images, generates a restored image using the preprocessed fingerprint image as an input image, calculates a restoration error between the input image and the restored image, and determines whether it is forged based on whether the restoration error exceeds a preset threshold, and the autoencoder structure includes an encoder step of encoding the features of the fingerprint image into a latent space by sequentially performing channel concentration, center frequency analysis, spatial concentration, and edge frequency concentration after extracting features from the fingerprint image;A non-contact forged fingerprint detection method characterized by comprising a decoder step that restores a fingerprint image by sequentially performing edge frequency restoration, spatial restoration, center frequency restoration, channel restoration, and feature restoration from the above latent space, wherein the encoder step is configured to remove noise and background of the fingerprint image by applying CBAM including spatial concentration and channel concentration, wherein the latent space is fixed as a latent map of size 4×4×2048, and the restoration error is combined with the PAD score to calculate a forgery probability, thereby simultaneously performing restoration-based judgment and PAD judgment, wherein the unsupervised learning-based artificial neural network refers to a second artificial intelligence neural network, and is executed by the forgery judgment unit of the forgery detection server. Claim 2 delete Claim 3 delete Claim 4 delete Claim 5 A non-contact forged fingerprint detection system using an unsupervised learning-based artificial neural network comprises: a user terminal that captures a fingerprint image in a non-contact manner; and a forged detection server that receives a fingerprint image transmitted from the user terminal and determines whether the fingerprint image is forged. The user terminal comprises: an image acquisition unit that captures a fingerprint image in a non-contact manner; a real-time forged detection unit that takes the captured fingerprint image as input and determines whether it is forged in real-time through a first artificial intelligence neural network simultaneously with the capture to primarily detect the possibility of forgery of the fingerprint image; and a transmission control unit that controls the transmission of a fingerprint image not detected as forged by the real-time forged detection unit to the forged detection server. The forged detection server comprises: an input unit that receives a fingerprint image transmitted from the user terminal; a preprocessing unit that performs image preprocessing including normalization, resizing, and region of interest extraction on the fingerprint image; and a forged determination unit that determines whether the preprocessed fingerprint image is forged using an autoencoder structure. The forgery determination unit includes a judgment result processing unit that notifies the user terminal of the forgery status when it is determined to be forgery by the forgery determination unit, and controls to allow subsequent authentication processing for the fingerprint image when it is not determined to be forgery. The forgery determination unit includes: an input module that receives a preprocessed fingerprint image as input; an encoder module that encodes the features of the fingerprint image into a latent space through a feature extraction flow that is learned in an unsupervised learning manner using only normal fingerprint images and includes channel concentration, center frequency analysis, spatial concentration, and edge frequency concentration; and a decoder module that reconstructs the fingerprint image by performing edge frequency restoration, spatial restoration, center frequency restoration, channel restoration, and feature restoration from the latent space.A non-contact forged fingerprint detection system comprising a restoration module that calculates whether a fingerprint is forged based on a restoration error and a PAD score between an input fingerprint image and a reconstructed fingerprint image, wherein the unsupervised learning-based artificial neural network is an autoencoder structure trained in an unsupervised learning manner using only normal fingerprint images, and is implemented in the forgery judgment unit of the forgery detection server.

Images