System for restoring nipple prints in images

Abstract

To provide an automatic fingerprint identification system that restores a nipple print by generating a restored image that does not contain any artifacts or parasitic nipple prints that may be present in the original image. The automatic fingerprint identification system includes a data processing device 600 configured to reconstruct a papilla print image in an image, the data processing device 600 performing the following steps: (a) applying a convolutional encoding neural network to an image including at least one papilla print image and a direction field map of the papilla print image as input data to generate an encoded vector; and (b) applying a convolutional decoding neural network to the encoded vector to generate a reconstructed image.

Description

[Technical Field]

[0001] The present invention relates to a system for recovering a fingerprint in an image. [Background technology]

[0002] Fingerprint identification is a method of identifying individuals using fingerprints (also known as "nipple prints," which includes fingerprints and palm prints). This method is used in connection with the investigation of crimes and criminal cases, particularly in forensic anthropometry services.

[0003] A fingerprint is a pattern formed by the dermatoglyphics of the skin on the hands and fingers when they come into contact with a surface. Dermatoglyphs are shallow ridges arranged in lines or spirals, formed by ridges of the skin on the palms of the hands, soles of the feet, and pads of the fingers. They are unique to each individual, and the pattern they form serves as an "anthropometric identification card" that identifies the individual.

[0004] The collection of fingerprints by forensic services is common practice in criminal investigations. In this context, fingerprints take the form of digital, palmar, or plantar "impressions," commonly referred to as dermatoglyphs or "nipple impressions" derived from dermatoglyphs.

[0005] There are three types of "nipple prints": visible nipple prints, latent nipple prints, and molded nipple prints. Visible nipple prints are marks that are directly visible without external intervention. They are called "positive" when they are formed by the deposition of a substance, such as blood, oily substances, or ink-stained finger marks. They are called "negative" when they are formed by the removal of a substance, for example, marks on a dusty surface. Latent nipple prints are marks that are not visible to the naked eye and can only be observed using a detection method, for example, due to sweat deposits or nipple secretions present on the nipple ridge. Molded nipple prints are three-dimensional marks that result from pressing a finger against a soft surface.

[0006] To identify an individual from a nipple print, the print must be compared with a large number of previously acquired fingerprints (1:N) from multiple individuals. Due to the complex nature of fingerprints and the large number of comparisons required for identification, the identification process can take a long time, despite the computational resources of current data processing systems. To reduce the time required for this operation, a common approach is to classify fingerprints into different classes based on their morphological features (dermatoglyphs). Examples of these morphological features include the general shape of the skin print (e.g., loop, arch, spiral orientation), according to the categories defined by Henry Falz, Francis Galton, and Edward Henry, as well as the overall pattern of wrinkles and "miniature" features consisting of single points and / or discontinuities along the wrinkles (e.g., wrinkle terminations, bifurcations), wrinkle shape, pores, scars, etc.

[0007] Fingerprint identification systems are classified as manual, semi-automated, or fully automated. Due to the increasing size of databases and the increasing computational power of data processing equipment, Automated Fingerprint Identification Systems (AFIS) are becoming increasingly widely adopted. These allow for the ability to quickly and efficiently analyze a list of candidate fingerprints that may match the owner of the fingerprint to be identified.

[0008] However, images of nipple prints used to identify individuals using automatic identification systems can be affected by artifacts such as distortions and foreign objects. The presence of these artifacts can reduce identification performance. These artifacts can arise from the conditions under which the nipple prints were formed and / or the collection method. For example, the presence of a graphic pattern on the surface on which the nipple prints are present can interfere with the recognition of the morphological features of the nipple prints. If the substrate is a sheet of paper containing text or photographs, the fingerprint may overlap the text or image, and these underlying elements may interfere with the graphic elements representing the fingerprint's morphological features. A fingerprint image may contain multiple fingerprints that partially overlap each other. Pre-correction and / or segmentation operations are typically required to distinguish the fingerprints from the underlying surface background or from each other if they overlap.

[0009] FR 3 102 600 A1 [IDEMIA IDENTITY & SECURITY FRANCE [FR]] 30.04.2021 describes a method for segmenting an image representing at least one nipple print. The method includes first applying a convolutional neural network to the image to generate an orientation map and a first segmentation mask. Next, the orientation map and the first segmentation mask are combined using a convolutional neural network to obtain a second segmentation mask. Finally, the second segmentation mask is applied to the image of the nipple print to perform segmentation.

[0010] Saponara et al. (2021). Fingerprint Image Reconstruction with a Convolutional Neural Network Autoencoder Architecture. IEEE Access, pp. 147888-147899, describes a convolutional autoencoder neural network for reconstructing fingerprint images from degraded or complex images. Summary of the Invention

[0011] State-of-the-art segmentation or reconstruction techniques can detect fingerprints in an image by providing a focused image of the fingerprint. This focused image may take the form of an image cropped from the region corresponding to the fingerprint or a synthetic image that "artificially recreates" the fingerprint's morphological characteristics, among other things. However, these techniques may not properly remove interfering signals, such as graphic or text elements or fragments of other papillae prints that may affect the fingerprint itself, or they may do so at the cost of significantly losing important information about the fingerprint's morphological features. The negative impact of this is that these unwanted signals and information loss can hinder the ability of an automated identification system to identify the fingerprint. The system may generate erroneous results or fail to identify fingerprints that match a reference fingerprint database.

[0012] Forensic experts are known to characterize fingerprints with a level of insight that automated systems cannot possess. In particular, they can identify fingerprint characteristics from interfering elements within fingerprint images, characteristics that may not be modeled or may be inadequately modeled by the algorithms of automated identification systems. However, the inflexibility of current identification processes prevents them from considering relevant information that a human operator, such as a forensic fingerprint expert, can independently establish.

[0013] The invention relates to a system for restoring a nipple print in an image according to claim 1, the dependent claims being advantageous embodiments. [Brief explanation of the drawings]

[0014] [Figure 1] Figure 1 is a schematic diagram of a fingerprint. [Figure 2] Figure 2 shows an example of a nipple print image containing artifacts and an example of the superposition of two nipple prints. [Figure 3] FIG. 3 is a schematic diagram illustrating an example of an orientation field map of the nipple print of the example image of FIG. [Figure 4] FIG. 4 is a flow chart of the procedure carried out in accordance with the present invention. [Figure 5] FIG. 5 is a schematic diagram illustrating an example of a segmentation map of the nipple print in the example image of FIG. [Figure 6] Figure 6 shows a schematic diagram of the data processing unit. DETAILED DESCRIPTION OF THE INVENTION

[0015] Referring to Figure 1, in image I100 of a nipple print **100**, the nipple print 100 appears as a pattern formed by the traces left on the surface by the skin print of a finger. This pattern represents the curves of the ridges of the papillae 101 and the apexes 102, which are papillae or epidermal folds present on the finger pad. These curves take on a variety of geometric shapes, primarily lines, loops, and spirals.

[0016] In this specification, "nipple print" means a visible, latent or molded trace of a nipple print related to a fingerprint or palm print (including acquisition by an electronic contact acquisition device or recording on a paper medium after application of ink), and for convenience it is taken to include "fingerprints."

[0017] Depending on the conditions under which the nipple print was formed and / or the method of collection, the image of the nipple print 100 may be affected by artifacts or may contain multiple overlapping nipple prints. In a first example, referring to Figure 2(a), the artifact is a graphic and / or text element 201 (represented in the form of a horizontal black band) contained in the background consisting of the surface on which the nipple print 100 was formed (in this case, a sheet of paper with typed text printed on it). In a second example, referring to Figure 2(b), the image contains two partially overlapping nipple prints 100, 202.

[0018] Referring to Figure 3, the curvature of the papillary ridge 102 of the papillary print 100 shown in Figure 1 can be represented in the form of an orientation field map 300 of the papillary ridge 102. The orientation field map (also known as a ridge flow map or orientation map) of the papillary print represents the local orientation of the papillary ridge at each pixel or group of pixels in the image representing the papillary print. These orientations are typically expressed as angles between 0° and 180° relative to a reference direction (usually horizontal) for the image. In this representation, adjacent papillary ridges oriented at 0° and 180°, respectively, are indistinguishable from each other. The orientation field map can be obtained manually by a human operator or automatically using image processing techniques such as those described in Hong, L., & Jain, A. (1999). Fingerprint Image Classification. Proceedings of the Scandinavian Conference on Image Analysis, Vol. 2, pp. 665-672.

[0019] With reference to Figure 4, the present invention relates to an automated fingerprint identification system including a data processing device 600 adapted to recover a nipple print impression 100 in an image 1100. The system performs the following steps: (a) 401 applying a convolutional coding neural network using as input data an image I100 containing at least one nipple print 100 and a direction field map 300 of the nipple print 100 to generate an encoded vector O401; (b) 402 Apply a convolutional decoding neural network to the encoded vector O401 to generate a restored image O402**.

[0020] Unlike conventional segmentation techniques, the system of the present invention restores, or reveals, the nipple print 100 by providing a restored image O402 that is free of any artifacts or parasitic nipple prints that may be present in the original image I100. In other words, the system of the present invention allows one to "focus" on the nipple print 100 of interest in image I100, ideally producing a restored image O402 that represents only the nipple print 100 to be identified.

[0021] The map 300 of orientation regions of the nipple print 100 is obtained using any suitable method, preferably not automatically by applying algorithm-based image processing techniques.

[0022] In certain preferred embodiments, a direction field map 300 of the papilla print 100 is manually determined from an image I100 of the papilla print 100. In particular, the direction field map 300 can be manually created by a human operator, such as a forensic fingerprint examiner. Such a map 300 is likely to represent ridge directions in the papilla trace 100 that automated image processing algorithms cannot identify due to artifacts present in the papilla trace 100 or overlapping with other papilla traces. This map 300, along with the flexibility of the process 400 of the first aspect of the invention, makes it possible to obtain a reconstructed image O402 of the papilla print 100 that includes morphological features that may be hidden by the algorithms of an automated identification system. These characteristics may be crucial in an identification task.

[0023] In a particular embodiment, referring to FIG. 5, the convolutional coding neural network also receives as input a segmentation map 500 of the nipple print 100. The function of the segmentation map 500 is to mask characteristics of portions of the image I100 that are determined not to belong to the nipple print 100. The segmentation map 500 is typically a binary map, meaning that pixel values ​​are either 0 or 1. In the two examples of FIG. 5, the segmentation map 500 displays only the portion of the image I100 that corresponds to the nipple print 100.

[0024] The use of 500 segmentation maps is advantageous when there are a large number of artifactual and / or extraneous papillae prints overlapping the primary 100 papillae prints, making it very difficult to accurately distinguish their morphological features. The combination of 300 direction field maps of the 100 papillae prints and 500 segmentation maps improves the performance of the encoding and decoding convolutional neural networks, strengthening their "focus" on the I100 image characteristics that are important for recovering the 100 papillae prints for identification purposes.

[0025] Similar to the 300 orientation field map of the papilla print 100, the 500 segmentation map is obtained using an appropriate method. Preferably, it is not obtained automatically. Ideally, the 500 segmentation map is obtained using image processing methods assisted by a human operator, such as a forensic fingerprint expert. Examples of segmentation processes include: Watershed; Mask R-CNN, described in He et al. (2017). Mask R-CNN. In Proceedings of the IEEE international conference on computer vision, pp. 2961-2969; and GrabCut, described in Rother et al. (2004). GrabCut: Interactive foreground extraction using iterative graph cuts. ACM Transactions on Graphics (TOG), 23(3), 309-314. The 500 segmentation map can also be generated manually by a human operator, such as a forensic fingerprint expert.

[0026] The convolutional encoding neural network encodes the image I100 as an encoded vector O401, and the convolutional decoding neural network reconstructs the restored image O402 from the encoded vector O401. These two networks can constitute two parts of the same convolutional neural network architecture, under supervised or unsupervised learning, particularly in the form of a convolutional autoencoding neural network.

[0027] In a particular embodiment, the convolutional encoding neural network and the convolutional decoding neural network are residual convolutional neural networks, specifically ResNet-type networks. The architecture of residual convolutional neural networks is described in He et al. (2016) Deep residual learning for image recognition, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770-78.

[0028] The convolutional neural network encoder and the convolutional neural network decoder are trained in any suitable manner. In some embodiments, the convolutional neural network encoder and the convolutional neural network decoder are pre-trained on a set of fingerprint images that have been degraded by adding graphic and / or text elements and / or fingerprints of other people. The training is supervised using a cost function (e.g., Manhattan L1 distance) between the restored image and the original, undegraded image.

[0029] Referring to Figure 6, the system includes a data processing device. The processing device 600 is responsible for automatically performing a sequence of arithmetic or logical operations to perform a task or operation. This device is commonly referred to as a computer and comprises one or more central processing units (CPUs) 601 and / or one or more graphics processing units (GPUs) 602, a physical remote communication module 603, one or more physical input / output modules 604 for exchanging data with external devices, a temporary storage medium 605 such as random access memory (RAM), a non-transitory storage medium 606, and a communication bus (not shown) for transferring data between the device's internal components.

[0030] The data processing** device 600 is capable of executing program modules that, when executed, cause the device 600 to implement the method of the first aspect of the invention. The program modules may be written in any programming language, compiled or interpreted. They may be part of a software solution that is a collection of executable instructions, code, scripts, or the like, or may include a database.

[0031] In an embodiment, the data processing device 600 is further configured to perform the following steps: (c) Identifying the nipple print 100 in the recovered image O402 by comparison with a set of reference fingerprints.

[0032] The fingerprint identification step can be performed using any suitable method, for example, using an automatic identification system such as those described in EP 0 366 850 A1 [MORPHO SYSTEM LTD CORP [FR]] September 5, 1990, US 5 465 303 A [AEROFLEX SYST CORP [US]] November 7, 1995, or US 2003 / 091724 A1 NEC CORP [US] May 15, 2003.

[0033] References EP 0 366 850 A1 [MORPHO SYSTEM LTD CORP [FR]] May 9, 1990. US 5 465 303 A [AEROFLEX SYST CORP [US]] November 7, 1995. US 2003 / 091724 A1 NEC CORP [US] May 15, 2003. FR 3 102 600 A1 [IDEMIA IDENTITY & SECURITY FRANCE [FR]] April 30, 2021. Hong, L., & Jain, A. (1999). Fingerprint image classification. Proceedings of the Scandinavian Image Analysis Conference, Vol. 2, pp. 665-672. (2004). "GrabCut": Interactive Foreground Extraction Using Iterative Graph Cuts. ACM Transactions on Graphics (TOG), 23(3), 309-314. He et al., (2016) Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.770-778. He, et al. (2017). Mask R-CNN. Proceedings of the IEEE International Conference on Computer Vision, pp. 2961-2969. Saponara et al. (2021). Fingerprint Image Reconstruction Using a Convolutional Neural Network Autoencoder Architecture. IEEE Access, pp. 147888-147899.

Claims

1. The automatic fingerprint identification system comprises a data processing device (600) configured to reconstruct a papilla print impression (100) in an image (I100), and performs the following steps: (a) applying a convolutional coding neural network to receive as input an image (I100) containing at least one nipple print (100) and a direction field map (300) of the nipple print (100) as input data to generate a coded vector (O401) (401); (b) A convolutional decoding neural network is applied to the coded vector (O401) to generate a reconstructed image (O402) (402).

2. In the automatic fingerprint identification system according to claim 1, the orientation field map (300) of the papilla print impression (100) is determined manually from an image (I100) of the papilla print impression (100).

3. In the automatic fingerprint identification system according to claim 1 or 2, the encoding convolutional neural network further takes in as input data a segmentation map (500) of the papilla print (100).

4. In the fingerprint automatic identification system as claimed in claim 1, the convolutional encoding neural network and the convolutional decoding neural network are residual convolutional neural networks.

5. 2. The system for automatic fingerprint identification according to claim 1, wherein the image (I100) containing at least one papilla print impression (100) contains at least two partially overlapping papilla print impressions.

6. An automatic fingerprint identification system according to claim 1, wherein the image (I100) containing at least one papilla print impression (100) contains a background consisting of graphic and / or textual elements.

7. In the automatic fingerprint identification system according to claim 1, the convolutional encoding neural network and the convolutional decoding neural network are pre-trained using a set of images of nipple prints that have been degraded by adding graphic and / or textual elements and / or a set of images of nipple prints of others.

8. 10. The automated fingerprint identification system according to claim 1, wherein the data processing device (600) is further configured to perform the following steps: (c) Identifying the papilla print (100) in the reconstructed image (O402) by comparison with a set of reference fingerprints.

Images