Method for restoring a papillary trace in an image

Abstract

A method, implemented by a data processing device, for restoring a papillary trace in an image, the method comprises the following steps: (a) Generating an encoding vector by applying a convolutional neural encoding network taking, as input data, an image comprising at least one papillary trace and a map of the orientation field of said papillary trace; (b) Generating a restored image by applying a convolutional neural decoding network to the encoding vector.

Description

Title of the invention: Method for restoring a papillary trace in an image technical field

[0001] The present invention relates to a method and a system for restoring a papillary trace in an image. Technical background

[0002] Dactyloscopy is a method of identifying individuals based on the analysis of fingerprints, also known as "papillary prints," which include "fingerprints" and "palm prints." This method is notably used by forensic anthropometry services in the context of an offense, misdemeanor, felony, or judicial investigation.

[0003] Fingerprints are patterns formed by the traces left on surfaces by dermatoglyphics of the fingers and / or palms. Dermatoglyphics are the superficial furrows formed on the palms, soles, and fingertips by the dermal ridges and arranged in lines or spirals. They are unique to each individual, and the patterns they form constitute an anthropometric "identity card" by which the individual can be identified.

[0004] The recording of fingerprints by forensic services is a common practice during judicial investigations. In such recordings, fingerprints take the form of digital, palmar or plantar "traces," and are commonly called "papillary traces" in reference to the papillary ridges or dermatoglyphs.

[0005] Three types of "papillary traces" are distinguished: visible papillary traces, latent papillary traces, and molded papillary traces. Visible papillary traces are those directly visible without external intervention. They can be "positive" when formed by the deposition of material, such as traces left by fingers stained with blood, grease, or ink. They can be "negative" when they result from the removal of material, such as traces on a dusty surface. Latent papillary traces are invisible to the naked eye and observable only by a development method. They are, for example, the result of a deposition of sweat or papillary secretions present on the papillary ridges. Molded papillary traces are three-dimensional traces resulting from the pressing of fingers onto a malleable surface.

[0006] Identifying an individual from a fingerprint requires comparing that fingerprint with numerous other fingerprints previously acquired from several individuals (1:N) and generally stored in a database. Because fingerprints are complex drawings and the number of comparisons required for identification can become very high, the identification process can remain lengthy despite the computing power of currently available data processing devices. To reduce the time required for this operation, it is known to classify fingerprints into different classes based on certain morphological characteristics, called dermatoglyphs. For example, these morphological characteristics might include the overall shape of the dermatoglyph (orientation of loops, arches, spirals, etc.).) according in particular the categories of Henry Faulds, Francis Galton and Edward Henry, the overall outline of the ridges, the "minutiae" constituted by singular points and / or discontinuities along the ridges (termination of a ridge, bifurcation...), the shape of the ridges, the pores or even the scars. .

[0007] Fingerprint identification methods and systems are either manual, semi-automatic, or fully automatic. Because the size of databases and the computing power of data processing devices are constantly increasing, automatic fingerprint identification systems (AFIS – “Automated Fingerprint Identification System”) are being used more and more frequently. They allow for the rapid and efficient analysis of a list of candidate fingerprints likely to match a fingerprint whose owner is to be identified.

[0008] However, the images of fingerprints used to identify individuals using automatic identification systems may be affected by artifacts such as distortions and / or elements foreign to the fingerprints. The presence of these artifacts is likely to impair identification performance. These artifacts may originate from the conditions under which the fingerprints were formed and / or from the methods of collection. For example, the surface on which the fingerprint is located may have graphic patterns that could hinder the recognition of the morphological characteristics of the fingerprint.If the substrate is a sheet of paper containing writing or a photograph, the fingerprint may be superimposed on the writing or image; these underlying elements may then interfere with the graphic elements representing the morphological characteristics of the fingerprint. Images of fingerprints may also include several fingerprints partially overlapping each other. Prior correction and / or segmentation operations are generally necessary. to allow the distinction of papillary traces from the background of the surface on which they are present, or from each other when they are superimposed.

[0009] FR 3 102 600 Al [IDEMIA IDENTITY & SECURITY FRANCE [FR]] 30.04.2021 describes a method for segmenting an image representing at least one fingerprint. The method comprises, first, the generation, using convolutional neural networks applied to the image, of an orientation map and a first segmentation mask. Next, a second segmentation mask is obtained by combining the orientation map and the first segmentation mask through the application of a convolutional neural network. Finally, the second segmentation mask is applied to the image of the fingerprint to segment it.

[0010] Saponara et al. (2021). Recreating fingerprint images by convolutional neural network autoencoder architecture. IEEE Access, p. 147888-147899, describes a convolutional neural network autoencoder for the reconstruction of fingerprint images from degraded and / or complex images. Summary of the invention

[0011] Prior art segmentation or restoration methods make it possible to reveal a papillary trace in an image by providing a focused image of the papillary trace. This focused image may, in particular, be in the form of a cropped image of the area of ​​the focused image corresponding to the papillary trace or a synthetic image "artificially recreating" the morphological characteristics of the papillary trace. However, these methods do not always allow for the proper elimination of extraneous signals such as graphic or script elements or fragments of other papillary traces, which are likely to affect the papillary trace itself, or only at the cost of a significant loss of information relating to the morphological characteristics of the papillary trace.One negative consequence is that these spurious signals or information losses can disrupt the ability of an automatic identification system to identify the fingerprint. This system may then produce erroneous results or fail to identify a fingerprint that nevertheless has a match in a database of reference fingerprints.

[0012] It is well known that a forensic expert demonstrates a level of skill in characterizing a fingerprint that an automated system is incapable of. In particular, they can discern, in an image of fingerprints, the characteristics of a fingerprint among a set of extraneous elements, whereas these same characteristics would not be, or would be poorly, modeled by an algorithm of an automated identification system. However, the lack of flexibility of current identification methods prevents them from taking into account relevant information that may be established independently by a human operator such as a forensic expert in fingerprinting.

[0013] A first aspect of the invention relates to a method, implemented by a data processing device, for restoring a papillary trace in an image; the method comprises the following steps: (a) Generate an encoding vector by applying a convolutional neural encoding network taking as input an image comprising at least one papillary trace and a map of the orientation field of said papillary trace; (b) Generate a restored image by applying a convolutional neural decoding network to the encoding vector.

[0014] According to some embodiments, the orientation field map of the papillary trace is determined manually from the image of a papillary trace.

[0015] According to some embodiments, the convolutional neural encoding network also takes, as input data, a segmentation map of the papillary trace.

[0016] According to some embodiments, the encoding convolutional neural network and the decoding convolutional neural network are residual convolutional neural networks.

[0017] According to some embodiments, the image comprising at least one papillary trace comprises at least two partially superimposed papillary traces.

[0018] According to some embodiments, the image comprising at least one background composed of graphic and / or scriptural elements.

[0019] According to some embodiments, the convolutional neural network for encoding and the convolutional neural network for decoding are previously trained on a set of images of papillary traces degraded by the addition of graphic and / or scriptural elements, and / or foreign papillary traces.

[0020] A second aspect of the invention relates to a data processing device comprising means for implementing a process according to any one of the embodiments.

[0021] A third aspect of the invention relates to a method for identifying a papillary trace in an image, said method comprising the following steps: (a) Generate a restored image of an image comprising at least one papillary trace using a restoration process according to any embodiment of the first aspect of the invention; (b) Identify the papillary trace of the restored image by comparison with a set of reference fingerprints.

[0022] A fourth aspect of the invention relates to an automatic system for identifying fingerprint traces comprising a configured data processing device for the implementation of a process according to any one of the embodiments of the first aspect of the invention. Brief description of the drawings

[0023] [Fig. 1] is a schematic representation of a papillary trace.

[0024] [Fig.2] is a representation of two examples of images of papillary traces, one including artifacts, the other a superposition of two papillary traces.

[0025] [Fig.3] is a schematic representation of examples of field map orientation for papillary traces of the example images of [Fig.2].

[0026] [Fig.4] is a flowchart of a method according to the first aspect of the invention.

[0027] [Fig.5] is a schematic representation of examples of segmentation map for the papillary traces of the example images in [Fig.2].

[0028] [Fig.6] is a schematic representation of a data processing device according to the second aspect of the invention. Detailed description of the implementation methods

[0029] With reference to [Fig. 1], in an image 1100 of a papillary trace 100, the papillary trace 100 is presented as a pattern formed by the traces left on the surfaces by the dermatoglyphs of a finger. This pattern represents the curvatures of the furrows 101 and the ridges 102 of the papillary or epidermal folds present on the pulp of a finger. These curvatures take various geometric forms, primarily lines, loops, and spirals.

[0030] For the purposes of this disclosure, "fingerprints" means any visible, latent or molded fingerprints, and also the fingerprints of an individual that may be acquired during an acquisition campaign using an electronic contact acquisition device or during a recording on a paper medium after prior inking of the fingers.

[0031] Depending on the conditions under which the fingerprints were formed and / or the methods of their collection, an image of a fingerprint 100 may be affected by artifacts or include several superimposed fingerprints. In a first example, with reference to [Fig. 2] (a), the artifacts may be graphic and / or scriptural elements 201 (represented as horizontal black bands) in the background, which is the surface on which the fingerprint 100 was formed, in this case a sheet of paper with typewritten text. In a second example, with reference to [Fig. 2] (b), the image includes two partially superimposed fingerprints 100, 202.

[0032] With reference to [Fig. 3], the curvatures of the papillary ridges 102 of a papillary trace 100 of [Fig. 1] can be represented in the form of maps 300 of the orientation field of said papillary ridges 102. An orientation field map A ridge flow map (or orientation map) of a fingerprint represents the local orientations of the ridges for each pixel or group of pixels in the image representing the fingerprint. These orientations are generally expressed as an angle between 0° and 180° relative to a reference direction of the image, usually the horizontal direction. According to this representation, which is common in the field, two neighboring ridges oriented at 0° and 180°, respectively, are indistinguishable from one another. An orientation field map can be obtained manually by a human operator or automatically using image processing methods such as those described in Hong, L., & Jain, A. (1999). Classification of fingerprint images. In Proceedings of the Scandinavian Conference on Image Analysis, Vol. 2, pp. 665–672.

[0033] With reference to [Fig.4], a first aspect of the invention relates to a method 400, implemented by a data processing device, for restoring a papillary trace 100 in an image 1100, the method 400 comprises the following steps: (a) Generate 401 an encoding vector O401 by applying a convolutional neural encoding network taking, as input data, an image 1100 comprising at least one papillary trace 401 and a map 300 of the orientation field of said papillary trace 101; (b) Generate 4 02 a restored image O 4 02 by applying a convolutional decoding neural network to the encoding vector O 4 01.

[0034] Unlike prior art segmentation methods, the process of the first aspect of the invention makes it possible to restore, or rather reveal, the papillary trace 100 by providing a restored image O 4 02 free of all artifacts or extraneous papillary traces that may be present in the original image 1100. In other words, the process 4 00 of the first aspect of the invention makes it possible to "focus" on the papillary trace 100 of interest in the image 1100 by ideally producing the restored image O 4 02 which represents only the papillary trace 100 whose identification is sought.

[0035] The map 300 of the orientation field of the papillary trace 100 is obtained using any suitable method. Preferably, it is not obtained automatically, in particular by applying an algorithmic image processing method.

[0036] According to some preferred embodiments, the orientation field map 300 of the papillary trace 100 is determined manually from the image 1100 of a papillary trace 100. In particular, the orientation field map 300 can be produced manually by a human operator such as a forensic fingerprint expert. Such a map 300 is capable of representing ridge orientations of the papillary trace 100 that an automatic processing algorithm the image cannot be discerned due to the presence of artifacts or other papillary traces superimposed on said papillary trace 100. Thanks to this card 3 00 and the flexibility of the method 4 00 of the first aspect of the invention, it is then possible to obtain a restored image O 4 02 of the papillary trace 100 including morphological characteristics which a processing algorithm of an automatic identification system would have obscured even though they may be decisive during an identification operation.

[0037] According to certain embodiments, with reference to [Fig. 5], the encoding convolutional neural network further takes as input a 500 segmentation map of the papillary trace 100. The function of the 500 segmentation map is to mask the parts of the image 1100 whose features are considered not to belong to the papillary trace 100. The 500 segmentation map is generally a binary map, that is, one whose pixel values ​​are 0 or 1. In the two examples in [Fig. 5], the 500 segmentation maps reveal only the parts of the image 1100 corresponding to the papillary trace 100.

[0038] The use of a 500 segmentation map can be advantageous when the number of artifacts and / or extraneous papillary traces superimposed on the main papillary trace 100 is significant, particularly to the point that the possibility of precisely distinguishing their morphological characteristics becomes very limited. The combination of a 300 orientation field map and a 500 segmentation map of the papillary trace 100 allows convolutional neural networks for encoding and decoding to amplify the "focus" on the image 1100 features relevant for restoring the papillary trace 100 for the purpose of its identification.

[0039] As with the 300 map of the orientation field of the papillary trace 100, the 500 segmentation map is obtained using any suitable method. Preferably, it is not obtained automatically. Ideally, the 500 segmentation map is obtained by an image processing method assisted by a human operator such as a forensic fingerprint expert. Examples of segmentation processing include: watershed; Mask R-CNN described in He, et al. (2017). Mask R-CNN. In Proceedings of the IEEE international conference on computer vision, pp. 2961-2969; GrabCut described in Rother et al. (2004). "GrabCut" interactive foreground extraction using iterated graph cuts. ACM Transactions on Graphs (TOG), 23(3), 309-314. The 500 segmentation map can also be produced manually by a human operator such as a forensic expert in fingerprinting.

[0040] The encoding convolutional neural network functions to encode the image 1100 in the form of an encoding vector O 4 01, and the decoding convolutional neural network functions to reconstruct a restored image O 4 02 from the encoding vector O 4 01. These two networks can constitute the two parts of the same convolutional neural network architecture, notably in the form of a self-encoding convolutional neural network under supervised or unsupervised learning.

[0041] According to some embodiments, the encoding convolutional neural network and the decoding convolutional neural network are residual convolutional neural networks, in particular of the ResNet type. The architecture of a residual convolutional neural network is described in He et al., (2016) Deep residual learning for image recognition, In Proceedings of the IEEE conference on computer vision and pattern recognition, p.770-78.

[0042] The encoding convolutional neural network and the decoding convolutional neural network are trained using any suitable method. In some embodiments, the encoding convolutional neural network and the decoding convolutional neural network are first trained on a set of images of papillary traces degraded by the addition of graphic and / or textual elements, and / or foreign papillary traces. The training can be supervised using a cost function, for example a Manhattan distance L1, between the restored images and the original undegraded images.

[0043] With reference to [Fig. 6], a second aspect of the invention relates to a data processing device 600 comprising means for implementing a method 400 according to any one of the embodiments. The processing device 600 is responsible for automatically executing sequences of arithmetic or logical operations to perform tasks or actions. This device, commonly called a computer, may include one or more central processing units (CPUs) 601 and / or one or more graphics processing units (GPUs) 602, a physical remote communication module 603, one or more physical input / output modules 604 for exchanging data with external devices, a transient storage medium 605 such as random access memory (RAM), a non-transient recording medium 606, and communication buses (not shown) for transferring data between the internal components of the device.

[0044] The data processing device 600 allows the execution of one or more program modules comprising instructions which, when the program module(s) are executed, cause said device 600 to execute the method of the first aspect of the invention. The program module(s) may be written in any programming language, compiled or interpreted. They may be part of a software solution, that is to say, a collection of executable instructions, code, scripts or other elements, and / or databases.

[0045] A third aspect of the invention relates to a method for identifying a papillary trace 100 in an image 1100, said method comprising the following steps: (a) Generating a restored image O 4 02 of an image 1100 comprising at least one papillary trace 100 using a restoration method 400 according to any embodiment of the first aspect of the invention; (b) Identify the papillary trace 100 of the restored image O 4 02 by comparison with a set of reference fingerprints.

[0046] The step of identifying the papillary trace can be implemented using any suitable method. By way of example, it can be implemented using an automatic identification system as described in EP 0 366 850 A1 [MORPHO SYSTEM LTD CORP [FR]] 09.05.1990, US 5 465 303 A [AEROFLEX SYST CORP [US]] 07.11.1995, US 2003 / 091724 A1 NEC CORP [US] 15.05.2003.

[0047] To this end, a fourth aspect of the invention relates to an automatic fingerprint identification (FFI) system comprising a data processing device configured for implementing a method (400) according to any one of the embodiments of the first aspect of the invention. The data processing device may be that of the automatic fingerprint identification system or an external device communicating with it. References Literature patent

[0048] EP 0 366 850 Al [MORPHO SYSTEM LTD CORP [FR]] 05 / 09 / 1990.

[0049] US 5,465,303 A [AEROFLEX SYST CORP [US]] 07.11.1995.

[0050] US 2003 / 091724 Al NEC CORP [US] 05.15.2003.

[0051] FR 3 102 600 Al [IDEMIA IDENTITY & SECURITY FRANCE [FR]] 04.30.2021. Non-patent literature

[0052] Hong, L., & Jain, A. (1999). Classification of fingerprint images. In Proceedings of the scandinavian conférence on image analysis, Vol. 2, p. 665-672.

[0053] Rother et al. (2004). " GrabCut" interactive foreground extraction using iterated graph cuts. ACM transactions on graphies (TOG), 23(3), 309-314.

[0054] He et al., (2016) Deep residual leaming for image récognition, In Proceedings of the IEEE conférence on computer vision and pattern récognition, p.770-778.

[0055] He, et al. (2017). Mask R-CNN. In Proceedings of the IEEE international conférence on computer vision, p. 2961-2969.

[0056] Saponara et al. (2021). Recreating fingerprint images by convolutional neural network autoencoder architecture. IEEE Access, p. 147888-147899.

Claims

Demands

1. A method (400), implemented by a data processing device, for restoring a papillary trace (100) in an image (1100), the method (400) comprises the following steps: (a) Generating (401) an encoding vector (O401) by applying a convolutional neural encoding network taking, as input data, an image (1100) comprising at least one papillary trace (100) and a map (300) of the orientation field of said papillary trace (100); (b) Generating (402) a restored image (O402) by applying a convolutional neural decoding network to the encoding vector (O401).

2. Method (400) according to claim 1, wherein the map (300) of the orientation field of the papillary trace (100) is determined manually from the image (1100) of a papillary trace (100).

3. Method (400) according to any one of claims 1 to 2, wherein the convolutional encoding neural network further takes as input data a segmentation map (500) of the papillary trace (100).

4. Method (400) according to any one of claims 1 to 3, wherein the encoding convolutional neural network and the decoding convolutional neural network are residual convolutional neural networks.

5. Method (400) according to any one of claims 1 to 4, wherein the image (1100) comprising at least one papillary trace (100) comprises at least two partially superimposed papillary traces.

6. Method (400) according to any one of claims 1 to 5, wherein the image (1100) comprising at least one papillary trace (100) comprises a background composed of graphic and / or scriptural elements.

7. Method (400) according to any one of claims 1 to 6, wherein the encoding convolutional neural network and the decoding convolutional neural network are pre-trained on a set of images of papillary traces degraded by the addition of graphic and / or scriptural elements, and / or foreign papillary traces.

8. Data processing device (6 00) comprising means for implementing a method (4 00) according to any one of claims 1 to 7.

9. A method for identifying a fingerprint trace (100) in an image (1100), said method comprising the following steps: (a) Generating a restored image (O 4 02) of an image (1100) comprising at least one fingerprint trace (100) using a restoration method (400) according to any one of claims 1 to 7; (b) Identifying the fingerprint trace (100) of the restored image (O 4 02) by comparison with a set of reference fingerprints.

10. Automatic fingerprint identification system comprising a data processing device including means for implementing a method (400) according to any one of claims 1 to 7.

Images