Method for classifying biometric features represented by an input image

Abstract

To provide a method for classification of a biometric trait represented by an input image.SOLUTION: The invention provides a method for classifying a biometric trait represented by an input image in an architecture where a server 1 and a client 2 are connected to an extended network 10 such as the internet for data exchange. The method comprises causing data processing means 21 of the client to implement a step of determining, for each predefined set of possible general patterns of biometric traits, whether the biometric trait presents the general pattern, by means of a convolutional neural network.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to the field of biometrics and in particular proposes a method for classifying biological features represented by an input image by means of at least one convolutional neural network, as well as a method for training the convolutional neural network. [Background technology]

[0002] Biometric authentication / identification consists of recognizing an individual based on their biological characteristics, such as their fingerprint (fingerprint recognition), palm print, retinal fingerprint, iris or face (facial recognition).

[0003] Traditional biometric approaches utilize characteristic information from biological characteristics, called features, extracted from acquired biometric measurements, and training / classification is performed based on comparison of these features.

[0004] In particular, for fingerprint recognition, fingertip images are processed to extract fingerprint features that can be classified into three categories. Level 1 defines the general pattern of such a fingerprint (one of four or five classes: right loop, left loop, arc, possibly protruding arc and whorl) and the overall layout of the ridges (in particular, an orientation map called the "Ridge Flow Matrix" (RFM) map is obtained, which represents the general direction of the ridges at each point of the fingerprint). -Level 2 defines specific points on the fingerprint, called minutiae, that make up "events" along the ridges (ridge ends, bifurcations, etc.). Traditional recognition methods essentially use these features. -Level 3 defines more complex information such as the shape of ridges, skin pores, scars, etc.

[0005] The process of extracting features (in the form of a feature map) from a fingerprint is called "encoding." These features make it possible to generate a signature, called a "template," that encodes useful information for the final stage of classification. More specifically, classification is performed by comparing the obtained feature map with one or more reference feature maps associated with known individuals.

[0006] A major problem lies in the response time when an identification needs to be performed, insofar as it is necessary to make as many comparisons as there are individuals in the database of known individuals.

[0007] To save time, it was proposed to extract level 1 features (general patterns of fingerprints) using deep learning. More specifically, fingerprints are classified by neural networks among several known types in a way that limits the number of individual candidates and therefore the number of comparisons that can be made.

[0008] Thus, the document Ruxin Wang, Congying Han, Yanping Wu and Tiande Guo; Fingerprint Classification Based on Depth Neural Network proposes a so-called "fuzzy" classification of fingerprints from their orientation map, obtained by preprocessing.

[0009] However, this method has proven ineffective for damaged fingerprints, especially so-called latent fingerprints such as those found at crime scenes. In fact, obtaining an orientation map is very difficult, since two latent images of the same fingerprint of identity may have significant rotations, obscurations, and a set of distortions relative to each other.

[0010] The paper "Fast Fingerprint Classification with Deep Neural Networks" by Michelsanti, Daniel; Ene, Andreea-Daniela; Guichi, Yanis; Stef, Rares; Nasrollahi, Kamal; Moeslund, Thomas B. demonstrates that the VGG-S neural network allows for the direct evaluation of the entire fingerprint pattern without preprocessing. This technique allows for the elimination of the need for an orientation map, but in practice it has proven to be poor for latent fingerprints, and its performance is only suitable for well-oriented, high-quality fingerprints.

[0011] Therefore, it is desirable to have a more efficient classification solution for biometric images, especially latent fingerprints, to extract their features. Summary of the Invention

[0012] According to a first aspect, the invention relates to a method for classification of biometric characteristics represented by an input image, said method comprising, by means of processing client data: (a) for each predetermined set of possible general patterns of a biological characteristic, determining whether the biological characteristic has the general pattern by a convolutional neural network (CNN).

[0013] According to advantageous and non-limiting features, The CNN has at least one residual connection.

[0014] The CNN has at least one so-called dense block with all possible residual connections.

[0015] The CNN includes multiple dense blocks, and the size of the feature maps of the dense blocks decreases from the input to the output of the CNN, and / or the number of layers per dense block increases from the input to the output of the CNN.

[0016] The rate of reduction in the size of the dense block feature maps increases from the input to the output of the CNN.

[0017] The method includes a pre-training step (a0) of the parameters of the CNN by server data processing from a database of images of already classified biological features.

[0018] Step (a0) preferably comprises random deactivation of the layer of dense blocks with a probability of about 10%.

[0019] The method includes the step (b) of processing the input image to extract features of interest from the biometric characteristics represented by the input image.

[0020] The biometric characteristics are fingerprints, and the features of interest extracted from at least one fingerprint represented by the input image include minutiae positions and / or orientations.

[0021] At least one biometric characteristic represented by the input image is of an individual, and the method further includes step (c) of identifying or authenticating the individual by comparing a feature of interest extracted from the biometric characteristic represented by the input image with a reference biometric characteristic feature having the predetermined pattern(s) determined in step (a) as exhibited by the biometric characteristic represented by the input image.

[0022] The biometric characteristic is a fingerprint, and the predetermined set of possible general patterns includes the general patterns left loop, right loop, arc, and whorl.

[0023] According to a second and third aspect, the present invention relates to a computer program product comprising code instructions for the execution of a method according to a first aspect of classification of a biometric feature represented by an input image, and storage means readable by a computing device, on which the computer program product comprises coded instructions for the execution of the method according to the first aspect of classification of a biometric feature represented by an input image. [Brief explanation of the drawings]

[0024] Other characteristics and advantages of the invention will become apparent on reading the following description of preferred embodiments, which description is given with reference to the accompanying drawings, in which: [Figure 1] FIG. 1 is a diagram of an architecture for implementing the method according to the invention. [Figure 2] FIG. 2 shows the steps of a preferred embodiment of the method according to the invention. [Figure 3] FIG. 3 shows an example of a convolutional neural network architecture for the implementation of the method according to the invention. DETAILED DESCRIPTION OF THE INVENTION

[0025] Principles and Architecture According to two further aspects of the present invention, it is proposed that: A method for classifying a biological characteristic represented by an input image by means of a convolutional neural network (CNN) and, where applicable, by extraction of features of interest from the biological characteristic represented by the input image. -Training method for the CNN configuration.

[0026] The input or training data is of image type and is a representation of a biometric characteristic (in other words an image of the biometric characteristic, i.e. the image on which the biometric characteristic appears), advantageously chosen from fingerprints, palm prints, retinal scans, etc., and generally any fingerprint that can be represented by an image and that can be uniquely associated with an individual. Preferably, the biometric characteristic is a fingerprint, and this example will be used in part of the following description (as will be shown, the method is particularly useful in the case of latent fingerprints).

[0027] The term "classification" refers to the determination of at least one "general pattern" of the biometric characteristic represented by the input image, i.e., a Level 1 feature of the fingerprint. This classification is distinguished herein from the identification / authentication of the individual possessing the biometric characteristic (i.e., the determination of identity information associated with the fingerprint), which may also be referred to as classification of the biometric characteristic.

[0028] The general pattern defines the overall structure of the fingerprint and makes it possible to distinguish between several types of biometric characteristics.

[0029] It is therefore assumed that there is a known set of general patterns of possible biometric characteristics. Generally, in the case of fingerprints, this set conforms to the so-called "Henry system" proposed in 1900 and typically includes (preferably only) the following general patterns: - Left-sided loop ("left loop" in the Henry system) - The fingerprint has a "core" and a "delta" located below and to the left of the core. - Right-hand loop (Henry system "right loop") - The fingerprint has a core and a delta located below and to the right of the core. - Arch ("Arrow" in the Henry system) - Fingerprint has a delta and optionally a core above and below. - Whorl (Henry system's "whorl") - The fingerprint has two cores and two deltas.

[0030] It should be noted that the present invention is not limited to these four general patterns, inasmuch as other general fingerprint patterns such as "double loop" or "protruding arc" may exist. The whorl may then be distinguished as two patterns depending on the direction of rotation. Conversely, the general left loop and right loop patterns may be grouped under the general pattern "loop."

[0031] One skilled in the art would be able to adapt this method to any known set of possible general patterns for any biological characteristic.

[0032] The principle underlying the method is not to consider the classification as a selection of a general pattern of the fingerprint of the input image from among a predetermined set of possible general biometric characteristic patterns (i.e., multi-class classification, in other words, classification into multiple classes), but rather to determine for each predetermined set of final general patterns of biometric characteristics whether the biometric characteristic exhibits this pattern or not (i.e., multiple binary classifications where each class does not exclude the others, also called multi-label classification).

[0033] For example, instead of determining whether the general pattern of a fingerprint is a right loop, a left loop, an arch, or a whorl (i.e., a choice between four general patterns), the determination becomes whether it is a right loop type, whether it is a left loop type, whether it is an arch type, and whether it is a whorl type.

[0034] Therefore, it is not theoretically impossible for a biometric characteristic to be identified as belonging to several types, i.e., even if the majority of fingerprints are clearly identified as having a single general pattern, the biometric characteristic may have two general patterns at the same time.

[0035] For n possible common biometric patterns in the given set, 2 n-1 possible combinations (as will be seen below, it is not possible for a biometric trait to have no general patterns), eg, for four general patterns there are 15 combinations.

[0036] These two n The possible combinations of numbers -1 are the "more complex" 2 for n initial classes (one general pattern). n -n-1 classes can be understood as the sum of the n-1 classes (if there are at least two common patterns, e.g., one left-hand loop + one right-hand loop, then double loop).

[0037] Such multiple classification choices actually significantly improve the performance of CNN implementations. In other words, the need to always select one general pattern (i.e., to the exclusion of others) can confuse neural networks, but the present solution no longer has this requirement. The benefits are particularly seen in latent fingerprints with distortions and occlusions, where many general patterns are possible candidates.

[0038] The final extraction of features of interest that may follow typically consists of "encoding" the fingerprint represented by the input image, i.e. the extracted features of interest are typically "biometric" features, i.e. "final" features that make it possible to construct a fingerprint template in the form of a biometric characteristic of an individual with a view to identifying / authenticating that individual, see below. In this respect, in the case of fingerprints, the features of interest typically describe minutiae, i.e. they include the position and / or orientation of the minutiae. However, it will be understood that the method is not limited to this embodiment and that any final biometric characteristic of interest may be extracted.

[0039] The method is implemented within an architecture such as that shown in Figure 1, having a server 1 and a client 2. Server 1 is a training device (which implements the training method) and client 2 is an operation device, e.g., a user terminal (which implements the classification method and, if applicable, feature extraction).

[0040] Preferably, the server 1 is that of a security solutions provider and the client 2 is a consumer device, in particular a biometric characteristic scanner, but also a smartphone, a personal computer, etc. The client device 2 advantageously includes optical acquisition means 23 in such a way that it directly acquires said input images, typically for direct processing, although alternatively the input images are loaded onto the client device 2 (e.g. latent fingerprints may be photographed at the crime scene by a forensic team and these photographs may then be loaded onto a dedicated client 2 device in a forensic laboratory).

[0041] In either case, each device 1, 2 is typically a remote computer equipment connected to an extended network 10 such as the Internet for data exchange. Each includes data processing means 11, 21 of the processor type and data storage means 12, 22 such as computer memory, e.g. flash memory or hard disk.

[0042] The server 1 stores a training database, i.e. a set of biometric characteristic images that have been previously classified (i.e. for each predefined set of possible generic biometric characteristic patterns it is already known whether the fingerprint exhibits this generic pattern or not), as opposed to the so-called input images that it actually attempts to process.

[0043] method In summary, referring to Figure 2, the present method for classifying a biometric characteristic represented by an input image essentially comprises step (a) of determining, for each predetermined set of possible general biometric characteristic patterns, by a convolutional neural network, CNN, whether the biometric characteristic pattern has that general pattern. In other words, a number of Boolean values ​​are obtained for the possible general biometric characteristic patterns (previously referred to as n) in the predetermined set. Thus, the output of the CNN is typically a vector of dimension n or 2n (if there is also a score for each pattern, representing the probability that the fingerprint exhibits that pattern).

[0044] In optional step (b), the input image is advantageously processed in such a way as to extract features of interest of the biometric characteristics represented by the input image.

[0045] As can be seen, the method may further comprise a step (c) for identifying or authenticating at least one individual by comparing the features of interest extracted from the biometrics represented by the input image with features of reference biometrics having the general pattern(s) determined in step (a) as presented by the biometrics represented by the input image, which will be understood to be a subgroup of the set of reference biometrics, thereby limiting the number of comparisons made.

[0046] CNN The CNN can be of many types, including traditional CNNs (a direct successor of CONV convolutional layers, BN batch normalization layers, and NL nonlinear layers).

[0047] According to a preferred embodiment, the CNN is of the residual network type. Such residual networks can be seen to be very effective for pure binary classification (determining whether a general pattern exists or not) and much lighter than many image processing CNNs, such as detection CNNs.

[0048] A residual network, or RESNET, is a CNN that has at least one "residual connection" (a "skip connection" or simply "shortcut"), i.e., at least one layer has connections that are "short-circuited," similar to those found in pyramidal neurons in the brain.

[0049] In fact, if adding layers makes the model more complex, some of these layers may have a negative impact on the model's performance. Residual connections ensure that if no useful transformations are learned, one layer must at worst learn discriminative information, avoiding degrading the performance of other layers. The operating principle behind residual networks is to add pointwise inlets and outlets to convolutional layers, allowing signals to propagate from superficial layers to deeper layers. As will be explained, this network produces excellent results, especially in pure classification.

[0050] Conventionally, within such a ResNet for implementing the present method, e.g., within a block, there is a main branch (short-circuited by residual connections) with multiple convolutional layers possibly followed by nonlinear layers (e.g., ReLU) and / or batch normalization layers. The output of this branch is typically added pointwise to the input image (by residual connections) and passed through a final dimensionality reduction layer (typically an AveragePool or MaxPool layer).

[0051] A fully connected layer allows the generation of an output vector (i.e., the classification result), each binary output may include its activation function, in particular one of softmax type, see below.

[0052] Preferably, referring to Figure 3, the CNN is also of the "densely connected" network type (or simply dense network, or DENSENET), i.e., the CNN not only has at least one residual connection, but also includes, within at least one so-called dense block, all possible residual connections, i.e., each residual connection connecting the input of the first layer of a block to the output of the second layer of a block downstream of the first layer. As will be seen below, the concept of layer should be understood here as being limited to convolutional layers (i.e., no nonlinear or batch normalization layers are short-circuited) or "convolutional units", insofar as a 1x1 convolutional layer can be added before an "activation" convolutional layer (usually 3x3) in order to reduce the number of feature maps at the input, in particular.

[0053] In other words, for a dense block with k layers, - k residual connections that short-circuit the first layer, the second layer, ... the kth layer, respectively; - k-1 residual connections short-circuiting the first and second layers, the second and third layers, ... the k-1th layer and the kth layer, respectively; - k-2 residual connections that short-circuit the first to third layers, the second to fourth layers, ... k-th - second layer to k-th layer, respectively; -... - The first and second layers, the second and third layers, ... the kth - there is one residual connection shorting the first layer and the kth layer respectively.

[0054] Therefore, the number of residual connections for a dense block with k layers is Σ k It is understood that for i, and a total of k(k+1) / 2 connections (direct or residual), excluding the k "normal" connections (from one layer to the next), there are k(k-1) / 2.

[0055] It is understood that a dense block with a constant size, i.e. the size of the input feature map, is the same as the size of the output feature map since there is no pooling layer (dimensionality reduction) within the dense block.

[0056] Advantageously, as in this example, the size of the dense block feature maps decreases from the input to the output of the CNN and / or the number of layers per dense block increases from the input to the output of the CNN.

[0057] In the example of Figure 2, there are three dense blocks: a first dense block with a feature map size of 128x128, a second dense block with a feature map size of 64x64, and a third dense block with a feature map size of 16x16. To enable size reduction of the feature maps between the two blocks, a "transition layer" may be precisely placed between the two dense blocks (typically including a pooling layer and / or a batch normalization layer).

[0058] As will be explained, dense blocks consisting of a slightly more complex set of roughly 6-48 such base layer "units" can be used as "layers," in the sense of elementary units, such as a series of 1x1 convolutional layers and 3x3 convolutional layers (each optionally followed by a ReLU and / or batch normalization layer).

[0059] Preferably, the CNN performs "thinning," i.e., it exhibits a very rapid reduction in the size of the feature map (mathematically, the rate of reduction in the size of the dense block feature map increases from the input to the output of the CNN, specifically, a bisection between the first and second dense blocks, then a quadruple division between the second and third dense blocks), and therefore there is an associated loss of accuracy insofar as determining general patterns requires the CNN to only "engage" in the details (typically single points of the core and delta type) rather than the entire image. Such a rapid reduction in the size of the feature map has the added benefit of limiting the memory footprint of the CNN, thus allowing it to be easily loaded onto simple client devices 2.

[0060] Furthermore, certain layers may be randomly disabled during training in known ways (see below), which is called "dropout" (usually with a given probability of 0% to 50% and 10% in the example of FIG. 2). When one layer is deactivated, adjacent layers are directly connected. In practice, many layers contribute minimally to the result and can therefore be disabled without any effect, but since the total number of parameters is reduced, the memory footprint of the CNN is also found to be reduced, as mentioned above.

[0061] training Advantageously, the method begins with a step (a0) of training the parameters of the CNN from a database of images of pre-classified biometric characteristics (i.e. known for each general pattern of the predetermined set of possible general patterns, if any), using the data processing means 11 of the server 1.

[0062] This training can be performed in the conventional way: the training cost function can consist of a conventional data fitting (mean double error) and total variation normalization.

[0063] As will be explained, the training step (a0) may include random deactivation of layers according to a predetermined dropout rate (probability that a layer is disabled) (e.g., 10%).

[0064] It should be noted that, in general, it is assumed that there is a reference fingerprint associated with the generic pattern, and that the generic pattern is the generic pattern exhibited by this reference fingerprint. This can be interpreted to mean that all other generic patterns in the given set of possible generic patterns are not exhibited by this reference fingerprint. It is also possible to find rare reference fingerprints that can introduce more generic patterns, if applicable.

[0065] Conventionally, in all cases, one can start with a database of high-quality fingerprints, and an augmentation algorithm may be implemented to double the size of the training base, and latent fingerprints may be artificially generated (e.g., by deforming high-quality fingerprints to create occlusions therefrom) to ensure the robustness of the CNN to common acquisition imperfections.

[0066] The trained CNN may be stored as needed on the data storage means 22 of the client 2 for use in classification. Note that the same CNN can be embedded in multiple clients 2 and only one training is required.

[0067] Identification / Authentication As will be explained, in step (b), the input image may be processed in such a way as to extract features of interest from the biometric characteristics represented by the input image, which may, among other things, include the position and / or orientation of minutiae in the case of a fingerprint.

[0068] Preferably, the method further comprises step (c) of identifying or authenticating at least one individual by comparing features of interest extracted from the biometrics represented by the input image with features of reference biometrics, which may be performed in any manner known to those skilled in the art.

[0069] For example, Client 2 may store fingerprint features of one or more individuals entered into police files as reference fingerprints in a manner that allows for the identification of the individuals who left their latent fingerprints. If the extracted features match those expected for a known individual, the identity of that individual is transmitted to the police.

[0070] Alternatively, client 2 may transmit the extracted features to a remote database of reference biometric features for identification of individuals.

[0071] As will be explained, this classification allows in step (c) the features of interest extracted from the biometric characteristic represented by the input image to be compared only with features of the reference biometric characteristic having the general pattern(s) determined in step (a) as presented by the biometric characteristic represented by the input image, thus saving time. It should be noted that if the classification in step (a) is incorrect, it can be expected that the result of the comparison will not be successful (i.e., no corresponding reference fingerprint is found), but the features of interest extracted from the biometric characteristic represented by the input image can nevertheless be compared with other features of the reference biometric characteristic.

[0072] It should be noted that steps (b) and / or (c) may also be performed as a second classification step of the biometric characteristics, for example by one or more dedicated CNNs, and may, if applicable, include a process for the refinement of the isolated fingerprints. See in particular French Patent Application No. 1860323.

[0073] result: Tests based on 100 latent fingerprints using the CNN according to Figure 2 showed correct classification in 98.86% of cases, effectively reducing the number of reference fingerprints tested for identification / authentication by 31%.

[0074] computer program products According to a second and third aspect, the present invention relates to a computer program product (in particular on data processing means 11, 21 of the server 1 and / or of the client 2) comprising code instructions for the execution of a method for classifying biological characteristics represented by an input image, as well as to storage means (memory 12, 22 of the server 1 and / or of the client 2) readable by a computing device on which said computer program product is located.

Claims

1. 1. A method for classifying a biometric feature represented by an input image, said method comprising the steps of:

1. A method comprising: (a) performing multi-label classification of an input image by a convolutional neural network (CNN) to determine, for each generic pattern of a predetermined set of non-exclusive possible generic patterns of biometric features, a Boolean expression as to whether the biometric feature exhibits the generic pattern.

2. The method of claim 1 , wherein the CNN has at least one residual connection.

3. The method of claim 2 , wherein the CNN includes at least one dense block with all possible residual connections.

4. 4. The method of claim 3, wherein the CNN comprises a plurality of the dense blocks, and wherein the size of the feature maps of the dense blocks decreases from the input to the output of the CNN, and / or the number of layers per dense block increases from the input to the output of the CNN.

5. The method of claim 4 , wherein the rate of decrease in the size of the feature maps of the dense blocks increases from the input to the output of the CNN.

6. 2. The method of claim 1, comprising a step (a0) of pre-training the parameters of said CNN from a database of images of pre-classified biometric features by data processing means of a server.

7. 7. The method of claim 6, wherein the CNN has at least one residual connection, the CNN includes at least one dense block with all possible residual connections, and wherein step (a0) includes random passivation of a layer of the dense block.

8. 8. The method defined by claim 7, wherein the random passivation of the layer of dense blocks has a probability of about 10%.

9. 2. The method of claim 1, further comprising the step (b) of processing the input image in such a way as to extract features of interest of the biometric characteristics represented by the input image.

10. The method of claim 9 , wherein the biometric characteristics are fingerprints, and the features of interest extracted from at least one fingerprint represented by the input image include minutiae positions and / or orientations.

11. 10. The method of claim 9, wherein the biometric characteristics represented by the input image are of an individual, the method further comprising step (c) of identifying or authenticating the individual by comparing the features of interest extracted from the biometric characteristics represented by the input image with features of reference biometric characteristics having the general pattern(s) determined in step (a) as exhibited by the biometric characteristics represented by the input image.

12. The method of claim 1 , wherein the biometric characteristic is a fingerprint and the predetermined set of possible general patterns includes the general patterns left loop, right loop, arc, and whorl.

13. 10. A non-transitory computer readable program comprising code instructions for the execution of the method of claim 1 for said classification of biometric characteristics represented by an input image when said program is executed on a computer.

14. 10. A non-transitory storage means readable by a computing device, wherein a computer program on said computing device comprises code instructions for the execution of the method of claim 1 for said classification of a biometric characteristic represented by an input image.

Images