Method and device for determining fraud in a biometric image recognition system.

Abstract

The present invention relates to a method for detecting fraud in a biometric recognition system characterized in that it comprises the following steps: obtaining a first image of a subject's area of ​​interest in a first wavelength band; obtaining a second image of the subject's area of ​​interest in a second wavelength band; encoding the first image by a first neural encoder to obtain a first vector representation of the first image; encoding the second image by a second neural encoder to obtain a second vector representation of the second image; calculating a similarity measure between the first and second vector representations; and determining fraud if the similarity measure is less than a predefined threshold. [Fig. 2]

Description

Title of the invention: Method and device for determining fraud in a biometric image recognition system.

[0001] The invention relates to a method for determining fraud in a biometric image recognition system based on the analysis of at least two images of the same object, for example a face, obtained in at least two distinct wavelength bands.

[0002] Many biometric person recognition systems are based on the analysis of images of the person. Generally, a particular part of the person, the area of ​​interest, is used for recognition. This can be the face, but also the person's fingerprint or iris, this list being non-exhaustive.

[0003] Systems based on the analysis of a single image, typically in the visible spectrum, of the area of ​​interest allow for a reasonable, though imperfect, recognition rate. However, such systems are susceptible to fraud. They can be fooled by images constructed for this purpose and presented in place of the actual person's area of ​​interest.

[0004] One way to make these systems more robust is to base the recognition on a pair of images of the area of ​​interest obtained in different wavelength bands, for example, a first image is obtained in the visible spectrum and a second image is obtained in the infrared spectrum. This is just one example; other wavelength bands could be considered. The two images are then submitted to a fraud detector, either separately or concatenated. These systems improve fraud detection. For example, a face image presented in place of the person's actual face, in a photograph or on a screen, might produce a realistic image of the face in the visible spectrum but a black image in the infrared spectrum.

[0005] In these systems, the fraud detector is typically a classifier using neural network technology trained to recognize fraud attempts. A first classifier may specialize in the visible spectrum, while a second classifier may specialize in the infrared spectrum. In this case, the output of the two classifiers is then consolidated to obtain the final fraud detection result. Alternatively, the two images are concatenated. For example, the infrared image is added as an additional component to the red, green, and blue components of the visible image. Alternatively, the infrared image replaces one of the components to obtain, for example, an RGIR image, i.e., red, green and infrared. It is then this image combining the two bands of different wavelengths that is provided as input to a single classifier to detect fraud.

[0006] These systems offer improved fraud detection, but still suffer from shortcomings. In particular, these classifiers are trained on specific facial poses, expressions, and known fraud techniques. When faced with a new pose, expression, or fraud technology, their results are unpredictable.

[0007] The invention presented aims to solve this problem. Description of the invention

[0008] To this end, the invention proposes subjecting the input images to encoding. Each image is encoded using an encoder based on a neural network. The result of the encoding is a data vector representing the image. Fraud is then detected by calculating the similarity between the encoded representations of the images in each wavelength band. The encoders are jointly trained to maximize the similarity of the encoded representations of images from the same person and to minimize the similarity when this is not the case.

[0009] This approach yields better fraud detection results compared to known classifier-based systems. In particular, when faced with a new fraud technology that was not taken into account during training, the risk of fraudulent images producing similar representations is low. Furthermore, the invention improves robustness to shooting conditions.

[0010] A method for determining fraud in a biometric recognition system is thus proposed, characterized in that it comprises the following steps: - obtaining a first image of an area of ​​interest of a subject in a first wavelength band; - obtaining a second image of the subject's area of ​​interest in a second wavelength band; - encoding by a first neural encoder of the first image to obtain a first vector representation of the first image; - encoding by a second neural encoder of the second image to obtain a second vector representation of the second image; - calculation of a similarity measure between the first vector representation and the second vector representation; and - determination of fraud if the similarity measure is below a predefined threshold.

[0011] According to some embodiments, the first image and the second image are captured at the same instant.

[0012] According to some embodiments, the first image and the second image are captured by the same camera on the same sensor.

[0013] According to some embodiments, the similarity measure is a normalized scalar product.

[0014] According to some embodiments, the first neural encoder and the second neural encoder are trained jointly.

[0015] According to some embodiments, the first neural encoder and the second neural encoder are trained so as to maximize the similarity of the first and second vector representations for authentic images and to minimize the similarity of the first and second vector representations for fraudulent images or images not from the same acquisition.

[0016] According to some embodiments, the first band of wavelengths is in the visible spectrum, in particular between 380 and 780 nm, and the second band of wavelength is in the infrared spectrum, in particular between 800 nm and 960 nm.

[0017] A computer program product is also proposed comprising instructions for implementing the process according to the invention, when this program is executed by a processor.

[0018] A non-transient recording medium readable by a computer is also proposed on which a program is recorded for the implementation of the method according to the invention when this program is executed by a processor.

[0019] A fraud detection device is also proposed in a biometric recognition system characterized in that it comprises a processor configured to execute the following steps: - obtaining a first image of an area of ​​interest of a subject in a first wavelength band; - obtaining a second image of the subject's area of ​​interest in a second wavelength band; - encoding by a first neural encoder of the first image to obtain a first vector representation of the first image; - encoding by a second neural encoder of the second image to obtain a second vector representation of the second image; - calculation of a similarity measure between the first vector representation and the second vector representation; - determination of fraud if the similarity measure is below a predefined threshold.

[0020] The invention thus provides greater robustness to shooting conditions, such as the orientation of the face relative to the camera. Indeed, unlike a classifier, the pose—that is, the orientation of the face relative to the camera—is unknown during training and therefore can be mishandled. With the invention, however, since the pose is identical for both views, it has little influence on the similarity between the two encodings. Similarly, the invention provides greater robustness to changes in expression.

[0021] This program may use any programming language (for example, an object-oriented language or other), and may be in the form of interpretable source code, partially compiled code or fully compiled code.

[0022] Another aspect relates to a non-transient storage medium for a computer-executable program, comprising a set of data representing one or more programs, said one or more programs comprising instructions for, when said one or more programs are executed by a computer comprising a processing unit operationally coupled to memory means and an input / output interface module, to execute all or part of the process described above. Brief description of the drawings

[0023] Other features, details, and advantages of the invention will become apparent upon reading the detailed description below. This description is purely illustrative and should be read in conjunction with the accompanying drawings, in which: Fig. 1

[0024] [Fig.1] illustrates a fraud detection system according to the prior art; Fig. 2

[0025] [Fig.2] illustrates a fraud detection system according to one embodiment of the invention; Fig. 3

[0026] [Fig. 3a] and [Fig. 3b] illustrate the learning of a fraud detection system according to a first example of an embodiment of the invention; Fig. 4

[0027] [Fig.4] illustrates the main steps of a fraud detection process according to an example of an embodiment of the invention. Fig. 5

[0028] [Fig.5] illustrates a schematic block diagram of an information processing device for the implementation of one or more embodiments of the invention. Detailed description

[0029] Fig. 1 illustrates a prior art fraud detection system.

[0030] The fraud detection system is typically integrated into a larger application designed for person recognition. This recognition is based on one or more images of a specific area of ​​interest of the person to be recognized. This area of ​​interest is a face in the embodiment shown. However, the invention applies equally to other areas of interest such as the fingertip for fingerprint recognition, or an image of the person's eye for iris recognition.

[0031] Fraud attempts are essentially aimed at deceiving the recognition system. Typically, a person seeks to be identified by the system as another person. For example, a person presents a photograph of an authorized individual in an attempt to gain access to a building or room under the control of the recognition system. This photograph may be presented as a printed image or displayed on a tablet screen, for example.

[0032] Other fraud attempts simply consist of avoiding identification. This might be the case for individuals wanted by the authorities, for example. In this case, the fraud attempt aims to alter the appearance of the area of ​​interest through makeup, accessories such as special glasses, or even objects placed in the mouth to modify the contour of the individual's cheeks.

[0033] The fraud detection system takes as input an image 101 of the area of ​​interest of the person to be identified. The image is typically taken by a camera. The subject can then be illuminated to ensure proper lighting of the area of ​​interest.

[0034] The simplest systems use a single image 10¹ taken by a camera in the visible spectrum, that is, in a wavelength band between 380 and 780 nm. Such an image 10¹ is typically a color image comprising three components: red, green, and blue. This means that each point in the image is defined by three different numerical values: a red value, a green value, and a blue value. These images are called RGB.

[0035] More sophisticated systems couple the image in the visible wavelength range with a second image taken in a different wavelength band. For example, this second wavelength band could be the ultraviolet band, or the infrared wavelength band between 800 nm and 860 nm, or a band centered on 940 nm, for example between 920 nm and 960 nm. The non-limiting embodiment example uses the infrared band as the second wavelength band. The choice of the second wavelength band is made according to the visual rendering of the area of ​​interest in that band; the choice is therefore open.

[0036] Choosing infrared as the second wavelength band has the advantage of being outside the visible spectrum. Most fraud attempts are designed to deceive the system in the visible range, and choosing a second wavelength band outside the visible spectrum generally makes it possible to thwart them. Infrared is also close to the visible spectrum, which facilitates focusing, particularly in the case of a single camera capturing both the visible and infrared spectrums during the same acquisition.

[0037] The input image 101 can then have a fourth component added, corresponding to the image in the infrared range; this is then referred to as a four-component RGB IR image. Alternatively, the infrared image replaces one of the components of the visible image, for example, blue; this is then referred to as an RGIR image. The input image is then a combined image, combining the two wavelength ranges.

[0038] Alternatively, the visible and infrared images are not combined. The system illustrated in [Fig. 1] is then duplicated. A first system processes the visible image and the second processes the infrared image; the output of the two systems is used to obtain the final result.

[0039] It is possible to use two different cameras to obtain the visible and infrared images. These cameras must be close together and synchronized to minimize any possible difference in exposure between the two images.

[0040] Advantageously, a single camera is used, allowing both wavelength bands to be obtained on a single sensor. This solution provides the best results because it guarantees synchronization and a unique pose for the subject between the two images.

[0041] The input image 101 is provided to an encoder 102 to obtain a vector representation of the input image. This encoder is a neural network, for example using the architecture called EfficientNet described in the article: “EfficientNet: Rethinking Model Scaling for Convolutional Neuronal Networks” by Mingxing Tan and Quoc V. Le.

[0042] The vector representation of the image is then submitted to a classifier 103 to produce a result 104. The classifier is also a neural network. The result is binary and gives the system's determination of the authenticity of the input image 101, that is, whether this image is fraudulent or not. Alternatively, the result is a real number, for example between 0 and 1, which gives a probability of fraud on the input image 101.

[0043] Such a system is trained on a set of images known to be authentic (non-fraudulent) and images known to be fraudulent. The fraudulent images used during training are generated using fraud techniques. known. This is one of the weaknesses of these systems. Faced with a new fraud technique not considered during the system's training, the outcome is unpredictable. The system according to the invention therefore improves the performance of such systems and makes them less sensitive to changes in pose and facial expression.

[0044] Figure 2 illustrates a fraud detection system according to one embodiment of the invention.

[0045] This system takes as input two images 201 and 211 in two different wavelength bands of the subject, and more precisely of the subject's area of ​​interest. Advantageously, these images are taken simultaneously, minimizing the difference in viewing angle. As explained previously, the best results are obtained with a single camera that allows simultaneous capture of the two wavelength bands by the same sensor. In this way, perfectly synchronized image capture and precisely the same viewing angle are guaranteed.

[0046] In the embodiment example, a single camera is used which produces an RGIR image, that is to say comprising three components, a red component, a green component and an infrared component.

[0047] Each image is then encoded by an encoder 202, 212, specialized for the wavelength band used. Encoder 202 is specialized for encoding RG images, while encoder 212 is specialized for infrared images. The specialization of the encoders is achieved by training these neural networks on images in the given wavelength band.

[0048] The result 203, 213 of each encoder is a vector representation of their respective input image 201, 211.

[0049] Instead of submitting the vector representations of the images to a classifier as in the prior art, the decision 204 regarding the fraudulent appearance of the input images is here deduced from a similarity measure 202 between the two vector representations 203, 213 associated with the two input images in their respective wavelength bands.

[0050] The similarity measurement between the vector representations 203, 213 of the input images is performed, for example, by a normalized dot product of the vectors constituting these vector representations. This normalized dot product corresponds to a cosine between the directions of the vectors. The result of the dot product is then compared to a predefined threshold above which the similarity between the vectors is considered to define authentic images. Below the threshold, the input images are considered fraudulent. In the embodiment shown, the value of the threshold is set at 0.38.

[0051] Any other measure of similarity between vectors can be used as an alternative to the normalized dot product, for example a measure of the Euclidean distance between vectors.

[0052] The method thus implemented has the advantage of being able to detect fraudulent images that might be generated using new fraud techniques. Indeed, it is unlikely that the vector representations resulting from these new fraud techniques would be similar in the two wavelength bands analyzed.

[0053] The method thus described can be generalized to more than two wavelength bands by generating a vector representation of each analyzed band. The similarity measurement can then be performed pairwise between the obtained vector representations. Alternatively, the average of the vector representations can be calculated, and then the dot products between the vectors and this average calculated. The result can then be produced by comparing each similarity measurement to the predefined threshold, the images being determined to be authentic if none of the similarity measurements exceeds the threshold. Alternatively, an average of the similarity measurements can be compared to the predefined threshold to obtain the result.

[0054] Figures 3a and 3b illustrate the learning of encoders 202 and 212 used by the fraud determination system illustrated in [Fig.2].

[0055] The system in [Fig. 3a] takes the input images 301 and 311 corresponding to the input images 201 and 211, the encoders 302 and 312 corresponding to the encoders 302 and 312, to generate the vector representations 303 and 313. The two encoders 302 and 312 are trained jointly so as to minimize or maximize the similarity 304 of the vector representations 303 and 313 as a function of the input images 301 and 311.

[0056] Figure 3b illustrates the processing of input images during system training. Image pairs (321, 331), (322, 332), and (323, 333) are input image pairs used during training. Image pairs (321, 331) and (322, 332) are genuine image pairs, while image pair (323, 333) is fraudulent. The training consists of maximizing the similarity of the vector representations of images 321 and 331 on the one hand, and the similarity of the vector representations of images 322 and 332 on the other. This is illustrated by links 341. Conversely, the similarity of the vector representations of images 321 and 332, 322 and 331, 322 and 333, 323 and 332, 323 and 333 is minimized.The aim is therefore to maximize the similarity between the images of the same pair of authentic images and to minimize the similarity between the images belonging to different pairs of images, represented by arrows 342, as well as between the two images of the same fraudulent pair, represented by arrow 343.

[0057] In the embodiment example, this learning principle is translated by the use, for example, of the following loss function: rOOSRl * V* i / i V / sbniZfR^RG,) , v .. / sifruZu^ZRo) \ Luuooj A = L^-logexpi —7--I +LlogLj^p[--?--- J + AwLfe!.^ --?--J

[0059] The indices i and j correspond to image pairs that can belong to the set of genuine image pairs, called live, or to the set of fraudulent image pairs, called spoof. The function sim is the similarity function. The Z values ​​are the vector representations indexed by the wavelength band and the index of the image pair. [Math.l] T is a "temperature" parameter, for example with a value of 0.1 in the example implementation, and [Math.l] J ^SpOOJ is a coefficient regulating the relative importance of the different terms, for example with a value of 0.5 in the implementation example.

[0060] This contrastive learning method is performed conventionally by creating sets of image pairs. Within each set, pairs of genuine images, pairs of fraudulent images, and pairs of cross-referenced images are created, where the images originate from different acquisitions. The loss function described above encourages the formation of similar vector representations for the genuine image pairs and dissimilar representations for all other pairs.

[0061] Figure 4 illustrates the main steps of the fraud determination process according to one embodiment of the invention.

[0062] In step 401, a first image of the area of ​​interest of a subject is obtained. This first image corresponds to a first wavelength band. In the embodiment shown, this first wavelength band is the visible spectrum.

[0063] In step 402, a second image of the same area of ​​interest of the subject is obtained. This second image corresponds to a second wavelength band. In the embodiment shown, this first wavelength band is the infrared spectrum.

[0064] In step 403, the first image is encoded using a first neural encoder to produce a vector representation of the first image. This first neural encoder is trained on images corresponding to the first wavelength band.

[0065] In step 404, the second image is encoded using a second neural encoder to produce a vector representation of the second image. This second neural encoder is trained on images corresponding to the second wavelength band.

[0066] In step 405, the vector representations of the two images are compared using a similarity function, for example, a normalized dot product. The result of this similarity function is used to determine whether the input images are genuine or fraudulent. If the vector representations are sufficiently similar, for example, by comparing the result of the similarity function to a threshold, the input images are determined to be genuine.

[0067] Figure 5 illustrates a schematic block diagram of an information processing device 500 for implementing one or more embodiments of the invention. The information processing device 500 may be a peripheral device such as a microcomputer, a workstation, or a mobile telecommunications terminal. The device 500 includes a communication bus connected to:

[0068] - a central processing unit 501, such as a microprocessor, denoted CPU;

[0069] - a 502 random access memory, denoted RAM, for storing the executable code of the method for carrying out the invention as well as registers adapted to record variables and parameters necessary for the implementation of the method according to embodiments of the invention; the memory capacity of the device can be supplemented by an optional RAM memory connected to an expansion port, for example;

[0070] - a read-only memory 503, denoted ROM, for storing computer programs for the implementation of the embodiments of the invention;

[0071] - a 504 network interface is normally connected to a communication network on which digital data to be processed is transmitted or received. The 504 network interface can be a single network interface, or composed of a set of different network interfaces (e.g., wired and wireless, interfaces, or different types of wired or wireless interfaces). Data packets are sent on the network interface for transmission or are read from the network interface for reception under the control of the software application running in the 501 processor;

[0072] - a 505 user interface for receiving input from a user or for display information to a user;

[0073] - a storage device 506 as described in the invention and noted HD;

[0074] - a 507 input / output module for receiving / sending data to / from external devices such as hard drives, removable storage media or others.

[0075] The executable code can be stored in read-only memory 503, on the storage device 506 or on a removable digital medium such as, for example, a disk. According to one variant, the executable code of the programs can be received by means of a communication network, via network interface 504, in order to be stored in one of the storage means of the communication device 500, such as the storage device 506, before being executed.

[0076] The central processing unit 501 is adapted to command and direct the execution of instructions or portions of software code of the program or programs according to one of the embodiments of the invention, instructions which are stored in one of the aforementioned storage means. After power-up, the CPU 501 is capable of executing instructions from the main RAM 502, relating to a software application. Such software, when executed by the processor 501, causes the execution of the processes described.

[0077] In this embodiment, the device 500 is a programmable device that uses software to implement the invention. However, alternatively, the device 500 can be implemented, in whole or in part, in hardware (for example, in the form of a specific integrated circuit or ASIC).

[0078] Naturally, to satisfy specific needs, a person competent in the field of the invention may apply modifications to the preceding description.

[0079] Although the present invention has been described above with reference to specific embodiments, the present invention is not limited to specific embodiments, and modifications which fall within the scope of the present invention will be obvious to a person versed in the art.

[0080] Although described through a number of detailed embodiments, the proposed method and the equipment for implementing the method include various variants, modifications, and improvements that will be obvious to those skilled in the art, it being understood that these various variants, modifications, and improvements form part of the scope of the invention, as defined by the following claims. Furthermore, different aspects and features described above may be implemented together, separately, or substituted for one another, and all the different combinations and subcombinations of aspects and features form part of the scope of the invention. In addition, some of the systems and equipment described above may not incorporate all the modules and functions described for the preferred embodiments.

Claims

Demands

1. A method for determining fraud in a biometric recognition system characterized in that it comprises the following steps: - obtaining a first image of a subject's area of ​​interest in a first wavelength band; - obtaining a second image of the subject's area of ​​interest in a second wavelength band; - encoding the first image by a first neural encoder to obtain a first vector representation of the first image; - encoding the second image by a second neural encoder to obtain a second vector representation of the second image; - calculating a similarity measure between the first vector representation and the second vector representation; and - determining fraud if the similarity measure is less than a predefined threshold.

2. A method according to claim 1, characterized in that the first image and the second image are captured at the same instant.

3. Method according to claim 2, characterized in that the first image and the second image are captured by the same camera on the same sensor.

4. A method according to any one of claims 1 to 3, characterized in that the similarity measure is a normalized scalar product.

5. A method according to any one of claims 1 to 4, characterized in that the first neural encoder and the second neural encoder are driven jointly.

6. A method according to claim 5, characterized in that the first neural encoder and the second neural encoder are trained to maximize the similarity of the first and second vector representations for authentic images and to minimize the similarity of the first and second vector representations for fraudulent images or images not from the same acquisition.

7. A method according to any one of claims 1 to 5, characterized in that the first band of wavelengths is in the visible spectrum, in particular between 380 and 780 nm, and the second

8.

9.

10. the wavelength band is in the infrared spectrum, specifically between 800 nm and 960 nm. Product computer program comprising instructions for implementing the process according to any one of claims 1 to 7, when this program is executed by a processor. Non-transient computer-readable recording medium on which is recorded a program for implementing the method according to any one of claims 1 to 7 when this program is executed by a processor. Fraud detection device in a biometric recognition system characterized in that it comprises a processor configured to execute the following steps: - obtaining a first image of an area of ​​interest of a subject in a first wavelength band; - obtaining a second image of the subject's area of ​​interest in a second wavelength band; - encoding by a first neural encoder of the first image to obtain a first vector representation of the first image; - encoding by a second neural encoder of the second image to obtain a second vector representation of the second image; - calculation of a similarity measure between the first vector representation and the second vector representation; - determination of fraud if the similarity measure is below a predefined threshold.

Images