Method and device for determining fraud in a biometric image recognition system.

Abstract

The present invention relates to a method for determining fraud in a biometric recognition system, characterized in that it comprises the following steps: obtaining a first image of an area of ​​interest of a subject in a first wavelength band; obtaining a second image of the area of ​​interest of the subject in a second wavelength band; encoding by a first neural encoder the first image to obtain a first vector representation of the first image; encoding by a second neural encoder the second image to obtain a second vector representation of the second image; calculating a similarity measure between the first vector representation and the second vector representation; and determining fraud if the similarity measure is less than a predefined threshold.[Fig. 2]

Description

Title of the invention: Method and device for determining fraud in a biometric image recognition system.

[0001] The invention relates to a method for determining fraud in a biometric image recognition system based on the analysis of at least two images of the same object, for example a face, obtained in at least two distinct wavelength bands.

[0002] Many biometric recognition systems of a person are based on the analysis of images of the person. Generally, a particular part of the person, the area of ​​interest, is used for recognition. This can be the face, but also the fingerprint or the iris of the person, this list being non-exhaustive.

[0003] Systems based on the analysis of a single image, typically in the visible spectrum, of the area of ​​interest allow a reasonable, although improvable, recognition rate. Such systems, however, prove to be susceptible to fraud. They can be fooled by images constructed for this purpose and presented in place of the person's area of ​​interest itself.

[0004] One way to make these systems more robust is to base the recognition on a pair of images of the area of ​​interest obtained in different wavelength bands, for example a first image is obtained in the visible spectrum and a second image is obtained in the infrared spectrum. This is an example, other wavelength bands can be considered. The two images are then submitted to a fraud detector, separately or concatenated. These systems make it possible to improve fraud detection. For example, a face image presented in place of the person's real face, on a photograph or a screen, can produce a realistic image of the face in the visible spectrum but a black image in the infrared spectrum.

[0005] In these systems, the fraud detector is typically a classifier using neural network technology trained to recognize fraud attempts. A first classifier may be specialized in the visible spectrum while a second classifier may be specialized in the infrared spectrum. In this case, the result of the two classifiers is then consolidated to obtain the final result of the fraud detection. Alternatively, the two images are concatenated. For example, the infrared image is added as an additional component to the red, green and blue components of the visible image. Alternatively, the infrared image replaces one of the components to obtain, for example, a RGIR image, i.e. red, green and infrared. It is then this image combining the two different wavelength bands which is provided as input to a single classifier to detect fraud.

[0006] These systems offer better fraud detection, but still suffer from inadequacies. In particular, these classifiers are trained on given facial poses, expressions and for known fraud techniques. Faced with a new pose, expression or fraud technology, their result is unpredictable.

[0007] The invention presented aims to solve this problem. Statement of the invention

[0008] To this end, the invention proposes to subject the input images to encoding. Each image is encoded using an encoder based on a neural network. The result of the encoding is a vector of data representative of the image. Fraud is then detected by a similarity calculation between the encoded representations of the images in each wavelength band. The encoders are trained jointly to maximize the similarity of the encoded representations of the images coming from the same person and to minimize the similarity when this is not the case.

[0009] This approach makes it possible to obtain better fraud detection results compared to known systems based on classifiers. In particular, when faced with a new fraud technology that has not been taken into account during training, the risk that fraudulent images produce similar representations is low. In addition, the invention improves robustness to shooting conditions.

[0010] A method for determining fraud in a biometric recognition system is thus proposed, characterized in that it comprises the following steps: - obtaining a first image of an area of ​​interest of a subject in a first wavelength band; - obtaining a second image of the subject's area of ​​interest in a second wavelength band; - encoding by a first neural encoder of the first image to obtain a first vector representation of the first image; - encoding by a second neural encoder of the second image to obtain a second vector representation of the second image; - calculating a similarity measure between the first vector representation and the second vector representation; and - determination of fraud if the similarity measure is below a predefined threshold.

[0011] According to some embodiments, the first image and the second image are captured at the same time.

[0012] According to certain embodiments, the first image and the second image are captured by the same camera on the same sensor.

[0013] According to some embodiments, the similarity measure is a normalized dot product.

[0014] According to some embodiments, the first neural encoder and the second neural encoder are jointly trained.

[0015] According to certain embodiments, the first neural encoder and the second neural encoder are trained so as to maximize the similarity of the first and second vector representations for authentic images and to minimize the similarity of the first and second vector representations for fraudulent images or images not originating from the same acquisition.

[0016] According to certain embodiments, the first wavelength band is in the visible spectrum, in particular between 380 and 780 nm, and the second wavelength band is in the infrared spectrum, in particular between 800 nm and 960 nm.

[0017] A computer program product is also provided comprising instructions for implementing the method according to the invention, when this program is executed by a processor.

[0018] Also provided is a non-transitory recording medium readable by a computer on which a program is recorded for implementing the method according to the invention when this program is executed by a processor.

[0019] There is also provided a fraud determination device in a biometric recognition system characterized in that it comprises a processor configured to execute the following steps: - obtaining a first image of an area of ​​interest of a subject in a first wavelength band; - obtaining a second image of the subject's area of ​​interest in a second wavelength band; - encoding by a first neural encoder of the first image to obtain a first vector representation of the first image; - encoding by a second neural encoder of the second image to obtain a second vector representation of the second image; - calculating a similarity measure between the first vector representation and the second vector representation; - determination of fraud if the similarity measure is below a predefined threshold.

[0020] The invention therefore makes it possible to be more robust to shooting conditions, such as the orientation of the face relative to the camera. Indeed, unlike a classifier, the pose, that is to say the orientation of the face relative to the camera, not seen during learning, is an unknown piece of data, which may be poorly processed, whereas with the invention, the pose being identical for the two views, it has little influence on the similarity between the two encodings. Similarly, the invention makes it possible to be more robust to changes in expression.

[0021] This program may use any programming language (for example, an object language or other), and be in the form of interpretable source code, partially compiled code or fully compiled code.

[0022] Another aspect relates to a non-transitory storage medium for a computer-executable program, comprising a data set representing one or more programs, said one or more programs comprising instructions for, upon execution of said one or more programs by a computer comprising a processing unit operatively coupled to memory means and to an input / output interface module, to execute all or part of the method described above. Brief description of the drawings

[0023] Other characteristics, details and advantages of the invention will appear on reading the detailed description below. This is purely illustrative and must be read in conjunction with the appended drawings, in which: Fig.l

[0024] [Fig.l] illustrates a fraud detection system according to the prior art; Fig. 2

[0025] [Fig.2] illustrates a fraud detection system according to one embodiment of the invention; Fig. 3

[0026] [Fig. 3a] and [Fig. 3b] illustrate the training of a fraud detection system according to a first exemplary embodiment of the invention; Fig. 4

[0027] [Fig.4] illustrates the main steps of a fraud detection method according to an exemplary embodiment of the invention. Fig. 5

[0028] [Fig.5] illustrates a schematic block diagram of an information processing device for implementing one or more embodiments of the invention. Detailed description

[0029] [Fig.l] illustrates a fraud detection system according to the prior art.

[0030] The fraud detection system is typically integrated into a broader application intended for the recognition of a person. This recognition is based on one or more images of an area of ​​interest of the person to be recognized. This area of ​​interest is a face in the exemplary embodiment. But the invention applies in an identical manner to other areas of interest such as the tip of the finger for fingerprint recognition, or an image of the person's eye for iris recognition.

[0031] Fraud attempts are essentially aimed at trying to deceive the recognition system. Typically, a person seeks to be identified by the system as another person. For example, a person presents the photograph of an authorized person to try to gain access to a building or a room under the control of the recognition system. This photograph may be presented in the form of a “paper” image or displayed on a tablet screen, for example.

[0032] Other fraud attempts simply involve avoiding identification. This may be the case for people wanted by the authorities, for example. In this case, the fraud attempt aims to modify the appearance of the area of ​​interest through make-up, accessories such as special glasses, or even objects placed in the mouth to modify the contour of the individual's cheeks.

[0033] The fraud detection system takes as input an image 101 of the area of ​​interest of the person to be identified. The image is typically taken by a camera. The subject can then be illuminated to ensure correct illumination of the area of ​​interest.

[0034] The simplest systems use a single image 101 taken by a camera in the visible range, that is to say in a wavelength band between 380 and 780 nm. Such an image 101 is typically a color image comprising three components: red, green and blue. This means that each point of the image is defined by three different digital values, a red value, a green value and a blue value. These images are called RGB.

[0035] More sophisticated systems couple the image in the visible wavelength range with a second image taken in another different wavelength band. For example, this second wavelength band may be the ultraviolet band, or the infrared wavelength band between 800 nm and 860 nm, or a band centered on 940 nm, for example between 920 nm and 960 nm. The non-limiting example of embodiment uses the infrared band as the second wavelength band. The choice of the second wavelength band is made according to the visual rendering of the area of ​​interest in this band, the choice is therefore open.

[0036] The choice of infrared as a second wavelength band has the advantage of being outside the visible spectrum. Most fraud attempts are in fact designed to deceive the system in the visible spectrum; choosing a second wavelength band outside the visible spectrum generally allows them to be thwarted. Infrared is also close to visible, which makes focusing easier, particularly in the case of a single camera capturing the visible spectrum and infrared during the same acquisition.

[0037] The input image 101 can then have a fourth component added corresponding to the image in the infrared domain; this is then referred to as a four-component RGB IR image. Alternatively, the infrared image replaces one of the components of the visible image, for example blue; this is then referred to as an RGIR image. The input image is then a combined image, combining the two wavelength domains.

[0038] Alternatively, the visible and infrared images are not combined. The system illustrated in [Fig.l] is then duplicated. A first system processes the visible image and the second processes the infrared image, the result of both systems is used to obtain the final result.

[0039] It is possible to use two different cameras to obtain the visible and infrared images. These cameras must be close and synchronized to minimize a possible difference in pose between the two images.

[0040] Advantageously, a single camera is used to obtain the two wavelength bands on a single sensor. This solution gives the best results because it guarantees the synchronization and uniqueness of the subject's pose between the two images.

[0041] The input image 101 is provided to an encoder 102 to obtain a vector representation of the input image. This encoder is a neural network, for example using the architecture called EfficientNet described in the article: “EfficientNet: Rethinking Model Scaling for Convolutional Neuronal Networks” by Mingxing Tan and Quoc V. Le.

[0042] The vector representation of the image is then submitted to a classifier 103 to produce a result 104. The classifier is also a neural network. The result is binary and gives the system's determination of the authenticity of the input image 101, namely whether this image is fraudulent or not. Alternatively, the result is a real number, for example between 0 and 1 which gives a probability of fraud on the input image 101.

[0043] Such a system is trained on a set of images that are known to be authentic (non-fraudulent) and images that are known to be fraudulent. The fraudulent images used during training are generated from fraud techniques known. This is one of the weaknesses of these systems. Faced with a new fraud technique not considered during the training of the system, the result is not predictable. The system according to the invention then improves the performance of such systems and makes it less sensitive to changes in pose and facial expression.

[0044] [Fig.2] illustrates a fraud detection system according to one embodiment of the invention.

[0045] This system takes as input two images 201 and 211 in two different wavelength bands of the subject and more precisely of the area of ​​interest of the subject. Advantageously, these images are taken at the same time while minimizing the difference in viewing angle. As explained previously, the best results are obtained with a single camera allowing the simultaneous shooting of the two wavelength bands by the same sensor. In this way, the perfect shots are guaranteed to be perfectly synchronous and at exactly the same viewing angle.

[0046] In the exemplary embodiment, a single camera is used which produces an RGIR image, i.e. comprising three components, a red component, a green component and an infrared component.

[0047] Each image is then encoded by an encoder 202, 212, specialized for the wavelength band used. The encoder 202 is specialized for encoding RG images, while the encoder 212 is specialized for infrared images. The specialization of the encoders is obtained by training these neural networks on images in the given wavelength band.

[0048] The output 203, 213 of each encoder is a vector representation of their respective input image 201, 211.

[0049] Instead of submitting the vector representations of the images to a classifier as in the prior art, the decision 204 as to the fraudulent aspect of the input images is here deduced from a similarity measure 202 between the two vector representations 203, 213 associated with the two input images in their respective wavelength bands.

[0050] The measurement of similarity between the vector representations 203, 213 of the input images is carried out, for example, by a normalized scalar product of the vectors constituting these vector representations. This normalized scalar product corresponds to a cosine between the directions of the vectors. The result of the scalar product is then compared to a predefined threshold above which the similarity between the vectors is considered to determine authentic images. Below the threshold, the input images are considered fraudulent. In the exemplary embodiment, the value of the threshold is set to 0.38.

[0051] Any other measure of similarity between vectors can be used as an alternative to the normalized scalar product, for example a measure of the Euclidean distance between vectors.

[0052] The method thus implemented has the advantage of being able to detect fraudulent images which would be generated using new fraud techniques. Indeed, it is unlikely that the vector representations resulting from these new fraud techniques will be similar in the two wavelength bands analyzed.

[0053] The method thus described can be generalized to more than two wavelength bands by generating a vector representation of each analyzed band. The similarity measurement can then be made two by two between the vector representations obtained. Alternatively, the average of the vector representations can be calculated, then the scalar products between the vectors and this average calculated. The production of the result can then be done by comparing each similarity measurement to the predefined threshold, the images being determined as authentic if none of the similarity measurements exceeds the threshold. Alternatively, an average of the similarity measurements can be compared to the predefined threshold to obtain the result.

[0054] Figures 3a and 3b illustrate the training of the encoders 202 and 212 used by the fraud determination system illustrated by [Fig.2].

[0055] The system of [Fig. 3a] takes the input images 301 and 311 corresponding to the input images 201 and 211, the encoders 302 and 312 corresponding to the encoders 302 and 312, to generate the vector representations 303 and 313. The two encoders 302 and 312 are trained jointly so as to minimize or maximize the similarity 304 of the vector representations 303 and 313 as a function of the input images 301 and 311.

[0056] [Fig. 3b] illustrates the processing of input images during training of the system. Image pairs (321, 331), (322, 332) and (323, 333) are input image pairs used during training. Image pairs (321, 331) and (322, 332) are authentic image pairs, while image pair (323, 333) is fraudulent. Training consists of maximizing the similarity of the vector representations of images 321 and 331 on the one hand, as well as the similarity of the vector representations of images 322 and 332 on the other hand. This is illustrated by links 341. Conversely, the similarity of the vector representations of images 321 and 332, 322 and 331, 322 and 333, 323 and 332, 323 and 333 is minimized.It is therefore a question of maximizing the similarity between the images of the same pair of authentic images and minimizing the similarity between the images belonging to different pairs of images, represented by the arrows 342, as well as between the two images of the same fraudulent pair, represented by the arrow 343.

[0057] In the exemplary embodiment, this learning principle is translated by the use, for example, of the following loss function: rOOSRl * V* i / i V / sbniZfR^RG,) , v .. / sifruZu^ZRo) \ Luuooj A = L^-logexpi —7--I +LlogLj^p[--?--- J + AwLfe!.^ --?--J

[0059] The indices i and j correspond to the image pairs that can belong to the set of authentic image pairs, called live, or to the set of fraudulent image pairs, called spoof. The sim function is the similarity function. The Z values ​​are the vector representations indexed by the wavelength band and the index of the image pair. [Math.l] T is a “temperature” parameter, for example with a value of 0.1 in the exemplary embodiment, and [Math.l] I ^SpOOJ is a coefficient regulating the relative importance of the different terms, for example of value 0.5 in the example implementation.

[0060] This contrastive learning is carried out in a conventional manner by constituting batches of image pairs. In each batch, we constitute authentic image pairs, fraudulent image pairs and crossed image pairs, where the images come from different acquisitions. The above loss function encourages the formation of similar vector representations for authentic image pairs and dissimilar ones for all other pairs.

[0061] [Fig.4] illustrates the main steps of the fraud determination method according to one embodiment of the invention.

[0062] In a step 401, a first image of the area of ​​interest of a subject is obtained. This first image corresponds to a first wavelength band. In the exemplary embodiment, this first wavelength band is the visible spectrum.

[0063] In a step 402, a second image of the same area of ​​interest of the subject is obtained. This second image corresponds to a second wavelength band. In the exemplary embodiment, this first wavelength band is the infrared spectrum.

[0064] In a step 403, the first image is encoded using a first neural encoder to produce a vector representation of the first image. This first neural encoder is trained on images corresponding to the first wavelength band.

[0065] In a step 404, the second image is encoded using a second neural encoder to produce a vector representation of the second image. This second neural encoder is trained on images corresponding to the second wavelength band.

[0066] In a step 405, the vector representations of the two images are compared by a similarity function, for example a normalized scalar product. The result of this similarity function is used to determine whether the input images are authentic or, on the contrary, fraudulent. If the vector representations are sufficiently similar, for example by comparing the result of the similarity function to a threshold, it is determined that the input images are authentic.

[0067] [Fig. 5] illustrates a schematic block diagram of an information processing device 500 for implementing one or more embodiments of the invention. The information processing device 500 may be a peripheral such as a microcomputer, a workstation or a mobile telecommunications terminal. The device 500 comprises a communication bus connected to:

[0068] - a central processing unit 501, such as a microprocessor, denoted CPU;

[0069] - a random access memory 502, denoted RAM, for storing the executable code of the method of implementing the invention as well as the registers adapted to record variables and parameters necessary for implementing the method according to embodiments of the invention; the memory capacity of the device can be supplemented by an optional RAM memory connected to an expansion port, for example;

[0070] - a read-only memory 503, noted ROM, for storing computer programs for the implementation of the embodiments of the invention;

[0071] - a network interface 504 is normally connected to a communications network on which digital data to be processed are transmitted or received. The network interface 504 may be a single network interface, or composed of a set of different network interfaces (e.g., wired and wireless interfaces, or different types of wired or wireless interfaces). Data packets are sent over the network interface for transmission or are read from the network interface for reception under the control of the software application running in the processor 501;

[0072] - a user interface 505 for receiving inputs from a user or for display information to a user;

[0073] - a storage device 506 as described in the invention and noted HD;

[0074] - an input / output module 507 for receiving / sending data from / to external devices such as hard drive, removable storage media or others.

[0075] The executable code may be stored in a read-only memory 503, on the storage device 506 or on a removable digital medium such as for example a disk. Alternatively, the executable code of the programs may be received by means of a communications network, via the network interface 504, in order to be stored in one of the storage means of the communications device 500, such as the storage device 506, before being executed.

[0076] The central processing unit 501 is adapted to control and direct the execution of the instructions or portions of software code of the program or programs according to one of the embodiments of the invention, instructions which are stored in one of the aforementioned storage means. After power-up, the CPU 501 is capable of executing instructions from the main RAM memory 502, relating to a software application. Such software, when executed by the processor 501, causes the execution of the described methods.

[0077] In this embodiment, the apparatus 500 is a programmable apparatus that uses software to implement the invention. However, alternatively, the apparatus 500 may be implemented, in whole or in part, in hardware (e.g., in the form of a specific integrated circuit or ASIC).

[0078] Naturally, to satisfy specific needs, a person skilled in the art of the invention may apply modifications to the preceding description.

[0079] Although the present invention has been described above with reference to specific embodiments, the present invention is not limited to the specific embodiments, and modifications that fall within the scope of the present invention will be apparent to a person skilled in the art.

[0080] Although described through a number of detailed exemplary embodiments, the proposed method and the equipment for implementing the method include various variations, modifications and improvements which will be apparent to those skilled in the art, it being understood that these various variations, modifications and improvements are part of the scope of the invention, as defined by the following claims. In addition, different aspects and features described above may be implemented together, or separately, or substituted for each other, and all of the different combinations and sub-combinations of the aspects and features are part of the scope of the invention. Furthermore, some systems and equipment described above may not incorporate all of the modules and functions described for the preferred embodiments.

Claims

Claims

1. Method for determining fraud in a biometric recognition system characterized in that it comprises the following steps: - obtaining a first image of an area of ​​interest of a subject in a first wavelength band; - obtaining a second image of the area of ​​interest of the subject in a second wavelength band; - encoding by a first neural encoder of the first image to obtain a first vector representation of the first image; - encoding by a second neural encoder of the second image to obtain a second vector representation of the second image; - calculating a similarity measure between the first vector representation and the second vector representation; and - determining fraud if the similarity measure is lower than a predefined threshold.

2. Method according to claim 1, characterized in that the first image and the second image are captured at the same time.

3. Method according to claim 2, characterized in that the first image and the second image are captured by the same camera on the same sensor.

4. Method according to one of claims 1 to 3, characterized in that the similarity measure is a normalized scalar product.

5. Method according to one of claims 1 to 4, characterized in that the first neural encoder and the second neural encoder are trained jointly.

6. Method according to claim 5, characterized in that the first neural encoder and the second neural encoder are trained so as to maximize the similarity of the first and second vector representations for authentic images and to minimize the similarity of the first and second vector representations for fraudulent images or images not originating from the same acquisition.

7. Method according to any one of claims 1 to 5, characterized in that the first wavelength band is in the visible spectrum, in particular between 380 and 780 nm, and the second

8.

9.

10. wavelength band is in the infrared spectrum, namely between 800 nm and 960 nm. Computer program product comprising instructions for implementing the method according to one of claims 1 to 7, when this program is executed by a processor. Non-transitory recording medium readable by a computer on which is recorded a program for implementing the method according to one of claims 1 to 7 when this program is executed by a processor. Fraud determination device in a biometric recognition system characterized in that it comprises a processor configured to execute the following steps: - obtaining a first image of an area of ​​interest of a subject in a first wavelength band; - obtaining a second image of the subject's area of ​​interest in a second wavelength band; - encoding by a first neural encoder of the first image to obtain a first vector representation of the first image; - encoding by a second neural encoder of the second image to obtain a second vector representation of the second image; - calculating a similarity measure between the first vector representation and the second vector representation; - determination of fraud if the similarity measure is below a predefined threshold.

Images