IMAGE PROCESSING METHOD FOR AN IDENTITY DOCUMENT

Description

FIELD OF INVENTION

[0001] The present invention relates to the field of image processing by deep learning, or "deep-learning" for the purpose of verifying identity documents.

[0002] An identity document is assigned to a bearer. It includes at least one data page, or datapage for an identity document such as a passport, which is a page personalized to the document bearer and which includes a background (graphic), most often a photograph of the bearer and variable data relating at least to the bearer.

[0003] An identity document can be official, that is, issued by a state or an administrative authority; or unofficial.

[0004] The present invention relates primarily to official sovereign identity documents, but can also be applied to any official document, including property titles.

[0005] For the sake of brevity, the term "identity document" hereafter refers to any official or unofficial document, identity or otherwise, comprising invariant data, i.e., data whose content and position are common to at least two identity documents, regardless of the identity of the bearer of the identity document, for example the wording of a field such as "nationality" on a passport or a graphic security element such as a guilloche; and comprising variable data, i.e., data specific to an identity document, for example the name of the bearer of the identity document.

[0006] There are several natures identity documents, for example passports, identity cards, residence permits, driving licenses, subscription cards, etc., which come from all issuing countries.

[0007] For the same type of document, there may be different types of identity documents, for example an "ordinary" type of passport, a "service" type of passport and a "diplomatic" type of passport.

[0008] Some identity documents include error-prone fonts, which include deliberate and known errors, particularly in serifs.

[0009] The same type of identity document can include several models. For example, a type of "ordinary" passport for a given country may include several models, or versions, depending in particular on their date of manufacture.

[0010] Identity documents can be categorized into a set of classes, such that each class is a unique combination of, in particular, a nature, a type and a model of identity document.

[0011] An identity document can be authentic or false, that is to say counterfeit or falsified.

[0012] A counterfeit identity document is created entirely from scratch, for example by a state or by a counterfeiter. Depending on the resources invested in the forgery, it can be virtually undetectable or more or less blatant.

[0013] A falsified identity document is an authentic identity document in which one or more elements have been fraudulently altered.

[0014] Generally, falsifications usually involve either a modification of variable data, i.e., data relating to the bearer of the document or to the document itself, for example his surname, his age, the expiry date of the document etc., or the replacement of the photograph of the bearer of the authentic identity document with the photograph of a third party.

[0015] Identity documents containing error fonts are very difficult to falsify because the counterfeiter does not know the intentional errors.

[0016] There are also blank identity documents, that is, not yet officially personalized, which are stolen and whose personalization is fraudulent; such documents are called "stolen blanks".

[0017] However, authentic identity documents can present different variations, even for the same type.

[0018] For example, two distinct models of the same identity document may have manufacturing differences without losing their authenticity or validity at the same time.

[0019] Furthermore, when a new authentic identity document is issued, there is no such thing as a fake identity document.

[0020] Finally, there are few, if any, specimens of identity documents, especially official ones. And these specimens are filled with the same personalizations, making them difficult to use as a training ground for a deep learning algorithm.

[0021] For each of these reasons, and a fortiori for all of them, the verification of an identity document is relatively complicated, especially when the date of verification is close to the date of manufacture of said document or of the type of said document.

[0022] To date, the verification of an identity document is based on human analysis and / or on computer analysis based on a verification algorithm involving a large quantity of authentic identity documents and which requires adaptation to each model and each version of the model.

[0023] To address these problems, a solution based on artificial intelligence is proposed which aims to analyze at least the variable data of the identity document.

[0024] In this field, the document "Optical Font Recognition in Smartphone-Captured Images and its Applicability for ID Forgery Detection" (Chernyshova et al., 2018-10-18, XP081143908, DOI: 10.1117 / 12.2522955) is known. However, this document provides for a digital processing which aims to facilitate character recognition (OCR) and which covers all the characters of an alphabet.

[0025] Contrary to this teaching, the present invention aims at treating each characteristic individually, as described below. SUMMARY OF THE INVENTION

[0026] More specifically, according to one of its objects, the invention relates to a method for processing the image of a candidate identity document comprising a data page, the method comprising steps consisting of: acquire a digital image of the data page of the candidate's identity document.

[0027] It is essentially characterized by the fact that it also includes: assign a class or superclass to the candidate identity document by automatic classification of said digital image, by a machine learning algorithm previously implemented on a set of reference images during a learning phase, process the digital image to obtain a set of at least one intermediate image whose weight is less than or equal to the weight of said digital image, discriminate said intermediate image by a discriminator neural network, and emit an output signal from the discriminator neural network, the value of which is representative of the probability that the candidate identity document is an authentic document or a fake.

[0028] In one embodiment, the class or superclass assignment step is implemented by an unsupervised learning algorithm, by a pre-trained classifier neural network, preferably a convolutional neural network.

[0029] Prior to the output signal transmission stage, steps can be planned that consist of: extract a set of at least one zone from: a zone corresponding to all or part of the automatic reading zone of the candidate identity document, a zone corresponding to the top band of the candidate identity document, and a zone corresponding to the outer part of the candidate identity document, each zone comprising glyphs, of at least one of the same font or a similar graphic object, extract each glyph individually from the set of at least one zone, by optical glyph recognition, save each extracted glyph as an individual image, and analyze the fonts of the extracted glyphs.

[0030] In one embodiment, each intermediate image is obtained by one of the following: A processing step for each individual image by a generator neural network, A processing step for each individual image by a recurrent neural network, A post-processing step for each individual image.

[0031] In one embodiment, the first generating network is an auto-encoder, preferably a convolutional one.

[0032] In one embodiment, the automatic classification step includes assigning a predetermined class based on at least one of the following characteristics: The nature of the identity document; the country or state that issued the identity document; the type of identity document; the version or generation number of the identity document

[0033] Furthermore, a data augmentation stage can be anticipated during the learning phase.

[0034] In one embodiment, when said set of at least one area includes variable data, the process further includes a step of blurring said variable data.

[0035] In addition, a step can be planned to check the optical security features of the identity document.

[0036] According to another of its objects, the present invention relates to a computer program comprising program code instructions for executing the steps of the process according to the invention, when said program is executed on a computer.

[0037] Thanks to the present invention, learning can be carried out continuously, thus allowing the database to be enriched regularly, which limits the risks of drift.

[0038] The present invention does not require training the system completely ex nihilo; it can advantageously be implemented from pre-trained networks, in particular on identity documents or on ad hoc images including the glyphs implemented on identity documents.

[0039] Other features and advantages of the present invention will become more apparent from the following description, given by way of illustrative and non-limiting example and made with reference to the accompanying figures. The attached drawings illustrate the invention:

[0040] [ Fig. 1 ] illustrates one way in which a passport data page could be implemented, and [ Fig. 2 ] illustrates a confusion matrix according to an embodiment of the process according to the invention, [ Fig. 3 ] illustrates a glyph "A" at the input of a first generator network and at the output of the second discriminator network according to an embodiment of the invention, [ Fig. 4] illustrates one embodiment of the process according to the invention, [ Fig. 5 ] illustrates a schematic diagram of the invention, and [ Fig. 6 ] illustrates a schematic diagram of a GAN according to the invention. DETAILED DESCRIPTION

[0041] The present invention is based on a solution incorporating artificial intelligence. It includes a preliminary glyph training phase using authentic identity documents, followed by an exploitation phase using candidate identity documents, during which, preferably, no further training takes place. The training process is therefore preferably continuous, and the exploitation process is temporally separate from the training phase.

[0042] A glyph is defined as a graphic representation (among an infinite number of possibilities) of a typographic symbol, in other words, a character, particularly an alphanumeric one. For the purposes of this invention, a glyph can also be a non-meaningful graphic representation, such as a logo.

[0043] A typeface or font is understood to be a set of glyphs, that is, visual representations of characters of the same family, which groups together all the sizes and weights of the same family, whose style is coordinated, in order to form an alphabet, or the representation of all the characters of a language, complete and coherent.

[0044] The term "font" or "font" for short refers to the set of glyphs or characters corresponding to the same characteristics of size, weight and italics within the same typeface. For example :

[0045] Garamond is a typeface; Garamond Roman bold 12 point is a font.

[0046] As explained in more detail below, the following is meant: "source image": a digital image of an authentic identity document or a candidate identity document; "reference image": all or part of the set of source images; "individual image": individual glyph extracted from a reference image; "intermediate image": image obtained by one of: ∘ A processing step of an individual image by a generator neural network, ∘ A processing step of an individual image by a recurrent neural network, and ∘ A post-processing step of an individual image. Acquisition

[0047] The basic diagram of the invention is illustrated on the figure 5 and described in detail below.

[0048] We plan to acquire a digital image of the data page of an ID document. This image is called the "source image" for authentic ID documents used in the training phase and the "test image" for candidate ID documents used in the operational phase.

[0049] The digital image of the data page of a candidate identity document undergoes a series of image processing operations, described below, which allow the candidate identity document to be assessed as belonging to one class or another in order to identify the candidate identity document for the purpose of its authentication, that is to say, to consider it as probably true or as probably false, with a reasonable degree of confidence, that is to say, greater than a predetermined threshold value.

[0050] For the learning phase, it is planned to store in memory a set of source images which come from a plurality of images of authentic identity documents.

[0051] In this case, all or part of the source image set is stored as a database on a secure server and is referred to hereafter as the reference image set. Therefore, a step can be taken to select all or part of the source image set to generate the reference image set.

[0052] In this case, data acquisition is performed on a plurality of identity documents, defining a plurality of classes, to generate a set of reference images comprising, for example, tens of thousands of images. These images are obtained, for example, by extracting images from a video stream. Each video stream typically corresponds to the acquisition of the identity document under different conditions, for example, different lighting angles, different viewing angles with and without flash, different brightness levels, etc.

[0053] For the operational phase, we also plan to acquire a set of test images including at least one test image. The set of test images may include several test images, for example obtained by extracting frames from a video stream.

[0054] For source images and test images, the video stream is obtained, for example, by a communicating object, in particular a smartphone. Data increase

[0055] For the training phase, an optional data augmentation step is planned for each source image, for example extracted from the video stream, allowing the generation of a larger dataset by varying each image with: predetermined rules, in this case color rules, which allows us to anticipate different colorimetry from different cameras, geometric distortions which correspond to different viewing angles, different resolutions which allows us to anticipate different optical qualities of devices, or resampling to limit the size of the image for transfer, etc.

[0056] This amounts to simulating different conditions for acquiring an identity document.

[0057] This results in a set of reference images that includes, for example, several tens of thousands of reference images per identity document class, each identity document class being authentic. The set of reference images is preferably unique, that is, covering all identity document classes.

[0058] This large set of reference images was determined to allow for the subsequent characterization, during the operational phase, of candidate documents whose image transmitted for analysis is of low resolution or whose colors are not transcribed identically to those of the initial shot, particularly due to differences in the performance of the various acquisition systems on the market or even the aging of the document.

[0059] The reference image set is obtained, for example, by acquiring images of several classes of documents using a connected device. In this case, the acquisitions are made by a smartphone, which is an advantageous connected device for the exploitation of the invention.

[0060] For the exploitation phase, an optional data augmentation step can also be planned, in a similar or identical way to the data augmentation step for the learning phase.

[0061] The process then involves two main phases: a first classification phase to identify the document type, and a second font recognition phase to authenticate the document. Document authentication can be sequential with classification. Alternatively, when classification alone is insufficient to classify a candidate identity document into a single category, authentication may be necessary to assign a single category. For example, the classification of an identity document is sometimes ambiguous between two categories. For instance, a diplomatic passport may be difficult to distinguish from an ordinary passport. Attempting successive authentication against each category may result in a higher likelihood rate for one of the categories, thus leading to a decision. The comparison is made against categories that are labeled.Authentication can also be performed if the document type information is accessible elsewhere, for example if it is known at the time of authentication. Identification of the type of identity document

[0062] To determine the type of identity document, a classification step is planned for both the learning and exploitation phases.

[0063] A first step in the classification process involves classifying all the reference images for the training phase and the candidate images for the exploitation phase according to a set of predetermined classes, preferably using supervised learning and, for example, a convolutional neural network (CNN). In this case, we plan to use the VGG-16 neural network developed by the Visual Geometry Group at the University of Oxford, which is a pre-trained, multi-stage classifying neural network. Advantageously, a VGG-16 neural network can be easily installed on a smartphone because the size of the file representing the network's weights can remain manageable for installation processes, which is not necessarily feasible with more complex neural networks that have larger files.

[0064] Most document identification systems are based on reading and interpreting the first glyphs in the MRZ area, which contain the document type, the issuing country, and the date of issue of the identity document.

[0065] The classification used here allows the class of a document to be automatically identified even if the MRZ area is illegible or truncated and independently of the photo of the bearer of the identity document since this is not extracted for classification.

[0066] The classes are predetermined, for example, based on the intrinsic and / or extrinsic characteristics of the documents.

[0067] For example, for an identity document such as a passport: A first characteristic corresponds to the nature of the identity document: passport, identity card, etc.; a second characteristic corresponds to the country or state of origin of the identity document: for example France, Netherlands, etc.; a third characteristic corresponds to the type of identity document: diplomatic, ordinary, etc.; a fourth characteristic corresponds to the version or generation of the identity document: for example generation 6; etc.

[0068] Combining all or some of these characteristics allows us to predetermine a set of classes, for example: A first class for a 6th generation French diplomatic passport, A second class for a 6th generation French ordinary passport, A third class for a 6th generation Dutch diplomatic passport, Etc.

[0069] For other identity documents, for example a driving licence; other classes are defined possibly based on other characteristics, for example the country, the category of vehicles, the version, etc.

[0070] Each authentic document belongs to a unique class. This class can be learned or declared for the learning phase.

[0071] It is also possible to define in advance a set of superclasses such that each superclass is a combination of, in particular, a nature and at least one of: a set of at least one type; and a set of at least one identity document model. Indeed, it is sometimes difficult to quickly distinguish an ordinary passport from a diplomatic passport, or to differentiate between two different versions of the same type of document.

[0072] During operation, the neural network used, in this case a VGG-16 neural network, allows either a class to be assigned to a candidate identity document, or, failing that, a superclass. Other neural networks can be implemented, for example Inception v3, etc. Extraction

[0073] For classification, the extraction of certain predetermined areas from the data page of the identity document is planned.

[0074] In this case, the plan is to select the following 3 areas: an area corresponding to all or part of the MRZ, an area corresponding to the top band TOP of the identity document, and an area corresponding to the outer part EXT of the identity document.

[0075] The 3 planned zones can include different glyphs according to different fonts.

[0076] MRZ stands for Machine-Readable Zone, the area on an official document designed for automatic reading, identification, and validation. This area is standardized at 23.2 mm for all documents conforming to the recommendations of ICAO standard 9303-2 version 7. The width of the document's footer is that of the document itself (height and width considered in landscape orientation). By extension, if there is no MRZ as defined above, MRZ refers to any area with predefined width and height. The height is calculated from the bottom edge of the document, and the width from the left or right edge of the document when viewed in the reading direction.

[0077] The top banner TOP includes variable or fixed data, for example for a French passport, the terms "passport" and "French Republic", the passport number, the passport type and the issuing country code.

[0078] By extension, TOP means any area of ​​predefined width and height, the height being calculated from the top edge of the identity document and the width being calculated from the left or right edge of the identity document when it is viewed in reading direction.

[0079] The outer (EXT) portion of the identity document also includes variable or fixed data, typically a field header (fixed data) and the value of that field (variable data), for example, the field header "Eye Color" (fixed data) and the value of that field (variable data), for example, "green". It covers the height of the identity document, within a predetermined width stored in memory, the width being determined from the right or left edge of the identity document when viewed from the reading direction.

[0080] For example, the left side of an identity document usually includes a photograph of the bearer. In this case, the outer part (EXT) is the area shown in dotted lines on the figure 2 Conversely, sometimes the right-hand side of the identity document includes a photograph of the holder. In this case, the outer part (EXT) is located on the left side of the identity document (not shown).

[0081] Advantageously, the position of the variable data areas is known, for example from ICAO 9303 documents, accessible for example at the address https: / / www.icao.int / publications / pages / publication.aspx?docnum=9303.

[0082] Preferably, during the learning phase, a step of blurring variable data is planned, in order to avoid overfitting, that is to say, so that the system learns on the basis of the invariant characteristics of the image and not on text elements which are generally variable from one identity document to another.

[0083] We then plan a learning step by the neural network on at least one of these 3 zones MRZ, TOP and EXT, for all or part of the set of reference images.

[0084] We can plan to separate the data available for training into two datasets, one of which is used for training and the other set is used for qualifying the effectiveness of the training.

[0085] The learning phase aims to determine the class of a candidate identity document to which the neural network is applied.

[0086] However, neural methods are not 100% reliable, so the possibility of a candidate identity document belonging to several classes, or the impossibility of assigning a class to a candidate identity document, remains.

[0087] In the case where at least part of the set of reference images includes several identity documents of the same type but of different models, it is also possible to know how the candidate document is situated in the class of identity documents.

[0088] To determine the authenticity of a candidate's identity document classified in a single predetermined class, at least one of the following two processes must then be implemented: a process to detect possible manipulation of glyph fonts, a process to detect possible manipulation of the image.

[0089] For an identity document classified into several categories, at least one of the two processes described above is performed for each category of said document. The authentication result can ultimately determine the category of the identity document. Recognition of glyph fonts

[0090] A step is planned to recognize the glyph fonts on each of the extracted (unblurred) areas.

[0091] The goal is to separate and learn each glyph from each font in each zone. Each glyph is therefore learned individually. It should also be noted that there can be multiple fonts per zone.

[0092] As a general rule, the cast iron in the MRZ zone is standardized and should be OCR-B. However, there may be several variations of this cast iron, and sometimes it is not OCR-B.

[0093] Furthermore, fonts in other regions are unrestricted, and several countries use dedicated private fonts. Identity documents may be produced using special fonts, meaning fonts not commercially available as such. Some special fonts may be based on commercially available fonts that have been modified. A particular category of these fonts, known as "error fonts," involves intentionally omitting elements such as the serif bars of certain glyphs or adding them in unusual orientations to create a visible (but subtle) difference from the original font.

[0094] Finally, it should be noted that the printing method (inkjet, laser or other), in combination with the nature of the printing substrate (natural paper, synthetic, or plastic) can also slightly transform the printed appearance of the initial font.

[0095] During the training phase, each glyph in each alphabet of each identity document in the reference set—that is, each class—is trained. Each glyph is learned individually, for each font in each class, where the class of the authentic identity document is known. Unlike the previous training method, there is no blurring of the image areas used for this processing. Preferably, the training is performed on several authentic identity documents rather than a single specimen, so as to have the maximum number of different glyphs under different conditions, thus optimizing the training.

[0096] The glyph recognition step consists of extracting each of the glyphs from each text field by optical character recognition (OCR) 20, across all areas.

[0097] Each extracted glyph is saved as an individual image, corresponding to a segment of the source or test image. A segmented image is therefore a part of the source or test image containing a single glyph.

[0098] For example, we can implement the Tesseract software developed today by Google, which allows us to extract each glyph individually using bounding boxes.

[0099] Therefore, there is not necessarily a need to outline the glyphs. However, a glyph cleaning step can be planned, consisting of outlining the glyph, that is to say, isolating the glyph from the background.

[0100] At this stage, each individual glyph is therefore an individual image.

[0101] We then plan to create an intermediate image, according to any of the 3 variants below, each intermediate image being an individual image or a processing of a respective individual image. 1st variant: auto-encoder

[0102] In a first variant, we plan a first neural network 31 which is a generator network fed into the input by a set of representatives (instances) of a single glyph, extracted as individual images from the source image(s) or the test image, and configured to generate into the output a set of intermediate images.

[0103] In this case, each individual image input to the generator corresponds to an individual glyph.

[0104] In this case, the first network (generator) 31 is an auto-encoder, that is to say an artificial neural network used for unsupervised learning of discriminating features.

[0105] Preferably, a plurality of autoencoders are used, in this case connected in parallel, each autoencoder having its own kernel, allowing for its respective level of accuracy. Each autoencoder is fed the same image from the glyph font recognition step as input and outputs its own output image.

[0106] In this case, as many autoencoders are planned as there are glyph types (one autoencoder for "a", another autoencoder for "b", etc.). Similarly, a discriminator network, described later, is planned for each glyph type, as illustrated. figure 6 .

[0107] In this case, the intermediate image corresponding to the individual glyph from the glyph font recognition step and entered into the set of auto-encoders is the result of the superposition of the output images of the set of auto-encoders.

[0108] An autoencoder comprises two parts: an encoder 311 and a decoder 312. Preferably, the autoencoder is convolutional.

[0109] The encoder comprises a set of layers of neurons that process the input data in order to build new representations called "encoded" 33 which together constitute a new, more compact representation of the dataset, i.e. presenting fewer descriptors, which makes it possible to reduce the dimensionality of said dataset.

[0110] The decoder includes another set of neural layers that receive the representations of said dataset as output from the encoder and, in turn, process them in an attempt to reconstruct the input data.

[0111] In this case, each pixel of each glyph of each alphabet is an input. At the encoder's output, the representation of the dataset is more compact, for example, by a factor of 100.

[0112] For example, tests were carried out and are illustrated on the figure 3 The first line of the figure 3 illustrates a set of individual glyphs from the glyph recognition stage, in this case glyphs "A", as input to an auto-encoder 31.

[0113] The 2nd line of the figure 3 illustrates the set of encoded images 33, in this case encoded on 36 bytes and represented by color codes at the output of the auto-encoder 31.

[0114] The last line of the figure 3 illustrates the individual glyphs reconstructed by a discriminator network 32 from the encoded images 33.

[0115] Preferably, a data augmentation step is also planned for all 33-encoded images, for example by generating noise on the 33-encoded images. This allows, for example, the simulation of the aging of an identity document, how the identity document is perceived by different sensors, etc. This data augmentation is therefore distinct from the data augmentation described previously, which relates to the optical conditions under which identity documents are photographed.

[0116] Preferably, augmented 33-encoded images participate in the auto-encoder learning phase.

[0117] Each individual glyph from the glyph font recognition stage was entered into the auto-encoder.

[0118] In this first variant, the output of the auto-encoder is a set of intermediate images, each intermediate image corresponding to an individual glyph.

[0119] In this first variant, the auto-encoder 31 and the second discriminator network 32, adversary of the generator and described later as common to the 3 variants, constitute generative adversarial networks or GANs for the acronym of "generative adversarial networks" in English which are a class of unsupervised learning algorithms.

[0120] Classic GANs (https: / / en.wikipedia.org / wiki / Generative adversarial network) exhibit backpropagation from the discriminator network to the generator network. Unlike these classical GANs, a GAN within the meaning of the present invention also comprises a generator network and a discriminator network but does not exhibit backpropagation.

[0121] In this case, we have, per glyph, a generator network, in particular an auto-encoder 31, which learns the said unique glyph, extracted in the form of individual images from the source image(s), and which generates a set of respective intermediate images.

[0122] During the learning phase, by glyph, the discriminator network 32 is fed both by the set of individual images from the source image(s), which are "true" images, and by the respective set of intermediate images, which are variants of the "true" images.

[0123] During the learning phase, by glyph, the discriminator network 32 is also fed by a set of the same glyphs but known as "false".

[0124] We thus have triplet loss learning, or "triplet loss" (https: / / en.wikipedia.org / wiki / Triplet_loss), in which we define the distance, or margin, α as a predetermined threshold value.

[0125] According to the invention, there is therefore one GAN per glyph (one GAN for the "a", one GAN for the "b", etc.).

[0126] In a GAN, two networks are placed in competition. The first network 31 is a generator fed as input by a set of glyphs extracted as individual images from the source image or the test image, and configured to generate as output a set of intermediate images.

[0127] During the training phase, identical images are fed into and out of the autoencoder, allowing the neural network weights to be adjusted through backpropagation. The training phase sets the VGG16 network weights for each glyph in each font. This is equivalent to each extracted glyph being fed into its own discriminator network.

[0128] The learning process is complete when the number of values ​​recognized as true by the discriminator network exceeds a predetermined threshold value.

[0129] During operation, only the discriminator network is implemented. It is fed a set of test images and determines, based on its training, whether the glyph in the test image it receives as input is a "true" or a "false".

[0130] We can also foresee a step consisting of transforming the representation space of the intermediate image.

[0131] In this case, the plan is to convert the intermediate image from the RGB space to the CIE 1976 L*a*b* color space, commonly known as CIELAB, which is a cylindrical coordinate visualization of colors. Alternatively, spherical, logarithmic, or other encoding methods could be used.

[0132] Transforming the representation space amplifies weak signals, thus making it easier to detect counterfeit identity documents. In this case, the CIELAB transformation detects colored patches present on the overlay of different images in counterfeit identity documents. This is particularly useful when the authentic font is an "error font" and intentionally differs, for example, by the presence or absence of serif strokes on certain character edges. 2nd variant: LSTM

[0133] In a second variant, a recurrent neural network, or LSTM (Long Short-Term Memory), is used. This network is fed input with a set of glyphs extracted as individual images from the source or test image, and outputs a set of intermediate images, each corresponding to a very good segmentation of the glyph obtained at the end of the glyph recognition stage, through co-segmentation. This variant is particularly applicable when acquiring image or video sequences of the original identity document, where a sequence can be reconstructed between glyphs of the same category.

[0134] The intermediate image is the result of an assembly of superimposed images taken from a sequence of source or test images.

[0135] In this variant, co-segmentation implies that each intermediate image obtained from the output of the LSTM is obtained by processing a sequence of segmented images, where each segmented image in the sequence includes the same individual glyph from the glyph recognition step.

[0136] A sequence of segmented images represents the evolution of a glyph over the time of their acquisition. For a given sequence, each segmented image in the sequence corresponds to a given glyph from a respective source image, where each respective source image corresponds to a unique augmented source image.

[0137] We therefore have a sequence of segmented images as input to the LSTM. The output of the LSTM is a single intermediate image, derived from the input sequence of segmented images and corresponding to the most probable contours of the glyph. These are therefore the most distinguishable or identifiable contours in relation to the original. 3rd variant: OCR post-processing

[0138] In a third variant, a post-processing step is planned for the image resulting from the glyph recognition step.

[0139] In this variant, each intermediate image obtained from the post-processing is obtained by processing a single source image or segmented test image, which includes the same individual glyph from the glyph recognition step.

[0140] The post-processing stage consists of improving the visual quality of the image from the glyph recognition stage, for example by rotation, resizing, straightening, cleaning up outliers, etc.

[0141] This step increases the dispersion of fake identity documents relative to the centroid of the cloud of real identity documents.

[0142] Besides the 3 variants described above, we can also foresee an invariant function, that is to say that we can foresee directly introducing the individual images of each individual glyph into the input of the discriminator network described below, so that each individual image is an intermediate image.

[0143] We can also plan to implement successively any two variants from the three variants above, or even all three variants, for a decision-making process using three times as many clues. Discriminator network 32

[0144] The authentication of an identity document is carried out by a neural network, called a discriminator network 32, whose input, during the learning phase, is fed by a set of intermediate images according to any of the previous variants.

[0145] For example, the discriminator network 32 is the discriminator network of a GAN described previously. As also described previously, one can predict a plurality of discriminator networks 32, one for each glyph type. However, for the sake of brevity, we will only describe one discriminator network 32.

[0146] Preferably, a convolutional neural network (CNN) is used as the discriminator network. In this case, the VGG-16 neural network is used. Training the discriminator network on the source images allows it to detect any manipulation of the glyph fonts in the test images, thus detecting any manipulation of the identity document. The discriminator network can also be trained on the test images.

[0147] The discriminator is configured to detect whether the sample is real or generated by the generator. The neural network's learning is "finalized" when the discriminator can no longer distinguish an encoded image (generated by encoder 311) from a real image, that is, one belonging to the set of reference images. Authentication

[0148] For each individual glyph of which an intermediate image feeds the input of the discriminator network 32, said discriminator network 32 presents two levels of output: a first level of output according to which the individual glyph is considered probably authentic, and a second level of output according to which the individual glyph is considered probably false.

[0149] An individual glyph is considered probably authentic when it corresponds to a glyph learned during the learning phase, that is, when its discriminator network no longer differentiates between the individual glyph and the corresponding learned glyph.

[0150] An identity document displays several glyphs in different fonts. Therefore, for a candidate identity document, it can be advantageous to implement a combination of the output levels of the discriminator networks for all intermediate images and generate an overall output signal corresponding to a predefined mathematical function indicating whether the candidate identity document is likely authentic or likely counterfeit with a higher level of certainty. This function can be implemented, for example, using majority voting, qualified voting, weighted voting, etc.

[0151] For example, a candidate identity document may be considered authentic by calculating a ratio of the number of glyphs recognized as valid to the number of glyphs recognized as invalid of said document, or to the total number of characters recognized, possibly followed by a thresholding step of said ratio.

[0152] The overall result can also be weighted by assigning individual weights to glyphs based on the zone from which they are extracted. For example, an "A" glyph from the MRZ zone may have a different weight than an "A" glyph from the TOP zone.

[0153] Neural network learning can also be used to optimize these relative weights.

[0154] The present invention can be implemented on glyphs other than printed glyphs; for example, it is applicable to perforated fonts or logos. Indeed, a small, sufficiently contrasting logo is similar to a character in a font.

[0155] Advantageously, the present invention can be implemented by a communicating object, in particular a smartphone.

Claims

1. Method for processing an image of a candidate identity document comprising a data page containing several glyphs from several fonts, the method comprising a step consisting of - acquiring a digital image of the data page of the candidate identity document, optionally by a communicating object, each digital image being further extractable from a video stream recorded by said communicating object; characterised in that it further comprises the steps of - assigning a class or superclass to the candidate identity document by automatically classifying the digital image using a machine learning algorithm implemented on a set of reference images during a learning phase in which each glyph is learned individually for each font in each class, - extracting a set of glyphs from the digital image and saving each glyph as an individual image, - process each individual image to obtain a set of at least one intermediate image, each intermediate image corresponding to an individual glyph, each intermediate image being obtained by at least one of the following steps: A step of processing each individual image by a neural network generative neural network, a step of processing each individual image by a recurrent neural network, and a step of post-processing each individual image, - discriminate each glyph of each font of said intermediate image by a respective discriminator neural network, each individual glyph being considered likely to be authentic when it corresponds to a glyph learned during a learning phase, and - emitting an output signal by combining the output signals of the discriminator neural networks, the value of which is representative of the probability that the candidate identity document is an authentic identity document or a forgery; the method comprising a prior learning step for each of the glyphs of each of the alphabets of each of the identity documents in a reference set, each glyph being learned individually, for each font of each class, where the class of each authentic identity document is known; the learning being preferably implemented on several authentic identity documents; the authentication of an identity document being performed by the discriminative neural network set, whose input is fed with a set of intermediate images during the learning phase.

2. Method according to claim 1, wherein the step of assigning a class or superclass is implemented by an unsupervised learning algorithm, by a pre-trained classifier neural network, preferably a convolutional neural network.

3. Method according to any of the preceding claims, comprising, prior to the output signal transmission step, steps consisting of - extracting a set of at least one area from - an area corresponding to all or part of the machine-readable zone (MRZ) of the candidate identity document, - an area corresponding to the top banner (TOP) of the candidate identity document, and - an area corresponding to the outer part (EXT) of the candidate identity document, each area comprising glyphs, from at least one same font, - individually extracting each glyph from the set of at least one area, by optical glyph recognition, and - comparing each extracted glyph with its corresponding glyph learned during the learning phase; the method optionally further comprising, when said set of at least one area comprises variable data, a step of blurring said variable data during the learning phase.

4. Method according to any of the preceding claims, wherein The step of processing each individual image by a recurrent neural network produces a set of intermediate images, each of which corresponds to a segmentation of the glyph obtained at the end of the glyph recognition step, by cosegmentation.

5. Method according to any of the preceding claims, in which the generator neural network is an autoencoder, preferably a convolutional autoencoder.

6. Method according to any of the preceding claims, wherein the automatic classification step comprises assigning a predetermined class based on at least one of the following characteristics the nature of the identity document; the country or state of origin of the identity document; the type of identity document; the version or generation number of the identity document7. Method according to any of the preceding claims, further comprising a data augmentation step during the learning phase.

8. Method according to any of the preceding claims, wherein the output signal is one of - a majority vote, - a qualified vote, and - a weighted vote output signals from discriminative neural networks.

9. Method according to any of the preceding claims, further comprising a step of checking optical security features of the identity document.

10. Method according to any of claims 3 to 9, comprising a step of assigning individual weights to the glyphs according to the area from which they are extracted.

11. Method according to any of the preceding claims, comprising a step of considering a candidate identity document to be authentic by calculating a ratio of the number of glyphs recognised as valid to the number of glyphs recognised as invalid in said document, or to the total number of recognised characters, possibly followed by a step of thresholding said ratio.

12. A computer programme comprising programme code instructions for performing the steps of the method according to any of the preceding claims, when said programme is executed on a computer.