Method for preventing duplication of digital sample images

Abstract

A computer-implemented method for training an artificial intelligence-based neural network is provided. At least one digital image of at least one security document is provided as a reference. A set of digital training images is provided, which are altered relative to the digital image of the security document. The set of digital training images includes a first subset of positive digital training images having a visual effect of alteration such that an unbiased human observer may consider a copy of each digital training image to represent a security document or multiple security documents. The set of digital training images includes a second subset of negative digital training images. A ground truth is provided to the neural network as to whether a copy of each digital training image is considered to represent one or more security documents.

Description

[Technical field]

[0001] The present invention relates to a method for training an artificial intelligence based neural network used to classify whether a document is considered a security document without authenticating its security features, a computer implemented method, a banknote detector and a computer program product for copy protection of at least one security document. [Background technology]

[0002] Security-relevant documents such as flight tickets and banknotes are often subject to counterfeiting and other copying activities. One approach to countering document counterfeiting involves evaluating the authenticity of the suspect document. However, this approach is a late-stage activity, since by the time the authentication process of the suspect document is performed, the original document has already been copied. Therefore, measures related to authentication are not desirable.

[0003] Reproduction of an original document may be performed using a scanning device, a printer, and / or a copier. Reproduction may also be considered as a data copy process, such as data conversion. In this regard, if a document is reproduced, it is desirable to avoid the act of reproduction itself if it could potentially be considered as an original security document. In this case, reproduction is avoided before it is made. Such methods exist for special security features included in security documents. That is, a suspicious document is evaluated for whether it has security features that were originally included in the original security document. However, these approaches are undesirable for several reasons. First, the evaluation of security features requires complex and sophisticated equipment. Second, the evaluation process must include specific information about the security features. If a counterfeiter knows the evaluation process, he can obtain the information. Third, a counterfeiter may attack security features or modified features in the target document, which may lead to errors in the results of the evaluation method.

[0004] To prevent the above actions, various security features exist. For example, they may include printed graphical design elements that are recognized by special detectors in the aforementioned devices. The detectors may react accordingly and either refuse to process or interfere with the undesired action, such as printing a degraded image. Such graphical elements may be designed to look as if they were part of the artwork of the security document. An example of the use of such elements may be found in U.S. Patent 5,845,008. In other cases, special signals that are barely detectable to the human eye may be added to the printed design, which may be recognized by special detectors in the aforementioned devices. These detectors may then trigger a reaction as explained above. An example of the use of such elements may be found in U.S. Patent 6,449,377.

[0005] However, these security features have inherent vulnerabilities. Graphical design elements, despite attempts to make them look like part of the artwork, are often easily recognized by skilled observers for their security purpose. As a result, even slight modifications may prevent specialized detectors from identifying them and interrupting the desired actions of the copier. Also, these elements may be applied by legitimate users to other documents not intended for protection, thereby preventing people from completing actions such as scanning, copying, or printing these documents.

[0006] Special signals such as digital watermarks can have the undesirable property of making the printed document appear distorted. In the case of artwork on banknotes, this can be particularly undesirable. The distortion can be reduced, at the expense of signal strength. A compromise is usually required.

[0007] Artificial intelligence combined with machine learning is increasingly being used for applications such as facial recognition and other object identification. In such applications, there may be an infinite number of potential images that need to be recognized reliably. For example, an application trained to recognize images of gray squirrels themselves may encounter either vast variations in the size, pose, age, color tone, lighting, or a host of other individual characteristics of gray squirrels. Applications designed to reliably recognize individual human faces will have to face similar variability, at a minimum increasing the application's computational complexity and computing resource needs. Summary of the Invention [Problem to be solved by the invention]

[0008] The objective technical problem to be solved is considered to be the provision of a method for training an artificial intelligence based neural network applicable to classify digital images as being security documents or not without authenticating their security features, and a copy protection method, the security documents using such trained neural networks being an improvement over the prior art. [Means for solving the problem]

[0009] According to the invention, the neural network is not trained to authenticate digital images, in particular digital images of security documents. Moreover, the inventive method for copy protection of security documents does not replace any authentication process. In contrast, the inventive copy protection method can generally represent an additional means that can be applied to evaluate whether a copy of a digital sample image is recognized as a security document by an unbiased human observer.

[0010] This problem is solved by the subject matter of the independent claims. Preferred embodiments are set out in the dependent claims and in the following description, each of which may represent an aspect of the invention either individually or in combination. Advantages and preferred embodiments explained with respect to the indicated devices are likewise transferable to the corresponding methods and vice versa. [Brief description of the drawings]

[0011] [Figure 1] 1 is a simplified schematic diagram of a method 100 for training an artificial intelligence-based neural network 150. [Diagram 2] FIG. 2 is a simplified schematic diagram of a method 200 for copy protection of at least one security document 110. [Diagram 3] Simplified schematic diagram of a banknote detector 300 [Figure 4] A simplified schematic diagram of a computer program product. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] The present invention uses an inventively trained artificial intelligence based neural network to determine whether a digital image of a document may be copied / replicated, and therefore does not exploit the presence of security features representing codes for copy protection. Thus, documents to be protected from copying do not need to include codes for copy protection, in particular to prevent copying of digital images. According to the present invention, the design of the copy protected document does not need to be distorted by the use of additional codes for copy protection, which also reduces the risk of counterfeiters identifying the areas of the protection codes. The absence of a code for copy protection on the document also reduces the risk of the code for copy protection being hacked and used fraudulently on other items to stop copying illegally. The copy protection method of the present invention using the trained artificial intelligence based neural network of the present invention is particularly suitable for high throughput sorting and / or copy / copy solutions of security documents, in particular banknotes. This may be possible on a shorter time scale, since the decision on whether to copy the document or not is shorter than the general authentication method of security documents, which requires the authentication of a specific code for copy protection.

[0013] According to a first aspect, a computer-implemented method for training an artificial intelligence-based neural network is provided. The neural network can be applied to classify whether a digital image is to be considered as a security document (hereinafter also denoted A for reference). This means that the method can be configured to train the network to classify the digital image to be considered as a security document A or to classify the digital image not to be considered as a security document. The advantage of artificial intelligence-based neural networks, and in particular convolutional networks as deep learning models, mainly comes from the interpretation of the entire raw image, excluding normalization and standardization. The neural network then typically transforms the signal into a set of features based on which the classification is made. The set of features differs between models due to their architecture and training process, but can generally be interpreted as condensed information about the content of the image.

[0014] The method comprises providing at least one digital image A1 of at least one security document as a reference.

[0015] The method also includes providing a set of digital training images (hereinafter also denoted as B1 for reference purposes). Based on the digital training images, the neural network can be trained for the classification process. The digital training images are altered compared to the digital images of the security documents.

[0016] The set of digital training images includes a first subset of positive digital training images (also referred to below for reference purposes as B1-1) having modifications that have such a visual impact that an unbiased human observer would consider a copy of the respective digital training image to represent a security document or multiple security documents.

[0017] The set of digital training images also includes a second subset of negative digital training images (also denoted hereinafter as B1-2) having modifications that do not have such a visual impact that an unbiased human observer would consider a copy of the respective digital training image to represent a security document or multiple security documents.

[0018] The method further includes providing a ground truth for each digital training image to the artificial intelligence-based neural network. The ground truth represents at least one tolerance level of one or more unbiased human observers for whether a copy of the respective digital training image represents a security document or multiple security documents. In other words, the ground truth is used to train the neural network with respect to a classification decision process. The ground truth describes how a particular digital training image is interpreted in this training process. The at least one tolerance level includes one or more responses of the unbiased human observer with respect to a decision as to whether a copy of the respective training image represents a security document or not. For example, in the case of one unbiased human observer, the tolerance level represents the respective response of this human observer. In the case of two, three, four or more human observers, the responses of the two, three, four or more human observers are used to represent the tolerance level, respectively.

[0019] The method thus designed can advantageously train an artificial intelligence-based neural network on at least one digital training image of a security document. Thus, the neural network can be trained for each digital training image on the level of tolerance at which a copy of the particular digital training image would be considered a security document by an unbiased human observer. Thus, the neural network can be trained to make decisions when a copy of a digital image is considered a security document.

[0020] Within the context of all aspects of the present invention, terms such as may, could, could etc. also include the indicative / realis mood of the correlating verb. For example, the expression "data file is suitable for describing the characteristics of the image means of the digital code" also includes the indicative / realis mood of "data file is suitable for describing the characteristics of the image means of the digital code".

[0021] Within the context of the present invention, "image considered as a security document", "digital image considered as a security document", "copy of an image considered as a security document" or "copy of a digital image considered as a security document" means that an unbiased human observer can recognize / consider / interpret the copy of the (digital) image as a security document (at least to a certain level of tolerance). In other words, an unbiased human observer does not authenticate the security document, its (digital) image or its copy, but instead, an unbiased human observer, by viewing / observing or using the digital image, considers / has the impression / imagine, at least to a certain level of tolerance, that the (digital) image or its copy represents a security document or a sample of the security document. Thus, the characterization presented above can also be considered as relating to whether an unbiased human observer would accept the (digital) image or its copy as a sample of a security feature without an underlying authentication process.

[0022] Within the context of the present invention, a digital image refers to digital code, usually written in a computer language, and thus computer readable code, that represents a particular image as a data file, the data file being adapted to describe the characteristics of the image using the digital code.

[0023] The digital image of the security document may have a resolution in the range of 50 dpi to 2000 dpi, in particular in the range of 100 dpi to 1000 dpi, more particularly in the range of 200 dpi to 600 dpi, more particularly in the range of 300 dpi to 400 dpi.

[0024] Within the context of the present method, digital image reproduction refers to a hardcopy and / or printing process in which a digital image is physically processed to render it permanently visible on a printable medium for at least a certain period of time, such as years. Additionally, digital image reproduction may also include data processing, conversion or storage processes regarding the data underlying the respective digital image.

[0025] The digital training images may be modified compared to a reference. Within the context of the present invention, modified digital training images may be considered as digital training images of different or reduced quality compared to a reference. Examples of image modification / transformation include: ColorJitter extensions, including parameters such as brightness, contrast, saturation, and hue RandomAffine extensions that may include degree, translation, scale, shear, interpolation, and / or fill parameters Extension of GuassianBlur that can include kernel size and / or standard deviation parameters may be related to.

[0026] The altered digital training images that are labeled as recognizable as security documents may contain the following ranges of alterations / transformations compared to the reference security document: ColorJitter extensions, which may include modification of the parameters Brightness (e.g., ±5% range), Contrast (e.g., ±8% range), Saturation (e.g., ±8% range) and / or Hue (e.g., ±6% range); "RandomAffine extensions, which may include changing the parameters degree (e.g., 0.05 degrees), translation (e.g., between 0% and 1% of the image size horizontally or vertically), scale (e.g., increase or decrease the size by 1%), shear (e.g., shear parallel to the x-axis between -0.5 degrees and +0.5 degrees), interpolation (e.g., nearest neighbor interpolation), and / or fill (white);" An extension of GuassianBlur, which may involve changing the kernel size (e.g. 5x5) and / or standard deviation (e.g. 0.3) parameters.

[0027] Similarly, altered digital training images labeled as unrecognizable as security may contain the following ranges of alterations / transformations compared to the reference security document: · Changing parameters: brightness (e.g., greater than ±5% range), contrast (e.g., greater than ±8% range), saturation (e.g., greater than ±8% range), and / or hue (e.g., greater than ±6% range); RandomAffine extensions, which may include changing the parameters degree (e.g., more than 0.05 degrees), translation (e.g., more than 1% of the image size horizontally or vertically), scale (e.g., increasing or decreasing the size by more than 1%), and shear (e.g., shear parallel to the x-axis in the range greater than -0.5 degrees and greater than +0.5 degrees).

[0028] Alternatively, the digital training images may have similar qualities from a reproduction standpoint, such as printing, but may be altered or modified to distinguish them from the reference security document, for example by modifying / substituting design and / or text elements, different placement of design and / or text elements, etc.

[0029] According to one example, the reference security document may be a particular banknote, for example, including a portrait of the Queen of England. The digital training images may have similar qualities in terms of printing characteristics such as resolution, shape, size, color, etc. However, the portrait of the Queen of England may be replaced by another person, for example the Prime Minister of the United Kingdom or any other person. Provided to the concerned person, so that an unbiased human observer may consider the document to be a security document. Such modified digital training images may be considered, according to the present invention, as modified such that an unbiased human observer may consider a copy of the respective digital training image to represent a security document or documents. The modification may include degradation. According to another embodiment, the portrait of the Queen of England may be replaced by a portrait of the President of the United States or any unrelated person, so that an unbiased human observer may directly understand that the document cannot be considered a security document. In this case, the digital training images may still be considered modified or degraded, but an unbiased human observer would not consider a copy of the respective digital training image to represent a security document or documents.

[0030] The quality of the altered digital training image may be degraded. In this case, the digital training image may be based on a chemically or physically eroded training document B. This means that the quality of the training document may be chemically or physically degraded. For example, a chemically reactive substance may be applied or the training document may be scratched. The digital training image may be obtained after the underlying training document has been eroded. Thus, the digital training image may show changes caused by the erosion to the underlying training document.

[0031] Alternatively, the digital images may have been obtained based on unaltered training documents, but the digital images themselves may have been digitally eroded. For example, the digital images may have been modified / enhanced by applying digital filters. The modifications / enhancements may include ColorJitter enhancements, which include brightness, contrast, saturation, and / or hue parameters. RandomAffine enhancements may include degree, translation, scale, shear, interpolation, and / or fill parameters. GuassianBlur enhancements may include kernel size and / or standard deviation parameters. In this case, the digital training images may be digitally eroded digital images.

[0032] The modified digital training image differs from the digital image of the security document in at least one of the following: pixelation, resolution, definition, general aspect, shape, color, color distribution, image processing filters, and aspect ratio. For example, the digital training image may have a resolution, meaning pixel density per unit area, that is reduced compared to the reference resolution. Thus, the visual impact may be different. Nevertheless, when reproduced, the resolution of the digital training image may be sufficient for an unbiased human observer to consider the reproduction to represent the security document. General aspect refers to features of the digital image of the security document that are not similarly included in the digital training document. For example, the security document may partially show a particular landscape that includes several mountains. The digital training image may show a different landscape with the same number of mountains, or it may show in principle the same landscape but miss some of the mountains included in the security document. Aspect ratio refers to the general ratio of length to width of the security document and the digital training image. Definition refers to the total number of pixels horizontally and vertically for each item. Image processing filters can include noise reduction filters, blurring filters, so-called neural filters that use AI to process images, and similar digital filters.

[0033] The modified digital training image may differ from the digital image of the security document in terms of at least one of the following: perspective angle, underlying illumination, coloration, folds, or creases. Perspective angle may refer to the angle from which the digital training image appears to be acquired. For example, the security document may have a rectangular shape. Due to the perspective angle during acquisition of the digital training image (or due to the respective digital erosion), the digital training image may not have a rectangular shape. Furthermore, specifying a particular perspective angle may distort certain features in the digital training image. Illumination may refer to a luminance distribution present in the digital training document that is different from the luminance distribution exhibited by the security document when well illuminated in top view. Similarly, the digital training image may differ from the security document according to the color scheme, i.e. at least some specific colors, or more generally the distribution of colors across the digital training image. There may be folds or creases in the training document from which the digital training document was acquired. Thus, folds or creases are generally also perceptible in the digital training image.

[0034] Furthermore, the security document may include one or more graphic design features, such as portraits, e.g. a portrait of the Queen of England, architectural images (bridges, buildings, etc.), natural images (plants or parts thereof, such as leaves (so-called floral / botanical emblems or floral / botanical elements), or fauna / animals (so-called wildlife emblems or wildlife elements)). The modified digital training images may then differ from the digital image of the security document in that the digital training images may include at least one different design feature that replaces the corresponding design feature of the security document, e.g. a different design feature of the security document, such as a separate portrait of a separate, related or unrelated person, a separate related or unrelated architectural image of an unrelated person, or a separate related or unrelated image of the natural world. In this respect, the digital training images may be considered to be modified. The neural network may be trained in this respect, i.e. according to whether an unbiased human observer would consider the respective modified digital image to be relevant to the security document. According to one embodiment, an unbiased human observer may consider the modified image to be relevant to a security document if the graphic design feature is replaced by a related graphic design feature. For example, in the case of a British pound note, the portrait of Queen Elizabeth may be replaced by a portrait of the British Prime Minister, or a portrait of another member of the British Royal Family. According to another embodiment, an unbiased human observer may consider the modified image to be relevant to a security document if the graphic design feature is replaced by an unrelated graphic design feature. For example, in the case of a British pound note, the portrait of Queen Elizabeth may be replaced by a portrait of the President of the United States of America or another country. According to the present invention, architectural images may be considered to be related to each other if they belong to the same category, such as bridges, buildings, or if they belong to architectural images considered to represent the same country (e.g., Tower Bridge, Westminster Bridge and / or Westminster Abbey, Big Ben representing the United Kingdom, or the Eiffel Tower and Pont Neuf representing France).According to another embodiment of the present invention, nature images may be considered to be related to each other if they belong to the same category, for example plants or parts thereof (leaves, animals, etc.), or if they belong to the same category of nature images that are considered to represent the same country (for example, kangaroos, platypus and koalas representing Australia).

[0035] This means that the digital training image is generally not equal to the digital image of the security document. However, the difference is small enough that a copy of the digital training image is still considered to be a security document. In other words, the neural network is advantageously trained to reflect the discovery that, at least at a certain tolerance level, a digital image that differs from the digital image of the security document when copied may also be considered to represent a security document by an unbiased human observer. In this respect, the tolerance level may represent a false belief of the unbiased human observer. Although there may be significant differences between the digital training image and the security document, the observer may still consider a copy of the digital training image to represent a security document. When comparing the security document and the digital training image directly, such differences may be easily recognized. However, human recognition and memory have limitations. For example, it is well known that people often tolerate counterfeit banknotes that differ significantly in appearance from genuine banknotes. Thus, an unbiased human observer may generally consider different items to be the same, at least up to a certain degree (the tolerance level), if the differences are not too strong. For example, a human observer may perceive a counterfeit banknote as representing an original banknote. The neural networks of the present invention are trained to advantageously incorporate these differences and details about human perception and memory.

[0036] The tolerance level can be thought of as describing a similarity metric between the altered digital training images and a measure of how distinct they (the copies) are considered to be from each other by a respective number of human observers.

[0037] An unbiased human observer is not necessarily an expert in the field, but is considered to be a common user of security documentation.

[0038] The ground truth may represent the tolerance levels of at least four unbiased human observers. In this case, the ground truth may include at least five different tolerance levels. Since humans may judge differently whether a duplicate of a digital training image represents a security document, by increasing the number of crucially unbiased human observers, this uncertainty is incorporated into the training process. The ground truth therefore has the advantage of including an improved distribution regarding the tolerance levels. For example, if there are four unbiased human observers (labelers), these will generally result in five different tolerance levels. In this case, the distribution regarding the tolerance levels regarding whether a duplicate is considered to represent a security document or a multiple thereof, or whether this is considered to be a security document, will be determined. If not, it will be either 4 / 0, 3 / 1, 2 / 2, 1 / 3, 0 / 4. In other words, if four labelers label a digital training image, the training image may be labeled as perceptible as a security document just in case at least one labeler (or at least 25% of the labelers) labels this image as a security document. Alternatively, a training image can be labeled as recognizable as a security document in case at least two labelers (or at least 50% of the labelers) label this image as a security document.

[0039] Of course, finer segmentation of the tolerance levels can also optionally be correlated with information extracted from the evaluated digital image and / or a score determined on the basis of the extracted information. Thus, the decision mechanism when classifying digital images performed by the neural network can become more sophisticated.

[0040] Typically, each training image classified as the "first subset of positive digital training images (B1-1)" is assigned the number 1, and each training image classified as the "second subset of negative digital training images (B1-2)" is assigned the number 0. Thus, the deep learning model outputs a score, which is a continuous number between 0 and 1. This score represents the likelihood that a particular image is of a particular class. Thus, the closer the output score is to 1, the more likely the digital image will be recognized as a security document. In other words, the closer the score is to 0, the more likely the digital image will not be recognized as a security document. Typically, the threshold between the two classes will be 0.5 or higher.

[0041] This threshold may also be correlated with a tolerance level. If the tolerance level is low, e.g., less than 50% of the labelers classify the digital images into the first subset of positive digital training images, the threshold level may be determined to be 0.5 or higher. If the tolerance level is higher, i.e., more than 50% of the labelers classify the respective digital images into the first subset of positive digital training images, the output score threshold may be determined to be lower than 0.5.

[0042] Alternatively or additionally, this threshold can be adjusted based on the precision / recall requirements of each model, and thus the threshold may be at least 0.1 or more, 0.15 or more, 0.2 or more, 0.25 or more, 0.3 or more, 0.35 or more, 0.4 or more, 0.45 or more.

[0043] The security document may be a banknote. In this case, two digital images may be provided as two references, consisting of a front image and a back image of the banknote. Each positive digital training image may then have such a visual effect of modification that an unbiased human observer may consider a copy of the respective digital training image as representing the front image and / or the back image or images of the security document. Since the back of the banknote is also printed, and in particular printed in a different way than the front, the positive digital training image essentially corresponds to one of the two sides, or a combination of the two sides. In any case, such a digital training image is said to be a positive digital training image, since an unbiased human observer may consider a copy of a particular digital training image as representing a banknote, at least with respect to one side or various combinations. Therefore, both sides of the banknote must be provided as references, and the set of training images is adapted accordingly.

[0044] In all aspects of the present invention, an artificial intelligence (AI) neural network refers to a software or hardware-based technology, e.g., a computer-implemented algorithm, configured to make decisions. The AI ​​can also be configured to automatically utilize the provided data for an intended purpose and automatically provide the respective results to the user.

[0045] One or more digital images are used as input data, preferably a batch of two or more digital images, in particular 8 or more, 16 or more, or 32 or more digital images at a time, are used as input data for the neural network prediction model to output a prediction score between 0 and 1. In general, the closer the prediction output score is to 1, the more likely the respective digital image will be recognized as a security document (positive). Conversely, the closer the prediction output score is to 0, the more likely the respective digital image will be recognized as a respective digital image classified as not recognized as a security document (negative). The threshold level is generally 0.5 or higher, but may be adjusted based on the precision and recall of the respective model and / or based on the tolerance level. Thus, the threshold may be, for example, 0.1 or higher, 0.15 or higher, 0.2 or higher, 0.25 or higher, 0.3 or higher, 0.35 or higher, 0.4 or higher, or 0.45 or higher.

[0046] The digital sample image is preferably provided in a suitable image size or resized accordingly. Suitable image sizes are (64,128), (129,256), (192,384), (256,512), or (299,299). Preferably, the image size is the size used to train the predictive model, and preferably the image size may be (64,128).

[0047] The method can be implemented using a deep neural network having one or more neural layers. Each layer may exhibit a large number of neurons independent of other layers. Each layer can have a branched or non-branched architectural structure. Thus, the neural network can be advanced to perform more detailed training mechanisms.

[0048] The artificial intelligence based neural network of all the inventive aspects may be equipped with machine learning capabilities, and thus the neural network may be configured to learn from the training mechanism and generalize the decision process based on the provided digital training images.

[0049] The neural network of the present invention, including machine learning capabilities, can be configured to include multiple inputs to improve the process of rendering decisions, in other words, the network can be configured to recognize several similar inputs to improve the probability of accuracy in making a decision compared to the accuracy of making a decision based on a single input.

[0050] In summary, architectures and datasets suitable for image classification that are already available can be used for the present invention. As an example, the following predictive model architecture can be used because it performs well on similar problems and is flexible enough to be adapted to different tasks using fine-tuning:

[0051] ResNet The ResNet architecture (He et al. "Deep Residual Learning for Image Recognition", https: / / arxiv.org / abs / 1512.03385) won the ILSVRC (Russakovsky et al. "Imagenet Large-Scale Visual Recognition Challenge", https: / / arxiv.org / abs / 1409.0575) 2015 classification competition. At the time, researchers found that a network with more layers could potentially perform better for many problems. However, in practice, these deeper architectures were difficult to train because of the difficulty of updating weights in early layers far from the output (vanishing gradient problem). The introduction of residual blocks made it possible to train the 152-layer ResNet152.

[0052] The ResNet architecture works by adding skip layers in addition to the normal connections between layers (Gebrado. "Standard Form of Residual Neural Nets", https: / / commons.wikimedia.org / wiki / File:ResNets.svg). This increases the gradients during backpropagation, since the output of layer l-2 does not have to pass through the intermediate layer l-1 first, but directly influences the next layer l. The gradients, which would vanish with only normal connections, have a sufficiently large value in the presence of residual blocks.

[0053] There are several variants of ResNet, such as ResNeXt4 and Wide ResNet5. For this application, the original ResNet architecture with 50 layers may be chosen as it may provide a good balance between training effort and prediction performance.

[0054] Inception v3 The first version of the Inception Network (Szegedy et al. “Rethinking Inception Architectures for Computer Vision”, https: / / arxiv.org / abs / 1512.00567) (also known as GoogLeNet, Szegedy et al. “Going Deeper with Convolutions”, https: / / arxiv.org / abs / 1409.4842) introduced an Inception module that combines convolutions with different kernel sizes to use features at different scales. Additionally, auxiliary classifiers (small convolutional neural networks) were added as regularizers between some layers. These helped the network to generalize better to unseen images. This architecture won the ILSVRC 2014 competition (ImageNet Large Scale Visual Recognition Challenge 2014, https: / / image-net.org / challenges / LSVRC / 2014 / ).

[0055] In the second version, greater computational efficiency is achieved by factoring larger convolutions into smaller ones: for example, a convolution with a 5x5 kernel is split into two convolutions with a 3x3 kernel, thus reducing the number of parameters from 25 to 18.

[0056] Inception v3 incorporates ideas from previous versions and further improves them using factorization and modifications to the auxiliary classifier.

[0057] DenseNet DenseNet is the newest architecture of the three models (Huang et al., "Densely Connected Convolutional Networks," https: / / arxiv.org / abs / 1608.06993). It won the Best Paper Award at the 2017 CVPR10 conference ("CVPR 2017," https: / / cvpr2017.thecvf.com / ). It takes the ResNet idea of ​​skipping a single layer a step further by using Dense Blocks. In every Dense Block, each layer receives input from all preceding layers. The feature maps from all previous layers are concatenated and used as the input for the next layer. Thus, the last layer in a block has access to the entire collective information. In particular, DenseNet, which has more than 100 layers, especially 121 layers, can be used for this purpose.

[0058] Pre-training It has been shown that the features that a neural network learns by training on one dataset for a specific task are also meaningful for another task involving a different dataset. In particular, the first layer of the network is known to learn more generalizable features. Based on these findings, it is common to take a network trained on a large dataset such as ImageNet ("ImageNet Large-Scale Visual Recognition Challenge 2014", https: / / image-net.org / challenges / LSVRC / 2014 / ) or COCO (Lin et al. "Microsoft COCO: Common Objects in Context", https: / / arxiv.org / abs / 1405.0312) and then fine-tune its weights on a second dataset of interest. This approach of using pre-trained weights has been shown to be effective even when the second dataset is very small.

[0059] Thus, the training method of the present invention, according to a preferred embodiment, uses pre-training on a common large image dataset, thereby initializing the model weights to perform well on the respective image dataset, such as ImageNet or COCO, where ImageNet is the standard for benchmarking classification models.

[0060] Each predictive model architecture is then fine-tuned by training on a dataset that includes digital images of a first subset of training images that are recognized as security documents (positive training documents) and digital images of a second subset of training images that are recognized as security documents but not recognized as banknotes or security documents, respectively.

[0061] The method of the first aspect of the present invention can be performed on a closed set of multiple security documents. An artificial intelligence-based neural network can then be trained on all security documents of the closed set of security documents. The closed set can therefore include or consist of a certain number of security documents on which the neural network is trained. This can be advantageous when using the neural network for a certain type of security document where there is only a certain number of distinct elements. For example, the closed set of security documents can be determined by the different banknote denominations of a limited number of monetary systems.

[0062] Data Processing As already mentioned above, the input data relates to a batch of one or more, preferably eight or more, sixteen or more, or even thirty-two or more digital images that are used at one time in the predictive model, for example a batch containing 32 digital images shows particularly advantageous performance in the above example model.

[0063] According to a preferred embodiment of the training and copy protection methods, the digital images of each batch can be resized to a common size, for example (64,128), (128,256), (192,384), (256,512), (299,299), to favorably affect the weights. If pre-training using a common image dataset is performed, it is preferable to use the average standard deviation of the image dataset, such as ImageNet or COCO, for resizing. In other words, if the training data of the present invention is standardized in the same way as the pre-training image data assets, the pre-trained weights will work well on the training data of the present invention.

[0064] According to additional or alternative embodiments of the invention, the number of training images can be artificially increased using suitable extensions, for example ColorJitter extension, RandomAffine extension, GuassianBlur extension, which can include parameters brightness, contrast, saturation and / or hue. RandomAffine extensions may include parameters for degree, translation, scale, shear, interpolation and / or filling. GuassianBlur extensions may include parameters for kernel size and / or standard deviation. More specifically, during training, small perturbations can be applied to some or all of the training images. The strength of these perturbations is randomly selected from a predefined range. This range is carefully defined so that the perturbations are not so strong that they change the ground truth. This means that images that are considered acceptable as security documents, especially banknotes, will remain acceptable after applying the perturbations.

[0065] Thus, augmented digital training images that are labeled as recognizable as security documents may contain the following range of modifications / transformations: ColorJitter extensions, which may include changes to brightness (e.g., ±5% range), contrast (e.g., ±8% range), saturation (e.g., ±8% range) and / or hue (e.g., ±6% range); RandomAffine extensions, which may include changing the parameters degree (e.g., 0.05 degrees), translation (e.g., between 0% and 1% of the image size horizontally or vertically), scale (e.g., increase or decrease the size by 1%), shear (e.g., shear parallel to the x-axis between -0.5 degrees and +0.5 degrees), interpolation (e.g., nearest neighbor interpolation), and / or fill (white); An extension of GuassianBlur, which may involve changing the parameters of kernel size (e.g. 5x5) and / or standard deviation (e.g. 0.3). The aforementioned transformations can be applied using the torchvision framework.

[0066] However, our experimental results (see the Examples section) show that such artificial augmentation does not improve training. Instead, image augmentation can be used to evaluate the model.

[0067] A suitable learning rate, such as 0.005, 0.01, 0.05, or 0.1, can be used to train a predictive model. In other words, the learning rate defines the step size used to optimize the model learning process. Thus, the smaller the step size, the faster the optimization of the model learning process will be performed, resulting in better overall performance but at the expense of a higher overall computational load. According to our experiments, a step size of 0.005 showed the best performance.

[0068] optimizer In general, training a neural network is an optimization problem. This optimization can usually be performed via a variety of optimizers, of which stochastic gradient descent is the best known. Typically, an optimizer iteratively updates the model weights by computing the value of a loss function for a single batch, determining the gradient of this function, and taking steps (defined by a learning rate) towards the direction that minimizes the loss. For example, the Adam (Kingma et al. "Adam: A Method for Stochastic Optimization", https: / / arxiv.org / abs / 1412.6980) optimizer can be used as a suitable optimizer for the training method of the present invention. It is built on stochastic gradient descent, where each parameter has a separate learning rate, which is adjusted according to the progress of the training.

[0069] Hyperparameter Search According to additional or alternative preferred embodiments of the present invention, after establishing a functional model as a baseline, it is preferable to optimize the performance of the model, which can be done by trying different parameter combinations during training.

[0070] Additionally or separately, the training methods of the present invention may implement suitable text-based classifiers such as digits (e.g., denominations, letters or words in the case of banknotes) and / or ensemble strategies and models, such as using an ensemble of image classifiers only, or an ensemble of image and text-based classifiers, particularly using ensembles, which may provide better performance and be more robust.

[0071] The training methods of the present invention are generally evaluated using one or more common metrics, such as precision, precision (also called "positive predictive value"), recall (also called "sensitivity"), F1 and Matthew Correlation Coefficient (MCC).

[0072] A confusion matrix usually provides an overall view of the performance of a model, since it describes the prediction quality of a classifier and indicates where correct and incorrect predictions occur by displaying the ground truth (target) positive and negative classified digital images and the predicted (model) positive and negative classified digital images. Thus, the confusion matrix indicates the number of digital images classified as true positive (TP), false positive (FP), true negative (TN), and false negative (FN).

[0073] Each evaluation metric attempts to summarize the confusion matrix with a single value.

[0074] The evaluation metric "Accuracy" is related to the proportion of correct predictions, i.e. Accuracy = (TP + TN) / (TP + TN + FP + FN). In the case of balanced classes, i.e. the number of digital images in the positive and negative classes is the same, accuracy is a good measure of overall performance.

[0075] The evaluation metric "recall" relates to the proportion of true positives that can be predicted by the model. In other words, recall = TP / (TP + FN).

[0076] The evaluation metric "precision" relates to the proportion of true positives when predicting positives, i.e., precision = TP / (TP + FP).

[0077] The evaluation metric "F1" relates to the combination of precision and recall in a single metric and is defined as the harmonic mean of precision and recall, in other words, F1 = (2x precision x recall) / (precision + recall).

[0078] The evaluation metric Matthews Correlation Coefficient (MCC) provides robustness against class imbalance and merges several desirable model characteristics into a single number, whose value lies between [-1,+1], with -1 being the worst and +1 being the best. In other words, MCC = [(TP x TN)-(FFP x FN)] / [(TP + FP) x (TP + FN) x (TN + FP) x (TN + FN)].

[0079] As detailed in the Examples section below, in an experimental training set for classifying banknotes, the ResNet50 architecture combined with input image size (64,128), batch size 32, learning rate 0.005, and standard transforms performed best.

[0080] The copy protection method constructed according to the invention advantageously improves security: since the characteristics of the digital images on which the prediction of the classification of the digital images is based are not determined a priori, the criteria on which the digital images are analyzed are not known to the public and therefore to potential counterfeiters, so that counterfeiters cannot design digital images that circumvent the copy protection or make them easier to copy.

[0081] All features and embodiments disclosed with respect to the first aspect of the invention may be combined alone or with one of the second to fifth aspects of the invention, including each of their preferred embodiments, or in their (sub)combinations, subject to the resulting combinations, some of which will be reasonable for a person skilled in the art.

[0082] According to a second aspect, a computer implemented method for copy protection of at least one security document A is provided.

[0083] The method may include providing a digital sample image C1. The method may also include applying an artificial intelligence based neural network to classify the digital sample image into the first category or the second category. The neural network may be trained according to the methods previously described herein.

[0084] If the neural network determines that a copy of at least a portion of the digital sample image can be considered by an unbiased human observer to represent at least one security document or multiple security documents, the digital sample image may be classified into a first category. This prediction is related to a threshold value of the output score of the prediction model. If the threshold value is 0.5, the digital sample image that receives a score of 0.5 or more is classified into the first (first) category. Alternatively, if the threshold value is 0.1 or more, 0.15 or more, 0.2 or more, 0.25 or more, 0.3 or more, 0.35 or more, 0.4 or more, 0.45 or more, the digital sample image is classified into the first category if the score is at least 0.1 or more, 0.15 or more, 0.2 or more, 0.25 or more, 0.3 or more, 0.35 or more, 0.4 or more, or 0.45 or more, respectively. The threshold value can be lowered to increase the chance that documents that can be recognized as security documents, especially banknotes, are not copied. However, in this case, there is a trade-off of an increased number of digital images, which are generally considered to be less likely to be recognized as security documents, especially banknotes. To reduce the number of unauthorized copy protections, artificial intelligence detection can be used as a pre-filter. This allows only a portion of the original digital image to be authenticated in a later step using a suitable detector for identifying security features of copy protection images, such as Eurion or Omron Ring. In other words, only this portion of the digital image that is classified into a first category in the first artificial intelligence detection (without using authentication security features) is authenticated in a second step using common image security features. If a copy protection image security feature is identified in the second authentication step, the copy process is prevented. If the respective image security feature is not identified in the second authentication step of the image security features, the copy process is not prevented. Thus, the preferred embodiment combining artificial intelligence detection as a pre-filter and then applying copy protection authentication only to the positive portions of the first step facilitates reducing the portion of copy protection for those digital images that do not contain copy protection image security features.At the same time, the overall authentication time is reduced because the AI ​​detection takes less time than the subsequent authentication of the copy-protected image security features. Furthermore, the AI ​​pre-filter does not prevent the copying of documents that illegally have copy-protected image security features but are not considered recognizable as security documents.

[0085] Alternatively, the digital sample image may be classified into the second category if the neural network determines that for any part of the digital sample image, the reproduction is unlikely to be considered by an unbiased human observer to represent at least one security document or multiple security documents. This prediction also relates to a threshold value of the output score of the prediction model. If the threshold value is 0.5, the digital sample image receiving a score less than 0.5 is classified into the second category. Alternatively, if the threshold value is 0.1 or more, 0.15 or more, 0.2 or more, 0.25 or more, 0.3 or more, 0.35 or more, 0.4 or more, 0.45 or more, the digital sample image is classified into the second category if the score is less than 0.1, less than 0.15, less than 0.2, less than 0.25, less than 0.3, less than 0.35, less than 0.4, less than 0.45, respectively.

[0086] Additionally, the method may include preventing duplication of the digital sample image if the neural network classifies the digital sample image into the first category.

[0087] Within the context of this application, a digital sample image refers to a digital image of a sample document C. For example, there may be a sample document for which it is to be determined whether copying should be prohibited or permitted. Obviously, if an unbiased human observer could consider any part of the sample document to represent a security document, copying should be avoided. An effective means is then provided for avoiding copying of candidates that may be used inappropriately or illegally as (original) security documents.

[0088] Thus, a digital image of the sample document is obtained to obtain a digital sample image. This digital sample image is provided to a neural network. The neural network may be trained according to the above details so that it recognizes the digital image of at least one security document as a reference. The neural network may then be able to determine whether a copy of the digital sample image would be considered by an unbiased human observer to represent the security document or documents. During this process, the neural network may take into account differences between the digital sample image and the reference image. Although these differences may exist, the network may determine that, at least to a certain tolerance level, a copy of at least a portion of the digital sample image may be considered to represent a security document by an unbiased human observer. If this condition is determined to be true, at least to a certain tolerance level, the digital sample image may be classified into a first category. This means that the first category includes or consists of digital sample images that may be misused. Thus, for digital sample images classified into the first category, copying may be prevented in order to prevent misuse that can be achieved with the current method. Preventing copying may include preventing processing of the digital sample image such that a hard copy of the digital sample image cannot be obtained. Additionally or alternatively, preventative measures may include preventing the processing, conversion, or storage of data underlying the digital sample image.

[0089] Within the context of the method, the security document may be one of the following: a banknote, a check, a certificate, a ticket, a passport, or an airline ticket. For these types of documents, unauthorized duplication of the respective digital image poses significant risks for both economic and security-related reasons. These risks are avoided or at least mitigated by the copy protection methods described hereinabove.

[0090] If the neural network trained according to the invention classifies the digital sample image into a first category, preventing copying of the digital sample image may include activating a prohibiting means. The prohibiting means may be a software or hardware implemented structure. The prohibiting means may prohibit copying of the digital sample image by a copying means such as a printer or a copier. The prohibiting means may also be configured to prohibit storing of data or converting and processing of data. This may be achieved by a master / slave system, the prohibiting means may control such devices or data processing units included in a common data processing system. Additionally or alternatively, data representing the digital sample image may be modified by the prohibiting means such that it cannot be read or processed by a copying device, i.e. a printer or a copier. Modifying the data may include modifying the data to include a mark / attribute / flag, the mark / attribute / flag preventing copying of the data.

[0091] The digital sample image may be altered compared to the digital image of the security document. The digital sample image may be degraded in particular compared to the digital image of the security document. The altered digital sample image may differ from the digital image of the security document in terms of at least one of the following: perspective angle, underlying lighting, color shade, folds, or wrinkles. The details described with respect to the alterations (synonyms: degradation or modification) described herein within the context of the digital training images may be equally applicable to the altered digital sample images. Similarly, as in the case of the altered digital training images, the digital sample image may not be optimal due to various reasons, especially degradation. Furthermore, image acquisition may not be optimal due to lighting limitations or non-optimized positioning of the image acquisition means. Furthermore, the sample document itself may differ from the security document, for example due to the circulation or exchange of features, especially graphic design features. Such differences may therefore be present in the digital sample image itself. However, once the neural network is trained accordingly, it may be possible to determine whether a copy of the digital sample image would be considered relevant to the security document by an unbiased human observer. In other words, the neural network may have been trained using digital training images that have been altered relative to the reference so that the neural network is able to compensate for differences that exist between the altered digital sample images and the reference, and the neural network is configured to appropriately access the differences between the altered digital sample images and the reference so as to be compatible with assessing how an unbiased human observer would view a reproduction of at least a portion of the digital sample image relative to the reference.

[0092] The security document may include a front and a back. The digital sample image may then be classified into a first category if the neural network determines that a reproduction of at least a portion of the digital sample image may be considered by an unbiased human observer to represent the front and / or back of one or more of the security documents of the security document. Alternatively, the digital sample image may be classified into a second category if the neural network determines that no portion of the digital sample image may be considered by an unbiased human observer to represent the front and / or back of the security document or documents.

[0093] The ground truth provided in the method for training a neural network can represent a range of tolerance levels comprising or consisting of a first sub-range and a second sub-range, and if the neural network determines according to the first sub-range a tolerance level that a copy of at least a part of the digital sample image can be regarded by an unbiased human observer as representing at least one security document or multiple security documents, the method of copy protection can be modified insofar as the digital sample image falls into the first category.

[0094] Alternatively, the digital sample image may be classified into the second category if the neural network determines a tolerance level according to the second subrange such that no portion of the digital sample image may be deemed by an unbiased human observer to represent at least one security document or a copy of a plurality of security documents. In this scenario, the first subrange may be greater than the second subrange.

[0095] The determined tolerance level may be correlated with an output score threshold, i.e., the higher the threshold value, the lower the numerical tolerance level, and vice versa.

[0096] The tolerance level can be thought of as describing a similarity metric between the altered digital sample images and a measure of how well a respective number of human observers would distinguish between them (the copies).

[0097] This means that the neural network can be trained to determine a number of different tolerance levels for whether a copy of a digital sample image is considered to represent a security document by an unbiased human observer. Advantageously, the range of tolerance levels is distributed asymmetrically with respect to the classification mechanism of the digital sample images. In other words, only if the tolerance level is very low, the digital sample image may be classified into a second category, where copying is not necessarily prevented. According to a larger subrange of the tolerance level, the digital sample image may be classified into the first category instead, so that copying is prevented. In other words, if the first subrange is larger than the second subrange, there may be an asymmetry between the number of digital sample images classified into the first category and the number of digital sample images classified into the second category, at least assuming a uniform distribution of the digital samples with respect to the various tolerance levels.

[0098] In a simple scenario, an unbiased human observer may consider a copy of a digital sample image as representing a security document with a 50% tolerance level in the range from 0 to 1. Assuming that the first subrange is larger than the second subrange, this digital sample image will be classified into the first category with a tolerance level of 50%. The thresholds between the subranges can in particular be determined very low, for example 0.3 vs. 0.7 in the range from 0 to 1, or even 0.2 vs. 0.8 in the range from 0 to 1, for example 0.1 vs. 0.9 in the range from 0 to 1, and for the same range of tolerance levels, for example 0.01 vs. 0.99.

[0099] An asymmetric distribution regarding the classification of the digital sample images into the first or second category according to the determined tolerance level may also cause an asymmetry in the proportion between false positive events compared to false negative events. The number of false positives may be higher, in particular much higher, than the number of false negatives. Here, a false positive refers to a configuration in which the neural network determines that at least some copies of the digital sample image are considered by an unbiased human observer to represent a security document, even though the observer actually considers the copies not to represent a security document. A false negative may refer to a configuration in which the neural network determines that at least some copies of the digital sample image are not considered by an unbiased human observer to represent a security document, even though the unbiased human observer actually considers at least some copies of the digital sample image to represent a security document. The reason behind the asymmetry between false positives and false negatives may be the threshold regarding the tolerance level between the first and second subranges. If the threshold tolerance level between these subranges is low, the digital sample image is rather classified into the first category. Classification into the second category is a rare case. Therefore, the number of false negatives will be less, and in particular much less, than the number of false positives.

[0100] The particular sizes of the first and second sub-ranges with respect to the tolerance level may be the result of a training process applied to the neural network.

[0101] Alternatively, the neural network may be trained such that the first and second subranges are essentially the same size with respect to the overall range of tolerance levels. In this case, the asymmetry can be manually introduced by setting the respective threshold tolerance levels before or during use of the neural network for copy protection of security documents. The setting or adaptation of the threshold tolerance levels may be motivated, for example, by a classification of the digital sample images. If the second category includes or consists of digital sample images that should be prevented from being copied, the threshold tolerance levels may be adapted accordingly.

[0102] In another alternative, the threshold tolerance level may be adapted by the neural network itself based on its inherent machine learning capabilities.

[0103] The methods for copy protection of at least one security document can be based on a code, each code of the method having a binary size of between 100 kB and 50 MB, in particular between 200 kB and 10 MB, more in particular between 200 kB and 10 MB, more in particular between 500 kB and 1 MB. Due to the comparable small size of the code, the code has the advantage that it can also be implemented in non-high-end data processing devices such as scanning devices, printers, copiers, etc.

[0104] The method for copy protection of at least one security document can be configured to be executable within a time period of less than 60 seconds, in particular within a time period between 100 milliseconds and 30 seconds, in particular within a time period of less than 1 second. A method configured in this way can also be advantageously applied during regular data processing procedures in real time, such as the printing of digital images, with an acceptable time commitment.

[0105] In particular, the method for copy protection may be operable at a speed that does not substantially slow down the reproduction process, such as the printing process. In this case, the processing of the method according to the second aspect of the invention may be performed within the mechanical latency of the printing device. For example, according to one embodiment, when using a printer, such as an inkjet printer, this may mean that only a few lines of a digital image can be reproduced / printed before the reproduction / printing operation is stopped for copy protection. This embodiment nevertheless achieves the object of the invention. All features and embodiments disclosed with respect to the second aspect of the invention may be combined alone or in (sub)combination with any of the first or third to fourth aspects of the invention, including each of the preferred embodiments. The resulting combination of features is reasonable for a person skilled in the art.

[0106] According to a third aspect of the invention, there is provided a banknote detector comprising or consisting of a communication means and a data processing unit. The communication means may be configured to receive a digital sample image and provide said digital sample image to the data processing unit. The data processing unit may be configured to perform a method for copy protection of at least one security document. The banknote detector may be configured to prevent duplication of the digital sample image.

[0107] The banknote detector of the present invention can in particular be implemented in a device for reproducing sample documents, such as a copier, printer, etc. The banknote detector can thus advantageously prevent the reproduction process of a sample document or a digital sample image if the digital sample image falls into the above-mentioned first category.

[0108] The banknote detector of the present invention may be implemented in software. The banknote detector may be included in a device for replicating the digital sample image. Alternatively, the banknote detector may be configured as a cloud-based or server-based application.

[0109] According to an alternative, the banknote detector may be at least partially implemented in hardware. In this case, at least some of the functions of the banknote detector may be implemented by a hardware-based data processing component, such as a CPU or a network communication device coupled to the CPU. Even a neural network may be at least partially implemented in hardware, such as using a quantum computing device. Alternatively, the neural network may be implemented in software, for example by processor commands executed by a CPU.

[0110] The banknote detector of the present invention may be realized entirely as digital code and written in a computer language. Thus, the banknote detector can be easily integrated into the firmware of reproduction equipment, in particular copiers and printers. Furthermore, updating of the banknote detector can be easily achieved by providing an upgraded firmware version of such a device. Alternatively, such a device may only include the client part of the banknote detector, with the banknote detector itself being included in a cloud service or server. The client part may then be configured to communicate with the cloud or server to execute the banknote detector in the cloud or on the server with respect to a local digital sample image present on the client. In this case, data regarding the digital sample image may need to be transferred to the cloud or server.

[0111] The banknote detector of the present invention may further be configured to evaluate the digital sample image for authenticity if the digital sample image is classified into the first category. The digital sample image may be evaluated for at least one security feature included in at least one security document, i.e. an image security feature for copy protection. In particular, certain copy protection security features such as the so-called Eurion constellation, also called Omron ring, may be used in this respect. This embodiment is advantageous since it allows further verification whether a digital image classified into the first category, i.e. a digital image thought to represent a security document, actually represents a security document whose copying is prohibited. In other words, such a preferred embodiment of the present invention provides a cross-check of whether a digital image classified into the first category is a "true" positive security document, thus preventing the misuse of image security features that would be illegally applied to non-security documents. Considering that the authentication process is generally time-consuming compared to the pre-filter AI detection process, the preferred embodiment not only allows cross-checking true positive hits, but also saves overall time as the authentication process of the images. The security features need only be performed on a subset of the digital images classified into the first category.

[0112] The artificial intelligence neural network also differs from known mechanisms implemented in printers and copiers, since the copy protection method of the present invention does not (only) depend on specific copy protection security features (eurions), which require such specific copy protection security features to be present in the sample document. However, the usefulness of the present method for copy protection is much broader, since, firstly, the method does not depend on a true physical sample document, but instead on digital sample images, and, secondly, these digital sample images can be classified based on a fairly broad range of visual features contained in each digital sample image.

[0113] In contrast thereto, the method for training an artificial intelligence-based neural network according to the first aspect of the present invention and the method for copy protection of at least one security document according to the second aspect of the present invention do not rely on any security features contained within the security document and therefore do not authenticate the security document. Thus, misuse or modification of certain security features, in particular image security features for copy protection, does not affect the copy protection method of the present invention. As mentioned above, the present invention has the advantage, compared to general authentication methods, that the classification of a document for further processing, in particular for copying, printing or otherwise reproduction, can be performed more quickly than performing a complete authentication step for the code for copy protection. Thus, a more rapid neural network-based method can be applied first to evaluate whether a copy of a digital sample image is generally considered to be a security document.

[0114] The overall time is reduced by sequentially applying an artificial intelligence method that evaluates whether a copy of a digital sample image is generally considered to be perceptible as a security document in combination with an authentication method that is subsequently applied to each security feature, i.e., copy prevention is only performed for documents that have been classified as representing security documents in the first step of the image security feature artificial method.

[0115] In the first step of the artificial intelligence method, which evaluates whether copies of digital sample images are generally considered recognizable as security documents, the majority of the examined digital sample images can generally be classified into the second category as human. The observer would not consider copies of any part of the digital sample image as a security document. According to the preferred method of the invention, it is not even necessary to carry out the authentication method for this majority, since in the first step the inventive copy protection method has already excluded these documents from being generally recognized as security documents. Thus, the sequential application of the process saves time compared to the case in which a complete authentication of the entire set of digital sample images is carried out.

[0116] Furthermore, the document design is not distorted by the use of copy protection codes and the risk of hacking due to unauthorized application of copy protection codes is reduced. In principle, the assessment regarding the digital sample image is independent of the copy protection security features / codes. Therefore, if the digital sample image falls into the first category, copying should be prevented.

[0117] Additional aspects may also provide time savings if a security feature or characteristic of the underlying security document is misused on a document not intended to be marked as such. The use of the security feature or characteristic of the security document may be restricted to authorized authorities. If a feature or characteristic is used in an illegal way on or with a document not intended to be used as a security document, the method of the invention uses an AI detection method as a pre-filter in a first step to evaluate whether a digital image of this document contains the following content: The image security feature illegally applied for copy protection is classified into a first category (recognizable as a security document) or a second category (not recognized as a security document). If the digital image is classified into the second category, the method of the invention does not apply a subsequent authentication process, so that the image security feature illegally applied for copy protection cannot stop the duplication process. Only for digital images that can be recognized as security documents (classified into the first category) a subsequent authentication process for the image security feature for copy protection is applied, leading to the prevention of the duplication process. However, this is advantageous, since an unbiased person may consider a digital duplication of this digital image to represent a security document. Thus, the time-consuming authentication process can be performed only for misuse cases that fall into the first category, while the time-consuming authentication process can be completely avoided for misuse cases that fall into the second category.

[0118] To provide a more advanced evaluation of the document, including authentication, the banknote detector may further comprise an authentication means for evaluating one or more security features, including a code for copy protection, and may be configured to determine the authenticity of the digital sample image and to compare with a reference to identify a digital image based on a counterfeit document. For example, as an additional authentication means, the banknote detector may be configured to evaluate the digital sample image for one or more codes for copy protection, such as the so-called Eurion, also called Omron ring. If the additional authentication step determines that the document does not contain one or more codes for copy protection, further processing may be stopped and the document may be further classified as a counterfeit document. In this case, the combination of the copy protection method of the present invention in addition to the authentication method of the code for copy protection provides further advantages in the case where a code for copy protection, for example Eurion, is illegally hacked and illegally applied to a non-security document. According to the copy protection methods of the prior art, the copy protection code illegally applied to the document is authenticated and the copy operation is stopped, regardless of the nature of the document, i.e. whether an unbiased human observer would consider the (digital) image of the document or its copy as a security document. In contrast, the copy protection method of the second aspect of the invention requires that the digital image be classified into a first category or into a second category. Classification into the first category means that the image or a copy thereof can be regarded as an image or a copy thereof by an unbiased human observer, at least up to a certain level of tolerance. Classification into the second category means that the image or a copy thereof cannot be regarded as an image or a copy thereof by an unbiased human observer, at least up to a certain level of tolerance. If the digital image is classified into the second category, copy protection may not be provided, even if the image is authenticated by an additional authentication method. Thus, an illegally applied copy protection code does not allow for illegal copy protection.

[0119] Another advantage of combining the copy protection method of the second aspect of the invention with a prior art authentication method is that it is copy-proof if the (digital) image of the counterfeit document does not carry a copy protection code, and therefore cannot be reproduced, but could be thwarted by prior art authentication methods if it were to be regarded as a security document by an unbiased human observer.

[0120] Thus, the method for training an artificial intelligence based neural network according to the first aspect of the invention and the method for copy protection of at least one security document according to the second aspect of the invention can be applied sequentially or simultaneously, together with the method for evaluating the reliability of at least one security feature in order to identify digital images based on counterfeit documents and prevent the copying of those images. In the case of sequential application, the authentication method is applied subsequent to the inventive copy protection method based on the application of an artificial intelligence based neural network.

[0121] The banknote detector may be configured with a low end ARM type multi-core CPU or similar CPU commonly used in mobile devices. The device may further comprise a main memory in the range of 4MB to 8GB, more particularly in the range of 16MB to 2GB, more particularly in the range of 64MB to 512MB, more particularly in the range of 128MB to 256MB. The at least one security document copy protection method may be configured to be executable on a specified CPU type in a local or remote manner using a main memory of a specified size.

[0122] All features and embodiments disclosed with respect to the third aspect of the invention may be combined alone or in (sub)combinations with any of the first, second and fourth aspects of the invention, including each of the preferred embodiments, with the resulting combinations of features being reasonable for a person skilled in the art.

[0123] According to a fourth aspect, there is provided a computer program product comprising or consisting of instructions which, when executed by a data processing unit, cause the data processing unit to apply an artificial intelligence based neural network for classifying digital sample images into a first category or a second category, where the neural network may be trained according to the method as previously described herein and the classification process may be accomplished as previously described herein.

[0124] Computer program products can be stored encrypted and / or encrypted. Some of the underlying techniques and procedures need to be kept secret for security reasons. Therefore, if the code is stored encrypted, it is advantageous to prevent the underlying techniques and instructions from being disclosed.

[0125] Alternatively, the computer program product may be open access. In general, this does not pose any risk, since the program cannot actually be misused in the same way as a digital watermark detector or the like. In the case of a digital watermark, if the code is misused, a counterfeiter may be able to reconstruct the watermark signal and apply it to a fraudulent image. However, in the case of the copy protection method according to the second aspect of the invention, a copy of at least a part of the digital sample image will either be regarded by an unbiased human observer as representing a security document, or it will not. Thus, there is no similar risk in misusing the current computer program product.

[0126] All features and embodiments disclosed with respect to the fourth aspect of the invention may be combined alone or in (sub)combination with any of the first to third aspects of the invention, including each of the preferred embodiments of the invention, provided that the resulting combinations are such that some of the features will be reasonable for a person skilled in the art.

[0127] Further aspects and features of the present invention will become apparent from the following description of preferred embodiments of the invention which proceeds with reference to the accompanying drawings. FIG. 1 is a simplified schematic diagram of a method for training an artificial intelligence-based neural network applicable for classifying digital images as being or not being security documents. FIG. 2 is a simplified schematic diagram of a method for copy protection of at least one security document Figure 3 shows a simplified schematic diagram of a banknote detector. FIG. 4 shows a simplified schematic diagram of a computer program product.

[0128] All features disclosed below with respect to the exemplary embodiments and / or the accompanying drawings may be combined, alone or in any subcombination, with the features of the aspects of the invention, including the features of the preferred embodiments, provided that the resulting combination of features is reasonable for a person skilled in the art.

[0129] 1 shows a simplified schematic diagram of a method 100 for training an artificial intelligence-based neural network 150. The method 100 is described below with reference to a device-based neural network 150 for illustrative purposes. However, this should not be construed as limiting the method 100.

[0130] The neural network 150 is applicable to classify digital images as security documents 110 or not. The neural network 100 may be a deep neural network with multiple layers. In the method 100, the neural network 150 is trained based on three inputs. The neural network 150 may be a residual neural network (ResNet) that further utilizes skip connections or shortcuts across one or more layers. As described in the examples section, a residual neural network with 50 layers is used in the context of the present invention. Alternatively, the neural network 150 may utilize an Inception network that combines different kernel sizes and convolutions to use features at different scales. Inception v3 is used as described in the examples section. Alternatively, the neural network 150 may utilize a DenseNet architecture, where each layer of the dense block receives inputs from all previous layers, whereby the feature maps from all previous layers are concatenated and used as input for the next layer. In this case, the last layer of the dense block has access to the entire set information. As described in the examples section, a 121-layer DenseNet is used.

[0131] First, a digital image of the security document 110 is provided as a reference to the neural network 150. Thus, the security document 110 represents the reference for classifying a digital image as to whether it is considered a security document 110 or not.

[0132] Second, a set of digital training images 120 is provided to the neural network 150. The digital training images 120 may generally differ from the digital image of the security document 110. In particular, the digital training images 120 are modified relative to the digital image of the security document 110. The modification of the digital training images 120 may be in accordance with at least one of the aspects described herein above.

[0133] In certain examples, the alterations may include degradation based at least in part on the digital training image including one of worn ink, small holes, heat damage to a certain percentage of the surface, additional graffiti, stains, marks, tape, staples, or tears.

[0134] The set of training images 120 includes or consists of a first subset of positive digital training images 125 and a second subset of negative digital training images 130. In this regard, the positive digital training images 125 may have a visual effect of modification such that an unbiased human observer would consider a reproduction of each digital training image 125 to be representative of the security document 110 or multiple security documents 110. The negative digital training images 130 may have a visual effect of modification such that an unbiased human observer would not consider a reproduction of each digital training image 130 to be representative of the security document 110 or multiple security documents 110.

[0135] The digital images may be provided to show the same image size. Preferably, if pre-training with general image data is performed, the image size can be standardized to the image size used in the general image data so that the pre-trained weighting works better. The image size is one of (64,128), (128,256), (192,384), (256,512), or (299,299), preferably (64,128).

[0136] For training purposes, a suitable number of digital images are processed at once, e.g., 8 or more, 16 or more, 32 or more digital images at a time. In terms of computational load tradeoffs, batches containing 32 digital images have proven to perform well.

[0137] The learning rate should be chosen appropriately. In particular, learning rates with step sizes of 0.005, 0.01, 0.05, or 0.1 may be used. From the perspective of a trade-off with computational load, the learning rate step size may be 0.005.

[0138] The model learning process can be optimized as mentioned above. This optimization can usually be performed via various optimizers, among which stochastic gradient descent is the best known. Typically, an optimizer iteratively updates the model weights by calculating the value of a loss function for a single batch, determining the gradient of this function, and taking steps (defined by a learning rate) towards the direction that minimizes the loss. For example, the Adam (Kingma et al. "Adam: A Method for Stochastic Optimization", https: / / arxiv.org / abs / 1412.6980) optimizer can be used as a suitable optimizer for the training method of the present invention. It is built on stochastic gradient descent, where each parameter has a separate learning rate, which is adjusted according to the progress of the training.

[0139] According to additional or alternative preferred embodiments of the present invention, after establishing a functional model as a baseline, it is preferable to optimize the performance of the model, which can be done by trying different parameter combinations during training.

[0140] Additionally or separately, the training method of the present invention may implement text-based classifiers such as digits, e.g., denominations, letters or words, e.g., in the case of banknotes, and / or implementation of ensemble strategies and models, such as using an ensemble of image classifiers only, or image and text-based classifiers, particularly using ensembles, which may provide better performance and be more robust.

[0141] Reproduction may be considered in particular as a hardcopy type reproduction, for example by a printer or copier, or as a data conversion, storage or processing operation.

[0142] Naturally, it is of equal interest whether the image desired to be reproduced represents a single security document or several of them. Both events need to be considered.

[0143] The unbiased human observer has been described earlier.

[0144] Third, the ground truth 140 is provided to the neural network 150. The ground truth 140 represents at least one tolerance level of one or more unbiased human observers as to whether a copy of the respective digital training image 120 for each positive digital training image 125 and each negative digital training image 130 is considered by the neural network 150 to represent the security document 110 or a plurality of them. In other words, the digital training images 120 are classified into a first subset of positive digital training images 125 or a second subset of negative digital training images 130 using one or more labelers, in this case four labelers, per digital image. If at least one labeler, in this case two labelers, classifies the digital training image into either the first positive subset 125 or the second negative subset 130, the digital image will be classified.

[0145] The tolerance levels may be thought of as discrete values ​​that represent how an unbiased human observer would interpret a reproduction of the digital training image 120 in relation to the security document 110 .

[0146] According to the method 100, the neural network 150 may be trained to make a decision as to whether a copy of the provided digital training image 120 should be considered as a security document 110 or not. This decision is based on the acceptance level of at least one unbiased human observer. Thus, a predicted output score is assigned to the digital images, the score being indicative of the likelihood that the respective digital image can be recognized as a security document, in particular a banknote. The score is normalized to a range of 0 to 1. The closer the score is to 1, the more likely the digital image will be recognized as a security document. The closer the score is to 0, the less likely the digital image will be recognized as a security document. A threshold output score may be predefined such that digital images having a predicted output score representing the threshold or greater are considered recognizable as a security document. Conversely, digital images having a predicted output score representing less than the threshold are considered not recognizable as a security document. The threshold may be determined in correlation with the acceptance level. In other words, if the acceptance level is low, i.e. less than 50% of the labelers classify the digital image into the first positive subset, the threshold may be 0.5 or greater. On the other hand, if the tolerance level is high, i.e., more than 50% of the labelers classify the digital images into the first positive subset, the threshold may be less than 0.5.

[0147] Within the method 100, the security documents 100 and the set of digital training images 120 may each be a closed set. For example, for a given series of security documents, such as euro banknotes, the set of digital training images 120 may comprise or consist of different positive and negative digital training images 125, 130, respectively. In a further example, the security documents 110 and the digital training images 120 as references may be provided for different denominations of different currencies.

[0148] Illustratively, the set of digital training images 120 may include hundreds or thousands of each of the positive digital training images 125 and negative digital training images 130 .

[0149] 2 shows a simplified schematic diagram of a method 200 for copy protection of at least one security document 110. Again, for illustrative purposes, the method 200 is described with reference to a neural network 150. However, this should not be understood as a limitation of the method 200.

[0150] The neural network 150 may in particular be trained according to the method 100 for training an artificial intelligence based neural network 150. Thus, the neural network may be applicable to classify digital images that are considered as security documents 110. Since the neural network 150 is trained according to the training method 100, the neural network 150 recognizes at least one security document 110 as a reference. Of course, multiple security documents 110 can be trained with the neural network 150.

[0151] Within the method 200 for copy protection, a digital sample image 210 is provided to the neural network. The digital sample image 210 is typically different from the security document 110 previously provided to the neural network. In particular, the digital sample image 210 may be altered compared to the security document 110. The alterations may appear as described above.

[0152] The neural network 150 then classifies the digital sample image 210 into the first category if it determines that a copy of at least a portion of the digital sample image 210 could be considered by an unbiased human observer to represent at least one security document 110 or a plurality of security documents 110. Alternatively, the neural network 150 classifies the digital sample image 210 into the second category if it determines that a copy of any portion of the digital sample image 210 could not be considered by an unbiased human observer to represent at least one security document 110 or a plurality of security documents 110. In this respect, the neural network takes into account the differences between the digital sample image 210 and the security documents 110. However, based on the trained decision behavior, the neural network 150 can determine at least to what extent an acceptable level at which an unbiased human observer could consider a copy of at least a portion of the digital sample image 210 as representing a security document 110.

[0153] Additionally, the neural network prevents duplication of the digital sample image 210 if it is classified into the first category.

[0154] Optionally, the method 200 can be combined with a subsequently applied authentication process, preferably performed only for digital sample images 210 classified in the first category, i.e. digital sample images 210 perceptible as security documents. During the authentication process, the digital sample images of the first category are evaluated with respect to certain image security features for copy protection, such as eurion. This authentication process applied subsequently to the image security features provides a cross-check whether the positive evaluation of a document recognized as a security document with the rapid AI detection method is a "true" positive hit that actually constitutes the respective image security document for copy protection. Since authentication taking into account the image security features for copy protection requires a detailed evaluation of the entire security document, the authentication process takes a comparable amount of time. Thus, although the combination of the AI ​​detection process and the authentication process seems more complex, it also involves the advantage of an overall time saving. This is because only a small part of the digital sample images 210 classified in the first category needs to be authenticated in terms of the image security features for copy protection, and the majority of the digital sample images 210 can be conveniently omitted, since they can be classified in the second category.

[0155] 3 shows a simplified schematic diagram of a banknote detector 300. The banknote detector 300 is implemented in software within a copier 320 and is configured to perform the method 200 for copy protection of at least one security document.

[0156] In an exemplary scenario, a sample document 310 that a user of the copy machine 300 wishes to copy is provided to the copy machine 320. The copy machine 320 is configured to obtain a digital sample image 325 based on the sample document 310. The digital sample image 325 is provided to a communication means 330, for example a communication interface of the banknote detector 300. The communication means 330 is configured to provide the digital sample image 325 to a data processing unit 335. The data processing unit 335 comprises an artificial intelligence based neural network 340. The neural network 340 is configured to execute the copy protection method 200 described above. The neural network 340 classifies the digital sample image 325 into a first or second category. If the digital sample image 325 is classified into the first category, the banknote detector activates the prohibition means 350. The prohibition means 350 is configured to interrupt the desired copy process with an interrupt 355. Thus, copying of the digital sample image 325 can be avoided. If the digital sample image 325 is classified into the second category by the neural network 340, the inhibiting means 350 is not activated. Thus, the copying process is not prevented and a copy 310a of the sample document 310 may be made.

[0157] 4 shows a simplified schematic diagram of a computer program product 400. The computer program product may in particular comprise or consist of instructions which, when executed by a data processing unit, cause the data processing unit to perform steps related to the classification of the digital sample images. Furthermore, the computer program product 400 may comprise or consist of instructions which, when executed by the data processing unit, cause the data processing unit to prevent duplication of the digital sample images classified in the first category.

[0158] Alternatively, the data processing unit may be executed based on instructions contained in active inhibiting means which may be arranged and configured to prevent copying of digital sample images classified in the first category.

[0159] Although the invention has been described with reference to specific embodiments, it is not limited to these embodiments, and further alternatives will no doubt occur to those skilled in the art that are within the scope of the invention as claimed.

[0160] example: The training dataset contains 6,000 digital images, 3,000 of which are labeled as positive (positive class deemed perceivable as security documents, i.e., banknotes) and 3,000 are labeled as negative (negative class deemed not recognizable as security documents, i.e., banknotes). For the purpose of post-training testing / evaluation of the present invention, 500 digital images per class are hidden.

[0161] In this dataset, some digital images belonging to the positive class are related to real banknotes (reference banknotes) without any degradation at all, some digital images have slight degradation but are still considered to belong to the positive class by the labeler, and others have severe degradation so the labeler classifies them into the negative class. Only for banknotes with degradation, four labelers were used to classify each digital image. In this case, each digital image was labeled into the positive class if at least two labelers, i.e. 50%, labeled the image into the positive class.

[0162] The 50-layer deep learning models ResNet50, 121-layer Inceptionv3 and DenseNet121 were used.

[0163] Pre-training was done using the popular image dataset ImageNet.

[0164] In this example, the model weights are initialized to perform well on the ImageNet dataset, which is the standard for benchmarking classification models. The model is then fine-tuned by training it on a training dataset, which is resized to match the standard ImageNet dataset.

[0165] In this example, the Adam optimizer is used, which builds on stochastic gradient descent and has a separate learning rate for each parameter, adjusting these as training progresses.

[0166] In this case, each digital image in the positive class was assigned the number 1, and each digital image in the negative class was assigned the number 0. The deep learning model outputs a score, a continuous number between 0 and 1, that represents the likelihood that a particular image is of a particular class. More specifically, the closer this score is to 0, the more likely it is that the bill would be deemed not recognizable as a bill by an unbiased human observer. And vice versa: if the cup has a score closer to 1, the bill would be deemed more likely to be recognized as a bill by an unbiased human observer.

[0167] By default, the threshold between the two classes is defined to be 0.5. However, this threshold can be adjusted based on your precision / recall requirements. With the default threshold, the predicted class of an image is determined by rounding down the score to 0 if it is less than or equal to the threshold (say 0.5) and rounding up the score to 1 if it is greater than the threshold, say 0.5.

[0168] Table 1 below shows the various parameters and their possible values ​​that were used as experimental examples according to the training method of the present invention.

[0169] [Table 1]

[0170] Our experiments reveal that different models perform similarly on different metrics, i.e. models that perform well on one metric also perform well on others.

[0171] This makes it easier to select the "best" model, as there are fewer trade-offs to consider between metrics.

[0172] After searching through all parameter combinations, it was found that the following set of parameters in Table 2 performed optimally in the task of classifying digital images considering whether they could be perceived as banknotes or not.

[0173] [Table 2]

[0174] The confusion matrix shows 256 TN, 19 FP, 9 FN, and 266 TP.

[0175] This model has an accuracy value of 94.9% (ranked 1st in the model ranking), an F1 value of 95.0% (ranked 1st in the model ranking), an MCC value of 0.899 (ranked 1st in the model ranking), a recall value of 96.7% (ranked 4th in the model ranking), and a precision value of 03.3% (ranked 32nd in the model ranking).

[0176] The training method of the present invention may further comprise training on text classifiers and / or ensembles, in particular ensembles including only image classifiers or only image and text based classifiers.

Claims

1. A computer-implemented method (100) for training an artificial intelligence based neural network (150) applicable to classify digital images as being or not being security documents (110, A) without authenticating security features, comprising: a) providing as a reference at least one digital image (A1) of at least one security document (110, A); b) providing a set of digital training images (120, B1), said digital training image (120, B1) being altered compared to said digital image (A1) of said security document (110, A); said set of digital training images (120, B1) comprising a first subset of positive digital training images (125, B1-1) having modifications that have such a visual impact that an unbiased human observer would consider a copy of each digital training image (125, B1-1) to be representative of said security document (110, A) or multiple security documents (110, A); said set of digital training images (120, B1) includes a second subset of negative digital training images (130, B1-2) having modifications that do not have such a visual impact that an unbiased human observer would consider a copy of each digital training image (125, B1-1) to be representative of said security document (110, A) or multiple security documents (110, A); c) providing a ground truth (140) for each digital training image (120, B1) to the artificial intelligence based neural network (150) in step b), said ground truth (140) representing at least one acceptance level of one or more unbiased human observers as to whether a copy of each said digital training image (120, B1) is considered to be indicative or not indicative of said security document (110, A) or security documents (110, A); The method includes:

2. The quality of the at least one modified digital training image (120, B1) in step b) is degraded; The degradation of the quality of the digital training images may be (B) training documents that have been chemically or physically compromised; or A digital image of training document (B), which has been digitally eroded. The method (100) of claim 1, based on

3. The method (100) according to any one of claims 1 to 2, wherein the at least one modified digital training image (120, B1) in step b) differs from the digital image (A1) of said security document (110, A) with respect to at least one of the following: resolution, definition, general aspect, shape, colour, colour distribution and aspect ratio.

4. The method (100) according to any one of claims 1 to 3, wherein at least one modified digital training image (120, B1) in step b) differs from the digital image (A1) of said security document (110, A) with respect to at least one of the following: perspective angle, underlying lighting, coloration, creases, wrinkles.

5. said security document (110,A) having one or more graphic design features, in particular portraits, architectural images or nature images; at least one digital training image (120, B1) of step b) differs from the digital image (A1) of said security document (110, A) in that at least one design feature has been replaced by a different design feature, in particular a different portrait or a different architectural image or a different natural image, The method (100) according to any one of claims 1 to 4.

6. The method (100) of any one of claims 1 to 5, wherein the ground truth (140) represents the tolerance levels of at least four unbiased human observers, the ground truth (140) comprising or consisting of at least five different tolerance levels.

7. said security document (110,A) being a banknote, In step a), two digital images are provided, each comprising a front image (A2) and a back image (A3), in step b), each positive digital training image (125, B1-1) has a modification having such a visual impact that an unbiased human observer would recognize a copy of the respective digital training image (125, B1-1) as a front image (A2) and / or a back image (A3) of said security document (110, A) or of a plurality of security documents (110, A), The method (100) according to any one of the preceding claims.

8. The method is performed using a deep neural network (150) having one or more neural layers, In particular, each layer exhibits a large number of neurons independent of other layers, and / or each layer has a branched or non-branched architectural structure; The method (100) according to any one of the preceding claims.

9. The method (100) of any one of claims 1 to 8, wherein the artificial intelligence based neural network (150) has machine learning capabilities.

10. The method (100) is performed on a closed set of a plurality of security documents (110, A), and the artificial intelligence based neural network (150) is trained for each security document of the closed set of security documents (110, A). The method (100) according to any one of the preceding claims.

11. The neural network (150) using a residual neural network that further utilizes skip connections and / or shortcuts to skip over at least one layer, preferably said residual neural network comprising 50 layers or more; or Use an Inception network that combines convolutions with different kernel sizes, preferably two convolutions with 3x3 kernels, or Use a DenseNet with dense blocks, where each layer receives input from all previous layers, preferably using a 121-layer DenseNet. The method (100) according to any one of the preceding claims.

12. A general image dataset is also provided, 12. The method (100) of claim 11, wherein the neural network (150) is pre-trained on a large-scale generic image dataset prior to training the neural network with the digital images (120) of steps a) and b).

13. 13. The method (100) according to claim 11 or 12, wherein the digital images (120) of steps a) and b) are provided with the same standardized size, preferably in the case of pre-training, the digital images (120) of steps a) and b) are provided with the same standardized size as the image data used in the pre-training, preferably the image size is one of (64,128), (128,256), (192,384), (256,512) or (299,299), more preferably (64,128).

14. The method (100) of any one of claims 1 to 13, wherein the method (100) uses a batch size of 8 or more, 16 or more, or 32 or more digital images (120) in total for steps a) and b).

15. The method (100) according to any one of the preceding claims, wherein the method (100) uses a learning rate of one of 0.005, 0.01, 0.5 or 0.1, preferably 0.

005.

16. 16. The method (100) of claim 1, wherein the method (100) uses an ensemble of image classifiers in step b).

17. The method (100) of any one of claims 1 to 16, wherein said method (100) further uses one or more text classifiers.

18. 20. The method (100) of claim 17, wherein the method (100) uses an ensemble of one or more text classifiers.

19. A computer-implemented method (200) for preventing copying of at least one security document (110, A) without authenticating a security feature, comprising: a) providing a digital sample image (210, C1); b) applying an artificial intelligence based neural network (150) to classify said digital sample images (210, C1) into a first category or a second category, The neural network (150) is trained according to the method (100) of any one of claims 1 to 18, if the neural network (150) determines that a reproduction of at least a portion of the digital sample image (210, C1) could be regarded by an unbiased human observer as representing the security document (110, A) or multiple security documents (110, A), then the digital sample image (210, C1) is classified into the first category; If, for all of the digital sample images (210, C1), the neural network (150) determines that a copy cannot be considered to represent the security document (110, A) or multiple security documents (110, A) by an unbiased human observer, the digital sample images (210, C1) are classified into the second category; and c) preventing duplication of the digital sample image (210, C1) if the neural network (150) classifies the digital sample image (210, C1) into the first category; method.

20. 20. The method (200) according to claim 19, wherein the digital sample image (210, C1) is altered in comparison with the digital image (A1) of the security document (110, A), and the altered digital sample image (210, C1) differs from the digital image (A1) of the security document (110, A) in terms of at least one of the following: perspective angle, underlying lighting, coloration, creases, wrinkles.

21. said security document (110,A) having one or more graphic design features, in particular portraits, architectural images or nature images; said digital sample image (210, C1) differs from the digital image (110, A) of said security document (110, A) in that at least one design feature has been replaced by a different design feature, in particular a different portrait or a different architectural image or a different natural image, 21. The method (200) of claim 19 or 20.

22. The security document (110, A) includes a front and a back surface, If the neural network (150) determines that a reproduction of at least a part of the digital sample image (210, C1) can be regarded by an unbiased human observer as representing the front side and / or the back side of the security document (110, A) or a plurality of security documents (110, A), the digital sample image (210, C1) is classified into the first category of step b) of claim 19, If, for all of said digital sample images (210, C1), said neural network (150) determines that a copy cannot be regarded by said unbiased human observer as representing said front and / or said back side of said security document (110, A) or said plurality of security documents (110, A), said digital sample images (210, C1) are classified into said second category of step b) of claim 19, 22. The method (200) of any of claims 19 to 21.

23. In step c) of claim 1, the ground truth (140) represents a range of tolerance levels including or consisting of a first subrange and a second subrange; In step b) of claim 19, said digital sample image (210, C1) is classified into said first category if said neural network (150) determines an acceptance level according to said first subrange, as a reproduction of at least a part of said digital sample image (210, C1) can be regarded by an unbiased human observer as representing said security document (110, A) or a plurality of security documents (110, A), In step b) of claim 19, the digital sample images (210, C1) are classified into the second category if the neural network (150) determines that for all of the digital sample images (210, C1), a copy cannot be considered by an unbiased, impartial human observer to represent the security document (110, A) or the security documents (110, A), and the first subrange being greater than the second subrange; 23. The method (200) of any of claims 19 to 22.

24. A banknote detector (300) comprising or consisting of a communication means (330) and a data processing unit (335), the communication means (330) is configured to receive a digital sample image (210, 325, C1) and to provide the digital sample image (210, 325, C1) to the data processing unit (335); The data processing unit (335) is configured to execute a method (200) according to any of claims 19 to 23, A banknote detector (300) configured to prevent copying of said digital sample image (210, 325, C1).

25. The banknote detector (300) is implemented in software; The banknote detector (300) of claim 24, wherein the banknote detector (300) is included in an apparatus for replicating the digital sample image (210, 325, C1) or the banknote detector (300) is configured as a cloud-based or server-based application.

26. The bill detector (300) further configured to evaluate the digital sample image (210, 325, C1) for authenticity if the digital sample image (210, 325, C1) falls into the first category in step b) of claim 19, said digital sample image (210, 325, C1) is evaluated with respect to at least one security feature contained in said at least one security document (110, A); A banknote detector (300) according to claim 24 or 25.

27. 27. The banknote detector (300) of claim 26, wherein said digital sample image (210, 325, C1) is authenticated with respect to at least one image security feature for copy protection contained in said at least one security document (110, A).

28. A computer program product (400) comprising or consisting of instructions, when the program is executed by a data processing unit, causing said data processing device to perform step b) of claim 19.

Images