Method for authenticating a product by means of fourier patterns

Abstract

The invention relates to a method for authenticating a product in that a graphical code (4) is Fourier-transformed into a Fourier pattern (3), and the Fourier pattern (3) is arranged on the product. A mobile terminal (6) with a camera (8), a screen (7), and a computing unit is used, and an image of the Fourier pattern (3) is captured by the camera (8) and is Fourier-transformed back into the graphical code (4). The graphical code (4) is machine read and evaluated, augmented reality content assigned to the graphical code (4) is retrieved, and an augmented reality application is displayed on the screen (7), said application displaying the result of the evaluation of the graphical code (4).

Description

[0001] The invention relates to a method for authenticating a product.

[0002] German patent DE 10 2017 206467 A1 discloses the testing of a Fourier pattern using a mobile device. An image of the Fourier pattern is captured and subjected to a Fourier transformation. Subsequently, it is checked whether the transformed pattern contains a readable code. Furthermore, it discloses the ability to provide the user with instructions on how to adjust the zoom and flash settings for image capture.

[0003] EP 3 388 250 A1 proposes printed security features for product authentication that have such a high resolution that a simple copy leads to a significant degradation of the security feature's image quality. This degradation can be detected by machine. The problem arises that an enlarged copy of this security feature often does not result in such degradation, and machine recognition can be circumvented. During machine recognition, the size of the security feature is generally not measured, and therefore the resolution is not absolutely determined.

[0004] US patent 8442295 B2 proposes a hidden pattern as a marker for product recognition in augmented reality applications. This pattern is compared by an image processing system to predefined markers in a marker database. A disadvantage of this approach is the need to maintain such a marker database. Furthermore, the recognition process can be inaccurate.

[0005] US 8814048 B2 specifies the application of a graphical, machine-readable code to a product. However, graphical machine-readable codes, such as barcodes, have the disadvantage of being highly visible, which can negatively impact the product's aesthetic appeal. Furthermore, a barcode requires reference markers that must be located by the image processing system. Damaged or obscured reference markers can lead to non-recognition.

[0006] The object of the present invention is to provide an authentication method as mentioned above that reduces the problems mentioned above.

[0007] The problem is solved by a method mentioned at the outset with the features of claim 1.

[0008] First, a graphic code, such as a 1D or 2D barcode, a QR code, or similar, is Fourier-transformed. This Fourier pattern is then used as part of a security feature and applied to a product. The term "product" here is to be understood broadly; it can refer to the actual item being offered or sold, but also to the product's packaging or outer packaging.

[0009] A mobile device with a camera, a screen, and a processing unit is used. This mobile device is preferably a standard smartphone onto which apps can be downloaded from a typical app store, or smart glasses. However, the choice of mobile device is not limited to these options.

[0010] An image of the Fourier pattern is captured by the camera, and the captured image is inversely transformed into a Fourier pattern. The inversely transformed Fourier pattern again comprises the original graphic code, which is read by machine, and according to the invention, augmented reality content associated with the graphic code is then retrieved, and an augmented reality application is displayed on the screen.

[0011] Augmented Reality, or Augmented Reality, is defined here according to the terms found, for example, on Wikipedia: Augmented Reality is a computer-aided enhancement of the perception of reality that can, in principle, address all of the human senses. Generally, Augmented Reality only concerns the visual presentation of information; that is, images or videos are supplemented with computer-generated additional information. Virtual objects can also replace the recorded or live-streamed images through overlay and superimposition.

[0012] The invention utilizes the idea of ​​using a Fourier pattern as a security feature or as part of a security feature. The pattern is imperceptible to the human observer because it appears as a random pattern. At very high resolution, humans no longer perceive any structures, but only a homogeneous surface. The Fourier pattern can be hidden within other graphic elements, such as images or grayscale gradients. This allows for a largely inconspicuous and concealed marking of the product. It is therefore ideally suited for augmented reality applications.

[0013] Basically, to read the Fourier pattern, an image of the pattern captured by the camera and processed by an image processing program is inversely transformed into a Fourier transform, and the amplitude of the inverse Fourier transform is then used for evaluation. The inverse Fourier transform is a graphical code, preferably a barcode or similar, which can be used for barcode recognition.

[0014] The Fourier pattern has the very advantageous property of translation invariance, meaning that the Fourier pattern does not need to be positioned in a specific location relative to the camera or smartphone in its x and y directions. Instead, the camera can be positioned anywhere within the Fourier pattern in the x and y directions, or it can even capture only sections of the pattern. A Fourier inverse transformation into graphical code is still possible. This makes the detection of the security feature by the image processing program exceptionally robust.

[0015] Preferably, the evaluation of the graphic code yields an identifier, serial number, or similar. If no Fourier pattern is present in the captured image, no code is recognized. The evaluation of the graphic code can be configured to achieve a low false-positive rate, thus minimizing the risk of misidentification.

[0016] The identifier can preferably be used to identify the product via a database query. Once the product is identified, augmented reality content is retrieved and displayed.

[0017] Preferably, a barcode, in particular a Data Matrix or a QR code, is used as the graphic code. Such codes can be reliably read.

[0018] A product-specific graphical code is particularly preferred.

[0019] It is also conceivable that a unique graphic code is used. A unique code is a code that occurs at most once in a series of similar products. Bonus programs or competitions associated with the unique graphic code are particularly likely to be displayed to the user on the screen.

[0020] Ideally, a smartphone or smart glasses are used as the end device.

[0021] Furthermore, product reference data can be retrieved to aid in the positioning of augmented reality content within the display. Product reference data preferably consists of product shapes or the product's graphic design. This data allows for a very precise determination of the product's position and orientation within the camera image, thus enabling optimal positioning of the augmented reality content.

[0022] The Fourier pattern is preferably applied to the product at a resolution of at least 300 dpi, preferably at least 600 dpi, and preferably at least 800 dpi. This resolution incorporates copy protection such that the graphic code in the Fourier pattern is no longer readable after scanning and printing with a standard printer, or at least it is detectable that the product is a copy. Additionally or alternatively, reflective coatings can be applied to the Fourier pattern to disrupt the copying or scanning process.

[0023] Preferably, the augmented reality application guides the user to place the mobile device in a predetermined position relative to the Fourier pattern and to authenticate the Fourier pattern.

[0024] Especially in the case of high-resolution Fourier patterns, it can be difficult to read the hidden Fourier pattern from the captured image if the distance between the camera and the Fourier pattern is too great, or if the image is blurry or poorly lit. Therefore, user guidance can be provided that, by displaying augmented reality content such as frames, arrows, or instructions, guides the user to position the mobile device in the predetermined location relative to the Fourier pattern.

[0025] It is particularly advantageous to have the lighting on the mobile device switched on during the scanning of the Fourier pattern. This exposes the Fourier pattern to favorable lighting conditions and makes it easier for the image processing program to capture. User guidance via augmented reality applications is significantly more effective than conventional user guidance using brochures, operating instructions, or similar materials.

[0026] Another form of user guidance can consist of an animated coordinate system on the product in an augmented reality application, which can be used to align the mobile device to the Fourier pattern.

[0027] The resolution of the Fourier pattern can be determined and compared with stored data, which is particularly advantageous. For this purpose, the size of the product packaging is preferably stored in the database. If the product is identified by the augmented reality application, its size can be determined simultaneously, and the resolution of the captured Fourier pattern can then be calculated. If the determined resolution of the Fourier pattern does not match the stored data, it is highly likely that the product is counterfeit.

[0028] The invention is described with reference to an exemplary embodiment in five figures. These show: Fig. 1 shows a view of a product package with a Fourier pattern according to the invention, Fig. 2 shows a live stream of the package in Fig. 1 on the display of a smartphone, Fig. 3 shows the Fourier pattern in Fig. 1 , Fig. 3 legs from the Fourier pattern in Fig. 3a Fourier-transformed 2D barcode, Fig. 4 a representation of the live stream in Fig. 2 with an augmented reality application, Fig. 5 a representation of an interactive user guidance, Fig. 6a - 6d four different packagings of a product, each with a Fourier pattern and each with a serial number encoded in the Fourier pattern.

[0029] In Fig. 1 The image shows the packaging (1) of a pair of headphones. The packaging (1) is shown in a top view and has a square outline.

[0030] A security feature 2 is located in the upper right corner of the packaging. This security feature 2 is firmly affixed to the packaging 1. The adhesive strength of the security feature 2 on the packaging 1 is so strong that it is impossible to remove the security feature 2 without destroying the packaging 1. The security feature 1 in Fig. 1 comprises a Fourier pattern 3. The production of this type of Fourier pattern 3 is described, for example, in EP 3 388 250 A1. The Fourier pattern 3 is basically the Fourier transform of a Figur 3b barcodes 4 shown, in particular a 2D barcode 4.

[0031] In the Fig. 3a is the Fourier pattern of the Fig. 1 depicted. In the Fig. 3b is the Fourier inverse transform of Fourier pattern 3 in the Fig. 3a The inverse Fourier transform has the property that the original barcode 4 appears twice.

[0032] The 2D barcode 4 is a machine-readable, graphical 2D code. To convert it into a Fourier pattern 3, the graphical 2D code is embedded in a real amplitude function of a two-dimensional, discrete, complex function G(fx, fy) with an fx frequency coordinate and an fy frequency coordinate. Complex numbers or complex functions can generally be represented either as the sum of their real and imaginary parts or in polar coordinate notation as an amplitude function and a phase function. One way to generate the Fourier pattern 3 is to provide the 2D barcode 4 as the amplitude function of the two-dimensional, discrete, complex function G(fx, fy). The amplitude function has either the function value 0 or the function value 1 across the two frequency coordinates fx and fy.

[0033] It is conceivable that the two-dimensional, discrete, complex function G( fxfy ) will be further processed.

[0034] The two-dimensional, discrete, complex function G(fxfy) is Fourier-transformed. The resulting Fourier transform g(x,y) is binarized to form a two-dimensional image. The binarization process is discussed in EP 3 388 250 A1. Essentially, the real part of the Fourier transform g(x,y) is determined and binarized using a threshold value. Binarization means that each pixel of the image is assigned either the value 1 or the value 0. The discussed binarized Fourier transform g(x,y) corresponds to the one described in the Fig. 3a The Fourier pattern shown is also referred to as Fourier pattern 3.

[0035] The Fourier pattern 3 is preferably printed on the security label 2 or directly on the packaging 1 at a resolution of at least 300 dpi, preferably at least 600 dpi, preferably at least 800 dpi. The high resolution of the Fourier pattern 3 provides copy protection, since conventional photographing or scanning of the Fourier pattern 3 and reprinting it on a standard printer produces a printed image that destroys the very fine and numerous details and substructures of the Fourier pattern 3, making reconstruction, i.e., inverse Fourier transformation, of the 2D barcode impossible or only partially possible. This has the effect that the inverse Fourier transformation of a copied Fourier pattern of the Fig. 3a The 2D barcode 4 is generated, which appears significantly fainter, i.e., faded, and the Fourier-transformed 2D barcode 4 thus allows conclusions to be drawn, due to its faintness, that a counterfeit product may be present. In particular, the 2D barcode 4 of a copied Fourier pattern is significantly fainter in the outer regions, which represent high spatial frequencies, because high spatial frequencies are incompletely or not at all transmitted during the copying process. In the Fig. 3a und 3b is the result of the Fourier inverse transform of Fourier pattern 3 of the Fig. 3a It is represented as a 2D barcode 4. The 2D barcode 4 is so bold and sharp that it can be easily read by a barcode reader integrated into a smartphone and confirms the authenticity of the headphones or the product in general.

[0036] According to the invention, the process of authenticating the product is integrated into an augmented reality application.

[0037] Augmented Reality, as defined here, for example, on the Wikipedia website, essentially refers to the concept of "Augmented Reality." It is a computer-aided enhancement of the perception of reality, which, in principle, engages all human senses. However, Augmented Reality is often understood to mean only the visual presentation of information. This involves overlaying images, videos, or even a live stream with computer-generated additional information or virtual objects.

[0038] In Fig. 4 In the augmented reality application, a thank you message (9) "Thank you for buying headphones" is spoken and inserted as an animation into the live stream of the recorded packaging (1). For this, the product is first captured with the Fourier pattern (3) by a rear camera (8) of a smartphone (6). The captured image is then displayed in a live stream on a screen (7) of the smartphone (6) according to... Fig. 2 As shown, an app is downloaded to smartphone 6 that enables the recognition of the Fourier pattern 3. The user of smartphone 6 receives instructions via the augmented reality application on how to hold the smartphone 6 relative to the packaging 1 of the headphones and at what approximate distance to the packaging 1, or rather to the Fourier pattern 3, the smartphone 6 camera should be positioned. Maintaining the correct distance between the camera 8 and the Fourier pattern 3 is particularly important, as if the distance is too great, not all the details of the Fourier pattern 3 can be captured, and the reconstruction of the 2D barcode 4 is no longer possible.On the other hand, the image becomes blurry if the camera 8 is brought too close to the Fourier pattern 3, and thus the necessary subtleties and details of the Fourier pattern 3 are also lost, which can also make it more difficult to transform the Fourier pattern 3 back.

[0039] For example, it is conceivable that camera 8 detects the packaging 1 of the smartphone 6, thereby calculating the relative position of the packaging 1 to the smartphone 6. After detecting the position of the packaging 1, the augmented reality application animates a coordinate system on the surface of the packaging 1. Even if the packaging is moved, the coordinate system of the packaging is updated as an augmented reality application. The user then receives instructions on where to move the smartphone within the coordinate system. It is also conceivable that the coordinate system is projected onto the surface such that the origin is located in the center of the Fourier pattern 3 and the x-axis and y-axis are perpendicular to the sides of the Fourier pattern 3. Of course, other configurations are also possible.

[0040] When the smartphone 6 has reached its correct position relative to Fourier pattern 3, Fourier pattern 3 will be displayed according to the Fig. 3a und 3b The resulting 2D barcode is reverse-transformed and read using a standard barcode reader, either implemented on the smartphone or already integrated into the downloaded app. The read information, such as a serial number, can be compared with information stored in a database, and if there is a match, the data is processed. Fig. 4 The feature shown, number 9, is also displayed in the augmented reality application in the product's live stream.

[0041] The information encoded in barcode 4 can be a serial number, a combination of letters and numbers, or something similar. The serial numbers are designed to identify the product. Valid serial numbers are stored in a database. This database can be stored on the smartphone 6, or the smartphone can be connected to a central server via the internet. The database is then stored on the central server, and the scanned serial number is compared to the stored serial number. If there is a match, the product is authenticated. If the serial number is not found in the database, the product is counterfeit. In this case, Fig. 4 Instead of a "Thank You 9" message, a different, more suitable animation is displayed. If a valid serial number is detected, a bonus program or contest associated with that serial number can be run within the augmented reality application. For example, specific prizes can be assigned to certain serial numbers, or the user can receive a discount for each detected serial number.

[0042] Fig. 5 An augmented reality application is shown in the live stream of product 1, featuring an interactive user guide. First, the Fourier pattern 3 is detected in the streamed image. The augmented reality application then places a box 11 around the Fourier pattern 3 and issues an instruction to align the camera 8 of the smartphone 6 more closely and centrally with the Fourier pattern 3. If the camera 6 is moved too close to the Fourier pattern 3, a new instruction will appear on the display 7 within the augmented reality application, for example, instructing the camera 8 to move back slightly.

[0043] The Fourier pattern 3 in the Fig. 5 It is copy-protected by a high resolution of 800 dpi. It is printed on the packaging at such a high resolution that a copy of the Fourier pattern 3 made with a standard commercial copier will be significantly degraded in resolution, rendering the copied Fourier pattern 3 either illegible or at least making the copy detectable by a deterioration in quality.

[0044] To examine the Fourier pattern 3 and distinguish it from copies, a high-resolution image of the Fourier pattern 3 must be captured. In the case of the smartphone 6, this is generally only possible if the smartphone 6 is brought into close proximity to the Fourier pattern 3, which is typically less than 20 cm away.

[0045] To enable even an untrained user to position the smartphone 6 in a favorable position for testing the Fourier pattern 3, instructions are displayed to the user 3 using augmented reality user guidance.

[0046] In the Figur 5 This is a prompt with an arrow: "Please move camera here". If the user moves the smartphone in the correct direction, indicators confirming the movement may appear, such as a green checkmark. Conversely, if the user moves the device in the wrong direction, other indicators may appear, such as a red exclamation mark.

[0047] The augmented reality application can react to user movement with further augmented reality cues as feedback. Such user guidance is significantly superior to existing user guidance systems, as it can intuitively convey the desired position to the user using augmented reality and provide feedback in case of incorrect movement.

[0048] Fortunately, Fourier pattern 3 is copy-protected due to its high resolution, and product reference data, retrieved from a database, for example, contains the product's physical size, i.e., its external dimensions. In this case, the augmented reality application can define a coordinate system on the product and thus determine an absolute size of Fourier pattern 3. Using this absolute size, the augmented reality application can determine its resolution by identifying the position of the graphic code within the Fourier transform. This determines the spatial frequency in position space. The augmented reality application can recognize the Fourier pattern 3 as valid if it possesses the expected resolution. Otherwise, it is recognized as invalid. In this way, the application is protected against enlarged copies of Fourier pattern 3, and the authentication of product 1 is improved.

[0049] The illustrations in the Figuren 6a bis 6d Figure 1 shows a series of identical products 1, namely headphones and product packaging, each encoding a Fourier pattern 3 with a different individual identifier in the form of a serial number. The serial number is located below the product packaging 1 in the Fig. 6a-6d Listed below. Each product (1) in the series receives a unique serial number, which is contained in the identifier within the graphical code. A Fourier pattern (3) is used as the graphical code. This allows for the unambiguous identification and authentication of each individual product (1) within the augmented reality application. Firstly, the user can be assured within the augmented reality application that they are viewing a genuine product. Secondly, it is also possible to implement bonus programs or contests within the augmented reality application. For example, specific serial numbers could be assigned bonus payments or similar rewards, which are authorized by scanning the serial numbers. Bezugszeichenliste

[0050] 1. Packaging 2. Security element 3. Fourier pattern 4. Barcode 6Smartphone 7Display 8Camera 9Thanks 11 boxes

Claims

1. Method for authenticating a product by a graphic code (4) being Fourier-transformed into a Fourier pattern (3), and the Fourier pattern (3) being arranged on the product, a mobile terminal (6) having a camera (8), a screen (7) and a computing unit being used, and an image of the Fourier pattern (3) being recorded by the camera (8), and the recorded image being inversely Fourier-transformed into the graphic code (4), the graphic code (4) being read by machine and being evaluated and augmented reality content allocated to the graphic code (4) being retrieved and an augmented reality application being depicted on the screen (7), said application depicting a result of the evaluation of the graphic code (4), wherein the augmented reality application interactively guides the user and reacts to movements of the user, characterised in that information is given to the user in the augmented reality application as to how they are to move the mobile terminal to form the Fourier pattern, and an accuracy of the movement is evaluated by a reply.

2. Method according to claim 1, characterised in that a barcode, in particular a data matrix or QR code, is used as the graphic code.

3. Method according to claim 1 or 2, characterised in that a product-specific graphic code is used.

4. Method according to claim 1 or 2, characterised in that a unique graphic code is used.

5. Method according to claim 4, characterised in that bonus programs or competitions allocated to the unique graphic code are depicted to the user.

6. Method according to one of the preceding claims, characterised in that a smartphone or a pair of data glasses are used as the terminal.

7. Method according to one of the preceding claims, characterised in that the Fourier pattern (3) is printed onto the product with a resolution of at least 300 dpi.

8. Method according to one of the preceding claims, characterised in that the Fourier pattern (3) is printed onto the product with a resolution of at least 600 dpi.

9. Method according to one of the preceding claims, characterised in that the Fourier pattern (3) is printed onto the product with a resolution of at least 800 dpi.

10. Method according to one of the preceding claims, characterised in that the augmented reality application guides the user to move the mobile terminal (6) into a predetermined position relative to the Fourier pattern (3), and the Fourier pattern (3) is then inversely Fourier-transformed.

11. Method according to one of the preceding claims, characterised in that illumination on the mobile terminal (6) is switched on while recording the Fourier pattern (3).

12. Method according to one of the preceding claims, characterised in that the augmented reality application determines a coordinate system on the product, by means of which the mobile terminal (6) is adjusted on the Fourier pattern (3).

13. Method according to one of the preceding claims, characterised in that dimensions of the Fourier pattern (3) are determined and are compared to the recorded Fourier pattern (3), and a resolution of the Fourier pattern (3) is determined, and an authentication is only carried out with sufficient resolution.

14. Method according to one of the preceding claims, characterised in that the Fourier pattern (3) is covered with a reflective layer.

Images