Computer-implemented method for supporting the verification of a physical document

Abstract

The invention relates to a computer-implemented method for supporting a verification of a physical document, the document comprising: at least one entry in human-readable form, said entry being located in an associated position on the document, and a machine-readable element which contains one or more specifications which are each linked to an associated entry of the at least one entry. The method comprises: capturing at least one image of the document by means of a camera (104), reading the specifications out of the machine-readable element by means of a reading unit (104), determining the positions of the entries linked to the read-out specifications in the at least one captured image, and displaying the at least one captured image on a display device (106), the read-out specifications each being shown in human-readable form at or adjacent to the determined position of the entry linked to the corresponding specification. A computer program, a machine-readable storage medium and a device (100) for supporting a verification of a physical document are also disclosed.

Description

[0001] Computer-implemented method to support the verification of a physical document

[0002] The invention relates to a computer-implemented method for supporting a verification of a physical document as well as a corresponding computer program, a corresponding machine-readable storage medium and corresponding devices.

[0003] Physical documents such as identity documents can contain not only human-readable information (e.g., last name, first name, and date of birth in text form), but also a machine-readable code such as a barcode or a 2D code. This code can contain a copy of the information contained in the information and include security features such as encryption to protect against manipulation or forgery. By reading the information from the machine-readable code and comparing it with the information contained in the information, the authenticity of the document can be verified.

[0004] Methods and devices for reading such codes are known from the state of the art, in which the extracted information is displayed, for example, in a table. To verify the document, an examiner can compare the information contained in the entries on the document with the displayed data. However, such a procedure is complex and error-prone, particularly against manipulation in which entries on the document are only slightly changed or swapped (for example, last name and first name).

[0005] Summary

[0006] The object of the invention is therefore to provide technical means to support the verification of a physical document, with which the verification can be simplified and its reliability improved.

[0007] To solve the problem, according to a first aspect, a computer-implemented method for supporting the verification of a physical document is provided. The document has at least one entry in human-readable form and a machine-readable element. The entry(s) is / are arranged at a respective position on the document. The machine-readable element contains one or more pieces of information, each of which is linked to an entry of the at least one entry.The method comprises capturing at least one image of the document using a camera, reading the information from the machine-readable element using a reading device, determining the positions of the entries linked to the read information in the at least one captured image, and displaying the at least one captured image on a display device, wherein the read information is displayed in human-readable form at or adjacent to the determined position of the entry linked to the corresponding information.

[0008] According to a second aspect, a computer program for supporting a verification of a physical document is further provided, which computer program comprises instructions which, when the program is executed by a processor, cause the processor to carry out the inventive method for supporting a verification of a physical document according to one of the embodiments described herein.

[0009] According to a third aspect, a machine-readable storage medium for supporting verification of a physical document is also provided, which comprises instructions which, when executed by a processor, cause the processor to carry out the inventive method for supporting verification of a physical document according to one of the embodiments described herein.

[0010] Furthermore, according to a fourth aspect, a device is provided which has a camera, a display device and a control unit, wherein the control unit is configured to carry out the inventive method for supporting a verification of a physical document according to one of the embodiments described herein.

[0011] According to a fifth aspect, a device is provided for supporting verification of a physical document, wherein the document has at least one entry in human-readable form and a machine-readable element, and the machine-readable element contains one or more pieces of information, each linked to an entry of the at least one entry. The device has a readout device, an augmented reality unit, and a display device. The readout device is configured to capture an image of the document and to capture an item of information from the machine-readable element. The augmented reality unit is configured to generate an overlay image in which the captured image of the document is overlaid with the captured item of information in human-readable form over or next to the entry linked to the corresponding item of information.The display device is configured to display the overlay image.

[0012] The method according to the invention can be carried out using any suitable computer device, wherein the camera, the reading device, and the display device can each be part of the computer device and / or an external device suitably connected to the computer device. In particular, the method can be carried out using a device according to the invention according to one of the embodiments described herein.

[0013] The method serves to support the verification of a document to be checked, for example, by a checking person (e.g., a border or police officer). The verification can be performed manually and / or partially or fully automated (e.g., by the inventive device according to the fourth or fifth aspect). The verification can be part of the inventive method or, for example, be performed subsequently to it. The verification can serve, for example, to check the authenticity of the document.

[0014] The document is in physical form and can be, for example, in paper form or as a card (e.g., in credit card format and / or as a chip card). The document contains one or more entries in human-readable form, for example, in text form (written text) and / or human-readable symbol form (e.g., in pictogram form). Each of the entries is arranged at a corresponding position on the document and can contain one or more pieces of information (e.g., last name, first name, and date of birth).

[0015] The machine-readable element contains (e.g. stores or represents) the one or more pieces of information in a form that is (generally) unreadable for humans, i.e., for example, can only be read using technical aids such as the reading device or the reading apparatus. Preferably, the machine-readable element is an optoelectronically readable code, such as a one-dimensional barcode or a two-dimensional 2D code, in particular a matrix code such as a QR code or a DataMatrix code. In this case, the camera can be used as a reading device for reading the information from the machine-readable element, i.e., the reading device can be the camera or comprise it. In particular, the information represented in the optoelectronically readable code can be read from the at least one image recorded by the camera.In other embodiments, the machine-readable element may, for example, be an electronically readable element such as a memory chip. In such cases, the information can be read from the machine-readable element using an electronic reading device, for example, a chip card reader (e.g., before or after capturing the at least one image).

[0016] The information contained in the machine-readable element is each linked to a corresponding entry on the physical document. The information can, for example, contain information relating to the associated entry. The information can, in particular, be target information which is intended to correspond to information in the associated entry, i.e. in the case of a genuine (not forged) document it is intended to match (e.g. be identical) to the information in the associated entry. For example, an entry can contain a name (e.g. first name or surname) and the machine-readable element can contain a corresponding target name. Alternatively or additionally, the information contained in the machine-readable element can, for example, also contain verification information for the associated entry, e.g. a checksum.

[0017] The at least one recorded image may be a single image (e.g. a photograph) or comprise a plurality of images, for example a plurality of photographs or preferably a plurality of video images (ie moving images recorded in immediate succession).

[0018] To display the at least one recorded image with the read-out information displayed, an overlay image can be generated in which the recorded image of the document with the read-out information is superimposed in human-readable form over or next to the entry linked to the corresponding information, for example by means of the control unit and / or the augmented reality unit of a device according to the invention.

[0019] The positions of the entries linked to the read-out information in the at least one recorded image (i.e., the location or area at / in which the corresponding entry is located in the image) can be determined by means of the control unit and / or the augmented reality unit of a device according to the invention. The positions of the entries linked to the read-out information in the at least one recorded image can be determined, for example, by detecting the corresponding entries and / or an associated entry indicator (e.g., a symbol or label such as "Last name:" or "First name:") in the at least one recorded image, for example, by means of computer-assisted text, image, and / or pattern recognition.

[0020] Preferably, the positions of the entries are determined by detecting a reference element arranged on the document in the at least one recorded image, for example using position information known in advance regarding the relative position(s) of the at least one entry on the document relative to the reference element on the document. This position information can be stored in advance (i.e., before carrying out the method according to the invention) in a memory and read out therefrom. The position information can characterize the relative position(s) of the at least one entry on the document relative to a reference element on the document, for example, specify a distance vector (e.g., relative coordinates).

[0021] Determining the positions of the entries linked to the read-out information in the at least one captured image may include detecting the reference element in the at least one captured image and determining the position of the reference element in the at least one captured image. The reference element may be detected, for example, using computer-assisted text, image, and / or pattern recognition.

[0022] The positions of the entries linked to the read-out information in the at least one captured image can be determined based on the position of the reference element and the stored position information. For example, a relative position of the entry in question (e.g., a distance vector) specified by or derived from the position information can be added to the position of the reference element to determine the position of the entry in question.

[0023] Preferably, the reference element is the machine-readable element or a part thereof, for example the optoelectronically readable code or a part thereof (e.g., a specific area or point such as a center point, an edge, or a corner of the machine-readable element). In other embodiments, the reference element can also be another element on the document (e.g., a specific entry, a security feature, or a photo) or a specific area or point of the document, e.g., a center point, an edge, or a corner of the document. In some embodiments, two or more reference elements can also be used to determine the positions of the entries linked to the read information.

[0024] Determining the positions of the entries linked to the read-out information in the at least one recorded image can comprise detecting a scaling and / or orientation of the document in the at least one recorded image. Depending on the perspective from which the at least one image of the document was taken, the document can be depicted in different sizes and / or at different angles. The scaling and / or orientation of the document can be detected based on a part of the document, in particular based on the reference element and / or the machine-readable element. Detecting the scaling and / or orientation of the document can comprise determining one or more dimensions (e.g.edge lengths) of the document or part thereof (in particular the reference element and / or the machine-readable element) and / or determining one or more angles of rotation of the document or part thereof (e.g. based on the one or more dimensions and / or an orientation of the edges of the document or part thereof).

[0025] Based on the detected scaling and / or orientation of the document, the position information (e.g., the relative positions characterized by it) can be converted, for example, to determine derived (e.g., transformed) relative positions. The conversion may include one or more scaling, rotation, and / or translation transformations.

[0026] In some embodiments, position information for a plurality of document types can be stored in the memory. The method can comprise determining a document type of the document in order to read the position information for the respective document type from the memory. Preferably, the document type is read from the machine-readable element. In other embodiments, the document type can also be determined based on the at least one recorded image, for example based on an entry on the document that indicates the document type, and / or based on an arrangement and / or number of the entries and / or other elements on the document. The at least one image can comprise a plurality of video images. The method can comprise tracking the positions of the entries linked to the read information in the plurality of video images using a tracking method, e.g.to determine tracked positions of the relevant entries for each of the video images. The tracking can be performed, for example, using an optical flow (e.g., a vector field of velocities of one or more points in the video images projected into the image plane) and / or by determining an acceleration of the camera using an acceleration sensor. Based on the optical flow and / or the acceleration, the positions of the entries can be suitably converted to determine the tracked positions. Alternatively or additionally, the tracking can also comprise repeatedly determining the positions of the relevant entries in the video images, e.g., as described above.

[0027] The plurality of video images can be displayed on the display device, with the extracted information being displayed in human-readable form at or adjacent to the tracked position of the entry associated with the corresponding information. Preferably, the video images are displayed (substantially) in real time, for example, with a delay of less than one second, in one example of less than 200 ms, between capture and display of a video image.

[0028] The information may be contained in the machine-readable element in encrypted form, e.g., represented in encrypted form in the machine-readable code. The information may be encrypted using an asymmetric cryptographic method, e.g., signed using a private key. This may, for example, make it more difficult to manipulate the information in the machine-readable element.

[0029] In some embodiments, the method can comprise comparing a read-out piece of information with an item of information in the entry on the document linked to the corresponding read-out piece of information, e.g. by means of the control unit or a verification unit of a device according to the invention. For example, it can be checked whether a read-out target piece of information matches the item of information in the entry in question. In one example, the item of information in the entry on the document is in text form and the method comprises reading the item of information in the entry using computer-aided text recognition. The authenticity of the document can be verified based on the comparison. For example, a discrepancy between the read-out piece of information and the item of information in the entry in question can indicate that the document may have been forged or tampered with.The method may comprise displaying a verification indicator on the display device indicating a comparison result for one or more (preferably all) pieces of information read out. The verification indicator may be linked to a single piece of information, a plurality of pieces of information, or all pieces of information. For example, a first verification indicator (e.g., a check mark) may be displayed if the corresponding piece(s) of information read out are each compatible with the piece of information in the respective entry (e.g., match / are identical to it), and a second verification indicator (e.g., a warning symbol such as an exclamation mark or cross) may be displayed if at least one of the corresponding piece(s) of information read out is each incompatible with the piece of information in the respective entry (e.g., differs from it).Alternatively or additionally, a third verification indicator may be displayed to indicate that the result of the comparison is ambiguous, for example to indicate that more detailed human review is required.

[0030] Alternatively or additionally, the method may comprise displaying a status indicator on the display device indicating whether retrieved information (e.g., information linked to a specific entry or all retrieved information) is displayed on the displayed image, for example, to alert a user that the information is ready for verification (e.g., manual reconciliation). The status indicator may, for example, be a highlight (e.g., a frame) displayed above, around, or next to the relevant information(s) and / or entry(s).

[0031] The device according to the fourth or fifth aspect of the invention can have a reading device for capturing an image (in particular a plurality of images) of the document and / or for reading information (in particular a plurality of information) from the machine-readable element. The reading device can be or comprise a camera. Alternatively or additionally, the reading device can be or comprise a reading device, for example an optoelectronic or (purely) electronic reading device (e.g., a chip card reader). The reading device can be fully or partially integrated into the control unit and / or the augmented reality unit of the device, for example, functionality for reading data from an optoelectronically readable code in a captured image. The display device can be or comprise a screen, in particular a pressure-sensitive screen (touchscreen).The display device may be configured to display a captured image and / or an overlay image (e.g., a captured image with the information displayed).

[0032] The device may comprise further components, in particular an acceleration sensor configured to determine an acceleration acting on the device. The acceleration sensor may further be configured to determine an orientation of the device or a part thereof, in particular the camera, for example based on the acceleration due to gravity. Alternatively or additionally, the device may comprise a verification unit configured to compare a read-out item with an item in the entry on the document linked to the corresponding read-out item. The verification unit may be fully or partially integrated into the control unit of the device.

[0033] The device may have a control unit, which may be implemented as hardware and / or software. The control unit may, for example, comprise a processor and a storage medium, wherein the storage medium contains instructions (e.g., the computer program according to the invention) which, when executed by the processor, cause it to provide the functionality described herein, for example, to execute the method according to the invention in whole or in part according to one of the embodiments described herein. The control unit may be configured to control other components of the device, in particular the camera and / or the display device, in order to provide the functionality described herein.

[0034] Alternatively or additionally, the device may comprise an augmented reality unit configured to generate an overlay image in which a captured image of the document is overlaid with a read-out item of information in human-readable form above or next to the entry linked to the corresponding item of information (in particular with a plurality of read-out items of information in human-readable form, each above or next to the entry linked to the corresponding item of information). The augmented reality unit may also be implemented as hardware and / or software, as described for the control unit. In some embodiments, the control unit may comprise the augmented reality unit (or vice versa). The augmented reality unit may, for example, be embodied in the form of hardware and / or software within the control unit.In other embodiments, the device may comprise a control unit and an augmented reality unit, each embodied as a standalone unit or device. The control unit may be configured to control the augmented reality unit to provide the functionality described herein. In one example, the device does not comprise a control unit; instead, the control unit may be embodied, for example, as an external unit that controls the device according to the invention.

[0035] The augmented reality unit can be configured to carry out one or more steps of the method according to the invention according to one of the embodiments described herein, in particular determining the positions of the entries linked to the read-out information in a recorded image (e.g. as described above), detecting a reference element, detecting a scaling and / or orientation of the document, converting position information, tracking positions by means of a tracking method and / or displaying a verification indicator and / or a status indicator.

[0036] In a preferred embodiment, the device is designed as a smartphone or augmented reality glasses.

[0037] Description of implementation examples

[0038] Further embodiments are explained in more detail below with reference to the figures of a drawing. Herein:

[0039] Fig. 1 is a schematic representation of an apparatus for supporting verification of a physical document according to an exemplary embodiment;

[0040] Fig. 2 is a flowchart of a computer-implemented method for supporting verification of a physical document according to an exemplary embodiment;

[0041] Fig. 3 is a schematic representation of a physical document and the device of Fig. 1 with an overlay image of the physical document displayed thereon according to an exemplary embodiment; and

[0042] Fig. 4 shows a schematic representation of a machine-readable storage medium with a computer program for supporting verification of a physical document according to an exemplary embodiment. Fig. 1 shows a schematic representation of a device 100 for supporting verification of a physical document according to an exemplary embodiment in a side cross-sectional view.

[0043] The device 100 comprises a control unit 102 with a processor 102A and a storage medium 102B. The processor may, for example, be a central processing unit (CPU), a graphics processing unit, a field programmable gate array (FPGA), and / or an application-specific integrated circuit (ASIC), or may comprise one or more of these elements. The storage medium 102B may comprise non-volatile memory and / or volatile memory. The storage medium 102B may contain instructions that can be executed by the processor 102A to provide the functionality described herein. The control unit is configured to execute a method for supporting verification of a physical document according to one of the embodiments described herein, for example, the method 300 of Fig. 3.In one example, the computer program 402 and / or the position information 410 from Fig. 4 are stored on the storage medium 102B.

[0044] The device 100 further comprises a readout device 104, which in the example of Fig. 1 is embodied as a camera, for example, a CCD or CMOS camera. The readout device 104 is configured to capture images of a physical document and read information from an optoelectronic code on the document from these images.

[0045] The device 100 also has a display device 106 for displaying images, which can be configured, for example, as a touch-sensitive LCD or OLED screen (touchscreen). In one example, the device 100 is configured as a smartphone, wherein the inventive functionality of the control unit 102 can be provided as a computer program in the form of an application (app). The readout device 104 and the display device 106 are connected to the control unit 102, e.g., via a suitable data connection, and can be controlled by it.

[0046] Fig. 2 shows a flowchart of a computer-implemented method 200 for supporting verification of a physical document according to an exemplary embodiment. The method 200 can be carried out, for example, using a device according to the invention according to one of the embodiments described herein. The method is described below by way of example using the device 100 from Fig. 1 and the physical document 300 schematically shown in Fig. 3. However, this is not to be understood as limiting, and the method 200 can also be carried out with other physical documents and / or other devices. Furthermore, the method 200 is not limited to the sequence indicated by the flowchart in Fig. 2. As far as technically possible, the steps of the method 200 can be carried out in any order and / or at least partially simultaneously.

[0047] The method 200 can be used, for example, to assist a human examiner (e.g., a border or police officer) in verifying the authenticity of the document 300. The document 300 is schematically illustrated in Fig. 3 together with the device 100, wherein the device 100 is shown in Fig. 3 in plan view, so that only the display device 106, but not the control unit 102 and the reading device / camera 104, can be seen.

[0048] The document 300 can, for example, be an identification document (e.g., a passport or identity card), a residence permit (e.g., a visa), an authorization document (e.g., an access authorization), a certificate, or another document. The document 300 is in physical form, e.g., as a card, and contains a plurality of entries 302, which, in the example of Fig. 3, contain information on a document type (“visa”), surname (“Mustermann”), first name (“Erika”), date of birth (“01 / 01 / 1970”), a secondary provision (“authorized to work”), and a document number (“0123456789”). The entries are each applied at a predetermined position on the document in human-readable form (e.g., text form in Fig. 3).

[0049] In addition, the document 300 has a machine-readable element 304, which in the example of Fig. 3 is embodied as an optoelectronically readable code, e.g., as a two-dimensional DataMatrix code. The machine-readable element 304 contains (e.g., represents) a plurality of pieces of information, each of which is linked to one of the entries 302. In the example of Fig. 3, these pieces of information are each target pieces of information to which the pieces of information in the entries 302 are intended to correspond (i.e., if the document is authentic, the target pieces of information in the machine-readable element 304 are identical to the pieces of information in the entries 302). The pieces of information in the machine-readable element 304 are encrypted, e.g., digitally signed with a secret key, so that they cannot be easily tampered with. The method 200 includes, in step 202, capturing an image 306 of the document 300 using the reading device or camera 104 of the device 100.Preferably, in step 202, a video is recorded with a plurality of video images of the document 300.

[0050] In step 204, the information 308 is read from the machine-readable element using a reader. In the example of Figs. 1 and 3, the camera 104 serves as a reader for the optoelectronically readable code 304, and the information 308 is extracted from one or more of the images 306 captured in step 202. The extraction of the information from the images 306 (e.g., the detection of data element blocks in the code 304, translation of the data element blocks into a suitable data format, and / or decryption of the data) can take place in the camera 104 itself or by the control unit 102 (in other words, parts of the reader functionality can be outsourced to the control unit 102).

[0051] In steps 206 and 208, an overlay image 306' is generated from one or more of the images 306 captured in step 202 and the information 308 read out in step 204, in which the corresponding captured image 306 is overlaid with the read out information in human-readable form (e.g., text form). For this purpose, the control unit 102 can be configured as an augmented reality unit that provides this functionality, or can comprise such a unit.

[0052] In step 206, the positions of the entries 302 linked to the information 308 read out in step 204 are determined in one or more of the images 306 captured in step 202. For this purpose, for example, a reference element on the document 300, such as the machine-readable element 304, can be detected in one or more of these images 306, and its position in the respective image can be determined. In some examples, this can occur during the reading in step 204. The positions of the entries 302 can be determined based on the position of the reference element in the respective image and position information that characterizes the relative positions of the entries 302 on the document relative to the reference element (e.g., specifying distance vectors between the entries and the reference element).This position information may, for example, be stored in advance on the storage medium 102B, for example as described below with respect to the storage medium 400 of Fig. 4.

[0053] In some embodiments, the perspective from which the document 300 is captured in the images 306 may be unknown or may vary, for example, across a plurality of images. To account for this, in step 206, a scaling and / or orientation of the document 300 may be detected for one or more of the images 306 captured in step 202, for example, based on the length and orientation of the edges of the document 300 and / or the optoelectronically readable code 304. The position information may be converted based on the detected scaling and / or orientation of the document, for example, by suitable scaling and / or rotation transformations of the distance vectors. The positions of the entries may be determined based on the position of the reference element and the converted position information, for example, by adding the transformed distance vectors to the position of the reference element.

[0054] Alternatively or additionally, the position of the entries can also be tracked using a tracking method. For this purpose, for example, the position of the entries in a first recorded image can be determined as described, while the position of the entries in later recorded images is not determined directly, but is indirectly predicted or estimated with the aid of the tracking method based on the position in the first recorded image. For this purpose, for example, an acceleration of the device 100 and / or the camera 104 measured by an acceleration sensor (inertial sensor, not shown) of the device 100 can be used. Alternatively or additionally, an object tracking method, in particular an object tracking method based on the optical flow in the recorded images, can also be used. In general, the tracking method can be an optical method, a sensor-based method, or a combination thereof.Sensor-based tracking can be performed, for example, using optical, acoustic, magnetic, and / or inertial sensors. In a preferred embodiment, a combination of optical and inertial sensors is used.

[0055] In some embodiments, the method 200 can be carried out by means of (e.g. on) a device according to the invention designed as a smartphone, and the detection of the scaling and / or orientation of the document, the conversion of the position information and / or the tracking method can be carried out entirely or partially with the aid of an augmented reality program interface (Application Programming Interface, API) such as the ARCore SDK for Google Android or the ARKit and / or RealityKit for Apple iPhone. In step 208, one or more of the images 306 recorded in step 202 are displayed on the display device 106. The information 308 read out in step 204 is displayed in human-readable form at or adjacent to the position of the entry (302) linked to the corresponding information 308, which position was determined in step 206.In other words, one or more overlay images 306' are displayed, in each of which a captured image 306 of the document 300 with the read-out information 308 is superimposed in human-readable form over or next to the entry linked to the corresponding information.

[0056] This is illustrated by way of example in Fig. 3, where an overlay image 306' is displayed on the display device 106, in which the corresponding information 308 from the machine-readable element 104 is displayed above each entry 302 on the document (indicated by an optional underline in Fig. 3). By comparing the entries 302 and the displayed information 308, the authenticity of the document 300 can be quickly and reliably verified. The read-out information 308 can also be displayed in tabular form, as illustrated in Fig. 3.

[0057] Once the overlay image 306' is displayed and / or the information 308 has been displayed, a status indicator may be displayed on the display device, for example, to inform the reviewer that the retrieved information is ready. The status indicator may, for example, be an underline (as shown in Fig. 3) or a frame around the relevant information 308 and / or the relevant entry 302, or a frame around the overlay image 306'.

[0058] In some embodiments, the method 200 may further comprise comparing one or more read-out pieces of information 308 with an item of information in the entry 302 linked to the corresponding read-out piece of information 308. For example, the textual information in the entries may be extracted from one or more of the images captured in step 202 using computer-aided text recognition. The comparison may, for example, comprise checking whether the relevant piece of information 308 is compatible with the item of information in the corresponding entry 302 (e.g., matches it or results in the correct checksum).

[0059] The result of the comparison can be displayed in the form of one or more verification indicators 310 on the display device 106. For example, as shown in Fig. 3, a verification indicator 310 can be displayed for the document 300 as a whole, indicating whether the document 300 is considered authentic or not (e.g., whether all extracted information 308 is compatible with the corresponding entry 302). Alternatively or additionally, a verification indicator 310 can also be displayed for each entry 302 (or groups of entries), indicating whether the information in the entry 302 is compatible with the relevant extracted information 308.

[0060] Fig. 4 shows a schematic representation of a machine-readable storage medium 400 for supporting verification of a physical document according to an exemplary embodiment. The storage medium 400 can be a non-volatile storage medium, such as a flash memory. The storage medium 400 can be used as the storage medium 102B of the device 100 or serve as a data carrier, e.g., to transfer the computer program 402 stored on the storage medium 400 to the storage medium 102B of the device 100.

[0061] A computer program 402 for supporting the verification of a physical document, such as the physical document 300 of Fig. 3, is stored on the storage medium 400. The computer program 402 includes commands 402-408 (e.g., instructions and / or program statements) that, when executed by a processor, such as the processor 102A of the device 100, cause the processor to perform the inventive method according to one of the embodiments described herein.

[0062] In the example of Fig. 4, the computer program 402 contains instructions 402-408 for executing the method 200 from Fig. 2, namely instructions 402 for executing step 202 (i.e. for capturing at least one image 306 of the document 300 by means of the camera 104), instructions 404 for executing step 204 (i.e. for reading the information 308 from the machine-readable element 304 by means of a reading device such as the camera 104), instructions 406 for executing step 206 (i.e. for determining the positions of the entries 302 linked to the read information 308 in the at least one captured image) and instructions 408 for executing step 208 (i.e. for displaying the at least one captured image 306 on a display device 106, wherein the read information 308 is in each case at or adjacent to the determined position of the entry 302 linked to the corresponding information 308 is displayed in human-readable form).

[0063] In addition, the storage medium 400 contains position information 410, based on which the relative positions of entries 308 on the corresponding document relative to a reference element on the document, such as a machine-readable element (e.g., the optoelectronically readable code 304 on the document 300), can be determined for a variety of document types (e.g., various types of identification documents and / or residence permits). For example, the distance vectors of all entries for documents of the corresponding types relative to the reference element can be stored in the position information 410.

[0064] The features disclosed in the above description, the claims and the drawings may be important for the realization of the various embodiments both individually and in any combination.

Claims

Claims 1. A computer-implemented method (200) for supporting verification of a physical document (300), the document (300) comprising: at least one entry (302) in human-readable form arranged at a respective position on the document (300), and a machine-readable element (304) containing one or more pieces of information (308) each linked to an entry (302) of the at least one entry (302), the method (200) comprising: - taking at least one image (306) of the document (300) by means of a camera (104), - reading the information (308) from the machine-readable element (304) by means of a reading device (104), Determining the positions of the entries (302) linked to the read-out information (308) in the at least one recorded image (306), and - displaying the at least one recorded image (306) on a display device (106), wherein the read-out information (308) is displayed in human-readable form at or adjacent to the determined position of the entry (302) linked to the corresponding information (308).

2. Method (200) according to claim 1, characterized in that the machine-readable element (304) is an optoelectronically readable code which represents the one or more items of information, and the reading device for reading the items of information (308) from the machine-readable element (304) is the camera (104) or comprises the camera (104).

3. Method (200) according to claim 1 or 2, characterized in that position information (410) is stored in advance in a memory (102B, 400), which characterizes the relative position(s) of the at least one entry (302) on the document (300) relative to a reference element (304) on the document (300), and determining the positions of the entries (302) linked to the read-out information (308) in the at least one recorded image (304) comprises: Detecting the reference element (304) in the at least one recorded image (306) and determining its position, and Determining the positions of the entries (302) linked to the read-out information (308) in the at least one recorded image (306) based on the position of the reference element (304) and the stored position information (410).

4. Method (200) according to claim 3, characterized in that the reference element is the machine-readable element (304) or a part of the machine-readable element (304).

5. Method (200) according to claim 3 or 4, characterized in that determining the positions of the entries (302) linked to the read-out information (308) in the at least one recorded image (306) comprises: Detecting a scaling and / or orientation of the document (300) in the at least one captured image (306); and Converting the position information (410) based on the detected scaling and / or orientation of the document (300).

6. Method (200) according to one of claims 3 to 5, characterized in that position information (410) for a plurality of document types is stored in the memory (102B, 400) and the method (200) further comprises determining a document type of the document (300) in order to read the position information (410) for the relevant document type from the memory (102B, 400).

7. The method (200) according to any one of claims 3 to 6, characterized in that the at least one image (306) comprises a plurality of video images and the method (200) further comprises tracking the positions of the entries (302) linked to the read-out information (308) in the plurality of video images by means of a tracking method.

8. Method (200) according to one of the preceding claims, characterized in that the information (308) is contained in the machine-readable element (304) in encrypted form.

9. The method (200) according to any one of the preceding claims, characterized in that the method (200) further comprises comparing a read-out item (308) with an item in the entry (302) on the document (300) linked to the corresponding read-out item (308).

10. The method (200) according to claim 9, characterized in that the information in the entry (302) on the document (300) is in text form and the method (200) comprises reading the information in the entry (302) by means of computer-aided text recognition.

11. Method (200) according to claim 9 or 10, characterized in that the method (200) further comprises displaying a verification indicator (310) on the display device (106) which indicates a result of the comparison for one or more read-out items of information (308).

12. The method (200) according to any one of the preceding claims, characterized in that the method (200) further comprises displaying a status indicator on the display device (106) which indicates whether read information (308) is superimposed on the displayed image (306).

13. A computer program (402) for supporting verification of a physical document (300), comprising instructions (404-408) which, when the program (402) is executed by a processor (102A), cause the processor (102A) to carry out the method (200) according to any one of the preceding claims.

14. A machine-readable storage medium (400) for supporting verification of a physical document (300), comprising instructions (404-408) which, when executed by a processor (102A), cause the processor (102A) to perform the method (200) according to any one of claims 1 to 12.

15. A device (100) comprising: a camera (104); a display device (106); and a control unit (102), wherein the control unit (102) is configured to carry out a method (200) according to any one of claims 1 to 12.

16. Apparatus (100) for supporting verification of a physical document (300), the document (300) comprising: at least one entry (302) in human-readable form, and a machine-readable element (304) containing one or more pieces of information (308) each linked to an entry (302) of the at least one entry (302), the device (100) comprising: - a reading device (104) configured to capture an image (306) of the document (300) and to read out an item of information from the machine-readable element (304), an augmented reality unit (102) configured to generate an overlay image (306') in which the captured image (306) of the document (300) is overlaid with the read-out piece of information (308) in human-readable form above or next to the entry (302) linked to the corresponding piece of information (308), and a display device (106) configured to display the overlay image (306').

17. Device (100) according to claim 15 or 16, characterized in that the device (100) is designed as a smartphone or augmented reality glasses.

Images