AR display of output security information

Abstract

The invention relates to a method for outputting a display of security information (211), the security information (211) being designed to unlock a lock, the lock being designed to protect access to a device (1) to be protected, comprising the following steps:-visually recording (S1) a device identification (12) of the device (1) to be protected; thereby creating device identification information; -transmitting (S2) the device identification information; -receiving (S3) the security information (211), the security information (211) being assigned to the device identification information (12); -outputting (S4) a visual display (21), the visual display (21) comprising at least a portion of an image (212) of the device (1) to be protected and a display of the security information (211); additionally outputting (S5) a visual aid (213) for unlocking the lock using the security information (211). The invention also relates to a related augmented reality device and to a corresponding method for transmitting security information using a related AR background system.

Description

Technical Field

[0001] The present invention relates to a method for outputting a display of safety information. The present invention also relates to a related augmented reality device and a corresponding method for sending safety information using a related AR background system. Background Art

[0002] In industrial automation systems, control devices are used, in particular in production environments. Even if remote access is possible in many cases via a data communication connection, direct local access to the local user interface plays an important role. In order to unlock access to critical functions, a device code, in particular a PIN or a password, must be entered. This is done in particular depending on the respective protection level and / or in particular to enable configuration functions.

[0003] Password memories are known in the prior art, in particular for PCs or mobile devices. These password memories enable the user to access the stored passwords after entering a master password. The user can in particular type the password or insert it into an input field by "copy & paste".

[0004] It is also known that web browsers, similar to password storage devices, store passwords and thereby automatically fill in fields in a web interface. When accessing a device remotely, it is known that the accessing device or the accessing user authenticates in a password-based manner or by means of a password.

[0005] Furthermore, the use of augmented reality (AR) in production is known. In particular, manuals can be displayed to service technicians in this way. Industrial solutions based on AR are also known, in which objects currently viewed by an AR user (e.g. a service technician) are automatically recognized or additional information is added to the local AR user's field of vision via AR glasses. Summary of the invention

[0006] The object of the present invention is to provide an improved solution for local access to devices protected by a code.

[0007] The invention is derived from the features of the independent claims. Advantageous further developments and embodiments are the subject matter of the dependent claims. Embodiments, possible applications and advantages of the invention are derived from the following description and the drawings.

[0008] The invention relates to a method for outputting a display of security information, wherein the security information is designed for unlocking a lock, wherein the lock is designed for access protection to a device to be protected,

[0009] The method has the following steps:

[0010] - visually record the device identification of the device to be protected, thereby creating device identification information;

[0011] - Send the device identification information;

[0012] - receiving security information, wherein the security information is assigned to the device identification information; and

[0013] - outputting a visual display, wherein the visual display includes at least a portion of an image of the device to be protected and a display of the security information; and

[0014] - outputting a visual aid for accessing the device using the security information.

[0015] The device to be protected is designed in particular as a device of an industrial automation system, in particular as a device in a production environment, in particular as a device, in particular as a control device. The lock in particular protects access to a local device interface of the device. Releasing the lock enables access to the local device interface of the device. The local device interface is in particular designed as a local user interface, so that no remote access via a data communication connection is involved.

[0016] Alternatively or additionally, the device is designed as an electronic lock of a control cabinet or a door to an equipment room. The lock is especially designed as an electronic lock. In this embodiment, the access protection is especially designed as access protection.

[0017] In summary, the access protection is designed in particular to:

[0018] - electronic locks for control cabinets; and / or

[0019] - an electronic lock on the door to the equipment room where the equipment is located; and / or

[0020] - management functions of the device, in particular the protection level of the device, and / or the configuration menu of the device; and / or.

[0021] The security information enables access to the device. Access to the device is thus protected, in particular blocked, by access protection, in particular access protection. The security information enables access protection, in particular access protection, to be released and thereby access to the device, in particular to a local device interface of the device, to be enabled.

[0022] The recording of the device identification of the device is in particular designed as a visual recording. The recording is carried out in particular by means of a camera. The user is in a position in which the device and in particular the device identification are in the field of view of the user's camera. The recording of the device identification is carried out by means of the method according to the invention. This can in particular be initiated by an action of the user or automatically as soon as the camera recognizes the device identification. In any case, the user must be in the vicinity of the device. The device identification must be visible from the user's position in order for the recording to be possible.

[0023] The device identification information can realize the unique identification of the device. The device identification is especially designed as a unique device appearance, device label, equipment label, logo or device ID. The device identification is especially a visual device identification. The device identification information is a file or data content, and the device identification is recorded in the form of the file or data content. The device identification information is especially a photo of the device identification or information obtained by image processing from the photo of the device identification.

[0024] The device identification is recorded, in particular, by a camera.

[0025] The security information is received by the receiving unit and sent by a password repository (password storage), which can also be referred to as a password storage unit or security code database. The security information is thus retrieved from the password repository. The security codes stored in the security code database are stored in particular in an encrypted manner.

[0026] In one embodiment, pure safety information is received and a display of safety information is created therefrom by the AR device (AR-enabled device). In an alternative embodiment, a display of safety information has been received and thus the safety information has been received in a processed form by the display.

[0027] A visual display is output, wherein the visual display includes at least a portion of an image of the device and a display of the security information. The image of the device particularly includes a real-time video playback of a field of view of the device. In addition to a portion of the image of the device or an image of the device as a whole, the security information is also output using the visual display. Thus, the security information is faded into the field of view of the user in relation to the image of the device, particularly next to the device and superimposed with the device.

[0028] The method according to the present invention provides the following advantages in particular: users who are unable to remember the corresponding security information, in particular service technicians and / or augmented reality (AR) users, do not need to manually carry the corresponding security information, in particular the security code, on a piece of paper, or ask other service technicians for the corresponding security information, for example by telephone.

[0029] This not only increases the security level, but also reduces the time required to unlock the necessary locks and access protection. The accesses made can also be recorded and thus traceable.

[0030] The invention exploits the possibilities, in particular those created in a production environment by means of industrial 5G system augmented reality, to simplify the entry of conventional security information, in particular device codes and / or PINs.

[0031] In this case, in particular, the visual aid is output in addition to the security information. The visual aid is therefore different from the security information itself. The visual aid being "suitable for using the security information" also includes that the visual aid makes it easier for the AR user to use the security information to unlock the lock, i.e., assists the AR user in this. Therefore, in particular, the visual aid is output additionally for unlocking the lock using the security information.

[0032] The output of the visual aid is therefore performed in addition to the output of the visual display, which includes a portion of the image of the device to be protected and the display of the security information.

[0033] The visual aid assists the user in correctly inputting the security information. Thus, the visual aid assists the user in unlocking the lock by correctly using the security information. The visual aid is particularly designed as an augmented reality output and is output via an AR-enabled device.

[0034] In one embodiment of the present invention, the security information is designed as follows:

[0035] - Facility code; and / or

[0036] - PIN code; and / or

[0037] - an unlock code; and / or

[0038] - password; and / or

[0039] - a numerical code; and / or

[0040] - Alphanumeric code.

[0041] As mentioned above, the security information enables access to the device by removing the access protection. The security information is especially designed as a series of characters that the user queries.

[0042] In a further development of the invention, the visual display is designed as an augmented reality display. According to the invention, the visual display comprises at least a part of an image of the device to be protected and a display of the security information.

[0043] As mentioned above, the image of the device includes in particular a real-time video playback of the field of view of the device. In addition to the image of a portion of the device or the image of the device as a whole, the security information is also output using a visual display. Thus, the security information is faded into the user's field of view in relation to the image of the device, in particular next to the device and superimposed with the device.

[0044] For the technical implementation of the output of the visual display, in particular, known augmented reality methods are used. The AR system fades the security information, in particular the corresponding code, into the field of vision of the AR user.

[0045] In another embodiment of the invention, the visual aid is designed as follows:

[0046] - enhancing the area of ​​the operating panel used to enter the security information; and / or

[0047] - enhancing the keys of the numeric keypad used to enter the security information; and / or

[0048] -Warning display.

[0049] In the sense of the present invention, augmentation may also be referred to as visual highlighting, in particular by means of augmented reality techniques.

[0050] In this embodiment, the corresponding operating panel is also highlighted, in particular after it has been recognized by the AR system, in particular the corresponding key of the numeric keypad is highlighted.

[0051] In another embodiment of the invention, the method according to the invention has the following steps:

[0052] -Record user actions on or associated with the device.

[0053] In this way, user operations performed by a user, in particular the input of security information, can be detected, in particular by detecting a key press by image evaluation.

[0054] It is also possible to monitor whether the AR user has actually made the correct input on the operation panel.

[0055] In a further embodiment of the invention, the visual aid is output in response to an evaluation of the user operation.

[0056] According to this embodiment, in particular, in response to the evaluation of a previously performed user input, a character is output as a visual aid. The output is performed in particular via a display and / or an AR system, via which the visual display is also output according to the invention. The character is in particular designed as an asterisk "*", which represents a previously performed user input.

[0057] The corresponding next character to be input or the next number to be input may also be highlighted by the visual aid in the form of a visually faded-in character string or number sequence, or only the next character or number to be input may be displayed.

[0058] In addition, if the AR user is monitored to see whether he has actually made a correct input on the operation panel, the visual aid can be used to indicate to the user that an incorrect input has occurred in the event of an incorrect input. This is particularly done through a warning display, in which the incorrect input is particularly presented in a red marking.

[0059] In another embodiment of the invention, the method according to the invention has the following steps:

[0060] - Before receiving the security message, send a verification message.

[0061] In particular, the verification information is sent after visually recording the device identification information.

[0062] The verification information confirms that the sender of the device identification information has the right to receive the security information. The sender of the security information checks the verification information before sending the security information.

[0063] In another embodiment of the present invention, the verification information is designed as follows:

[0064] -Device authentication information; and / or

[0065] - location information; and / or

[0066] - User authentication information; and / or

[0067] -Application information of the user's device.

[0068] Through the device authentication information, the device from which the device identification comes can be authenticated. The device authentication information is particularly designed as equipment authentication information. In particular, the device assigned to the device authentication information or the assigned input field is identified. This is especially done by object recognition or based on a barcode attached to the device, especially a two-dimensional barcode (2D-Barcode) or a QR code. Thus, it can be checked whether the AR user is actually located right in front of the corresponding device, and whether the input field is currently available, especially whether it is not blocked by a shield, or whether the device is currently waiting for the input of a security code corresponding to the device display.

[0069] The position information makes it possible to detect whether a user or an AR pair of glasses or an AR device used by the user is located in the vicinity of the device or input field. This can also be done by image evaluation. In addition, however, the position information can also be collected and checked by a positioning system, in particular RFID, NFC, WLAN, UWB, 5G mobile communication.

[0070] The wearer of the AR glasses (an embodiment of the AR device) can be identified and / or authenticated by means of the user authentication information. This is done in particular by biometric authentication, in particular by means of a camera, which for AR glasses recognizes iris patterns and / or fundus, or by means of an acoustic fingerprint (ultrasound) of the wearer or the head, whereby an acoustic fingerprint of the skull is captured and checked.

[0071] Information about applications, especially correct or incorrect applications of user devices, is provided through application information of user devices. User devices, which may also be referred to as User-Device, are especially mobile devices, especially smart devices or tablet computers with AR applications, or AR glasses. Through the application information, it is possible to check whether the AR glasses are worn correctly. To this end, the orientation of the AR glasses can be detected in particular by an orientation sensor or a touch sensor, in particular on the bridge of the nose and / or temples of the AR glasses. It can also be checked whether the eyes are actually at the expected distance or expected position relative to the AR glasses.

[0072] The security information stored in the security code database is stored in an encrypted manner. In a variant, the cryptographic key for decrypting the encrypted security information can be activated only when the authentication information has been checked and in particular the AR user is currently allowed to use the security information.

[0073] The invention also includes an augmented reality device, which is designed to perform a method according to any of the above claims. The device is a user device, which is designed to output an augmented reality display to a user. The augmented reality display extends the display of the reality located in the field of vision of the user by an extension, i.e., the augmentation. The part of the image of the device to be protected in the sense of the invention corresponds to the reality. The display of security information in the sense of the invention is displayed by the augmentation.

[0074] In one embodiment of the present invention, the augmented reality device is designed as follows:

[0075] - AR glasses; and / or

[0076] - Portable mobile devices, especially smartphones, smart watches or tablets.

[0077] The mobile device has in particular an AR application (AR-App) and in particular also has a user authentication function by fingerprint recognition.

[0078] The invention also includes a method for sending security information, wherein the security information is designed to unlock a lock, wherein the lock is designed to protect access to a device to be protected,

[0079] The method has the following steps:

[0080] - receiving device identification information, wherein the device identification information is designed to be a visual record of the device identification of the device to be protected;

[0081] - Check the device identification information;

[0082] - sending the security information according to the result of checking the device identification information, wherein the security information is assigned to the device identification information; and

[0083] - sending a further data set which comprises an additional visual aid for using the security information to access the device to be protected, wherein the visual aid is designed to be output in addition to the security information.

[0084] The present invention describes a method for sending safety information from the perspective of an AR background system. The present invention describes the above method for outputting a display of safety information from the perspective of a corresponding user device.

[0085] In one embodiment of the invention, the method for transmitting safety information comprises at least one of the following steps:

[0086] - sending a data set comprising a visual display, wherein the visual display comprises at least a portion of an image of the device to be protected and a display of the security information; and / or

[0087] - evaluating user actions on or in relation to the device; and / or

[0088] - receiving authentication information before sending the secure information; and / or

[0089] - checking the verification information, wherein the security information is also sent based on the result of checking the verification information.

[0090] The subsequent steps of the method for transmitting safety information correspond to the subsequent steps of the embodiment of the method for outputting a display of safety information, and the description details apply correspondingly.

[0091] The present invention also includes an AR background system, which is designed to execute the method for sending safety information according to the present invention.

[0092] In particular, data transmission between the AR device (in particular, AR glasses) and the AR background system is carried out via the 5G mobile communication system. However, WLAN communication is also used in particular. The data transmission between the AR device and the AR background system is carried out in particular in a cryptographically protected manner, in particular in an encrypted manner, in an integrity-protected manner and / or in an authenticated manner.

[0093] The invention also comprises a computer program product comprising a computer program, wherein the computer program can be loaded into a storage device of a computing unit, wherein the steps of the method according to the invention are performed by means of the computer program when the computer program is executed on the computing unit. BRIEF DESCRIPTION OF THE DRAWINGS

[0094] BRIEF DESCRIPTION OF THE DRAWINGS Characteristics and advantages of the invention emerge from the following description of a number of exemplary embodiments with reference to the accompanying schematic drawings.

[0095] in:

[0096] Figure 1 A flow chart showing a method according to the present invention; and

[0097] Figure 2 A schematic diagram of the apparatus to be protected and an augmented reality device is shown. DETAILED DESCRIPTION

[0098] Figure 1 The display for outputting the safety information 211 according to the present invention is shown (in Figure 2 ), wherein the security information 211 is designed to unlock a lock, wherein the lock is designed to unlock the device 1 to be protected (in Figure 2 Access protection presented in

[0099] The method has the following steps:

[0100] - Step S1: The device 1 to be protected ( Figure 2 Device identification 12 (presented in Figure 2 ) to visually record the device identification information;

[0101] -Step S2: sending the device identification information;

[0102] - Step S21: before receiving S3 the security information, sending verification information;

[0103] - Step S3: receiving security information 211, wherein the security information 211 is assigned to the device identification information 12;

[0104] - Step S4: Output visual display 21 (in Figure 2 ), wherein the visual display 21 includes an image 212 of the device 1 to be protected (in Figure 2 ) and display of at least a portion of the security information 211;

[0105] - Step S5: Output visual aid 213 (in Figure 2 ), for using the security information 211 to access the device to be protected (1); and

[0106] - Step S6: Recording user operations on or related to the device 1 to be protected.

[0107] Figure 2 Embodiments of the present invention are explained.

[0108] exist Figure 2 In, the security information 211 may be specified as a device code, a PIN code, an unlock code and / or a numeric code.

[0109] The visual display 21 is designed as an augmented reality display 21 which augments the display device 2 .

[0110] exist Figure 2 In , the visual aid (213) is designed to:

[0111] - enhancing the area of ​​the operating panel (13) for inputting the security information (211); and

[0112] -Enhance the keys of the numeric keypad used to enter the security information (211).

[0113] The visual aid 213 is output in response to the evaluation of the user operation.

[0114] For the industrial device 1, the user must enter a code on the keyboard 13, in particular to unlock service functions. For this purpose, the access code 211 (here: 317298 for the left eye) is faded in to the service technician as an AR user in his AR glasses 2.

[0115] exist Figure 2 In, the security information 211 may be specified as a device code, a PIN code, an unlock code and / or a numeric code.

[0116] exist Figure 2 Visual Aids 213 is designed to:

[0117] - enhancing the area of ​​the operating panel 13 for entering the security information 211; and

[0118] -Enhance the keys of the numeric keypad 13 used to enter the security information 211.

[0119] As a visual aid 213, the key corresponding to the next number to be entered (here the key "9") is visually marked in the AR glasses 2. For this purpose, it can be detected by the camera of the AR glasses 2 that four numbers have been entered (in particular, according to the display of four asterisks "****" in the device display 11). In addition, the device 1 can be identified, in particular by detecting the device appearance or the device nameplate 12.

[0120] The present invention can be fully implemented in the AR glasses 2 or other AR devices. Preferably, the AR glasses 2 or the other AR device is connected to an AR background system, in particular via WLAN or a 5G mobile communication network. This enables, in particular through object recognition, image detection and image evaluation in a background system, in particular in a backend or edge computing system or in a 5G Multi Access Edge Computing Infrastruktur (5G Multi Access Edge Computing Infrastruktur) of a mobile communication system. Similarly, additional information to be faded in by the AR glasses 2, in particular information for equipment maintenance, can be determined there and transmitted to the AR glasses 2, in particular by evaluating the image detected by the AR glasses 2 and by querying a code stored in a security code database in a manner assigned to a device located in the field of view.

[0121] In particular, data transmission between the AR glasses 2 and the AR background system can be carried out via the 5G mobile communication system. However, in particular, WLAN communication can also be used. The data transmission is carried out in particular in a cryptographically protected manner, in particular in an encrypted manner, in an integrity-protected manner and / or in an authenticated manner.

[0122] Although the present invention has been illustrated and described in more detail by way of embodiments, the present invention is not limited to the disclosed examples and other variations may be derived therefrom by those skilled in the art without departing from the scope of protection of the present invention.

Claims

1. A method for outputting a display of safety information (211), wherein: The security information (211) is designed to unlock a lock, wherein the lock is designed to protect access to the device (1) to be protected, The method comprises the following steps: - visually recording (S1) a device identification (12) of a device (1) to be protected, thereby creating device identification information; - sending (S2) the device identification information; - receiving (S3) the security information (211), wherein the security information (211) is assigned to the device identification information (12); - outputting (S4) a visual display (21), wherein the visual display (21) comprises at least a portion of an image (212) of the device (1) to be protected and a display of the security information (211); and - additionally outputting (S5) a visual aid (213) for unlocking the lock using the security information (211).

2. The method according to claim 1, in, The security information (211) is designed as follows: - Facility code; and / or - PIN code; and / or - an unlock code; and / or - password; and / or - a numerical code; and / or - Alphanumeric code.

3. The method according to any one of the preceding claims, in, The visual display (21) is designed as an augmented reality display.

4. The method according to any one of the preceding claims, in, The visual aid (213) is designed to: - enhancing the area of ​​the operating panel (13) for inputting said security information (211); and / or - enhancing the keys of the numeric keypad used to input said security information (211); and / or - a warning display indicating that an erroneous input of the security information (211) has been made.

5. The method according to any one of the preceding claims, The method has the following steps: - Recording (S6) user operations on or in relation to the device (1) to be protected.

6. The method according to claim 4 and claim 5, in, In response to the evaluation of the user operation, the visual aid is output (213).

7. The method according to any one of the preceding claims, The method has the following steps (S21): - Before receiving (S3) said security information, sending authentication information (211).

8. The method according to claim 7, in, The verification information is designed to: -Device authentication information; and / or - location information; and / or - User authentication information; and / or -Application information of the user's device.

9. An augmented reality device (2) designed to carry out the method according to any of the preceding claims.

10. The augmented reality device (2) according to claim 9, The augmented reality device is designed to: - AR glasses; and / or - Portable mobile devices, especially smartphones, smart watches or tablets.

11. A method for sending security information (211), wherein: The security information (211) is designed to unlock a lock, wherein the lock is designed to protect access to the device (1) to be protected, The method comprises the following steps: - receiving device identification information, wherein the device identification information is designed as a visual record of the device identification (12) of the device (1) to be protected; - checking the device identification information; - sending the security information (211) according to the result of checking the device identification information, wherein the security information (211) is assigned to the device identification information; and - sending a further data set comprising an additional visual aid (213) for accessing the device (1) to be protected using the security information (211), wherein the visual aid (213) is designed to be output in addition to the security information (211).

12. The method according to claim 11, The method has the following steps: - sending a data set, said data set comprising a visual display (21), wherein The visual display (21) comprises at least a portion of an image (212) of the device (1) to be protected and a display of the security information (211); and / or - evaluating user actions on or in relation to the device (1) to be protected; and / or - receiving authentication information before sending said security information (211); and / or - checking the verification information, wherein the security information is also sent according to the result of checking the verification information (211).

13. An AR background system, designed to execute the method according to any one of claims 11 to 12.

Images