Display device and display method

Abstract

The present invention relates to a display device comprising: a display device configured to display one or more buttons; a transparent and touch-sensitive layer disposed on the display device and configured to detect operation and position of operation of the display device; at least one detection device configured to detect buttons on the display device and operation and position of operation of the display device; and a computer device configured to test whether buttons on the display device detected by at least one detection device correctly match buttons to be displayed by the display device, and / or whether operations and positions detected by the layer and at least one detection device match each other and the displayed buttons. Furthermore, a display method and a computer program product are provided.

Description

Technical Field

[0001] The present invention relates to a display device and a method for displaying for the display device.

Background Art

[0002] In the prior art, display devices have already been realized in various ways by touchscreens. In such touchscreens, a touch-sensitive layer detects contact with an image screen by a finger or other body part or a separate input device.

[0003] During the operation of machines and equipment, in various usage cases, human safety is desired or even specified in order to avoid human danger, that is, to eliminate injuries. In order to ensure the safe operation of machines and equipment, unintentional contact must be prevented from triggering undesired actions or operations. For safety reasons, it is also inevitable to execute the actual intention of the operator without error. For this purpose, it must be ensured that the operator knows which functions are associated with the operation of the buttons. If the displayed control element has a different function depending on the situation, or if the displayed control element does not match the function executed thereby due to a defective display, there is a risk of danger.

[0004] Particularly, touch-user interfaces are suitable for variable displays, that is, different working elements with different functions depending on the operating state may be shown at the same location of the input area. Therefore, it is necessary for safety to recognize functions accompanied by errors in the hardware and software components required for presentation, or to plausibilize these functions. The errors that occur must be technically detectable before danger occurs, or recognizable to the operator. In some usage purposes, it is also desirable that a predetermined action can be triggered only by a qualified person.

[0005] A safe method for operating an industrial robot is described in European Patent No. 2 834 050. In this case, a graphic user interface with a touch display shows virtual work elements representing the functions or operating modes of the industrial robot on the touch display. After a work element is selected, additional work elements are displayed on the touch display in constantly changing positions, and only after such additional work elements have been operated is the function or operating mode activated. Thus, the function is executed only after additional operations of separately displayed work elements have been performed in predetermined, but constantly changing positions.

[0006] The drawback in this case is that a two-step method is used, namely, a method of confirmation by first operating the intended functional element and then operating a second button that is specifically displayed and constantly changing in its position. In the case of conventional touchscreen systems, in the context of industrial machinery operation, for example, if accidental contact may be indistinguishable from intentional contact, the lack of safety cannot be tolerated, and the operator cannot recognize without error what the intended work action will result in. This is particularly problematic when the functional element changes over time or depending on the work state. In addition, it must be possible to test that the information displayed for the function is also displayed correctly. Even though the evaluation of the input sensor superimposed on the display for the display position has already triggered a newly assigned control command, a faulty display controller may cause the display to freeze and signal a predetermined function previously assigned to the operating element.

[0007] U.S. Patent Application Publication No. 2012 / 019485 describes an information processing device that can be influenced by operator gestures. The technical solution proposed in such a document allows multiple operators to perform gesture operations in three-dimensional space, and each operator's three-dimensional gesture is precisely assigned to a target object of that three-dimensional gesture. For this purpose, a projector projects at least one selected target object onto the top surface of a table. When a pointer operated by an operator touches the top surface of the table, a device for recognizing two-dimensional coordinates recognizes the contact location and determines which object was designated by the operator. At this stage, the device for recognizing three-dimensional coordinates detects the pointer closest to the contact location, and the closest pointer is identified as the tracking target. Subsequently, the tracking target pointer is tracked by the device for recognizing three-dimensional coordinates to determine the operator's gesture pattern.

[0008] An interaction system described in U.S. Patent Application Publication No. 2016 / 314727 allows a user to interact with a data processing device. In this system, a depth camera is used, and the distance detected along an axis from the depth camera to the desktop edge is recognized by increasing distance, which is then used as the origin of a coordinate system, defining a projection area for representing the user interface. By monitoring end-user input to the desktop edge and the projected output, user interaction via the projected user interface can be harmonized when the camera or projector moves relative to the desktop surface.

[0009] Further display and interaction devices for the technical system are known pursuant to U.S. Patent Application Publication No. 2018 / 144673 A1, U.S. Patent Application Publication No. 2015 / 091811 A1, U.S. Patent No. 10,809,882 B1, Chinese Patent No. CN114 780 321 A, or U.S. Patent Application Publication No. 2016 / 063230 A1. [Overview of the project] [Problems that the invention aims to solve]

[0010] The objective of this invention is to overcome the shortcomings of the prior art and, in particular, to improve safety during operation using touchscreens on machinery and equipment. [Means for solving the problem]

[0011] Display devices, display methods, uses of display devices, and computer program products are provided. The object of the present invention is solved by the subject matter of the independent claims. Further advantageous configurations are found in the dependent claims. [Effects of the Invention]

[0012] Embodiments of the present invention are shown below. In principle, these embodiments can also represent devices or other forms of implementation (methods, uses, computer program products, etc.).

[0013] In a first embodiment, according to the first embodiment, a display device is provided comprising a display device, a transparent and touch-sensitive layer, at least one detection device, and a computer device.

[0014] The display device is configured to display one or more buttons. A transparent, touch-sensitive layer placed on the display device is configured to detect the operation and position of the operation of the display device. At least one detection device is configured to detect the buttons of the display device and the operation and position of the operation of the display device. The computer device determines whether the buttons of the display device detected by at least one detection device match the buttons that should be displayed by the display device. and The system is configured to test whether the operations and positions detected by the layers and at least one detection device match each other and the displayed buttons.

[0015] A further aspect of the present invention provides a method of displaying. The method of displaying on a display device includes displaying one or more buttons by a display device. Then, an operation and position on the display device is detected by a transparent and touch-sensitive layer disposed on the display device, and the buttons of the display device and the operation and position on the display device are detected by at least one detection device. Finally, it is determined whether the buttons of the display device detected by at least one detection device match the buttons to be displayed by the display device. and A computer device tests whether the operations detected by the touch-sensitive layer and at least one detection device match those detected by the buttons.

[0016] In this case, a dual justification, or testing, can be performed. On the one hand, it is tested whether the detected button roughly matches the button that should be displayed, thereby ensuring that the operator is accurately informed about the function currently associated with the operation. On the other hand, it is tested whether the operation and location detected by the two channels (touch-sensitive layer and detection device) match, thereby avoiding unintended activation due to, for example, electrostatic discharge, contamination, or condensation on the surface. Specifically, this is because the layer and the detection device utilize different techniques for detecting operations (this is characterized by diversity), and as a result, the individual effects of errors do not affect both systems simultaneously and in the same form.

[0017] This allows the display device or its computer system to determine whether the display is correct or not, and if an error is detected, it can prevent the activation of incorrect functions and avoid false activation.

[0018] The embodiments described below may relate to each of the aforementioned aspects of the present invention.

[0019] In a further embodiment, the computer device may transmit information or control signals depending on the test results.

[0020] This enables a safe process to occur within the machine or device to be operated. Specifically, a potentially dangerous function can be activated only when the presentation of the button corresponds to the function actually assigned in a control-technical manner, and the operator can be informed of the presence of a device error in the display device and the display device associated with danger can be stopped from operating. The control signal may be a control instruction or a control command.

[0021] In a further embodiment, the display device may be formed as a film, specifically as a liquid crystal display or a light emitting diode film. The transparent and touch-sensitive layer may also be formed as a film, specifically as a touch screen film. The detection device may be, for example, a camera or a light irradiation field film.

[0022] The advantages of such individual configurations are detailed below.

[0023] In a further embodiment, one of at least one detection device is a fingerprint sensor, and the fingerprint sensor is disposed below the display device, between the touch-sensitive layer and the display device, or above the touch-sensitive layer. The fingerprint sensor is configured to detect fingerprints.

[0024] By detecting fingerprints, for example, operation by hand can be ensured, and as a result, activation by the elbow or other body parts can be avoided.

[0025] In a further embodiment, the computer device is further configured to test whether the fingerprint detected by the fingerprint sensor matches the fingerprint stored in the data storage device, thereby enabling the qualification of the operator to be tested. By comparing the fingerprint with the stored data, biometrics can be used to enforce access restrictions for control or display purposes. The access restrictions can here relate to control as a whole, i.e. activation of the control console, or to the individual operation of functional elements. The test is advantageously carried out for each individual work act, so that for example two operators can very well alternately perform work acts via the display device in a timely manner, but only one of the operators can potentially activate a dangerous function, which is blocked for the other operator.

[0026] In a further embodiment, at least one detection device and a touch-sensitive layer are further configured to detect gestures during operation, and the computer device is further configured to test whether the gestures detected by the touch-sensitive layer and the at least one detection device match each other and match the gestures stored in the data storage device.

[0027] In this case, it is advantageous that various functions or actions can be activated to expand workability and operate the buttons in more than one form.

[0028] In a further embodiment, the display device includes two or three or more display parts, each display part being configured to display one part by each button. In a first embodiment, the display parts are arranged one above the other, and at least one display part is translucent. The buttons are caused by the overlap of the partial displays. The partial displays can be shifted spatially or temporally. In a second embodiment, the display parts are arranged side by side, and each button extends over each display part.

[0029] This allows the operator to recognize without a doubt that, through a display via two or more sub-displays, if the sub-displays do not correspond to each other, one of the sub-displays is malfunctioning, i.e., at least one of the display parts is defective, and therefore the use of the terminal is no longer permitted due to potential danger. The safety of the equipment can only be guaranteed, for example, if the display is guaranteed to be correct. Only in such a case can the user be confident in what the operation of the button will result in. By dividing a single button into two or more superimposed display parts, at the time of the appropriate selection of the display, if the displays do not correspond to each other, the user can be made aware that at least one of the displays is defective, and therefore the use of the terminal is no longer permitted due to potential danger. Technically, this is signaling implemented in a dual-circuit manner until it reaches the user's eye.

[0030] Further embodiments include the following characteristics, namely Changing the display location after a predetermined time. Having continuous motion and visible flashing via the aforementioned display device, The flashing is not visible, but it is detectable by the detection device. Having dynamic changes in brightness and / or color, and / or having a pattern that makes up dynamic changes in brightness and / or color, A button or the surrounding surface of a button having one or more of the following features is displayed.

[0031] Such characteristics of the display allow the operator to recognize that the display is operating outside of normal working parameters and that there is a potential error, making the work uncertain and ultimately unacceptable. However, these characteristics can also provide additional information and enable or facilitate additional functionality.

[0032] In a further embodiment, the computer unit is further configured to perform the aforementioned tests for two or more buttons.

[0033] This allows the display device to provide the user with multiple buttons simultaneously and without prior selection for various potentially hazardous functions, for example, for two directions of motion of a machine axis, or even for multiple machine axes. To enhance safety, simultaneous testing for multiple buttons is also advantageous for two-handed operations where the work is specified to be performed by one worker using both hands.

[0034] In a further embodiment, the computer device is further configured to consider a button operation detected by at least one detection device only after a predetermined time span has elapsed between the start of the button display and the button operation.

[0035] This prevents unintended operations, such as the display changing just before the display device is touched, followed by the operation of a button that has just appeared. This applies to both the dynamic possibility of the display changing as described above, and the possibility of a new button appearing in an area that was previously empty.

[0036] In a further embodiment, the display device shows the button in a way that differs from how it appears after a predetermined time span has elapsed, during a predetermined time span.

[0037] The advantage of this is that the operator can see when the button will become available for use. This helps to avoid invalid operations or surprisingly delayed function execution.

[0038] A second aspect of the present invention relates to the use of the aforementioned display device as a fail-safe indicator means in a control system for a potentially hazardous technical system, such as industrial equipment, machinery, or automobiles.

[0039] A further aspect of the present invention describes a computer program product which, when the program product is executed on the control device, can be loaded directly into the storage device of a programmable control device of a display device, together with program code means, in order to perform all the steps of the method described herein.

[0040] These embodiments illustrate possible modifications. The present invention is not limited to the modifications specifically shown, but rather various combinations of individual modifications are also possible. Specifically, the features of the apparatus can also be realized in the method, and vice versa.

[0041] For a better understanding of the present invention, it will be described in detail with reference to the following drawings. The drawings are significantly simplified schematic diagrams. [Brief explanation of the drawing]

[0042] [Figure 1] Figure 1 shows a display device based on an example embodiment. [Figure 2] Figure 2 shows an example including a partial view. [Figure 3] Figure 3 is a schematic flowchart based on an example embodiment. [Modes for carrying out the invention]

[0043] First, it should be noted that in embodiments described as various different things, the same parts will be given the same reference numeral or component numeral. The disclosures included in the entire description can be adapted to the same parts having the same reference numeral or component numeral. Furthermore, descriptions of selected positions in the description, such as top, bottom, and side, relate to the drawings that are directly described and illustrated, and when the position changes, these descriptions of positions can be adapted to the new position accordingly.

[0044] Figure 1 schematically shows an example of the structure of a display device 1. Such a display device includes a display device 2, a transparent and touch-sensitive layer 4, a detection device 5, and a computer device 6. The display device 2 shows, for example, three buttons 3 in Figure 1, but more or fewer buttons 3 may be shown. The buttons 3 are visible from above and are therefore also shown on the transparent and touch-sensitive layer 4 and the detection device 5.

[0045] Note that Figure 1 shows only one embodiment schematically.

[0046] The display device 2 displays at least one or more buttons 3.

[0047] A transparent, touch-sensitive layer 4 placed on the display device 2 detects the operation of the display device 1 and its position.

[0048] At least one detection device 5 detects the button 3 on the display device 1, as well as the operation and position of the operation of the display device 1. The detection device 5 here at least detects the presentation features of the button 3. The detection device 5 may also cover only a portion of the display device 1, for example, if it is an optical fingerprint sensor 7, or if it is a detection device that detects only one button 3. Position detection can be achieved by the known and appropriate use of the position of the detection device 5.

[0049] The detection device 6 tests whether the button 3 of the display device 2, as detected by at least one detection device 5, matches the button 3 that should be displayed by the display device 2. Alternatively, or in addition to this, the detection device 6 tests whether the operation and / or position detected by the touch-sensitive layer 4 and at least one detection device 5 matches each other and the displayed button 3.

[0050] The display device 2 may be formed, for example, in the form of a film. This may be a liquid crystal display or a light-emitting diode film. However, the display device 2 may also be other forms of display, such as a monitor, OLED or LCD display, or similar. The display device 2 displays buttons 3. Since each display as a button 3 is technically associated with an action or function, for example, when controlling equipment with the display device 1, operating a button 3 will initiate an action for the equipment. As will be further explained later, multiple functions or actions of a single button 3 may be assigned alternately. These may be context-dependent on the one hand, meaning that various actions can be activated by the button 3 depending on the operating and working conditions, and on the other hand, multiple linked actions can be assigned simultaneously and then activated at the time of operation.

[0051] The display device 2 can be configured to display at least button 3, i.e., in addition to button 3, i.e., the display interaction element, further elements may be displayed. These may include photo signals or video signals. These signals may indicate, for example, the operation of the equipment to be controlled. For example, measurements of the equipment to be controlled or ambient conditions, such as temperature and air humidity, may be displayed. Further, less reliable buttons may be provided. These buttons are used for non-critical functions, such as switching display settings, changing parameters, or similar. They may also display information related to communications, such as incoming phone calls, or text messages directed to the operator, such as emails or other text reports.

[0052] A transparent and touch-sensitive layer 4 may also be formed in the form of a film. The film is placed on the display device 2 and therefore must be transparent. This layer may be formed, for example, as a capacitive or resistive touchscreen film.

[0053] A transparent, touch-sensitive layer 4 placed on the display device 2 detects the operation of the display device 1 and its position. In other words, layer 4 can determine that the display device 1 is being operated, and also determine at what position on the display device 1 such operation is being performed.

[0054] In another configuration, the display device 2 and layer 4 are configured as an integrated component. In this configuration, this is a film that can, for example, indicate buttons and simultaneously detect operation and position. This is commonly known as a touchscreen and is widely used.

[0055] The detection device 5 may be formed in various forms. One example is a light-illuminating field film which may be placed above or below the transparent and touch-sensitive layer 4. In Figure 1, the detection device is placed above the transparent and touch-sensitive layer 4 and therefore must also be transparent or see-through. However, even when the detection device is placed below the transparent and touch-sensitive layer 4 and above the display device 2, the detection device must also be transparent or see-through. The detection device 5 may take the form of a film or a fingerprint camera and be placed below the display device 2, in which case it may not be necessary for the detection device 5 to be transparent or see-through.

[0056] The detection device 5 may be configured as another optical system, namely a camera, which can photograph the display device 2 and layer 4. The camera can photograph the display device 2 and layer 4 from various positions, for example, from below or from above. The user can also be additionally detected at this position.

[0057] The detection device 5 detects the button 3 on the display device 1, the operation of the display device 1, and the position of the operation. In other words, similar to layer 4, the detection device 5 can determine that the display device 1 is being operated, and at what position on the display device 1 such operation is performed. However, the detection device 5 can also detect the button 3 displayed by the display device 2, and its position.

[0058] In the display device 1, one or more of the detection devices 5 are formed as fingerprint sensors 7. The fingerprint sensors may be located directly below the display device 2, between layer 4 and the display device 2, or above layer 4. The fingerprint sensors 7 are configured to detect fingerprints.

[0059] Figure 1 shows an exemplary fingerprint sensor 7. The positions shown in Figure 1 indicate the arrangement on the detection device.

[0060] For example, two fingerprint sensors 7, positioned sufficiently far apart from each other, may be used for security-related tasks requiring two hands. For instance, a function requiring two hands may be stored as an action in one of the two fingerprint sensors 7, but this function can only be activated when the second fingerprint sensor 7 has also successfully read the fingerprint.

[0061] To implement restrictions for certain functions, the fingerprint read by the fingerprint sensor 7 can also be compared with a fingerprint stored in the data storage device 8. This ensures that, for example, only qualified personnel can shut down or disconnect the machine, or restart it after an error. Thus, for safety-critical work activities, it can be ensured that only a restricted circle of workers, such as particularly reliable, particularly qualified, or particularly tested individuals, can perform the prescribed actions. Their fingerprints are technically registered in advance. As for other workers, even if they are in the vicinity of the work terminal at the same time as qualified personnel, such operations are prevented.

[0062] Multiple detection devices 5 can also be used in the display device 1. These detection devices can be of various types, such as a light field film, three cameras, and two fingerprint sensors 7.

[0063] Such a fingerprint sensor 7 may be configured, for example, as a low-resolution camera integrated into the display. A similar sensor is also used in mobile phones.

[0064] One or more of the detection devices 5 and / or layers 4 can also detect gestures during operation. The computer device 6 can, on the one hand, test whether the detected gestures match each other, thereby enhancing security, and on the other hand, the computer device 6 can test whether the detected gestures match gestures stored in the data storage device 8. In the latter case, a function associated with a gesture that may differ from the normal function of button 3 can be activated. Button 3 may not normally activate a function, for example, to protect against unintended operation. In contrast, an operation involving a circular or pulling gesture activates button 3, for example, resulting in the unblocking of the remaining buttons 3. Alternatively, other functions may be assigned to simple operations and gesture operations, for example, to produce movement along the mechanical axis in one direction or opposite directions depending on the direction of finger movement. The advantage in this case is that multiple functions can be provided by just one detection device 5, such as a fingerprint sensor 7, and by just one button 3, for example, by a swipe motion (Streichbewegungen).

[0065] Computer device 6 may have the authority to transmit the buttons 3 to be displayed, and other information, to the display device 2. However, this may also be performed by other computer devices. Computer device 6 may transmit information or control signals depending on the test results. This may directly or indirectly affect the display on the display device 2. The transmitted information or control signals may, for example, notify the equipment to be controlled by the display device 1 of the operated function and action, thereby enabling control of the equipment. The control signals may be control instructions or control commands.

[0066] Figure 2 shows that the display device 2 may consist of two or more display sections 2a, 2b. Figure 2 shows an embodiment in which the display device 2 consists of two display sections 2a, 2b. Each of the display sections 2a, 2b displays a portion of button 3. In Figure 2, the display sections 2a, 2b are shown positioned one above the other. In this case, at least one display section 2a, or more specifically, at least one display section, i.e., all display sections except the bottommost display section, is semi-transparent, see-through, or transparent. This is suggested in Figure 2 by the hatching on the plane shown above. Button 3 is created by the overlapping of partial displays on the display sections 2a, 2b. If one of the display sections 2a, 2b fails, the operator can recognize this without suspicion and avoid potentially dangerous use.

[0067] The aforementioned altered presentation represents a presentation where a portion of the information has been shifted in location.

[0068] Partial displays can also be performed with a time stagger, so that the displayed parts alternate with each other. If one of the displayed parts malfunctions, an interruption in the display occurs. This interruption indicates a potentially dangerous situation, allowing the operator to avoid the operation.

[0069] Instead of the aforementioned modification in which the display sections are arranged vertically, a modification is also possible in which the display sections 2a and 2b are arranged side by side, and each button 3 extends over each of the display sections 2a and 2b. This can be achieved, for example, by executing an image dot matrix (Bildpunktraster) using various control units.

[0070] The display of button 3 can be further improved in various formats.

[0071] The surrounding surface (Umgebungsflaechen) is the surface displayed within the immediate periphery of the button. This can display content that can be assigned to button 3 or the function or action of the button, or it can provide suggestions for alternative functions or actions. Such functions or actions can be performed via button 3, for example, by gesture or alternative forms of operation, such as pressing it twice or multiple times, pressing and holding, pressing more firmly, or pressing with an input device, such as an input pen or glove.

[0072] One or more buttons 3 can change their display location after a predetermined period of time. This can, on the one hand, avoid damage to the display device 2, such as increased wear at that location, and on the other hand, counteract the skilled behavioral patterns of the operator, because the buttons 3 can be positioned in new locations, and therefore the operator must actively search for and consciously operate the currently desired button 3.

[0073] One or more buttons 3 can have continuous motion via the display device 2. This allows the operator to send sufficient attention to the display device so that they cannot miss any information or warnings. But most importantly, it allows the operator to recognize screen presentations that have been frozen due to technical defects, and thus invalid.

[0074] One or more buttons 3 or their surrounding surfaces can flash visibly. Flashing indicates that the display is functioning without errors. In this case, the absence of flashing means that there is an error in the display unit. Alternatively, additional information can be transmitted through flashing. This can occur through the fact that the buttons are flashing, but also through a predetermined rhythm or pattern, a predetermined frequency or speed. Such patterns and speeds can also be changed. Patterns can suggest states such as "ready," "working," or "error," and speeds can suggest, for example, the remaining time that a function is still available; the faster the button 3 flashes, the shorter the remaining time that is available.

[0075] The flashing may be invisible but detectable by the detection device 5. Examples include extremely high-frequency flashing, or long display phases combined with short non-display phases. Such flashing may be unnoticed by the operator but detectable by a suitably designed camera. Actively controlled flashing indicates to the detection device 5 that the display control is still active and that there are no system errors, i.e., no malfunctions in the display control.

[0076] Button 3 or its surrounding surface may have dynamic brightness changes and / or color changes, each with or without a pattern. These may be displayed for the operator, or, as described above, may be displayed in such a way that only a correspondingly formed detection device 5 can detect them, but the operator cannot.

[0077] By changing brightness or color, error-free functions or additional information can also be presented as described above.

[0078] The computer device 6 can also perform the aforementioned tests for two or more buttons 3 simultaneously. This is particularly beneficial when the display device is used for multiple work fields and these work fields are operated by multiple workers. For added safety, simultaneous testing for multiple buttons 3 is also advantageous for two-handed operations where the work is specified to be performed by one worker using both hands.

[0079] In order to test whether the operations detected by layer 4 and detection device 5 match, the computer unit 6 can only consider the detected operation of button 3 after a predetermined time span has elapsed between the start of the display of button 3 and the operation of the button.

[0080] This prevents unintended operations from occurring, as the display changes immediately before contact with display device 1, and then the button 3 that has just appeared is operated. This applies to both the dynamic possibility of the display changing as described above, and the possibility that the new button 3 appears in an area that was previously empty.

[0081] During such a time span, that is, as long as button 3 has not yet been evaluated as active, the display device 2 can indicate button 3 in a different way after the time span has elapsed, that is, when button 3 is active. This can be achieved through the aforementioned possibilities, such as flashing, color, brightness, etc.

[0082] The advantage of this is that the operator can see when button 3 becomes available for use and simultaneously recognize its assigned function in advance. This helps to avoid invalid operations.

[0083] Figure 3 shows a schematic flowchart. Method 100 will be described based on this flowchart. Any features are indicated by dashed lines. In principle, the features of the apparatus can also be applied to the method, and therefore any further detailed explanations in relation to the method will be omitted.

[0084] First, in step 110, at least one or more buttons 3 are displayed by the display device 2. Next, in step 120, the operation and position of the display device 1 are detected by a transparent and touch-sensitive layer 4 placed on the display device 2.

[0085] At the same time, but before or after step 120, in step 130, at least one detection device 5 detects the button 3 of the display device 2, the operation of the display device 1, and the position of the operation.

[0086] When a button is detected in step 130, the presentation of button 3, or a predetermined characteristic of the presentation, such as frequency, can also be detected. In step 140, the computer device 6 tests whether the button 3 of the display device 2, detected by at least one detection device 5, matches the button 3 that should be displayed by the display device 2.

[0087] This determines whether the display device 2 is functioning correctly. Furthermore, in step 140, the computer device 6 tests whether the operations detected by layer 4 and at least one detection device 5 match each other and the button 3. This justifies the operation through two channels, namely layer 4 and detection device 5. When the operation data from layer 4 and detection device 5 match spatially and temporally, the operation of button 3 is determined more reliably than if it were through only one channel.

[0088] Optionally, in step 150, the computer device 6 may transmit information or control signals in accordance with the results of test 140. This information or control signals may, for example, be useful for controlling the equipment, but may also include the results of test 140 and, consequently, trigger an error report.

[0089] One of the detection devices may be, or may include, an optical fingerprint sensor 7. In this case, step 130 may include a sub-step 131, in which a fingerprint can be detected. Alternatively, if there are multiple detection devices 5, for example, one of the detection devices 5, which is a fingerprint sensor 7, may perform fingerprint detection as the sole component of step 130.

[0090] If a fingerprint is detected in step 130, step 140 of the computer device 6 may include testing 141 whether the detected fingerprint matches a fingerprint stored in the data storage device 8. Thus, for safety-critical work activities, it can be ensured that only a restricted circle of workers, for example, particularly reliable, particularly qualified, or particularly tested individuals, can perform the prescribed activities. Their fingerprints are technically registered in advance. As for other workers, even if they remain around the work terminal at the same time as qualified individuals and can perform non-safety-critical work activities, such safety-critical operations are prevented.

[0091] This allows biometrics to be used for general access restrictions or to restrict access to individual work activities.

[0092] In addition, as an optional feature, both layer 4 and detection device 5 can detect gestures during operation in sub-step 121 or 131 in step 120 or 130. Computer device 6 can then, as sub-step 141 within the framework of step 140, test whether the detected gestures match each other and whether they match gestures stored in data storage device 8.

[0093] The buttons in step 110 can be displayed by two or more display sections 2a, 2b of the display device 2. In this case, each display section 2a, 2b is configured to display one part of each button 3, so that the buttons 3 are created by the overlapping of the partial displays. The partial displays can be shifted spatially or temporally.

[0094] Alternatively, the display sections 2a and 2b can also be configured so that each button 3 extends over each display section, with each button 3 displaying a single section.

[0095] The display 110 of button 3 has the following characteristics when displayed on button 3 or its surrounding surface: The display location changes after a predetermined time, there is continuous movement via the display device, there is visible flashing, the flashing is not visible but is detectable by the detection device 5, there is dynamic brightness change and / or color change, and / or there is a pattern that makes up dynamic brightness change and / or color change. This can be further improved.

[0096] Furthermore, in step 140, the computer device can test two or more buttons 3 simultaneously. This can be done truly simultaneously in a parallel process, or directly before or after.

[0097] The test in step 140 may be configured such that the operation of button 3 is considered only when a predetermined time span has elapsed between the start of the display of button 3 and the operation of the button. Button 3 may be displayed in a different form during the predetermined time span than after the predetermined time span has elapsed.

[0098] A further embodiment is a computer program product that includes instructions, the instructions causing the computer to perform one of the aforementioned methods when the computer executes the program.

[0099] A further aspect is a computer-readable medium on which computer program products are stored.

[0100] A further embodiment relates to the use of the display device 1 described above as a fail-safe indicator means in a control system for a potentially hazardous technical system, such as industrial equipment, machinery, or automobiles.

[0101] These examples illustrate possible implementation variations. Just to be clear, The present invention is not limited to the specific modifications shown.

[0102] The scope of protection is defined by the claims. However, the specification and drawings may be used to interpret the claims. 。

[0103] Finally, and this is purely formal, I would like to point out that, in order to better understand the structure, the elements are shown in parts not at a fixed scale, and / or enlarged and / or reduced. [Explanation of Symbols]

[0104] 1 Display device 2 Display device 2a,2b Display area 3 buttons 4. Transparent and touch-sensitive layer 5. Detection device 6. Computer equipment 7. Fingerprint sensor 8. Data storage devices How to display 100 110 Button Display 120 Detection of operation and position 121 Gesture Detection 130 Button and position detection 131 Gesture Detection 132 Fingerprint detection 140. Testing for positional accuracy and consistency 141 Gesture Test 150 Transmission of information and control signals

Claims

1. Display device (1), A display device (2) configured to display one or more buttons (3), A transparent and touch-sensitive layer (4) is disposed on the display device (2) and configured to detect the operation of the display device (1) and the position of the operation, The display device (1) has a button (3), and at least one detection device (5) configured to detect the operation of the display device (1) and the position of the operation, Whether the button (3) of the display device (2) detected by the at least one detection device (5) matches the button (3) that should be displayed by the display device (2), and / or Whether the operation and position detected by the layer (4) and the at least one detection device (5) match each other and the displayed button (3) A computer device (6) configured to test, A display device equipped with the following features.

2. The display device (1) according to claim 1, wherein the computer device (6) is further configured to transmit information or control signals in accordance with the results of the test.

3. The display device (1) according to claim 1 or 2, wherein the display device (2) is in the form of a film, specifically formed as a liquid crystal display or a light-emitting diode film.

4. The display device (1) according to any one of claims 1 to 3, wherein the transparent and touch-sensitive layer (4) is formed in the form of a film, specifically as a touchscreen film.

5. The display device (1) according to any one of claims 1 to 4, wherein one of the at least one detection device (5) is a camera.

6. The display device (1) according to any one of claims 1 to 5, wherein one of the at least one detection device (5) is a light irradiation field film.

7. The display device (1) according to any one of claims 1 to 6, wherein one of the at least one detection device (5) is a fingerprint sensor (7), and the fingerprint sensor is positioned directly below the display device (2), between the layer (4) and the display device (2), or above the layer (4), and is configured to detect a fingerprint.

8. The display device (1) according to claim 7, wherein the computer device (6) is further configured to test whether a fingerprint detected by the fingerprint sensor (7) matches a fingerprint stored in the data storage device (8).

9. The at least one detection device (5) and the layer (4) are further configured to detect gestures during operation, and The computer device (6) is further configured to test whether the gestures detected by the layer (4) and the at least one detection device (5) match each other and whether they match the gestures stored in the data storage device (8). A display device (1) according to any one of claims 1 to 8.

10. The display device (2) includes two or more display portions (2a, 2b), Each display section (2a, 2b) is configured to display one section when pressed by each button (3). The display portions (2a, 2b) are arranged vertically relative to each other, at least one display portion (2a) is semi-transparent, and the button (3) is created by the overlapping of the partial displays, The aforementioned partial display can be performed with a spatial or temporal shift. A display device (1) according to any one of claims 1 to 9.

11. The display device (2) includes two or more display portions (2a, 2b), Each of the aforementioned display sections (2a, 2b) is configured to display one section when a button (3) is pressed. The aforementioned display sections (2a, 2b) are arranged side by side, and each button (3) extends over each display section (2a, 2b). A display device (1) according to any one of claims 1 to 9.

12. The button (3) or the surrounding surface of the button has the following characteristics, namely Changing the display location after a predetermined time. Having continuous motion and visible flashing via the display device (2), The flashing is not visible, but it is detectable by the detection device (5). Having dynamic changes in brightness and / or color, and / or having a pattern that constitutes dynamic changes in brightness and / or color, The display device (1) according to any one of claims 1 to 11, wherein the display device (2) is configured to display a button (3) having one or more of the following:

13. The display device (1) according to any one of claims 1 to 12, wherein the computer device (6) is further configured to perform tests for two or more buttons (3) simultaneously.

14. The display device (1) according to any one of claims 1 to 13, wherein the computer device (6) is configured to consider the operation of a button (3) detected by at least one detection device (5) only after a predetermined time span has elapsed between the start of the display of the button (3) and the operation of the button.

15. The display device (1) according to claim 14, wherein the display device (2) displays the button (3) in a manner different from the state after the predetermined time span has elapsed during the predetermined time span.

16. Use of the indicator device (1) according to any one of claims 1 to 15 as a fail-safe indicator means in a control system for a potentially hazardous technical system, such as industrial equipment, machinery, or automobiles.

17. A method (100) of displaying on a display device (1), Display device (2) displays at least one or more buttons (3) (110), A transparent and touch-sensitive layer (4) placed on the display device (2) detects the operation of the display device (1) and the position of the operation (120). The operation of the button (3) of the display device (2) and the display device (1), as well as the position of the operation, are detected by at least one detection device (5) (130), and Whether the button (3) of the display device (2) detected by the at least one detection device (5) matches the button (3) that should be displayed by the display device (2), and / or Whether the operations detected by the layer (4) and the at least one detection device (5) match each other and the button (3) The computer device (6) is used to test it (140). A method (100) of displaying on a display device (1), including the above.

18. The display method (100) according to claim 17, further comprising transmitting information or control signals (150) in accordance with the results of the test (140) using the computer device (6).

19. A method (100) of displaying on a display device (1) according to claim 17 or 18, wherein the detection of the button (130) includes detecting a fingerprint (132) by a detection device (5) which is a fingerprint sensor (7).

20. The display method (100) according to claim 19, wherein the test (140) includes testing (141) whether the detected fingerprint matches a fingerprint stored in the data storage device (8).

21. The detection of the operation (120) and the detection of the button (130) include the detection of gestures during the operation (121, 131), and The test (140) includes testing (141) whether the gestures detected by the layer (4) and the at least one detection device (5) match each other and match the gestures stored in the data storage device (8), The method of display according to claims 17 to 20.

22. The display (110) is generated by two or more display portions (2a, 2b) of the display device (2), and the button (3) is generated by the overlapping of the partial displays, such that each display portion (2a, 2b) displays one portion when the button (3) is pressed, and the partial displays can be shifted spatially or temporally. A method of display according to any one of claims 17 to 21 (100).

23. The display (110) is generated by two or more display portions (2a, 2b) of the display device (2), and each display portion (2a, 2b) is indicated by each button (3) such that each button (3) extends over each display portion. A method of display according to any one of claims 17 to 21 (100).

24. The button (3) or the surrounding surface of the button has the following characteristics, namely Changing the display location after a predetermined time. Having continuous motion and visible flashing via the display device (2), The flashing is not visible, but it is detectable by the detection device (5). Having dynamic brightness changes and / or color changes, and / or Having a pattern that produces dynamic changes in brightness and / or color, The display method (100) according to any one of claims 17 to 23, wherein the display (110) of the button (3) is carried out so as to have one or more of the following.

25. A display method (100) according to any one of claims 17 to 24, wherein the test (140) tests two or more buttons (3) simultaneously.

26. The display method (100) according to any one of claims 17 to 25, wherein the test (140) considers the operation of button (3) only when a predetermined time span has elapsed between the start of the display of button (3) and the operation of button (3).

27. The display method (100) according to claim 26, wherein the display (110) shows the button (3) in a manner different from the state after the predetermined time span has elapsed during the predetermined time span.

28. A computer program product wherein, when the program product is executed on the control device, the computer program product can be loaded directly into the storage device of the programmable control device of the display device (1) together with program code means in order to perform all the steps of the method according to any one of claims 17 to 27.

Images