Method for locating industrial safety sensors in industrial plant using augmented reality

Abstract

The invention relates to a method for locating at least one real sensor, in particular a safety sensor or camera for industrial safety applications, in a real environment of an industrial plant by means of an augmented reality view based on the real environment of the industrial plant, at least one simulation parameter of the virtual sensor is visualized for the user in an augmented reality view by means of a display device, the virtual sensor being based on the real sensor to be positioned, the simulation parameter comprising at least one position parameter, and the position parameter comprising at least one position parameter. The at least one position parameter indicates a position and / or orientation of the virtual sensor in the augmented reality view, a change in the position parameter is obtained, the position parameter updated on the basis of the obtained change is visualized for the user in the augmented reality view by means of the display device, and the position parameter is displayed on the basis of the obtained change. Outputting the updated position parameter based on the virtual sensor and enabling positioning of at least one output parameter of the real sensor in the real environment.

Description

TECHNICAL FIELD

[0001] The present invention relates to a method, a display device and a system for positioning at least one real sensor, in particular a safety sensor or camera for industrial safety applications, in a real environment of an industrial plant by means of an augmented reality view. BACKGROUND

[0002] Industrial safety sensors or cameras for industrial safety applications allow for a safe environment perception, in particular a safe three-dimensional environment perception, whereby the safety and efficiency of industrial processes in an industrial plant can be improved. Examples of industrial safety sensors or cameras for industrial safety applications include ToF (Time of Flight) cameras, laser scanners, 2D cameras, 3D cameras, lidar (light detection and ranging) sensors, radar sensors, ultrasonic sensors, etc.

[0003] It is to be understood that an industrial plant is to be understood broadly here and can for example include a factory hall, a production plant, a warehouse, a logistics center, a livestock facility, a chemical industry plant, a waste incineration plant or a power plant. In the following, only reference is made to (real) sensors, wherein both industrial safety sensors and cameras for industrial safety applications are meant.

[0004] So far, positioning and configuring real sensors in a real environment of an industrial plant requires a very high level of technical understanding as well as a very good knowledge of the physical and technical operating principles of the sensors. Furthermore, real measurement data acquired with the pre-positioned real sensors is required to be displayed to the user in a pure virtual 3D configuration application. The user is then required to have almost perfect imagination in order to transfer the complex 3D transformations performed in the mind to the real application. It is to be understood that such a positioning and configuration process is only possible in case the real sensors have been pre-positioned and connected and real measurement data is available. However, for the initial positioning and configuration step, it is unclear whether the real sensors are even installed in a suitable position so that the sensing capabilities of the real sensors can first be used in the application or whether the detection area of the sensors is even sufficient for the intended application. SUMMARY

[0005] It is a basic object of the present invention to improve the positioning of real sensors in a real environment of an industrial plant, in particular to make it easier for a user to position.

[0006] The method having the features of claim 1 is provided to meet the stated object. Advantageous embodiments of the present invention can be gathered from the dependent claims, the description and the drawings.

[0007] In the method for positioning at least one real sensor, in particular a safety sensor or camera for an industrial safety application, in a real environment of an industrial plant by means of an augmented reality view according to claim 1, at least one simulation parameter of the virtual sensor is visualized for a user in the augmented reality view by means of a display device.

[0008] It is to be understood that when referring to (at least) one real sensor in the following, also a plurality of real sensors can be meant. Thus, the present invention equally relates to the positioning (and configuration) of a plurality of real sensors. The real sensor is preferably a ToF camera, a laser scanner, a 2D camera, a 3D camera, a laser radar sensor, a radar sensor or an ultrasonic sensor.

[0009] According to the present invention, the augmented reality view or augmented reality environment is based on a real environment of the industrial plant. The augmented reality view can for example be a view or representation of the real environment in the industrial plant, wherein a visualization of a simulation element can be inserted into the view or representation of the real environment.

[0010] According to the present invention, the virtual sensor is based on the real sensor to be positioned. This can mean that the virtual sensor mirrors the real sensor to be positioned, is a model of the real sensor and / or represents a virtual copy of the real sensor, such that properties and / or parameters of the virtual sensor can be transferred to properties and / or parameters of the real sensor and vice versa. For example, at least one property and / or parameter of the virtual sensor can be identical to a property and / or parameter of the real sensor to be positioned.

[0011] It is to be understood that each parameter mentioned in the following is also to be understood as a set of parameters. For example, a parameter can comprise a plurality of values or can be a vector or array comprising a plurality of elements. Further, it is to be understood that even if reference is made to one parameter, a plurality of parameters can be meant, in particular a plurality of parameters having the same name.

[0012] According to the present invention, the at least one simulation parameter comprises at least one position parameter, wherein the at least one position parameter is indicative of a position and / or orientation of the virtual sensor in the augmented reality view. It is to be understood that the position parameter simulates for a user in the augmented reality view a (future) real position of the real sensor in the real environment. The position parameter specifically represents an initial position parameter which can be changed and which can be updated based on the change.

[0013] In the method according to the application, further a change of the position parameter is obtained and the position parameter updated based on the obtained change is visualized for the user in the augmented reality view by means of the display device. In this respect, it is likewise possible that the change is obtained by means of the display device and the position parameter is updated by means of the display device. Additionally or alternatively, it is conceivable that at least one device separate from the display device, such as a computing unit or a processor, obtains the change of the position parameter, the change is captured by means of the camera or the user interface, the position parameter is updated based on the obtained change and the updated position parameter is transmitted to the display device and / or can be retrieved by the display device. The updated position parameter can then be visualized in the augmented reality view by means of the display device, for example by updating and / or adapting a previous visualization of the (old) position parameter or adding a new visualization of the (updated) position parameter. Finally, in the method according to the application, at least one output parameter is outputted, which is based on the updated position parameter of the virtual sensor and which enables a positioning of the real sensor in the real environment. It is to be understood that the output parameter can comprise the position parameter and / or a conversion parameter obtained by converting the position parameter. Additionally or alternatively, the output parameter can be indicative of a position and / or an orientation of the virtual sensor relative to the position marker or the display device.

[0014] According to the application, the output parameter enables a positioning of the real sensor in the real environment. In other words, the output parameter provides information based on which the real sensor can be positioned (manually or automatically) in the real environment.

[0015] In other words, the application is based on the idea that, by means of a spatial visualization in the augmented reality view, the user is provided with the possibility to plan, adjust and / or optimize the positioning and, preferably also, the configuration of the safety sensor (in particular three-dimensionally) from his own perspective. Using the augmented reality view, the user can easily and flexibly test at which position the real sensor should be positioned and / or installed (in the future) in the real environment of the industrial plant. The same or similar applies to the configuration of the protection and warning field of the laser scanner or the protection and warning volume of the camera or radar sensor. The size and the orientation of the virtual sensor can be easily and flexibly adjusted and its visualization in the augmented reality view can be easily and flexibly inserted and / or updated. The positioning (and, preferably also, the configuration) of the virtual sensor is then outputted by means of the output parameter and can be ideally transferred to the positioning of the real sensor in the real environment of the industrial plant.

[0016] In short, a model of at least one real sensor can be positioned (and / or configured) in the augmented reality view and the real sensor can then be positioned (and / or configured) in the real environment based on this positioning (and / or configuration) of the virtual sensor.

[0017] In contrast to the previous procedure, real sensors do not necessarily have to be available in this respect, and real data acquired with real sensors do not necessarily have to be available. The position of the virtual sensors can be easily and flexibly adjusted and optimized in the augmented reality view based on the specific three-dimensional space of the real environment. It is conceivable that the protection field, the field of view and / or the field of view (FOV) of the virtual sensors are additionally visualized. A so-called protection field is a part of the field of view, in which a warning signal can be output if the protection field is violated, for example, by a person or a robot, in particular an autonomous robot. The protection field of the virtual sensors can cover the entire field of view of the virtual sensors or only a part of the field of view of the virtual sensors. The same applies to the real sensors. It is also conceivable that a plurality of (different) real and corresponding virtual sensors can be selected in the method, so that the best sensors can be found for each application.

[0018] In general, the inherently complex process of positioning (and preferably also configuring) safety sensors in the environment of an industrial plant can be improved and, in particular, can be made simpler.

[0019] Furthermore, a plurality of virtual sensors can be positioned or run in a single method substantially simultaneously, so that, for example, gaps between the monitoring areas of the virtual sensors and real barriers, for example walls, in the real environment can be detected and eliminated. The plurality of sensors (and the corresponding protection fields) can also be optimally matched to one another. This can be particularly advantageous if seamless monitoring of large protection fields, which can only be covered by a large number of sensors, is desired or if targeted overlapping of protection fields is desired in order to achieve increased safety, for example via redundant monitoring in particularly critical areas of the industrial plant. In this respect, it is conceivable that in the augmented reality view, the overlapping areas of the protection fields of the plurality of virtual sensors can also be calculated and visualized, for example by different colors. Furthermore, the method can be implemented in a computer-implemented manner, in particular in a network-based manner, so that experts can be consulted by, for example, digital telecommunication.

[0020] According to one embodiment, the at least one position parameter of the virtual sensor comprises an x-coordinate value, a y-coordinate value, a z-coordinate value and / or an alignment angle of the virtual sensor. The alignment angle of the virtual sensor can comprise an azimuth angle and / or an elevation angle of a line of sight of the virtual sensor.

[0021] According to one embodiment, at least one analog parameter of the virtual sensor visualized in the augmented reality view further includes at least one configuration parameter indicating the configuration of the virtual sensor. Preferably, the at least one configuration parameter of the virtual sensor corresponds to at least one possible configuration of the real sensor. In this method, changes to the (initial) configuration parameters can then be obtained alternatively or additionally. By means of a display device, for example by adaptively adjusting a previous visualization or by adding a new visualization, the updated configuration parameters based on the obtained changes are visualized for the user in the augmented reality view. Furthermore, the updated configuration parameters based on the virtual sensor are output, and another output parameter of the real sensor can be configured in the real environment. It should be understood that the output parameter based on the updated position parameter and the other output parameter based on the updated configuration parameter can be output substantially together or offset in time, and can be output specifically as a single parameter and / or as a set of parameters.

[0022] According to one embodiment, at least one configuration parameter of the virtual sensor includes the virtual sensor's maximum field of view, a portion of the maximum field of view, orientation, size and / or width of the guard field or warning field, range, and / or resolution of the virtual sensor. The range of the sensor can mean the maximum width of the sensor's guard field in front of the background and / or into free space. Preferably, the configuration parameters of the virtual sensor are based on or correspond to possible configuration parameters of the real sensor, such that the configuration parameters of the virtual sensor can be transferred to the real sensor and vice versa.

[0023] According to one embodiment, the position of the virtual sensor can be changed by the user in the augmented reality view, and can be specifically shifted and / or rotated. In other words, the user can change the virtual sensor or the visualization of the virtual sensor in the augmented reality view, and in particular shift and / or rotate it, so that changes in position parameters can be obtained in this way.

[0024] According to one embodiment, the display device is a mixed reality display or an augmented reality display. Preferably, the display device is augmented reality glasses, a laptop computer, a smartphone, and / or a tablet computer. The display device is preferably mobile and can be moved within the real environment to change the perspective of the augmented reality view. This allows the user to continuously view new perspectives in the augmented reality view to check and optimize the positioning and configuration of the virtual sensor from all sides. Thus, for example, it is conceivable that the display device is carried by the user. If the user then moves within the real environment, the perspective of the augmented reality view can be changed, allowing the user to move around the virtual sensor and view it from all sides. In this way, for example, gaps between the protective field of the virtual sensor and barriers in the real environment can be detected and eliminated. Preferably, it is conceivable that the position of the display device in the real environment is known, and changes in the position of the display device in the real environment can be tracked, for example, by odometry. Preferably, screenshots of the augmented reality view can be taken at any point in time for documentation purposes.

[0025] According to one embodiment, the method further includes creating augmented reality views and / or creating virtual sensors.

[0026] According to one embodiment, an augmented reality view is created using a display device. For this purpose, the display device may include, for example, sensors, scanners, or cameras that utilize their scanning or capture of the real environment. Additionally or alternatively, it is conceivable that the augmented reality view is created because the display device detects location markers applied in the real environment, such as machine-readable codes, particularly QR codes or RFID tags. The location markers are preferably captured by a camera included in the display device or by an RFID reader. A model of the real environment can be obtained by detecting the location markers. In particular, additional information relating to the configuration parameters of virtual sensors, and which can then be read by the display device, can also preferably be stored on the location markers. For example, information regarding which real sensor should be used and correspondingly which virtual sensor should be used, which sensors or protective fields should be used and how many sensors or protective fields should be used, what sensor resolution the sensors should provide, whether there are limitations on positioning options, and so on, can be obtained via the location markers.

[0027] Furthermore, it is conceivable that the initial position of the display device can be detected in a model of the real environment by detecting location markers. For example, the position and orientation of the display device in the real environment can be calculated based on the optical distortion of the detected location markers. Additionally or alternatively, the position of the display device can be determined and tracked by means of markers and / or sensors (e.g., cameras or radio anchors) attached to the real environment. Additionally or alternatively, the display device may include a GPS module for the same purpose.

[0028] Obtaining a model of the real environment allows for the detection of overlap between the virtual sensor's field of view and / or protective field and real objects, and / or the shadows cast by real objects. In this way, real objects and barriers in the real environment can be taken into account. Preferably, such overlaps are identified based on the model of the real environment and visualized in the augmented reality view.

[0029] According to one embodiment, at least one simulation parameter, particularly a position parameter and / or configuration parameter, is visualized as at least one geometry in an augmented reality view. Preferably, the at least one simulation parameter is displayed in the augmented reality view as a polyhedron (e.g., a pyramid) or a cone. The simulation parameter is preferably displayed in the augmented reality view as a geometry shown in color. For example, it is conceivable that a volume defined by the geometry, at least one side surface of the geometry, and / or at least one edge of the geometry is displayed in black, gray, white, yellow, green, red, or any other possible color, or that all side surfaces and / or edges of the geometry displayed in the augmented reality view are displayed in black, gray, white, yellow, green, red, or any other possible color.

[0030] Preferably, at least one analog parameter, particularly a position parameter and / or configuration parameter, is additionally visualized as text in the augmented reality view. For example, it is conceivable to show the auxiliary elements that must be protected by the sensor, such as the tolerance range of the protection field, or the tolerance range of the protection volume, or the dimensions of the so-called undercreep volume (particularly related to a camera positioned very close to the surface (e.g., 30 cm above the ground)). Preferably, it is also conceivable to show which sensor can achieve which resolution at which distance. According to one embodiment, the dimensions (i.e., e.g., length, width, and / or height) of the protection field of at least one virtual sensor are visualized and / or the distances between the protection fields of multiple virtual sensors are visualized to each other and / or the distances between the protection fields of virtual sensors and markers or reference marks in the real environment are visualized (e.g., as numerical values ​​in each case).

[0031] In addition to 2D cameras, display devices can also include LiDAR sensors to detect surfaces in the real environment or otherwise acquire surface data. Therefore, a surface model can be calculated (in the background) during the positioning and configuration process. With the aid of the surface model, a clear distance (e.g., the shortest distance) between the virtual sensor's protective field and the real object or geometry can be output. Providing this information about distance can be advantageous in terms of security. Furthermore, particularly with the aid of the surface model, it is possible to continuously check for collisions between the configured protective field and the real geometry during the process, and potential collisions can be displayed in the visualization (e.g., by changing color).

[0032] According to one embodiment, the resolution of the virtual sensor is visualized, based on the range, as shadows, transparency gradients, and / or color gradients within the geometry. In this way, multiple configuration parameters of the virtual sensor can be visualized together for the user in a single geometric form.

[0033] According to one embodiment, changes to position parameters and / or configuration parameters are obtained through user input. Therefore, for example, it is conceivable that a user can input and / or select desired changes to position parameters and / or configuration parameters via a user interface (e.g., a GUI). Additionally or alternatively, preferably, the user can change the position parameters and / or configuration parameters via gestures or voice input, preferably detected by a camera or microphone. In other words, the user (particularly via gestures or voice control) can interactively change the position and / or configuration of a virtual sensor in an augmented reality view, and particularly shift, rotate, expand, widen, extend, reduce, and / or distort the position and / or protective field of the virtual sensor. Voice control can preferably be supported by translating voice commands into “machine code” using a so-called large language model.

[0034] Preferably, it is conceivable that the position parameters and / or configuration parameters can be changed incrementally or continuously.

[0035] In addition or alternatively, according to another embodiment, changes in position parameters and / or configuration parameters are obtained by detecting at least one mark, contour, and / or surface in the real environment, wherein the mark, contour, and / or surface is preferably captured by a camera included in the display device. In this regard, the position and / or protective field of the virtual sensor in the augmented reality view is preferably (automatically) oriented in the real environment based on the detected mark, contour, and / or surface. More precisely, the position and / or protective field of the virtual sensor is preferably changed based on the detected mark, contour, and / or surface, such that, and in particular, it is shifted, rotated, magnified, widened, extended, reduced, and / or distorted, such that the protective field of the virtual sensor terminates at the detected mark, contour, and / or surface and / or is centered thereon and / or has or presents a specific distance from the detected mark. In this way, the positioning and / or configuration of virtual sensors (especially multiple virtual sensors) in the augmented reality view can be simplified and / or accelerated. Furthermore, it is possible to prevent unwanted gaps or uncovered areas from remaining between the protective field of the virtual sensor and the barrier in the real environment.

[0036] According to one embodiment, the updated position parameters and / or updated configuration parameters are preferably fixed and / or immutable. The position of the virtual sensor, the orientation of the virtual sensor, the orientation of the guard field, and / or the size of the guard field can preferably be fixed, while the corresponding other degrees of freedom remain freely changeable. Subsequently, it is also conceivable that, preferably, the position of the virtual sensor and / or the guard field, particularly the corners, edges, and / or side surfaces of the virtual sensor's guard field, can be automatically aligned with markings, contours, and / or surfaces in the real environment as described herein, and can then be fixed and / or blocked for further modification. In other words, the position of the virtual sensor and / or the orientation of the guard field at the markings, contours, and / or surfaces can preferably no longer be changed by the user, such that only the other degrees of freedom of the virtual sensor can be changed by the user. In this way, gaps that might be overlooked by the user can be prevented from remaining between the guard field of the virtual sensor and the barrier in the real environment. This allows for the minimization or avoidance of errors as much as possible.

[0037] According to one embodiment, prohibitions on changing the position parameters and / or configuration parameters of a virtual sensor are visualized in the augmented reality view as a change in the color of the geometry. In this way, for example, if a change in the configuration parameters of the virtual sensor exceeds the tolerance defined for the configuration parameters of the real sensor, or if a change in the position parameters of the virtual sensor selects a location for the virtual sensor that cannot be transmitted to the real sensor (because the real sensor cannot be positioned and / or connected to the corresponding location in the real environment), this can be interactively and immediately displayed to the user.

[0038] According to one embodiment, if it is determined that the altered protective field of the virtual sensor collides with a real object, aligns with a marker in the real environment, and / or detects a marker, then the prohibition of alteration of the virtual sensor's position parameters and / or configuration parameters is visualized in the augmented reality view as a change in the color of the geometry.

[0039] According to another embodiment, the real sensor is positioned in the real environment based on the output parameters. With the real sensor positioned in the real environment, real measurement data is acquired, wherein the positioning and / or configuration of the real sensor in the real environment is verified by comparing simulated measurement values ​​acquired by the virtual sensor with real measurement values. For example, test markers and / or test objects with known dimensions, shapes, and / or patterns can be placed in the protective field of the positioned real sensor, and real measurement data can be acquired from the test markers and / or test objects. Furthermore, the measurement data acquired from the test markers or test objects is simulated using a virtual sensor. The comparison of the real measurement data with the simulated measurement data then allows verification of the positioning and / or configuration of the real sensor in the real environment. Therefore, it is preferably conceivable to use test markers to track or mark the ends of the protective field of the virtual sensor on a surface, particularly a floor surface, or the line of sight of the virtual sensor in the real environment. In the real measurement data, it must then be identifiable that the test markers frame the field of view of the real sensor, or are centered on the field of view of the real sensor, in order to verify the positioning and / or configuration of the real sensor in the real environment. Additionally or alternatively, it is conceivable that the optical distortion of the test object (which is identifiable in the measurement data) must be matched in order to verify the positioning and / or configuration of the real sensor.

[0040] According to one embodiment, a display device is used to identify and / or visualize for a user at least one difference between real measurement data and simulated measurement data in an augmented reality view. In response to the visualized difference, the positioning of the real sensor in the real environment is iterated using the augmented reality view by changing, and particularly by shifting and / or rotating, the position of the virtual sensor in the augmented reality view and outputting updated output parameters.

[0041] Another subject of the invention is a display device for locating at least one real sensor (particularly a safety sensor or camera for industrial safety applications) in a real environment of an industrial plant using an augmented reality view based on the actual environment of the industrial plant. The display device according to the invention is configured to visualize at least one simulated parameter of a virtual sensor for a user in an augmented reality view, wherein the virtual sensor is based on the real sensor to be located, and wherein the simulated parameter includes at least one position parameter indicating the position and / or orientation of the virtual sensor in the augmented reality view. The display device is also configured to acquire changes in the position parameter, visualize the updated position parameter based on the acquired changes for the user in the augmented reality view, and output the updated position parameter based on the virtual sensor, thereby enabling the location of the real sensor in the real environment.

[0042] Another subject of the invention is a system for locating at least one real sensor (particularly a safety sensor or camera for industrial safety applications) in a real environment of an industrial plant by means of an augmented reality view based on the real environment of the industrial plant, wherein the system includes the display device described herein and at least one real sensor.

[0043] It should be understood that the description of the method according to the invention also applies to display devices and systems. This is particularly true of the embodiments and advantages. Furthermore, it should be understood that, unless otherwise expressly stated, all features and embodiments disclosed herein can be combined. Attached Figure Description

[0044] The invention will now be described by way of example only, with reference to possible embodiments and accompanying drawings. As shown below:

[0045] Figure 1A A schematic diagram of a real sensor of a system according to an embodiment of the present invention is shown;

[0046] Figure 1B A schematic diagram of a display device according to an embodiment of the present invention is shown;

[0047] Figure 2A A schematic diagram of a display device according to an embodiment of the present invention is shown;

[0048] Figure 2B A schematic diagram of a display device according to an embodiment of the present invention is shown;

[0049] Figure 3A A front view of a display device according to an embodiment of the present invention is shown;

[0050] Figure 3B A front view of a display device according to an embodiment of the present invention is shown;

[0051] Figure 3C A front view of a display device according to an embodiment of the present invention is shown;

[0052] Figure 4A A front view of a display device according to an embodiment of the present invention is shown;

[0053] Figure 4B A front view of a display device according to an embodiment of the present invention is shown;

[0054] Figure 4C A front view of a display device according to an embodiment of the present invention is shown;

[0055] Figure 5A A front view of a display device according to an embodiment of the present invention is shown;

[0056] Figure 5BA front view of a display device according to an embodiment of the present invention is shown; and

[0057] Figure 6 A front view of a display device according to an embodiment of the present invention is shown. Detailed Implementation

[0058] Figure 1A A schematic diagram of an exemplary real sensor 10 is shown, which can be used in conjunction with methods, display devices, and systems according to embodiments of the present invention. The real sensor 10 preferably includes a 3D camera, particularly a ToF camera. The guard field 11 of the real sensor 10 corresponds to the monitoring area detected by the sensor and has a breadth that can be seen relative to a starting point at the real sensor 10 and / or can also be defined by x, y, and z coordinates in a global coordinate system. As shown in FIG1, the guard field 11 of the real sensor 10 may have a pyramidal geometry.

[0059] For examples of available real sensor 10, Table 1 shows the dimensions of the protection field and maximum field of view for certain ranges. Table 2 shows exemplary achievable ranges and reflectance values ​​of the real sensor 10 at certain object resolutions according to the performance level c (PLc) of the real sensor 10. The performance level c (PLc) is a measure of the reliability of the technical safety function and is determined in risk assessments according to machine instructions, such as the DIN ENISO 13849-1 standard for industrial robots. The values ​​in Table 2 indicate the range of object resolutions achievable for the body, hand, one or more arms or legs according to PLc. The values ​​of the real sensor from Tables 1 and 2 can be visualized in augmented reality views as at least one geometry, particularly as at least one geometry shown in color, and preferably as shadows, as transparency gradients, and / or as color gradients within the geometry.

[0060] Table 1:

[0061]

[0062] I Minimum measurable range

[0063] II The largest protective field without background

[0064] III Maximum protection field with background

[0065] IV Warning field

[0066] Table 2:

[0067]

[0068] Reflectance guaranteed: 4% to 2m, 18% to 4m, 60% to 7.3m

[0069] If the detection area is limited by visible background objects, such as walls or floors, a magnified 4-meter scanning area can be used.

[0070] I In mobile use, an enlarged scan area may affect device usability, so it is not recommended by default.

[0071] According to an embodiment of the present invention Figure 1B The display device 40, schematically shown, includes a camera 41 and a user interface 42. The display device 40 is configured to visualize simulated parameters of a virtual sensor 30 for a user in the augmented reality view 20. The virtual sensor 30 is in this respect based on a real sensor 10 to be positioned in the real environment of an industrial plant, such as via... Figure 1A As shown in the example, the simulation parameters include at least one position parameter indicating the position and / or orientation of the virtual sensor 30 in the augmented reality view 20, and at least one configuration parameter indicating the configuration of the virtual sensor 30, which ideally corresponds to, for example... Figure 1A Possible configurations of the real sensor 10 shown. The display device 40 can also be configured to create augmented reality views because the camera 41 of the display device 40 detects location markers applied in the real environment. Figure 1B (Not shown in the image).

[0072] Preferably, the display of the augmented reality view 20 and the user interface 42 can be implemented in a browser (typically available on a terminal device) for both the front end (e.g., a so-called graphical user interface GUI) and the back end (computation and transformation). In this regard, the front end may include functionalities such as spatial representation of 3D elements, field-of-view geometry, a guard field, resolution capabilities and / or detection capabilities of the guard field, and optionally, projection of the guard field onto the origin of the associated sensor. Further possible are so-called user interface (UI) elements for adding, removing, and manipulating 3D elements using so-called transformation handlers (also called handles), and UI elements for navigating (orbit control) in virtual space (i.e., moving the virtual camera). The back end may include functionalities such as calculating the correct position and orientation of individual 3D elements in virtual space, calculating the correct shape of the guard field (preferably truncated at the boundaries of the field of view), calculating the projection of the guard field onto the origin of the associated sensor, and responding to various user events and user inputs (triggered by UI elements). Modern web technologies can be used for implementation, such as "node.js" and / or the 3D library "Three.js". Therefore, the limited display size of mobile devices can be fully utilized. Similarly, it is conceivable that the augmented reality view 20 is implemented using the so-called Vuforia engine, a powerful software development kit (SDK) for creating augmented reality (AR) applications.

[0073] like Figure 1B As shown in the example, the position of the virtual sensor 30, along with its protective field 31 and maximum field of view 33, can be visualized in the augmented reality view 20 as multiple geometries, and in particular, pyramids. For example, in Figure 1B In the virtual sensor 30, the protective field 31 is visualized as a pyramid with side surfaces shown in color, and the maximum field of view 33 of the virtual sensor is visualized as a pyramid with edges shown in color.

[0074] Figure 1B The display device 40 shown is further configured to receive changes to position parameters and / or configuration parameters via user input through a user interface 42. For example... Figure 1B As shown, the user interface 42 may include a GUI with an image of the virtual sensor 30, which the user can interact with via a computer mouse or touchpad. Preferably, the image of the virtual sensor 30 in the user interface 42 can be changed for the user via a computer mouse, touchpad, and / or voice commands, and can be specifically shifted, rotated, magnified, enlarged, extended, reduced, and / or distorted, so that changes in position parameters and / or configuration parameters can be obtained as user input in this way.

[0075] Figure 2A and Figure 2B A display device 40 according to another embodiment of the present invention is shown schematically. Figure 2A and Figure 2B The display device 40 shown may include, with Figure 1B The display devices shown have the same or similar components and functions, and are configured to visualize the (initial) position of the virtual sensor 30 and its field of view and / or guard field for the user in the augmented reality view 20. Figure 2B As shown, the display device 40 is also configured to visualize, in the augmented reality view 20, the updated position parameters based on the obtained changes in the position parameters of the virtual sensor 30. Changes in the position parameters may specifically include displacement and / or rotation of the position and / or orientation of the virtual sensor 30. It should be understood that the visualization of the maximum field of view and / or guard field of the virtual sensor 30 may co-displace and / or co-rotate according to the displacement and / or rotation of the position and / or orientation of the virtual sensor 30. The display device 40 is also configured to output multiple output parameters based on the updated position parameters and updated configuration parameters of the virtual sensor 30. In this way, it is possible to locate and / or configure the real sensor 10 in the real environment without pre-locating and / or connecting the real sensor 10 in the real environment.

[0076] Figure 3A , Figure 3B and Figure 3C A front view of a display device 40 according to another embodiment of the invention is shown, illustrating how the protective field 31 of a virtual sensor 30 can be altered. A user can, for example, shift, rotate, zoom in, enlarge, lengthen, reduce, and / or distort an image of the virtual sensor 30 in a user interface 42 by clicking and dragging with a computer mouse. The visualization of the protective field 31 of the virtual sensor 30 in the augmented reality view 20 is then updated based on the displacement, rotation, zoom, enlargement, lengthen, and / or distortion obtained via user input, and specifically, the corresponding shift, rotation, zoom, enlargement, lengthen, reduction, and / or distortion.

[0077] Figure 4A , Figure 4B and Figure 4C A front view of a display device 40 according to another embodiment of the invention is shown, illustrating how the protective field of the virtual sensor 30 can be altered. A user can, for example, zoom in on the image of the virtual sensor 30 in the user interface 42 by clicking and dragging with a computer mouse. The visualization of the protective field 31 of the virtual sensor 30 in the augmented reality view 20 is then updated based on the zoom obtained through user input, and is zoomed in accordingly.

[0078] Figure 5A and5B A front view of a display device 40 according to another embodiment of the present invention is shown, illustrating how the position of a virtual sensor 30 visualized via a first visualization in augmented reality view 20 can be changed. For example, a user can add a second image of the virtual sensor 30 in user interface 42, and can shift the second image in user interface 42 by clicking and dragging with a computer mouse. In augmented reality view 20, a second visualization 60 is then similarly added to the position of the virtual sensor 30 and shifted accordingly.

[0079] exist Figure 6 The image shows a front view of a display device 40 according to an embodiment of the present invention, wherein changes in position parameters and / or configuration parameters are obtained by detecting markers 50 applied in a real environment. The markers 50 are preferably provided by a camera 41 included in the display device 40. Figure 6 (Not shown in the image) captures and marks the location of the real object or barrier 70 in the real environment. Virtual sensor 30 ( Figure 6 The position and / or protective field 31 of the virtual sensor 30 (not shown) is then preferably oriented (automatically) based on the detected mark 50. More precisely, the position and / or protective field 31 of the virtual sensor 30 is preferably changed based on the detected mark 50, such that, and in particular, it is shifted, rotated, enlarged, widened, lengthened, reduced, and / or distorted, so that the protective field 31 of the virtual sensor 30, in particular the corners, edges, and / or surfaces of the protective field 31 of the virtual sensor 30, terminates at and / or centers on the detected mark 50 and / or has or presents a specific distance from the detected mark 50. In this way, the positioning and / or configuration of the virtual sensor 30 in the augmented reality view 20 can be accelerated. Furthermore, it is possible to prevent discrepancies between the protective field 31 of the virtual sensor 30 and the visualization of the real object 70 in the augmented reality view 20, and therefore between the protective field 11 of the real sensor 10 in the real environment and the real object 70 in the real environment. Figure 6 Unwanted gaps or uncovered areas are maintained between the virtual sensor 30 (not shown) and the real object 70. The protective field 31 of the virtual sensor 30, based on the orientation of the marker 50 and / or the orientation of the marker 50, is then preferably fixed and blocked to prevent further alteration by the user, thereby reducing or avoiding errors when the position and / or protective field of the virtual sensor 30 are changed.

[0080] Then, output parameters are output, which are based on the updated position parameters of the virtual sensor 30 and enable the positioning of the real sensor 10 in the real environment. The output parameters can, for example, specify the orientation and radial distance, or the x, y, and z coordinates, at which the real sensor will be positioned in the global coordinate system. This can be obtained by detecting position markers applied to the real environment using a display device, starting from the position marker as an anchor point, from the position of the display device as an anchor point, or from the position of the marker 50 as an anchor point. The real sensor can be positioned manually or mechanically based on the output parameters, and in particular, mounted and / or attached to the real environment. The user can, for example, use a measuring tape to measure the x, y, and z coordinates output using the output parameters, starting from the position of the position marker or the display device (which may correspond to the user's position), and the real sensor can be attached to the corresponding position. Once the positioning of the real sensor has occurred, the data from the virtual sensor and the real sensor can be aligned. Based on this adjustment, it can be checked whether the real sensor has been correctly positioned and / or configured.

[0081] List of reference numerals

[0082] 10 real sensors

[0083] 11 Protective Fields for Real Sensors

[0084] 20 Augmented Reality Environments

[0085] 30 virtual sensors

[0086] 31. Protective Field for Virtual Sensors

[0087] 33 Virtual Sensor Maximum Field of View

[0088] 40 display devices

[0089] 41 cameras

[0090] 42 User Interface

[0091] 50 mark

[0092] Second visualization of the location of 60 virtual sensors

[0093] 70 real objects

Claims

1. A method for positioning at least one real sensor (10), in particular a safety sensor or camera for an industrial safety application, in a real environment of an industrial plant by means of an augmented reality view (20) based on the real environment of the industrial plant, wherein at least one simulated parameter of a virtual sensor (30) is visualized for a user in the augmented reality view (20) by means of a display device (40), wherein the virtual sensor (30) is based on the real sensor (10) to be positioned, wherein the simulated parameter comprises at least one position parameter, wherein the at least one position parameter is indicative of a position and / or orientation of the virtual sensor (30) in the augmented reality view (20), wherein a change of the position parameter is obtained, wherein the updated position parameter based on the obtained change is visualized for the user in the augmented reality view (20) by means of the display device (40), and wherein at least one output parameter is outputted, which is based on the updated position parameter of the virtual sensor (30) and enables positioning of the real sensor (10) in the real environment.

2. The method of claim 1, wherein, The at least one position parameter of the virtual sensor (30) comprises an x-coordinate value, a y-coordinate value, a z-coordinate value and / or an alignment angle of the virtual sensor (30).

3. The method of claim 1 or 2, wherein, The at least one simulated parameter of the virtual sensor (30) visualized in the augmented reality view (20) further comprises at least one configuration parameter, wherein a change of the configuration parameter is obtained, wherein the updated configuration parameter based on the obtained change is visualized for the user in the augmented reality view (20) by means of the display device (40), and wherein a further output parameter is outputted, which is based on the updated configuration parameter of the virtual sensor (30) and enables configuring of the real sensor (10) in the real environment.

4. The method of claim 3, wherein, The at least one configuration parameter of the virtual sensor (30) comprises a maximum field of view (33) of the virtual sensor (30), a portion of the maximum field of view (33), an orientation, a size and / or a width of a protection field or warning field (31), a range and / or a resolution of the virtual sensor (30).

5. The method according to any one of the preceding claims, wherein, The position of the virtual sensor (30) is changeable, in particular shiftable and / or rotatable, for the user in the augmented reality view (20).

6. The method according to any one of the preceding claims, wherein, The augmented reality view (20) is created by means of the display device (40).

7. The method according to any of the preceding claims, wherein, The display device (40) is a mixed reality display or an augmented reality display, such as an augmented reality glasses, a laptop, a smartphone and / or a tablet computer, and wherein the display device (40) is preferably mobile and movable in a real environment in order to change a perspective of the augmented reality view (20).

8. The method of any of the preceding claims, wherein, The augmented reality view (20) is created as: detecting a position marker, for example a machine-readable code, in particular a QR code, applied in the real environment by means of the display device (40), wherein the position marker is preferably captured by a camera (41) comprised by the display device (40), and obtaining a model of the real environment by detecting the position marker, wherein, by detecting the position marker, preferably an initial position of the display device (40) in the model of the real environment can be detected.

9. The method according to any of the preceding claims, wherein, The at least one simulation parameter, in particular the position parameter and / or the configuration parameter, is visualized in the augmented reality view (20) as at least one geometric shape, in particular as at least one geometric shape shown in color, wherein the at least one simulation parameter, in particular the position parameter and / or the configuration parameter, is preferably additionally visualized in the augmented reality view (20) as a text display.

10. The method according to claims 8 and 9, wherein, The resolution of the virtual sensor (30) is visualized in the geometric shape as a shading, a transparency gradient and / or a color gradient depending on the range.

11. The method of any of the preceding claims, wherein, The change of the position parameter and / or the configuration parameter is obtained by a user input, and / or wherein the change of the position parameter and / or the configuration parameter is obtained by detecting at least one marker (50), contour and / or surface in the real environment, wherein the marker (50), the contour and / or the surface is preferably captured by a camera (41) comprised by the display device (40), wherein the position and / or the protection field (31) of the virtual sensor (30) in the augmented reality view (20) is preferably oriented based on the detected marker (50), contour and / or surface in the real environment, wherein the position and / or the protection field (31) of the virtual sensor (30) is preferably changed, in particular displaced, rotated, enlarged, widened, elongated, reduced and / or distorted, based on the detected marker (50), contour and / or surface, so that the protection field (31) of the virtual sensor (30) ends with the detected marker (50), contour and / or surface and / or is centered on the detected marker (50) and / or has a certain distance to the detected marker (50).

12. The method of any one of the preceding claims, wherein, The updated position parameter and / or the updated configuration parameter is fixable and / or unchangeable.

13. The method of any of the preceding claims, wherein, The prohibition of a change of the position parameter and / or the configuration parameter of the virtual sensor (30) is visualized in the augmented reality view (20) as a color change of the geometric shape.

14. The method of any of the preceding claims, wherein, positioning the real sensor (10) in the real environment based on the output parameter, wherein the real sensor (10) positioned in the real environment acquires real measurement data, wherein a verification of the positioning and / or the configuration of the real sensor (10) in the real environment is performed by comparing the simulation measurement data acquired by the virtual sensor (30) with the real measurement data.

15. The method of claim 14, wherein, identifying and / or visualizing, by means of the display device (40), a difference between the real measurement data and the simulated measurement data for the user in the augmented reality view (20), and wherein, in response to the visualized difference, iteratively positioning and / or configuring the real sensor (10) in the real environment by means of the augmented reality view (20) by changing, and in particular by shifting and / or rotating, a position of the virtual sensor (30) in the augmented reality view (20) and outputting an updated output parameter.

16. A display device (40) for positioning at least one real sensor (10), in particular a safety sensor or camera for an industrial safety application, in a real environment of an industrial plant by means of an augmented reality view (20) based on the real environment of the industrial plant, wherein The display device (40) is configured to: visualize for a user in an augmented reality view (20) at least one simulated parameter of a virtual sensor (30), wherein the virtual sensor (30) is based on the real sensor (10) to be positioned, wherein the simulated parameter comprises at least one position parameter, wherein the at least one position parameter is indicative of a position and / or orientation of the virtual sensor (30) in the augmented reality view (20), obtain a change of the position parameter, visualize for the user in the augmented reality view (20) the position parameter updated based on the obtained change, and output an output parameter based on the updated position parameter of the virtual sensor (30) and enabling to position the real sensor (10) in the real environment.

17. A system for positioning at least one real sensor (10), in particular a safety sensor or camera for industrial safety applications, in a real environment of an industrial plant by means of an augmented reality view (20) based on the real environment of the industrial plant, wherein The system comprises the display device (40) according to claim 16, and the at least one real sensor (10).

Images