Method and device for testing the function of a system, use of a head-mounted display, head-mounted display, and method and device for visually displaying 3D data

Abstract

The invention relates to a method for testing the function of a system (1). According to the invention, real models (7, 8) of details of a virtual 3D model (39) of the system are created, the spatial position and orientation of said real models being captured by a 3D measuring device (11) during the function testing, wherein said determined orientations and positions can be used to manipulate virtual objects (3, 4) in the 3D model (39) in order to display a visual impression (20) in a virtual reality of handling or manipulating the real models (7, 8).

Description

[0001] Method and device for functional testing of a system, use of a head-mounted display, head-mounted display and method and device for visual representation of 3D data

[0002] The invention relates to a method for testing the functionality of a system.

[0003] It is known from practice that functional tests, particularly on pharmaceutical plants, are carried out on cardboard and / or wooden models before the often complex production is started.

[0004] The invention further relates to a device for testing the functionality of a system.

[0005] For the above-mentioned process, cardboard and / or wooden models are known in practice.

[0006] The invention further relates to a head-mounted display and its use.

[0007] Head-mounted displays are known in practice, for example, as VR (virtual reality) glasses, AR (augmented reality) and MR (mixed reality) glasses. These glasses are used for a variety of purposes, often in the entertainment industry, to visually present virtual, spatially assigned data alone or in conjunction with real scenes to a viewer in order to create a spatial impression. Head-mounted displays are described, for example, in the German edition of Wikipedia. Accordingly, a head-mounted display can be characterized as a visual output device that is worn on the head.Such a device can, for example, be configured to present images either on a screen close to the eye or by projecting them onto the retina in order to supplement (AR, MR) or replace (VR) a viewer's natural visual impression with an artificially generated impression.

[0008] The invention further relates to a method for the visual representation of 3D data and a corresponding device.

[0009] It is known from practice to use such methods and devices in the entertainment industry, in particular with the head-mounted displays already mentioned, in order to present three-dimensional data in an immediately tangible way.

[0010] The invention is based on the object of simplifying the functional testing of complex systems.

[0011] To achieve the stated object, the features of claim 1 are provided according to the invention. In particular, to achieve the stated object, in a method of the type described at the outset, the invention proposes that the system is represented as a virtual 3D model made up of virtual objects, wherein a real model of at least one virtual object is provided, and wherein a virtual body is aligned with the real model at regular intervals using a 3D position measurement, and the at least one virtual object is linked to the virtual body and brought into a desired positional relationship with the virtual body, wherein the link between the virtual body and the at least one virtual object is changed by a user.Thus, the invention enables a haptic experience of virtual modeling, allowing for real-world functional testing without requiring a complete, physical replica of the system under test. This can significantly simplify functional testing, since the entire system does not have to be constructed as a real model.

[0012] In general, the aforementioned connections can refer to only a subset of the degrees of freedom of movement of the respective objects or bodies, or they can require a complete fixation. For example, a desired positional relationship can also mean that only parts of the movement are reproduced (e.g., only the X and Y axes, no rotation). This applies to the hologram as well as to the target shoulder ring (Giove Port).

[0013] For example, the real model can be tilted relative to, say, a glass pane plane, so that when the link is activated, a corresponding virtual object would be pulled out of the (virtual) pane plane. Here, a boundary condition can be formulated that allows the orientation of this object with respect to the virtual body only within certain degrees of freedom and fixes it within the pane's degrees of freedom so that the virtual object, such as a shoulder ring or a glass port, remains in the pane. This way, disturbing visual impressions can be avoided.

[0014] For example, real objects that do not come into contact with a user in a particular test because they would be too far away can be omitted. The concept of activating a link, for example, enables a coupling of the virtual world to the real world, which makes details of the 3D model haptically tangible through appropriately positioned real models. The concept of deactivating a link, for example, makes it possible to exchange virtual objects and thus use a very limited supply of real models multiple times, for example at different locations in an industrial plant, especially when design details are used multiple times.

[0015] The user who activates or deactivates the links can, for example, be a viewer of the (virtual) system, in particular a person who performs a functional check or test of the system, or an assistant who maintains a 3D engine or, more generally, software that implements the invention. A 3D engine, also called a graphics engine, can be characterized, for example, as an integrated or externally stored program code that is responsible for calculating the graphics interface in parallel to the actual program.

[0016] A virtual object can, for example, be characterized as a functional and / or structural part of the 3D model. Examples can be static parts such as shoulder rings or boundary walls of an isolator in a special (pharmaceutical) plant, or movable parts such as doors, particularly of transfer ports or rapid transfer ports ("quick-change system" - RTP for short, also known as alpha-beta port system) or locks, or functional stations such as filling stations, closing stations, or material storage or manipulators. This list is not exhaustive. Other examples may be used to advantage.

[0017] A virtual body can be characterized, for example, as a rigid body that is formed from measuring points that are arranged in a fixed manner relative to one another.

[0018] For example, it can be provided that in order to change the link it is activated (or started) and / or deactivated (or ended). This makes it possible to create or release a spatial coupling between the virtual object and the virtual body in a virtual space. Since the virtual body is coupled to a real model via the 3D position measurement and its changes in position and location are necessarily reproduced in a virtual space, the link can thus - in the case of activation - couple the virtual objects of the 3D model to reality or - in the case of deactivation - separate the virtual objects from reality.

[0019] In an advantageous embodiment, it can be provided that linking the virtual body with the at least one virtual object comprises forcing a desired positional relationship of a position and / or an orientation of the virtual object to the virtual body. In this way, an impression of a virtual object moving along with a haptically perceivable real model can be created. This can be used, for example, to test the feasibility of real work steps on a 3D model. This forcing, particularly if it is time-limited, can also be used as a simple means of transferring a change to the real model, for example an ergonomic improvement, to the virtual object. The forcing can relate to all degrees of freedom of movement or to a subset of the degrees of freedom of movement, particularly to take boundary conditions into account.

[0020] In general, a position of a virtual object or a real model can be described, for example, by three coordinates of a selected point, in particular a center of gravity, midpoint or another distinguished or special point. A location of a virtual object or a real model can be described, for example, by angular information of an orientation in relation to rotations around the selected point to which the position refers, and / or by information on a position of another point on the virtual object or the real model, which can be in a fixed relationship to the selected point. A pose can, for example, be described by a position and an attitude.

[0021] For example, it can be provided that the position and / or orientation of the virtual object is set to the position and / or orientation of the virtual body. This allows for displacement-free following or movement.

[0022] Alternatively or additionally, the forcing can be triggered by executing a request. This allows for the exchange of virtual objects and / or real models during the linking process.

[0023] Alternatively or additionally, the forcing can be performed continuously, for example, recurringly, preferably automatically. This makes it possible, for example, to move the virtual object with the virtual body over a certain period of movement.

[0024] In an advantageous embodiment, it can be provided that, in a setup step, the virtual object to which a virtual body can be linked is defined. Thus, a set of real models can be expanded.

[0025] In an advantageous embodiment, it can be provided that in a setup step, for example, the one already mentioned, it is defined how a virtual object can be linked to a virtual body. This allows a precise alignment of the virtual body to the virtual object to be defined. For example, a position of markers that define a virtual body can be determined on a suitable virtual object. This allows new real models to be integrated subsequently.

[0026] In an advantageous embodiment, it can be provided that a virtual body can be linked to at least two virtual objects. Thus, operation of a complex arrangement comprising several virtual objects can be recreated or simulated by selectively docking the arrangement with various virtual objects to reality, for example, to real models. The docking results, for example, from the fact that the coupling between the virtual body and the real model is fixed.

[0027] In an advantageous embodiment, it can be provided that a link between a virtual body and a virtual object can be activated or started and / or deactivated or terminated independently of a link between another virtual body and another virtual object. It has been found that a link should be deactivated if the real model is to be practically unmoved in order to avoid image processing artifacts.

[0028] In an advantageous embodiment, it can be provided that a number of virtual, in particular linkable, objects is not smaller than a number of virtual bodies. The two numbers can therefore be the same, or the number of virtual objects can be larger, in particular larger than three times, than the number of virtual bodies. This makes it possible to simulate operating processes on complex arrangements such as production lines or insulators without having to set up the entire system. This saves time in preparation, material and space requirements. In addition, changes can be implemented more flexibly and transport of a structure can be carried out with little effort and is therefore not tied to a specific location. This saves costs.

[0029] In an advantageous embodiment, at least two real models can be provided. Thus, processing operations that relate two real models to each other, for example, the opening of a door, can be simulated.

[0030] It can be provided that a corresponding virtual body is realized in each case.

[0031] In an advantageous embodiment, it can be provided that at least two real models are brought into a spatial relationship to one another, which is predetermined by a virtual spatial relationship of at least two virtual objects.

[0032] It can be provided that the at least two virtual objects are linked to at least two virtual bodies that belong to the at least two real models.

[0033] In an advantageous embodiment, at least two real models can be positively guided relative to each other. A non-exhaustive list of examples of positive guidance includes an articulated connection of a door to its frame, for example, in a RTF, or a rail guide of a carriage.

[0034] In an advantageous embodiment, it can be provided that at least two real models have limited relative mobility. Such a limitation can arise, for example, when using an isolator glove in that a hand, to which a real model with markers can be attached, can only be inserted through a shoulder ring to which the glove is attached until the material of the glove is at its maximum tension.

[0035] In an advantageous embodiment, it can be provided that a user's mobility is restricted by at least one real model during use.

[0036] This makes it easy to test whether a movement can be performed in the simulated system. One example is testing the operating range of a shoulder ring used to keep a user away.

[0037] In an advantageous embodiment, it can be provided that the linking is deactivated for a preferably defined or indefinite period of time. Thus, an alignment of a real model, in particular with respect to another real model whose virtual body is already linked, can be carried out in order to align the real model with a virtual world, in particular the virtual 3D model to which the at least one virtual object belongs, in such a way that the other real model remains congruent with this virtual world.

[0038] In this case, it can be provided that deviations between the at least one virtual object and the virtual body are displayed when the link is deactivated. This can be used, for example, to bring a real model into a desired position so that a desired relationship to a virtual object is established.

[0039] In an advantageous embodiment, it can be provided that the link between the virtual body and the at least one virtual object is replaced by another link between the virtual body and another virtual object. This allows a real model to be reused for testing on other virtual objects of the 3D

[0040] Model. Thus, a complete assembly of the system is not necessary. This can save space and time for creating the real models and enable functional testing to be carried out at remote locations or by users who are far apart from each other. It also saves costs for manufacturing and assembling the models. There are also technological advantages; for example, a cross-section can be inserted into the virtual model or a hologram can be displayed for better understanding.

[0041] In an advantageous embodiment, it can be provided that when the link is replaced by another link, the 3D model is subjected to an isometric transformation until the virtual body and the other virtual object are aligned at least within a tolerance range. This enables the user to be moved in the virtual world without the user having to change their location in the real world. This makes it easy to use already constructed real structures for further tests without modifications. Adapting the existing real models to the position and / or orientation of the new virtual objects is then easy to carry out as described above.

[0042] Such an isometric transformation can, for example, involve rotation and / or translation. Thus, the relocation simply corresponds to any change of location in the real world.

[0043] Preferably, only isometric transformations are permitted which retain an orientation, i.e., for example, do not mirror. This blocks changes which have no equivalent in the real world. In an advantageous embodiment, it can be provided that a position of the real model is changed, preferably manually or automatically, until the associated virtual body is aligned with the at least one virtual object or with the other virtual object. This enables an alignment of the real models in such a way that a haptic impression in interaction with the real model coincides with a visual impression when viewing the virtual object.

[0044] For example, a shoulder ring can first be aligned with the 3D model by activating a link to the relevant virtual object.

[0045] Subsequently, another shoulder ring or another part, for example a door or a functional unit to be manipulated, can be modified as a real model in such a way that this real model is aligned with the virtual object corresponding to it and the real model positioned and / or aligned in this way is inserted into the virtual world.

[0046] It can be provided that a link between the virtual body and the virtual object or the other virtual object is subsequently activated.

[0047] This allows the virtual object to subsequently replicate the movement of the real model. This allows a viewer of the virtual world to have the feeling of actually moving or manipulating the virtual objects, as the viewer receives haptic or tactile sensory information that complements the visual sensory information.

[0048] In an advantageous embodiment, it can be provided that several virtual bodies are linked to a respective virtual object of the 3D model, wherein the individual links are changed independently of one another, in particular activated and / or deactivated. Thus, different real models, for example two shoulder rings, can be set independently of one another and / or individual real models can be selected as movable parts of the system, for which it is necessary to carry the virtual object, while other real models can be or remain usable as reference points in the real world to which the virtual world can dock.

[0049] In an advantageous embodiment, it can be provided that an update of the coordinates of the at least one virtual object is output. This can be used, for example, to edit design data of the 3D model. Thus, adjustments and ergonomically and / or process-economically necessary changes to the system can be easily made without having to create a new, complete model in the real world.

[0050] In an advantageous embodiment, the system can be used in the pharmaceutical sector, preferably for filling medications into containers and / or in conjunction with a protected space, preferably an isolator. This allows for easy testing of regulatory requirements and / or ergonomic constraints in workflows.

[0051] Alternatively or additionally, the stated object is achieved by a method for functional testing of a system, wherein the system is represented as a virtual 3D model made up of virtual objects, wherein a real model of at least one virtual object is provided, and wherein a virtual body is aligned with the real model at regular intervals using a 3D position measurement, and the at least one virtual object is linked to the virtual body and brought into a desired positional relationship with the virtual body, wherein a real model corresponding to the at least one virtual object is produced and provided with identifiable features, in particular markers, for a 3D position measurement, and wherein a correspondence between the identifiable features and the at least one virtual object is stored.This makes it easy to create and integrate details of the facility relevant to the tests and with which physical interaction is desired. The identifiable features can easily be used to generate the virtual body to be associated with the object.

[0052] This aspect can be advantageously combined with the previously described aspect. For example, by activating the link, a real model prepared for use by markers can be easily used in the method according to the invention. The markers can, for example, be realized by preferably two- or three-dimensional markers.

[0053] Preferably, the real model is created using an additive process, particularly from CAD data or other data of the virtual object. This allows for the most accurate realization of the 3D model, allowing even details to be experienced haptically. Another advantageous option is to use the real model and directly add markers to it to achieve an even better haptic experience.

[0054] Alternatively, a virtual object can also be derived from a 3D scan of a real model. This means that a prototype or sample from a manufacturer can be used directly without the need for CAD data, and / or the complex 3D printing of a complex object can be avoided. In an advantageous embodiment, it can be provided that the identifiable features are formed at predetermined positions on the real model. This enables quick and easy integration or creation of a correspondence between the virtual body, which can be given by the features, and the virtual object, on which the positions of the features can be noted.

[0055] In an advantageous embodiment, it can be provided that at least one position of the formed features is measured on the real model. This can be done, for example, with a 3D camera. The measurement allows for the application of any desired markers. This can simplify the preparation of the real models for use.

[0056] In an advantageous embodiment, it can be provided that an operator wears a glove and / or a hand tracking device (smartglove, metaglove, motion capture glove, fingertracking device). Alternatively, direct tracking of hands is also possible. The detection of hands is advantageous for the most realistic reproduction of manipulation actions in the virtual world. Insulator or shoulder gloves can also be used, for example, to create a realistic simulation of the resulting physical constraints.

[0057] In this case, it can be provided that a 3D position, in particular a position and a location, of one or more fingers and / or a hand and / or an arm is repeatedly determined. This makes it easy to integrate hands and / or arms with which manipulations are carried out and / or for which collision checks are required, into the virtual world. For example, this can be achieved by detecting a glove, in particular the one already mentioned, and / or a hand tracking device, in particular the one already mentioned. This makes it possible to represent realistic manipulation actions virtually.

[0058] In an advantageous embodiment, it can be provided that the 3D model represents a shoulder ring, to the position of which a real shoulder ring is adjusted, in particular in a preceding setup step and / or wherein an operator puts an arm through the shoulder ring, preferably in a manipulation glove attached to the shoulder ring.

[0059] In an advantageous embodiment, the viewer's recording pose can be defined relative to the shoulder ring. Thus, the viewer's position and position can be used as a reference for displaying the virtual objects.

[0060] In an advantageous embodiment, the at least one virtual object can be a door of a transfer port or a lock. Other details of a system that must be manipulated during use can also be used, for example, air samplers, agar plates, sampling devices, filling stations, pump bodies, hoses, and generally semi-stationary (for example, those whose mobility is restricted by joints or guides) or freely movable parts of the system.

[0061] In general, the real models can be divided into those that are movable with respect to a reference point, for example a boundary of the system and / or with respect to an access, in particular with respect to a shoulder ring, and those that are immobile. In an advantageous embodiment, it can be provided that the 3D model has a further virtual object for which a further real model is provided, wherein the real model is arranged so as to be movable relative to the further real model. In this way, individual real models can be used as reference points for connecting the virtual world and other real models can be used for location-accurate manipulations in a virtual world registered with reality.

[0062] In an advantageous embodiment, the further real model can be at least partially immobile and / or at least partially movable relative to a boundary of the facility. Thus, a boundary can be used as a reference point or reference surface to establish a correspondence between the virtual world and the real world.

[0063] A non-exhaustive list of examples of at least partially movable components of a system that can advantageously be used as a real model includes a shoulder ring and / or a door frame of a transfer port and / or a door, in particular of a transfer port or a lock door, and / or a lock and / or a Petri dish, an air sampler, at least one agar plate, a sampling, a pump body, at least one hose.

[0064] An embodiment of possibly independent inventive quality proposes a method for functional testing of a system, in particular as described above, to achieve the stated object, comprising the following steps: providing CAD data of the system, creating at least one real model for at least part of the CAD data, setting up the at least one real model in a 3D measuring device, displaying a virtual 3D model created from the CAD data by processing at least 3D measurement data from the 3D measuring device. This enables functional testing of a complex system with little material expenditure, supported by haptic impressions of the real model.

[0065] In an advantageous embodiment, it can be provided that a field of view of a head-mounted display is determined, preferably with the 3D measuring device. This enables the embedding of an observer and / or operator in a virtual scene of virtual objects of the CAD model.

[0066] In an advantageous embodiment, the 3D model can be displayed relative to the field of view of a head-mounted display. This enables a realistic viewing of the 3D model from a viewing position.

[0067] In an advantageous embodiment, it can be provided that a change to the at least one real model is automatically reproduced in the 3D model. This means that changes in the real world can be easily reproduced in the virtual world in which the 3D model is defined. A viewer can thus be given the impression that the virtual objects can be changed by - haptically perceivable - changing the associated real model, which corresponds, for example, to the linked virtual body. This makes it possible to carry out functional tests of complex industrial plants such as pharmaceutical plants with minimal use of material, time and space.

[0068] In this case, or generally, it can be provided that modified design data is generated and output from the modified 3D model or modified virtual objects. This allows for the specification of design changes that arose during the functional test.

[0069] In an advantageous embodiment, the 3D measuring device can be transported in a fixed measuring setup before installation. Thus, a device according to the invention can be easily transported to a remote location, for example, for an on-site functional test.

[0070] The object mentioned at the outset is achieved alternatively or additionally by a device for functional testing of a system, wherein the system is present as a virtual 3D model, with a 3D measuring device, at least one real model of a virtual object of the 3D model, a device for automatically integrating a virtual body captured by the 3D measuring device into the 3D model in relation to the real model, a device for automatically moving the virtual object along with the virtual body and a device for visually representing the 3D model, in particular a 3D engine. Means are thus provided for testing a system that can be experienced haptically and represented virtually.

[0071] In an advantageous embodiment, a means for activating and / or deactivating a link between the virtual body and the at least one virtual object can be provided. Thus, an operator can easily determine how the virtual world of the 3D model should be linked to reality.

[0072] In an advantageous embodiment, it can be provided that a head-mounted display is configured to generate a field of view of the 3D model. This allows a natural viewing position to be realized. In an advantageous embodiment, it can be provided that a device for generating a field of view of the 3D model is fed with measured values ​​from the 3D measuring device to a head-mounted display, in particular the one already mentioned. This enables viewing of the 3D model from a human observer position.

[0073] In an advantageous embodiment, a device for isometrically transforming the 3D model relative to the field of view can be provided. This enables a relocation of the 3D model or a virtual change of position of an operator of the system.

[0074] The object mentioned above is achieved alternatively or additionally by using a head-mounted display, in particular VR, XR, and / or AR glasses, and a 3D measuring device that is preferably stationary and / or operates independently of the head-mounted display, for creating a virtual view of a 3D model of a system in the head-mounted display, wherein individual virtual objects correspond to real models captured by the 3D measuring device. This provides a means for a haptically controllable, virtual functional test of a system represented as a 3D model.

[0075] In particular, this can be used for a functional test of a preferably pharmaceutical system, preferably in a method according to the invention, in particular as described above and / or below and / or claimed below, and / or in a device according to the invention, in particular as described above and / or below and / or claimed below. The invention can save considerable costs, time and space here, since pharmaceutical systems, in particular as controlled rooms or in controlled rooms such as RABS (restricted access barrier systems) or isolators, often have large spatial dimensions. This makes a traditional structure made of cardboard and / or wood complex.

[0076] The invention preferably uses a head-mounted display, for example VR, AR and / or XR glasses, with at least one marker, in particular with more than two markers for a preferably extrinsic determination of a recording pose. Three markers are often sufficient to clearly determine a position and orientation of a real model. However, it is advantageous to attach more than three markers, particularly in the case of more complex real models. This makes it possible to easily embed a field of view of the head-mounted display into a virtual space of the virtual 3D model using a 3D measurement. This gives a viewer a realistic impression of the system from their perspective.

[0077] The marker can be actively formed. This allows for individual detection of each head-mounted display. This allows devices to be easily replaced without the need for re-learning and / or re-aligning the system. It is also possible to consistently define a center as a reference for multiple codings. This consistent center can then be used to map the virtual body to a field of view and ultimately for embedding without re-learning for each marker.

[0078] Alternatively or additionally, the marker can also be designed to be passive. This makes it possible to increase the service life of the head-mounted display, as fewer resources are used during operation. Markers can also be attached in different positions on different head-mounted displays, thus enabling individual recording. A standardized fastening kit for connecting the glasses and markers allows embedding without any training or with very little effort. Another advantage of passive markers is their low weight, which, for example, has a positive effect on the wearing comfort of the head-mounted display.

[0079] The object stated at the outset is achieved alternatively or additionally according to the invention by a method for the visual representation of 3D data, wherein a field of view of a head-mounted display (display attached to the head), in particular VR and / or XR and / or AR glasses, is intrinsically determined on a time-recurring basis and information that moves along with the field of view is represented in the head-mounted display, wherein a recording pose of the head-mounted display that specifies the field of view is determined, preferably on a time-recurring basis, and compared with the field of view. In this way, a spatially precise embedding of the field of view in a virtual world is feasible with the available computing capacities of a head-mounted display.The invention has the advantage that a spatial relationship of the observer can also be determined to such virtual objects that are not currently in the field of view of the observer, defined by a recording pose of the head-mounted display.

[0080] An intrinsic determination can be characterized, for example, by the fact that associated sensors are moved and / or aligned in the direction of the field of view and / or that a calculation of the field of view can be carried out using on-board resources of the head-mounted display.

[0081] The field of view of the head-mounted display can, for example, be defined by the field of view of a viewer whose head position corresponds to a current recording pose of the head-mounted display when the display is in the operating position. The recording pose can, for example, refer to the position and orientation of a forward direction of the head-mounted display.

[0082] This method can be designed or carried out, for example, as part of a method according to the invention, in particular as described herein and / or claimed below, for testing the functionality of a system.

[0083] In an advantageous embodiment, the recording pose can be determined using a 3D measuring device that is designed independently of the head-mounted display and / or is stationary. This enables continuous and all-round detection of the head-mounted display and uninterrupted embedding of the field of view.

[0084] The 3D measuring device can, for example, comprise at least one or more cameras. In general, it can be said that the use of multiple cameras can, on the one hand, improve measurement accuracy and, on the other hand, is less susceptible to details being obscured by other details.

[0085] 3D measuring devices are known for spatially recording the position and attitude of real models. One possibility is to create two-dimensional images of the real models from different recording poses, to identify the respective models in these images, for example using attached markers, and then to solve a system of equations which describes these images as recordings of a common real model, with the shape, for example the position of the individual markers, as an unknown and the image positions being treated as input variables. Alternatives to this include the use of structured light, the pattern of which on the real models allows conclusions to be drawn about the attitude and position of the real models. Methods which use time-of-flight measurements of signals are also known.

[0086] In an advantageous embodiment, the recording pose can be determined using a measuring device that moves along with the image and / or is independent of the determination of the field of view. This reduces the equipment required for the device according to the invention.

[0087] In an advantageous embodiment, the intrinsic determination can be carried out using at least one moving sensor, in particular a camera and / or a motion and / or acceleration and / or position sensor. Thus, known systems for determining the field of vision and its changes during head movement can be used.

[0088] In an advantageous embodiment, the recording pose can be measured using active markers on the head-mounted display. Active markers offer the advantage of better distinguishability and easy switching of identifications.

[0089] In an advantageous embodiment, the recording pose can be measured using passive markers on the head-mounted display. Passive markers help save energy during operation and thus extend the device's service life.

[0090] In an advantageous embodiment, it can be provided that the recording pose is measured by means of a stationary measuring device, in particular by means of stationary cameras.

[0091] In an advantageous embodiment, it can be provided that the intrinsic determination of the recording pose is measured by means of moving cameras of the head-mounted display.

[0092] The object stated at the outset is achieved according to the invention alternatively or additionally by a method for the visual representation of a system and / or a method as part of a method described above and / or claimed below, wherein a field of view of a head-mounted display (display attached to the head), in particular VR and / or XR and / or AR glasses, is intrinsically determined over time and wherein an air flow is calculated and visually displayed as preferably moving 3D data in the head-mounted display. The advantage here is that the influence of a work process on an air flow is immediately apparent and / or controllable.

[0093] It is known to use air currents in controlled environments to prevent the transfer of contaminants to areas requiring special protection. The invention makes it possible to monitor these, since air currents are also influenced, for example, by mobile functional units and / or a user.

[0094] In an advantageous embodiment, it can be provided that the head-mounted display is connected to a preferably stationary processing unit for transmitting measurement data of the recording pose and / or image data for the head-mounted display. This enables computing routines to be outsourced to stationary units with greater capacity. The data transmission can be wireless or wired, for example. In an advantageous embodiment, it can be provided that the 3D data comprise a 3D model of a system and / or wherein the 3D data also comprise AR metadata relating to components of a system, in particular the system already mentioned. The use of a 3D model enables a realistic visual representation of a system in virtual space.The use of AR metadata makes it possible to display or display additional data beyond the mere image content, such as warnings, messages, or instructions. This allows, for example, a simple switch to a viewer's language or a change in display depending on the system's operating status. Air flow data can also be displayed as 3D data, particularly in the form of streamlines.

[0095] In an advantageous embodiment, it can be provided that the head-mounted display, in particular a head-mounted display according to the invention, for example as described above and / or claimed below, generates a superposition of a real field of view with a virtual representation of the 3D data. MX or AR applications are thus possible.

[0096] Alternatively or additionally, it can be provided that a real environment is shielded with the head-mounted display, in particular a head-mounted display according to the invention, for example as described above and / or claimed below. VR applications are thus possible.

[0097] In addition, one of the described methods can provide for a real model, for example one of the real models already mentioned, to be adjusted by a motor. This means that an adjustment to a position and / or orientation of a virtual object can be carried out more easily and / or more precisely. Alternatively or additionally, one of the described methods can provide for a real model, for example one of the real models already mentioned, to be adjusted preferably by a motor and / or automatically until a preferably automatically detected deviation in a position and / or orientation of a virtual body from a corresponding virtual object lies within a tolerance range. This means that an automatic integration of a real model into the method can be achieved.

[0098] These two designs can be used advantageously together or individually, for example, in the shoulder rings described in more detail below.

[0099] Alternatively or additionally, one of the described methods may provide for the generation of a virtual light beam, in particular with an automatic check to determine whether the virtual light beam is interrupted. Thus, a functional test can be performed even more closely to reality.

[0100] To achieve the object mentioned above, the invention further provides a device for the visual representation of 3D data, comprising a head-mounted display which is configured to determine a moving field of view, a device for determining a recording pose of the head-mounted display, and a device for comparing the recording pose with the field of view. Thus, a spatially accurate embedding of a field of view in a virtual world of virtual objects can be realized with low computational requirements for the head-mounted display. This can, for example, be used for the spatially correct display of

[0101] Information and messages can be made available.

[0102] The device can be designed, for example, as part of a device according to the invention for functional testing of a system, for example as described above and / or claimed below.

[0103] An advantageous embodiment can be implemented with a 3D engine for visually displaying the 3D data in the field of view. Known algorithms for controlling visual displays can thus be used.

[0104] An advantageous embodiment can be designed with a preferably stationary 3D measuring device for determining the recording pose. A stationary 3D measuring device can be designed with a larger spatial coverage, for example, by spatially distributing corresponding cameras.

[0105] An advantageous embodiment can be provided with a device for generating a virtual space for a virtual body captured with a 3D measuring device, for example, the one already mentioned. This allows the capture of real models and their changes. This can be used, for example, to connect a virtual world to a real world. An example of a 3D measuring device is the combination of the cameras Prime x 13 and / or Prime x 13W from OptiTrack (NaturalPoint , Inc . , P . O . Box 2317 , Corvallis , OR 97339 ) with the motion capture software Motive from OptiTrack .

[0106] An advantageous embodiment can alternatively or additionally be provided with a device for generating a virtual space for the visual representation of the 3D data. Thus, a scene can be provided for the generation of virtual visual impressions.

[0107] An advantageous embodiment can alternatively or additionally be provided with a device for establishing a correspondence between two virtual spaces, preferably the virtual spaces already mentioned, in particular for embedding the virtual bodies in the virtual space for the virtual representation. Thus, an easily manageable means is created for modifying virtual objects by manipulating real, corresponding models.

[0108] An advantageous embodiment can be designed with a device for embedding a field of view of the head-mounted display into the virtual space for the virtual representation. This enables a true-to-life visual representation of the virtual world through the eyes of a viewer, as if the viewer were actually visually perceiving the virtual world.

[0109] An advantageous embodiment can be provided with a device for calculating air flow, in particular for a visual representation of the air flow. Thus, air flows, in particular as streamlines, can be visualized.

[0110] An advantageous embodiment can be provided with means for carrying out a method according to the invention, in particular as described above and / or claimed below. Thus, a method for implementing the described methods is provided.

[0111] In an advantageous embodiment of one of the described devices, it can be provided that the device is designed with a device for motorized adjustment of at least one real model. This enables precise and / or automatic and / or remotely triggered adjustment. Alternatively or additionally, in an advantageous embodiment of one of the described devices, it can be provided that the device is designed with a device for preferably automatic determination of a deviation in a position and / or orientation of a virtual body from a corresponding virtual object. In this way, integration can be supported by computer assistance.

[0112] Alternatively or additionally, in an advantageous embodiment of one of the described devices, the device can be provided with a device for motorized adjustment of at least one real model until a preferably automatically detected deviation in a position and / or orientation of a virtual body from a corresponding virtual object lies within a tolerance range. Thus, a fully automatic or semi-automatic integration of real models is achievable.

[0113] Alternatively or additionally, in an advantageous embodiment of one of the described devices, the device can be equipped with a device for a collision check for a virtual light beam. This can enable a simulation of a light barrier, for example, a light grid. For example, the associated light barrier modules can be stored as virtual objects on the 3D model, and the light beam is automatically generated and monitored based on the location and position of the light barrier modules.

[0114] The invention will now be described in more detail with reference to exemplary embodiments, but is not limited to the exemplary embodiments. Further exemplary embodiments result from combining the features of one or more claims with one another and / or with one or more features of the exemplary embodiments.

[0115] It shows :

[0116] Figure 1 is a highly schematic representation of a device for functional testing of a system with a head-mounted display and a device for the visual display of 3D data,

[0117] Figure 2 is a more realistic single representation of a real model of the device according to Figure 1 with two adjustable shoulder rings and a 3D measuring device,

[0118] Figure 3 shows a further realistic representation of the arrangement according to Figure 2 in a side view of a glove,

[0119] Figure 4 is a further more realistic detailed representation of a real model of the device according to Figure 1 with a movable door,

[0120] Figure 5 shows an adaptation of a virtual object of a 3D model of the device according to Figure 1 to achieve a desired positional relationship and to establish a connection,

[0121] Figure 6 shows a modification of a real model in a device according to Figure 1 to obtain a desired positional relationship with respect to an associated virtual object and to establish a link,

[0122] Figure 7 shows a device for the visual representation of 3D data, Figure 8 shows a schematic representation of a 3D model in different orientations for a virtual change of location of a viewer,

[0123] Figure 9 shows a detailed representation of a 3D model of a pharmaceutical plant in plan view and

[0124] Figure 10 is a schematic representation of a real room and two virtual rooms to explain the invention.

[0125] Figure 1 shows a device, designated as a whole by 1, for testing the functionality of a system.

[0126] A virtual 3D model 2 of a system 1 is composed of virtual objects 3, 4 and provided in a first virtual space 5.

[0127] The virtual 3D model 2 is derived from CAD data of an industrial plant (not shown further).

[0128] This system is shown only very schematically with the virtual objects 3 and 4 in order to explain the functional principle of the method according to the invention.

[0129] In fact, the system comprises components. A preferred application of the invention provides for the system to be a pharmaceutical system, which may be designed, for example, in a protected space or an isolator for carrying out specific processes, such as transferring / filling medications or assembling dispensing devices for medicinal preparations.

[0130] The first virtual space 5 contains a large number of virtual objects 3, 4. There may be further virtual objects 63. Some of these virtual objects 3, 4 are represented in a real

[0131] Corresponding real models 7 and 8 are set up in room 6. No real models are set up for the other virtual objects 63.

[0132] These real models 7 , 8 are attached to special stands 9 so that they remain in a desired position in the real space 6 .

[0133] These stands 9 are not part of the virtual 3D model 2, since in it the virtual objects 3 and 4 are mounted on other structural details, such as glass walls, boundary walls, or tables. However, these glass walls, boundary walls, and tables are not present in the real space 6.

[0134] The real models 7 and 8 are also equipped with additional markers 10 which have no equivalent in the virtual objects 3 and 4.

[0135] These markers 10 are intended for position and orientation detection by a 3D measuring device 11 that is set up in the real space 6. The 3D measuring device 11 detects these markers 10 and forms their position in a second virtual space 12. In this second virtual space 12, therefore, the virtual objects 3 and 4 are not present, but only the positions of the markers 10, which are combined to form virtual bodies 13, 14, depending on which real models 7, 8 they are attached to.

[0136] A correspondence 15 is established between the first virtual space 5 and the second virtual space 12 , which connects spatial points of the first virtual space 5 with corresponding spatial points in the second virtual space 12 and vice versa .

[0137] In the real room 6, a head-mounted display 16 is also arranged, for example on the head of a user person (not shown).

[0138] This head-mounted display 16, for example a VR headset, is not necessarily represented in the virtual 3D model 2, but also has markers 17 in order to detect its location and position in the real space 6 and to display it in the second virtual space 12.

[0139] In other words, in the second virtual space 12 there exists a virtual body 18 which represents the head-mounted display 16 via the markers 17.

[0140] The head-mounted display 16 generates, as is known per se, a field of view 19 for generating a visual 3D impression for the user. This visual impression 20 is calculated from the measured attitude and position, i.e., the recording pose, of the head-mounted display 16 and the attitude and position of the virtual objects 3 and 4 using the aforementioned correspondence 15.

[0141] For the visual impression 20, the virtual objects 3 and 4 are used, so that, for example, the stand 9 and also the markers 10, 17 are not displayed.

[0142] The field of view 19 of the head-mounted display 16 is defined by a wearing position of the head-mounted display 16 on the head of a user and their natural field of view.

[0143] To generate the visual impression 20, the field of view data 21, the position and attitude data 22 from the second virtual space 12 and the object data 23 from the first virtual space 5 are schematically processed together in a manner known per se in Figure 1.

[0144] To determine the field of view 19, the head-mounted display

[0145] 16 may be equipped with its own sensors 46, not shown here.

[0146] For example, the head-mounted display 16 can be equipped with a number of cameras that capture the field of vision of a user of the head-mounted display 16 and calculate a position and location and / or a change in these values ​​based on spatial features such as edges and corners and the like. For this purpose, it is technically known to equip the head-mounted display 16 with the appropriate computing capacity itself, so that no external computing capacity is required.

[0147] Since the real models 7 and 8 are set up in the real room 6, a user in the real room 6 can feel these real models 7 and 8, even if the head-mounted display 16 is of the VR glasses type and thus prevents the view of the real models 7, 8.

[0148] In order to achieve a correspondence between this haptic impression and the visual impression 20, the real models 7 and 8 are initially aligned with the virtual objects 3 and 4 assigned to them. The method according to the invention now makes it possible to activate a link 24 between the virtual objects 3 and 4 on the one hand and the corresponding virtual bodies 13, 14 on the other hand. This link 24 has the consequence that the virtual object 3, 4, in the example the virtual object 4, is aligned with the virtual body 13, 14, in the example the virtual

[0149] Body 14 , is moved along with the correspondence 15 .

[0150] Accordingly, if the real model 8 is changed by a movement 25, this movement 25 is mapped in the second virtual space 12 by the 3D measuring device 11. Direct tracking can also be modified depending on the situation so that only certain axes follow and certain axes are "fixed," for example, to fulfill boundary conditions, as already mentioned.

[0151] The link 24 now causes the virtual object 4 to move equally in the first virtual space 5 according to the correspondence 15.

[0152] This results in an apparent movement 26 in the visual impression 20 .

[0153] For the user who generates the movement 25 by manipulating the real model 8, the impression thus arises that, due to the manipulation, the virtual object 4 executes an apparent movement 26.

[0154] If the link 24 is deactivated by the user or another person, manipulation of the real model 8 does not result in a change in the visual impression 20. This can be decided individually for the objects; it does not necessarily have to be the case everywhere.

[0155] Also arranged in the real space 6 is a head-mounted tracking device 27. The aforementioned user wears this head-mounted tracking device 27 like a glove in order to perform the aforementioned manipulation on the real models 7, 8. Markers 28 on the head-mounted tracking device 27 are also represented by the 3D measuring device 11 as a virtual body 29 in the second virtual space 12, so that a hand replica 30 appears in the visual impression 20.

[0156] The head-mounted display 16 can be connected wirelessly or by cable to a preferably stationary processing unit for transmitting measurement data of the recording pose and / or image data for the visual impression 20. A 3D engine (not shown in detail) is used to generate the visual impression 20. This forms a device for generating a field of view of the 3D model 39. A device for isometrically transforming the 3D model 39 relative to the field of view 19 operates in the first virtual space 6, as explained in more detail below with reference to Fig. 8.

[0157] Figure 2 shows, in a somewhat more realistic representation, the real model 7 from Figure 1 in a view from the front, and Figure 3 shows this arrangement from the side.

[0158] The 3D measuring device 11 is only shown in detail in both illustrations.

[0159] In fact, in the real space 6, typically three or even more than five cameras 31 are arranged on a special support structure 32.

[0160] The position and location of this camera 31 is precisely known.

[0161] To facilitate on-site assembly, the fully assembled support structure 32 can be transported to the site in a container or packaged as a whole. The invention offers the advantage that transporting a model of the system is not necessary. Rather, the assembled support structure 32 can be used as a mobile measurement setup.

[0162] The actual model 7 has two shoulder rings 33, 34, as are known on insulators for attaching insulator gloves 49. The shoulder rings 33, 34 are often formed in glass walls in the system. The position of the shoulder rings 33, 34 thus clarifies a position of a boundary of the system, for example, a protected or controlled space. These shoulder rings 33, 34 are variable in their position on the stand 9 horizontally and vertically.

[0163] The hand tracking device 27 is connected to one of the shoulder rings 33, 34 to form an isolator glove 49.

[0164] The respective other shoulder ring 33, 34 can similarly have a hand tracking device 27 for another hand of the user. During use, the user thus stands in front of the shoulder rings 33, 34 and reaches into one of these shoulder rings 33, 34 with both arms in order to operate the respective hand tracking device 27 with their hands.

[0165] In further embodiments, instead of the hand tracking device 27, the location and position or shape of the user's hand is detected using optical recognition algorithms.

[0166] The hand shape can also be detected using special sensors in the fingers 35, which will not be discussed further here.

[0167] For example, hand tracking devices 27 in the form of METAGLOVES “Quantum” from Manus, Floor 9, Kennedyplein 200, NL-5611 ZT Eindhoven are known and usable.

[0168] Figure 4 shows a more realistic representation of the real model 8 from Figure 1 .

[0169] It can be seen that the real model 8 consists of a moving part 36 and a stationary part 37.

[0170] The illustration shows an example of a door 47 , which can be used on a wall of an isolator, for example as a transfer port 48 or as a rapid transfer port or in a lock. The details of this door 47 are important for the explanation of the

[0171] Invention is irrelevant, the only important thing is that this transfer port 48 has a movable part 36 and a stationary part 37.

[0172] This movable part 36 can be gripped and opened by the user in order to carry out the movement 25 mentioned in Figure 1.

[0173] This results in the visual impression 20 representing a change in the associated multi-part virtual object 4, which corresponds to the opening of a door 47.

[0174] The user can thus, for example, check whether he or she can reach and operate a door 47 in an isolator through the shoulder rings 33, 34.

[0175] Figure 5 shows the arrangement of the already mentioned linkage 24 using the shoulder rings 33, 34 as an example.

[0176] This adjustment is made before the user begins the scheduled tests in order to exactly align the shoulder rings 33, 34 with their virtual counterparts.

[0177] Shown in the front row is the real model 7 and behind it is shown in dashed lines a corresponding virtual body 13 and shown in solid lines the corresponding virtual object 3 .

[0178] This representation is chosen for simplification. In fact, the real model 7, the virtual object 3, and the virtual body 13 are located in different spaces 6, 12, 5.

[0179] The projection lines 38 between the real model 7 and the virtual body 13 are intended to symbolize the 3D measurement via the 3D measuring device 11.

[0180] When a request is made, the virtual object 3 is brought to a position which corresponds to the position of the virtual body 13 via the correspondence 15.

[0181] In order to support this process, it can be provided that both the virtual body 13 and the virtual object 3 are reproduced in the visual impression 20.

[0182] For the right shoulder ring 34 this process has already been completed in Figure 5.

[0183] By activating the above-mentioned request, a desired positional relationship between the position and the attitude of the virtual object 3 to the virtual body 13 is enforced.

[0184] In Figure 6, the real model 7 is shown in the foreground and the corresponding virtual object 3 in the background.

[0185] The projection lines 38 again illustrate the effect of the 3D measuring device 11 .

[0186] To simplify the illustration, the virtual body 13 is not shown.

[0187] In the visual impression 20, the user person now sees that the virtual object 3 and the virtual body 13, which is also displayed, do not lie on top of each other.

[0188] The user or an assistant can now modify the stand 9 such that the virtual body 13 lies over the virtual object 3. The link 24 can then be activated. Compared to the procedure according to Figure 5, this procedure has the effect that the virtual object 3 is not modified by the activation of the link 24 and, in particular, remains unchanged with respect to other virtual objects 4.

[0189] Since the shoulder rings 33, 34 remain stationary, it is not necessary to activate the link 24, but it can be permanently deactivated.

[0190] The formation of individual links 24 between the virtual body 14 and the virtual object 4 allows the individual links 24 to be permanently deactivated for individual virtual bodies 13, 14. This makes it possible to perform such adjustments independently of the other virtual objects 3, 4.

[0191] The real model 7 can be adjusted by motor, for example until an automatically detected deviation 64 in a position and / or location of a virtual body 13, 14 (not shown, cf. Fig. 5) from a corresponding virtual object 3 lies within a tolerance range.

[0192] For this purpose, the device shown has a device for a motorized adjustment of at least one real model 7, 8 and a device for a preferably automatic determination of a deviation 64 in a position and / or orientation of a virtual body 13, 14 from a corresponding virtual object 3, 4 and a device for a motorized adjustment of at least one real model 7, 8 until an automatically and / or manually detected deviation 64 in a position and / or orientation of a virtual body 13, 14 from a corresponding virtual object 3, 4 lies within a tolerance range.

[0193] Figure 8 shows a schematic representation of a 3D model 39 of an industrial plant, for example, a pharmaceutical plant. The 3D model has a wall 40 in which, for example, three access points 41, 42, 43 are arranged.

[0194] Each of these accesses 41, 42, 43 can, for example, contain virtual equivalents of pairs of shoulder rings 33, 34.

[0195] The left illustration of Figure 8 shows a position in which a real model 7 of the access is brought into alignment with the virtual equivalent or into a desired positional relationship using a method according to Figure 6 or 5.

[0196] This state serves to test and verify the accessibility of the system leading to the 3D model 39 through this access 41.

[0197] If a different access 42, 43 is now to be tested, the 3D model 39 can be isometrically transformed, particularly relative to the virtual body 13, such that the further access 42 can be brought into alignment with the real model 7 or its virtual body 13. This changes the position of the 3D model 39 in the first virtual space 6 or the correspondence 15 between the first virtual space 6 and the second virtual space 12. The system can now be tested through this access 42. This situation is depicted in the middle part of Fig. 8.

[0198] In the right part of the picture, another real model 44 is shown, which can be set up in a separate real room 6 or in the same real room 6.

[0199] For this additional real model 44, there is thus an additional virtual body 14, which is generated either via the same 3D measuring device 11 or via an additional 3D measuring device 11 if the additional real model 44 is set up in a different (real) space. This makes it possible for two users to test the system simultaneously, without these users having to be in any real spatial relationship to each other.

[0200] These two users can, for example, perform a (virtual) handshake or check for handovers or mutual obstructions.

[0201] When changing from the left situation to the middle situation in Figure 8, the link 24 between the access 41 as a virtual object and the virtual body 13, 14 of the real model 7 is deactivated in order to replace it with a link 24 between the same virtual body 13 and another access 42 as a virtual object. When establishing the link 24, a tolerance range is specified within which a link 24 is accepted without the virtual object 3 having to be modified.

[0202] The users can now, for example, check the filling of medication in the protected space, here an isolator.

[0203] With the discussed structure, a method for functional testing of a system which is composed of the virtual 3D model 39 with a large number of virtual objects 3, 4 can be carried out, wherein a real model 7, 8 which is as realistic as possible is provided for each selected virtual object 3, 4. For example, these real models 7, 8 are created as a 3D print from the virtual objects 3, 4. The similarity should be such that an optical impression when touching the real model 7, 8 corresponds to a visual impression when viewing the virtual object 3, 4. The 3D measuring device 11 now repeatedly carries out a measurement of the real models 7, 8 in order to align an associated virtual body 13, 14 accordingly in the second space 12.

[0204] Here, the real models 7, 8 are provided with markers 10 to enable identification based on features. To enable the virtual objects 3, 4 to be moved along, provided a link 24 is activated, correspondences 15 are stored between these identified features, i.e., the markers 10, and the virtual objects 3, 4, so that virtual bodies 13, 14 can be moved with the virtual objects 3, 4.

[0205] The virtual objects 3, 4 can contain predetermined positions, for example, drill holes, where the markers 10 are applied. This makes it easy to arrange the markers 10 at positions on the real models 7 and 8, so that the virtual body 13, 14 can be easily located and aligned with respect to the virtual object 3, 4.

[0206] Alternatively, the markers 10 can also be applied arbitrarily to the real models 7, 8, and a position of the formed features, in particular the markers 10, can then be measured on the real model 7, 8.

[0207] The method for functional testing of a system 1 can therefore begin by first providing CAD data of the system by appropriate design, by processing these CAD data into a 3D model 39, by generating real models 7, 8 for this 3D model 39 or for part of the CAD data for selected details, in particular with 3D printing or an alternative manufacturing process, by setting up these real models 7, 8 in a 3D measuring device 11, and by displaying the 3D model 39 from the viewer's position in the visual impression 20.

[0208] If changes to the 3D model 39 are necessary during the test, for example, according to the procedure in Figure 5, these changes to the 3D model 39 can be output as modified design data at the end of the test. This can be the case, for example, if it turns out that certain details of the 3D model 39 are ergonomically unfavorable and therefore need to be redesigned.

[0209] During testing, the users can now perform any manipulations on the real models 7 , 8 in order to check their effects on the visual impression 20 at the level of the virtual objects 3 , 4 .

[0210] This is made possible by the link 24, which forces the virtual objects 3, 4 to be moved along with the associated virtual bodies 13, 14. In the exemplary embodiment presented, this is achieved with a device for automatically moving the virtual objects 3, 4 along with the associated virtual bodies 13, 14 and a 3D engine as a device for visually displaying the 3D model 39.

[0211] Figure 7 shows a head-mounted display 16 in a schematic representation with a 3D measuring device 11. This structure can be used in the structure according to Figure 1, but can also be operated independently of the structure according to Figure 1.

[0212] In this head-mounted display 16, the position and orientation of the field of view 19 is determined repeatedly over time, with moving information, for example virtual objects 3, 4 in the manner described or other location-related information such as warnings or work instructions being displayed, for example in the visual impression 20.

[0213] In this case, the field of view 19 is determined using intrinsic, i.e., integrated, means of the head-mounted display 16. This means that this field of view 19 can only be created with reference to real models 7, 8 that lie within the field of view 19.

[0214] In order to also enable alignment of the head-mounted display 16 with respect to the other real models outside the field of view 19, the 3D measuring device extrinsically determines a location and position of the head-mounted display 16 using markers 10. This information can then be associated with the field of view 19 in order to enable precise and almost uninterrupted or completely uninterrupted detection of a location and position of the head-mounted display 16 and thus a continuous visual experience in the visual impression 20.

[0215] It is known that controlled air flows are often used in isolators and other controlled environments in order to be able to remove potentially occurring contaminants in a controlled manner and to keep sensitive areas free of contamination.

[0216] The present invention now makes it possible to calculate such air flows as streamlines 45 and to represent their change, particularly during movements 25, in the visual impression 20.

[0217] Figure 9 shows another virtual 3D model 39 for use in the invention. It is a pharmaceutical system with a wall 40, accesses 41, 42, 43, 50, measuring stations 51, for example for measuring with agar plates and / or by means of particle measurement, a feed 52 from an automation, a discharge 53 to another automation, light barrier modules 54 with light beams 55 for monitoring the accesses 41, 42, 43, 50 for unexpected and / or unauthorized access, a sorting pot for the correct positional provision of stoppers or other components of packaging or medical dispensing devices, a material supply 57 for the sorting pot 56, which can be refilled, for example, via a transfer port 48, a transport and processing area 58, for example for the processing (filling and closing) of containers (vials, etc.), a filling station 59 for filling containers, a separating station 60 for separating bundles of containers fed through the feed 52, a placing station 61 for closing the containers with the aforementioned plugs, and a control station 62 for checking the filled and closed containers.

[0218] Some of the components require haptic contact with a user during a functional test and are therefore used as a real model 7, 8. For example, this is not necessary for the sorting pot 56—here, a virtual object 63 is sufficient. However, for the filling station 59 and the measuring stations 51, it is advantageous to use a real model 7, 8. This can be an actual product instead of the 3D printing described above. This can be easier for complex structures.

[0219] The light barrier modules 54 can also be present as real models 7, 8, but for example, without any function. In the virtual world, the light beam 55 is simulated, and it is checked whether a user virtually interrupts the light beam 55 and, if so, generates a signal.

[0220] Figure 10 shows another virtual 3D model 39 for use in the invention. Components and functional units that are functionally and / or structurally similar or identical to previous embodiments are designated by the same reference numerals and are not described separately. The statements regarding the previous embodiments therefore apply accordingly.

[0221] Figure 10 additionally shows the movement of the virtual body 13, 14 based on the 3D measurements 68 on the real models 7, 8.

[0222] In a setup step, it is defined with which virtual object 3, 4, 63 a virtual body 13, 14 can be linked, indicated here by a matching contour 69 with one or more shapes of the virtual objects 3, 4, 63. The virtual body 14 can thus be linked to at least two virtual objects 4, 63.

[0223] No virtual body can be linked to the virtual objects 65.

[0224] The setup step further defines how a virtual object 3, 4, 63 can be linked to a virtual body 13, 14. This is expressed by a spatial relationship of the contours 69 to the positions of the markers 10, which correspond to the positions of the markers on the real model 7, 8.

[0225] The link 66 can be deactivated after setup to activate a link 67 to the virtual object 63. From the moment this link is changed, the virtual object 63 is moved along with the real model 8.

[0226] The link 24 between a virtual body 13 and the virtual object 3 can be activated or terminated independently of the link 66 between another virtual body 14 and another virtual object 4, 63 (or also the optionally activatable link 67). The number of virtual, in particular linkable, objects 3, 4, 63, 65 is greater than the number of virtual bodies 13, 14.

[0227] At least two real models 7, 8 are provided, each of which has an associated virtual body 13, 14 realized.

[0228] The at least two real models 7, 8 are spatially related to one another, which is defined by a virtual spatial relationship between at least two virtual objects 3, 4, namely via the additional unlinked virtual objects 65. This spatial relationship is mediated via the at least two virtual bodies 13, 14 belonging to the at least two real models 7, 8.

[0229] The at least two real models 7, 8 can be guided relative to each other or absolutely forced or can be movable relative to each other to a limited extent.

[0230] In the case of forced guidance or limitation, a user is restricted in his or her mobility during use by at least one real model 7, 8.

[0231] In a method for functional testing of a system 1, it is thus proposed to create real models 7, 8 from details of a virtual 3D model 39 of the system, the spatial position and location of which is recorded with a 3D measuring device 11 during the functional test, wherein these specific locations and positions can be used to carry along virtual objects 3, 4 of the 3D model 39 in order to represent a visual impression 20 in a virtual reality for carrying along for manipulation of the real models 7, 8.List of reference symbols Device for testing the function of a system Virtual 3D model Virtual object Virtual object First virtual space Real space Real model Real model Stand Marker 3D measuring device Second virtual space Virtual body Virtual body Correspondence Head-mounted display Marker Virtual body Field of view Visual impression Field of view data Position and attitude data Object data Link Movement Apparent movement Hand tracking device Marker Virtual body Hand replica Camera Support structure Shoulder ring Shoulder ring Finger 36 movable part.

[0232] 37 stationary part

[0233] 38 Projection line

[0234] 39 3D model

[0235] 40 wall

[0236] 41 Access

[0237] 42 Access

[0238] 43 Access

[0239] 44 more real models

[0240] 45 Streamline

[0241] 46 Sensor

[0242] 47 Door

[0243] 48 Transport

[0244] 49 I solator glove

[0245] 50 additional access

[0246] 51 measuring stations

[0247] 52 Feed

[0248] 53 Transfer

[0249] 54 light barrier module

[0250] 55 light beam

[0251] 56 sorting pot

[0252] 57 Material stock

[0253] 58 Transport and processing area

[0254] 59 filling station

[0255] 60 Separation Station

[0256] 61 On set zs tat ion

[0257] 62 Control Station

[0258] 63 additional virtual object

[0259] 64 Deviation

[0260] 65 additional virtual object

[0261] 66 Link

[0262] 67 Link

[0263] 68 3D measurement

Claims

Claims 1. Method for functional testing of a system (1), wherein the system is represented as a virtual 3D model (2) made up of virtual objects (3, 4), wherein a real model (7, 8) is provided of at least one virtual object (3, 4), and wherein a virtual body (13, 14) is aligned with the real model (7, 8) using a 3D position measurement at regular intervals, and the at least one virtual object (3, 4) is linked to the virtual body (13, 14) and brought into a desired positional relationship with the virtual body (13, 14), wherein the link (24) of the virtual body (13, 14) with the at least one virtual object (3, 4) is changed by a user, in particular activated or started and / or deactivated or ended.

2. Method according to claim 1, characterized in that the linking (24) of the virtual body (13, 14) with the at least one virtual object (3, 4) comprises forcing a desired positional relationship of a position and / or an attitude of the virtual object (3, 4) to the virtual body (13, 14), in particular on its position and / or attitude and / or by activating a request and / or permanently.

3. Method according to one of the preceding claims, characterized in that in a setup step it is defined with which virtual object (3, 4) a virtual body (13, 14) can be linked and / or that in one or the setup step it is defined how a virtual object (3, 4) can be linked to a virtual body (13, 14).

4. Method according to one of the preceding claims, characterized in that a linkability of a virtual body (13, 14) with at least two virtual objects (3, 4) is established.

5. Method according to one of the preceding claims, characterized in that a link between a virtual body (13, 14) and a virtual object (3, 4) can be activated or started and / or deactivated or terminated independently of a link between a further virtual body (13, 14) and a further virtual object (3, 4).

6. Method according to one of the preceding claims, characterized in that a number of virtual, in particular linkable, objects (3, 4) is not smaller or larger, in particular larger than three times, than a number of virtual bodies (13, 14).

7. Method according to one of the preceding claims, characterized in that at least two real models (7, 8) are provided, in particular wherein in each case an associated virtual body (13, 14) is realized.

8. Method according to one of the preceding claims, characterized in that at least two real models (7, 8) are brought into a spatial relationship to one another which is predetermined by a virtual spatial relationship of at least two virtual objects (3, 4), in particular by means of at least two virtual bodies (13, 14) belonging to the at least two real models (7, 8).

9. Method according to one of the preceding claims, characterized in that at least two real models (7, 8) are forcibly guided and / or limitedly movable relative to each other are .

10. Method according to one of the preceding claims, characterized in that a user is restricted in his mobility during use by at least one real model.

11. Method according to one of the preceding claims, characterized in that the linking is deactivated for a preferably defined or indefinite period of time, in particular so that deviations between the at least one virtual object (3, 4) and the virtual body (13, 14) are displayed when the linking (24) is deactivated.

12. Method according to one of the preceding claims, wherein the link (24) of the virtual body (13, 14) with the at least one virtual object (3, 4) is replaced by another link of the virtual body (13, 14) with another virtual object (3, 4).

13. Method according to one of the preceding claims, wherein the 3D model (39) is subjected to an isometric transformation, in particular rotated and / or shifted, when the link (24) is replaced by another link (24), until the virtual body (13, 14) and the other virtual object (3, 4) are brought into alignment at least within a tolerance range.

14. Method according to one of the preceding claims, wherein a position of the real model (7, 8) is preferably changed manually or automatically until the associated virtual body (13, 14) is brought into alignment with the at least one virtual object (3, 4) or with the other virtual object, in particular wherein subsequently a link (24) between the virtual body (13, 14) and the virtual object (3, 4) or the other virtual object (3, 4) is activated.

15. Method according to one of the preceding claims, wherein a plurality of virtual bodies (13, 14) are linked to a respective virtual object (3, 4) of the 3D model (39), wherein the individual links (24) are changed independently of one another, in particular activated and / or deactivated.

16. Method according to one of the preceding claims, wherein an update of coordinates of the at least one virtual object (3, 4) is output, in particular for processing design data of the 3D model (39).

17. Method according to one of the preceding claims, characterized in that it is a system for the pharmaceutical sector, preferably for filling medicines into containers and / or in cooperation with a protected space, preferably an isolator.

18. Method for functional testing of a system 1, in particular according to one of the preceding claims, wherein the system is represented as a virtual 3D model (2) from virtual objects (3, 4), wherein a real model (7, 8) is provided from at least one virtual object (3, 4) and wherein temporally recurring to the real model (7, 8) a virtual body (13, 14) is aligned with a 3D position measurement and the at least one object is linked to the virtual body (13, 14) and brought into a desired positional relationship with the virtual body (13, 14), wherein a virtual body (13, 14) is assigned to the at least one virtual body A real model (7, 8) corresponding to the object (3, 4) is preferably produced using an additive process and is provided with identifiable features for a 3D position measurement, and a correspondence (15) is stored between the identified features, in particular markers (10, 17), and the at least one virtual object (3, 4).

19. Method according to one of the preceding claims, wherein the identifiable features are formed at predetermined positions of the real model (7, 8).

20. Method according to one of the preceding claims, wherein at least one position of the formed features is measured on the real model (7, 8).

21. Method according to one of the preceding claims, wherein an operator wears a glove and / or a hand tracking device (27) and / or wherein a 3D position of one or more fingers and / or a hand and / or an arm, preferably of the or a glove and / or the or a hand tracking device (27), is repeatedly determined.

22. Method according to one of the preceding claims, wherein the 3D model (39) represents a shoulder ring, to the position of which a real shoulder ring (33, 34) is adjusted, in particular in a preceding setting step and / or wherein an operator puts an arm through the shoulder ring (33, 34), preferably in a manipulation glove attached to the shoulder ring (33, 34).

23. Method according to one of the preceding claims, wherein the recording pose of the observer relative to the shoulder ring (33, 34) is defined.

24. Method according to one of the preceding claims, wherein the at least one virtual object (3, 4) is or has a door (47) of a transfer port (48).

25. Method according to one of the preceding claims, wherein the 3D model (39) has a further virtual object (3, 4) for which a further real model (7, 8) is provided, wherein the real model (7, 8) is arranged to be movable relative to the further real model (44).

26. Method according to one of the preceding claims, wherein the further real model (44) is at least partially immobile and / or at least partially movable relative to a boundary of the installation.

27. Method for functional testing of a system (1), in particular according to one of the preceding claims, with the following steps: providing CAD data of the system, creating at least one real model (7, 8) for at least part of the CAD data, setting up the at least one real model (7, 8) in a 3D measuring device (11), displaying a virtual 3D model (2) created from the CAD data by processing at least 3D measurement data of the 3D measuring device (11).

28. Method according to one of the preceding claims, characterized in that a field of view (19) of a head-mounted display (16) is determined, preferably with the 3D measuring device (11).

29. Method according to one of the preceding claims, characterized in that the representation of the 3D model (39) is carried out with respect to a field of view (19) of a head-mounted display (16).

30. Method according to one of the preceding claims, characterized characterized in that a change to the at least one real model (7, 8) is automatically reproduced on the 3D model (39), in particular wherein modified design data are generated and output from the modified 3D model (39).

31. Method according to one of the preceding claims, characterized in that the 3D measuring device (11) is transported in a fixed measuring structure before being set up.

32. Device for functional testing of a system (1), the system being in the form of a virtual 3D model (2), with a 3D measuring device (11), at least one real model (7, 8) of a virtual object (3, 4) of the 3D model (39), a device for automatically integrating a virtual body (13, 14) recorded with the 3D measuring device (11) in relation to the real model (7, 8) into the 3D model (39), a device for automatically moving the virtual object (3, 4) along with the virtual body (13, 14) and a device for visually displaying the 3D model (39), in particular a 3D engine.

33. Device according to claim 24, characterized in that a means for activating and / or deactivating a link between the virtual body (13, 14) and the at least one virtual object (3, 4) is formed.

34. Device according to claim 24 or 25, characterized in that a head-mounted display (16) is arranged to generate a field of view (19) on the 3D model (39).

35. Device according to one of claims 24 to 26, characterized in that a device for generating a field of view (19) on the 3D model (39) with measured values ​​of the 3D measuring device (11) is fed to the or a head-mounted display (16).

36. Device according to one of claims 24 to 27, characterized in that a device for isometric transformation of the 3D model (39) relative to the field of view (19) is formed.

37. Use of a head-mounted display (16), in particular VR, XR and / or AR glasses, and a 3D measuring device (11) that is preferably stationary and / or operates independently of the head-mounted display (16) for creating a virtual view of a 3D model (39) of a system in the head-mounted display (16), wherein individual virtual objects (3, 4) correspond to real models (7, 8) that are captured by the 3D measuring device (11), in particular for a functional test of a preferably pharmaceutical system, preferably in a method according to one of claims 1 to 23 and / or in a device according to one of claims 24 to 28.

38. Head-mounted display (16), in particular VR, AR and / or XR glasses, with at least one marker (10, 17), in particular with more than two markers (10, 17) for a preferably extrinsic determination of a recording pose.

39. Method for the visual representation of 3D data, in particular in a method according to one of the preceding claims 1 to 23, wherein a field of view (19) of a head-mounted display (16) (display attached to the head), in particular a VR and / or XR and / or AR glasses, is intrinsically determined in a time-recurring manner and information moving along with the field of view (19) is displayed in the head-mounted display (16), wherein a recording pose predetermining the field of view (19), preferably recurring, of the head-mounted display ( 16 ) and compared with the field of view (19).

40. Method according to one of claims 1 to 23 or 31, characterized in that the recording pose is determined with a 3D measuring device (11) which is designed independently of the head-mounted display (16) and / or is stationary, in particular comprising at least one or more cameras (31).

41. Method according to one of claims 1 to 23 or 31 or 32, characterized in that the recording pose is determined with a moving and / or from the determination of the field of view (19) independent measuring device.

42. Method according to one of claims 1 to 23 or 31 to 33, characterized in that the intrinsic determination is carried out with the aid of at least one moving sensor, in particular a camera (31) and / or a movement and / or acceleration and / or position sensor.

43. Method according to one of claims 1 to 23 or 31 to 34, wherein the recording pose is measured by means of active markers (10, 17) on the head-mounted display (16).

44. Method according to one of claims 1 to 23 or 31 to 35, wherein the recording pose is measured by means of passive markers (10, 17) on the head-mounted display (16).

45. Method according to one of claims 1 to 23 or 31 to 36, wherein the recording pose is determined by means of a stationary measuring device, in particular by means of stationary cameras (31) , is measured.

46. ​​Method according to one of claims 1 to 23 or 31 to 37, wherein the intrinsic determination of the recording pose by means of moving cameras (31) and the head-mounted display (16).

47. Method for the visual representation of a system and / or as part of a method according to one of the preceding method claims, wherein a field of view (19) of a head-mounted display (16) (display attached to the head), in particular VR and / or XR and / or AR glasses, is intrinsically determined in a time-recurring manner and wherein an air flow is calculated and visually displayed as preferably co-moving 3D data in the head-mounted display (16).

48. Method according to one of claims 1 to 23 or 31 to 39, wherein the 3D data comprise a 3D model (39) of a plant and / or wherein the 3D data comprise AR metadata relating to components of the or a plant.

49. Method according to one of claims 1 to 23 or 31 to 40, wherein with the head-mounted display (16), in particular a head-mounted display (16) according to claim 30, a superposition of a real field of view (19) with a virtual representation of the 3D data is generated and a real environment is shielded.

50. Method according to one of claims 1 to 23 or 31 to 41, characterized in that a real model (7, 8) is adjusted by a motor and / or that a real model (7, 8) is adjusted until a preferably automatically detected deviation (64) in a position and / or orientation of a virtual body (13, 14) from a corresponding virtual object (3, 4) lies within a tolerance range.

51. Method according to one of claims 1 to 23 or 31 to 42, characterized in that a virtual light beam (55) is generated, in particular wherein it is automatically checked whether the virtual light beam (55) is interrupted.

52. Device for the visual representation of 3D data, in particular as part of a device according to one of claims 24 to 28 and / or for use in a method according to one of claims 1 to 23 or 31 to 43, with a head-mounted display (16) which is set up to determine a moving field of view (19), with a device for determining a recording pose of the head-mounted display (16) and with a device for comparing the recording pose with the field of view (1).

53. Apparatus according to claim 44, comprising a 3D engine for visually displaying the 3D data in the field of view (19).

54. Device according to one of claims 44 or 45, with a preferably stationary 3D measuring device (11) for determining the recording pose.

55. Device according to one of claims 44 to 46, with a device for generating a virtual space for virtual bodies (13, 14) detected with the or a 3D measuring device (11) and / or with a device for generating a virtual space (5, 12) for the visual representation of the 3D data and / or with a device for establishing a correspondence (15) between the virtual spaces (5, 12), in particular for embedding the virtual bodies (13, 14) in the virtual space (5, 12) for the virtual representation.

56. Device according to one of claims 44 to 47, with a device for embedding a field of view (19) of the head-mounted display (16) in the virtual space (5, 12) for the virtual representation.

57. Device according to one of claims 44 to 48, comprising a device for calculating an air flow, in particular for a visual representation of the air flow.

58. Device according to one of claims 24 to 28 or 44 to 49, with a device for a motorized adjustment of at least one real model (7, 8) and / or with a device for a preferably automatic determination of a deviation (64) in a position and / or orientation of a virtual body (13, 14) from a corresponding virtual object (3, 4) and / or with a device for a motorized adjustment of at least one real model (7, 8) until a preferably automatically detected deviation (64) in a position and / or orientation of a virtual body (13, 14) from a corresponding virtual object (3, 4) lies within a tolerance range.

59. Device according to one of claims 24 to 28 or 44 to 50, with a device for a collision check for a virtual light beam (55).

60. Apparatus according to any one of claims 44 to 50, comprising means for carrying out a method according to any one of claims 1 to 23 or 31 to 43. REPLACEMENT SHEET (RULE 26)

Images