METHOD FOR DISPLAYING A VIRTUAL OBJECT

Description

[0001] The invention relates to a method for fading in a computer-generated, virtual object into the field of vision of a real building interior perceived by a person, in particular into the field of vision of a transparent display unit, wherein the fading in of the virtual object takes place as a function of the current position and the direction of view of the person, and the fading in of the virtual object additionally takes place as a function of the presence of a visual or movement obstacle for the person, according to the preamble of claim 1, as well as a device for carrying out the inventive method according to claim 7.

[0002] Methods of this type are used in so-called "mixed reality (MR)" or "augmented reality (AR)" systems. In this case, a computer-generated, virtual object is superimposed into a person's perceived field of vision of a real environment, depending on the person's current position and viewing direction. Various sensors such as acceleration sensors (accelerometers or G-sensors), sometimes in combination with magnetometers and / or gyroscopes, and GPS systems are conventionally used to determine this. The person's current position and viewing direction initially determine whether a virtual object is superimposed at all.The positioning and size of the virtual object in the person's field of vision are then adjusted depending largely on the current position of the observing person, particularly the distance to the observing person, in order to create the realistic illusion of a virtual distance. For this purpose, display units in the form of data glasses are used in particular. These devices are worn like glasses and are capable of displaying virtual objects in the real world and positioning these objects in relation to real objects in space. Such data glasses are also referred to as "augmented reality" glasses or "mixed reality" glasses. A person wearing such glasses sees the environment as if through normal glasses, but (holographically virtual) objects can be displayed in their field of vision.Alternatively, it is also conceivable to use lenses that act as a playback unit to display virtual objects in the person's field of vision.

[0003] The application of this technology, and especially the correct display of virtual objects for a person moving inside a building, is associated with additional difficulties. Due to walls, doors, room dividers, or even furniture, the virtual objects displayed may or may not be visible to the person depending on the person's position and viewing direction. One possible application is in the education and training of police, fire, or military personnel. These personnel move through the building interior in varying lighting conditions and, in some cases, under acoustic irradiation, and must react correctly to various scenarios, which are realized by displaying virtual objects.For example, as part of a training course, police officers could be trained to search a building for perpetrators, with the perpetrators being positioned by a training manager as virtual objects behind walls, doors, pieces of furniture and the like, and being visible or hidden depending on the position and direction of view of the trainee.

[0004] A generic method is described in US 2019 / 371072 A1. US 2002 / 0196202 A1 and US 2003 / 0210228 A1, for example, describe a method in which emergency personnel moving inside a building during an emergency situation are presented with information in data glasses based on a building plan provided to the incident commander as a digital image. Such information can include warnings about potential hazards or navigation aids in the form of arrows and the like. The current position of the emergency personnel is determined using sensors and displayed to the incident commander in his digital image of the building plan. Displaying virtual objects while taking into account possible obscuration by structural features or furniture is not possible with a method according to US 2002 / 0196202 A1 and US 2003 / 0210228 A1.US 2002 / 0191004 A1 describes another method for displaying virtual objects for training emergency services.

[0005] US 2018 / 0330184 A1 describes a method for creating a digital image of the surfaces of a building's interior. The interior is measured using a distance sensor. The result of the distance measurements is a three-dimensional data point cloud, from which the contours of the floors, ceilings, and walls of the interior spaces are reconstructed using mathematical methods. US 2018 / 0330184 A1 does not provide for the display of virtual objects.

[0006] The aim of the invention is therefore to provide a method for superimposing a computer-generated, virtual object into the field of vision of a real building interior perceived by a person, in which the virtual object can be correctly superimposed according to the person's current line of sight, taking into account obscuring elements such as walls, doors, room dividers or even furniture.

[0007] This objective is achieved by the features of claim 1. Claim 1 relates to a method for superimposing a computer-generated, virtual object into the field of vision of a real building interior perceived by a person, in particular into the field of vision of a transparent display unit, wherein the superimposition of the virtual object takes place depending on the current position and the direction of view of the person, and the superimposition of the virtual object additionally takes place depending on the presence of a visual or movement obstacle for the person, wherein the invention proposes that the visual or movement obstacle is determined based on a digital image of surfaces of the building interior formed from planar surface elements,by carrying out a horizontal stratification between a floor level of the building interior and an assumed eye level level with a plurality of layer layers of a given layer thickness, and horizontal surface elements or a totality of adjacent horizontal surface elements in the same layer layer are identified as obstructions to sight or movement, if horizontal secondary surface elements can be found in their peripheral area which are located more than one layer layer below the horizontal surface element or the totality of adjacent horizontal surface elements, and vertical surface elements are identified as obstructions to sight or movement.

[0008] The starting point of the method according to the invention is the creation of a digital image of the surfaces of the building interior facing the room interior, which is formed from planar surface elements. One possibility is for such a digital image to already exist in the form of a digital building plan, whereby the digital building plan already includes the positioning of furniture in addition to walls, room dividers, or doors. Another possibility for creating a digital image of the building interior is to determine it by measuring the building interior with a camera capable of measuring depth information. In this case, the camera capable of measuring depth information can be mounted on the playback unit with which the building interior is walked through. A camera capable of measuring depth information is also referred to as a 3D camera and can be designed, for example, as a ToF ("Time-of-Flight") camera.ToF cameras are cameras that not only capture a 2D image but also measure depth information for each captured pixel. Depth information refers to information about the distances between the individual objects in a scene and the ToF camera. Using well-known mathematical methods, the resulting "scans" of depth information can be used to create a digital image that approximates the surfaces of the building's interior using surface elements. These surface elements can be planar surface elements in the form of triangles, quadrilaterals, or hexagons that seamlessly reproduce the surface to be approximated. The common corner points of these surface elements are also called vertices, the side of the surface elements facing the viewer is called "faces," and the totality of the vertices is called a "mesh."Another possibility using known mathematical methods is to assign an orientation to the surface elements by calculating a surface normal to the surface element in question. The method according to the invention makes use of this possibility by first determining a digital image of surfaces of the building interior formed from planar surface elements. According to the invention, however, only those surface elements are subsequently used that lie between a floor level of the building interior and an assumed eye level plane, which defines a maximum height for the subsequent method steps. An eye level plane is generally understood to be a horizontal plane that is at eye level. The exact height is not important here; in the context of the present method, it is assumed to be 1.80 m, for example.The digital image of the building's interior is thus, in a sense, "cut off" at the eye-level plane. Furthermore, the invention determines those surface elements whose surface normals run horizontally or vertically, i.e., surface elements whose orientation is vertical or horizontal. Furthermore, the invention creates a horizontal layering between the floor level of the building's interior and the assumed eye-level plane, using a plurality of layers of a predetermined thickness.

[0009] On the one hand, vertical surface elements that are part of the digital image are now identified as visual or movement obstacles according to the invention. These visual or movement obstacles generally represent walls, room dividers, or tall pieces of furniture such as cupboards and the like. On the other hand, horizontal surface elements or a set of adjacent horizontal surface elements in the same layer are identified as visual or movement obstacles according to the invention if adjacent horizontal surface elements can be found in their peripheral area that are located more than one layer below the horizontal surface element or the set of adjacent horizontal surface elements. These visual or movement obstacles generally represent low pieces of furniture such as tables and the like.Using the surface elements of the digital image thus determined as visual or movement obstacles, the virtual object can be easily displayed depending on the presence of the visual or movement obstacle, as will be explained in more detail below.

[0010] The layer thickness for the horizontal stratification can be specified with a value of approximately 5–30 cm. This layering allows, for example, horizontal surface elements located in vertically successive layer layers to be identified as steps. Furthermore, for better visualization of the elevation, particularly in a two-dimensional projection of the digital image, it is also possible to display the horizontal surface elements in different layer layers with different colors. Therefore, it is also proposed that the digital image be created as a two-dimensional image with the surface elements identified as obstructions to vision or movement.

[0011] To improve the digital image, "oblique" surface elements—i.e., surface elements that are identified as neither horizontal nor vertical surface elements—can be considered. They could be assigned to horizontal or vertical surface elements if they meet specified criteria, for example, if their surface normals deviate from the corresponding surface normals within specified tolerance ranges.

[0012] With the help of the visual or movement obstacles present as planar surface elements of the digital image, the display of the virtual object can be carried out in a simple manner depending on the presence of a visual or movement obstacle. This is done by calculating, within the framework of a calculation algorithm, connecting lines between an eye point of the person corresponding to the current position and direction of view and image points of the virtual object and determining, within the distance between the eye point and the image points of the virtual object, the existence of intersection points of the connecting lines with the visual or movement obstacle present as planar surface elements, whereby if an intersection point for an image point of the virtual object is present, the image point in question is not displayed or is not visible in the person's field of vision.and if an intersection point for a pixel of the virtual object is missing, the pixel in question is visibly displayed. The eye point is a term from central perspective and conventionally refers to a point in space that corresponds to the position of a person and from which the "visual rays" originate. Connecting lines are first calculated between this eye point and pixels of the virtual object to be displayed. Subsequently, it is determined whether, within the distance between the eye point and the pixels of the virtual object, there are intersection points of the connecting lines with a visual or movement obstacle present as a planar surface element. According to the invention, it is thus checked whetherwhether the line connecting a pixel to the eye point intersects a surface element considered to be a visual or movement obstacle. If this is the case, the relevant pixel of the virtual object is not displayed or is not visible in the person's field of vision. A non-visible display can be achieved, for example, via a transparent display, in which the relevant pixel is also not visible to the observer. If no such intersection exists, the relevant pixel is displayed visibly. The calculation process, beginning with the calculation of the connecting line and determining whether an intersection exists, requires only a few milliseconds per pixel. The calculation process can thus be completed quickly enough to enable real-time applications and frequent repetition as the observer moves through the building interior.

[0013] However, the visual or movement obstacles present as planar surface elements of the digital image can also be used as navigation aids. The virtual objects displayed depending on the presence of a visual or movement obstacle are navigation aids that are displayed in the person's field of view of the display unit. These navigation aids can, for example, show emergency personnel the fastest route to a destination while taking movement obstacles into account, or enable safe movement inside a building in poor visibility conditions.

[0014] A further challenge in correctly displaying virtual objects in the field of view of the playback unit of a person moving through the interior of a building is the exact localization of the person within the building and the comparison of the coordinate system of the data glasses with that of the digital image. In principle, this is done in a conventional manner: the virtual object is displayed whenever the person is viewing an area of ​​space in which the coordinates of the object to be displayed lie, and the person's current position is within a predefined reference range, or the person has reached a certain, predefined position. If a person located within the reference range views an area of ​​space in which the coordinates of the object to be displayed lie, they are presented with a view that is essentially predictable.A suitable virtual object can be generated for these views. This object is made up of pixels and stored in a database. If a person wearing a display device such as data glasses enters the building, the data glasses are switched on and then calibrated. The current position and direction of view of the initially stationary person is determined using position and / or motion sensors or a camera. Starting from the known initial position and direction of view, changes in position and direction of view are then measured in the conventional way using various sensors such as acceleration sensors (accelerometers or G-sensors), sometimes in combination with magnetometers and / or gyroscopes or camera images, and the respective current position and direction of view are determined.The viewing direction can be determined using magnetic field sensors, IMU sensors, or image-based orientation methods. For example, orientation can be determined by comparing two images taken at different times. The horizontal and vertical shift between the two images is determined and used to calculate the horizontal and vertical orientation change. The image shift can be calculated using prominent image points (so-called "features") by determining the position (x and y position in the image) of such a feature in the first and second images and then determining the shift. This is based on the assumption that the relationship between image dimensions (width, height) and image shift is the same as the relationship between viewing angle (also known as "field of view") and orientation change.This calculation can be performed either directly on the processing unit of the smart glasses or on a processing unit of an external device. Based on a so-called AHRS (Attitude Heading Reference System) algorithm, additional orientation information can be collected using an accelerometer, gyroscope, and magnetometer, which is then combined with the results of the image-based solution.

[0015] However, these methods can sometimes be error-prone, so that the current position of the person can only be determined inaccurately as the person continues to move and the coordinate system of the data glasses no longer matches that of the digital image, which leads to incorrect overlays of the virtual object.It is therefore proposed that a camera be used to create a sequence of camera images of the real environment perceived by the person, for example as RGB images, at a predetermined time interval. Using known methods for tracking image features between a recorded camera image and a reference image associated with a known recording position, a displacement vector is determined based on the image features. If the displacement vector falls below a predetermined displacement value, the known recording position associated with the corresponding reference image is assigned to the person as the current position, thus synchronizing the coordinate systems of the data glasses and the digital image. As already mentioned, when viewing a spatial area within the interior of a building, the person is presented with a view that is essentially predictable.From these views, reference images can be manually created from known recording positions and for known viewing directions and stored in a computer unit's database. The respective known recording positions are assigned to these reference images. These reference images can then be continuously compared with the camera images currently recorded of the person. If an approximate identity with one of the reference images is determined, the corresponding recording position can be used as the person's current position. These reference images can therefore also be referred to as "spatial anchors" or "optical markers" in digital image processing terms. The camera images recorded of the person can also be used for room recognition, with the computer unit comparing the camera images with the reference images, thus allowing the room in which the person is located to be immediately recognized.

[0016] Furthermore, it is proposed that 3D images corresponding to the person's current position and viewing direction be created using a camera capable of measuring depth information, and that an updated digital image be determined from the resulting 3D camera images using known mathematical methods. The mathematical methods mentioned could be those already described in connection with the creation of the digital image. Since the current view of a spatial area can also change compared to a previously created digital image, for example, if a piece of furniture is moved or a door is closed, it can be advantageous if the digital image is updated based on the real environment currently perceived by the person.

[0017] Furthermore, the invention relates to a device comprising a portable, transparent display unit for displaying a computer-generated, virtual object in the field of vision of a real building interior perceived by a person, in particular in the field of vision of the transparent display unit, as well as a computer unit connected to the display unit via a bidirectional data connection, which is designed to carry out the method according to the invention.

[0018] The invention will be explained in more detail below using exemplary embodiments with the aid of the accompanying figures. Fig. 1 a schematic overview of the basic procedure of the method according to the invention, Fig. 2 a schematic view of a floor plan of a building interior being measured using data glasses to generate a digital image and reference images, Fig. 3 a schematic overview of the inventive generation of visual or movement obstacles, Fig. 4 an example of a real digital image of a building interior, and the Fig. 5 a schematic representation to explain the horizontal layering to determine the visual or movement obstacles.

[0019] First, the Fig. 1 Reference is made to FIG. 1, which shows a schematic overview of the basic process of the method according to the invention. First, a digital image of the surfaces of the building interior facing the room interior must be created. According to the embodiment of the Fig. 1 For this purpose, a playback unit 1 in the form of data glasses is used, which is equipped with a camera for recording two-dimensional images, with a 3D camera for recording images with depth information and with sensors for determining position and viewing direction and in which Fig. 1 indicated in the top right. With such a display unit 1, the interior of the building is walked through, which, for example, has a floor plan according to the Fig. 2 The Fig. 2 schematically shows several positions of a person wearing such data glasses 1, which the person successively assumes while walking through the rooms. The result of the walking through is a multitude of images, provided with depth information, of the room views that the person sees while walking through, which are sent to a computer unit 2. The computer unit 2 is usually an external device such as a stationary or portable computer or tablet. Using known mathematical methods, a digital image can be obtained in the computer unit 2 from the thus obtained "scans" of depth information, which approximates the surfaces of the building interior facing the person using surface elements F. These surface elements F are in the Fig. 2 as planar surface elements F in the form of triangles, which seamlessly reproduce the surfaces to be approximated. Another possibility of known mathematical methods is to assign an orientation to the surface elements F by calculating a surface normal to the respective surface element F. In this way, surface elements F can be identified as vertically oriented surface elements Fv or as horizontally oriented surface elements Fh, from which visual or movement obstacles can subsequently be reconstructed, as will be shown later on the basis of the Fig. 4 will be explained. The result of the "scan" is a three-dimensional mesh of the building's interior. As soon as a certain difference in the height coordinate is detected, a new 3D model is generated, as this is interpreted as a change of floor. By visualizing the previously scanned areas, the user is informed of what has already been captured and where there are still gaps in the scan. Once the user is satisfied, the 3D models for each floor are saved on the server (tablet application) or locally. In addition, one or more reference images can be captured of each room, for example, as RGB images, which will later serve as recognition points ("optical markers") and stored in a database of computer unit 2. These reference images are assigned the respective known recording positions, as they are defined in the Fig. 2 indicated by the diamonds. This process can also be used to determine which rooms are connected. This allows for immediate restriction of possible room changes, which can be helpful in determining the current room in the positioning process.

[0020] Once the digital image of the building's interior, the reference images, and the visual and movement obstacles have been determined, the computer unit 2 is ready to display virtual objects stored in a database of the computer unit 2. Using proprietary software, the virtual objects can be placed and events planned on the digital image using user-friendly and efficient methods such as a computer mouse or touch controls. The position and orientation of all virtual objects are saved relative to the digital image. Preferably, a preview can also be played directly on the computer unit 2. The preview can also be played using the digital image for alternative viewing angles.This allows the sequence of events of a "scenario" to be planned first during training or operational planning and then discussed later with the people who will ultimately carry out the application.

[0021] During such training or deployment, a person moves through the building interior wearing a display unit 1, for example in the form of data glasses, which is equipped with a camera for capturing two-dimensional images, a 3D camera for capturing images with depth information, and sensors for determining position and viewing direction. This display unit 1 is in the Fig. 1 indicated top left. During training or deployment, the scenario information is transferred from computer unit 2 to the application of playback unit 1. For the scenario to run correctly, the current coordinate system of playback unit 1 must be synchronized with the one used to create the digital image. Previously recorded reference images are used for this purpose. If the camera of playback unit 1 delivers a previously recorded image for a view, the deviation between the two coordinate systems can be calculated. The reference images can also be used to continue the scenario in a limited way if playback unit 1 can no longer orient itself, since their recognition can be used to determine the person's position and orientation. Alternatively, it would also be conceivable to create so-called "spatial anchors" at multiple locations.By comparing the anchor transformations, a difference between the starting points and the orientation can also be calculated.

[0022] The position of the playback unit 1 can be displayed on the computer unit 2 in the digital image of the room plan.

[0023] With the help of the visual or movement obstacles present as planar surface elements F of the digital image, the display of a virtual object can now be carried out in a simple manner depending on the presence of a visual or movement obstacle. This is done by calculating, within the framework of a calculation algorithm, connecting lines between an eye point of the person corresponding to the current position and direction of view and image points of the virtual object and determining, within the distance between the eye point and the image points of the virtual object, the existence of intersection points of the connecting lines with the visual or movement obstacles present as planar surface elements F, whereby if an intersection point for an image point of the virtual object is present, the image point in question is not displayed or is not visible in the person's field of vision.and if there is no intersection point for a pixel of the virtual object, the pixel in question is visibly displayed. The virtual object is thus displayed either completely, partially, or not at all, depending on the presence of a visual or movement obstacle.

[0024] However, the visual or movement obstacles present as planar surface elements F of the digital image can also be used as a navigation aid, in that the virtual objects displayed depending on the presence of a visual or movement obstacle are navigation aids that are displayed to the person in the field of view of the playback unit 1. These navigation aids can, for example, show emergency services the fastest route to a destination while taking movement obstacles into account, or enable safe movement inside a building in poor visibility conditions.

[0025] In the Fig. 1 An arrow can also be seen which, in the event that no virtual objects are to be displayed, leads back to the generation of a digital image. This arrow is intended to indicate that, using the camera attached to the playback unit 1 and capable of measuring depth information, 3D images corresponding to the current position and direction of view of the person can be created, from which an updated digital image can be determined. Since the current view of an area of ​​a room can also change compared to a previously created digital image, for example if a piece of furniture is moved or a door is closed, it can be advantageous if the digital image is updated based on the real environment currently perceived by the person.

[0026] Subsequently, the Fig. 4 and 5The determination of visual and movement obstacles from the digital image is explained. As already mentioned, using known mathematical methods, a digital image can be obtained from the previously described scans of the building's interior, known as a "mesh," which approximates the surfaces of the building's interior using planar surface elements F ("faces"). Using an application running on computer unit 2, the meshes obtained from the data glasses 1 are processed, and a building plan is created from them. The vertices and faces are divided into floors based on their height. Afterwards, all vertices and faces above an eye-level level A, which corresponds to the average eye height, are removed from the scanned meshes for each floor, and any faulty faces are removed.The exact elevation of the eye-level plane A is not important; for the purposes of this procedure, it is assumed to be 1.80 m, for example. The digital image of the building's interior is thus, in a sense, "cut off" at eye-level plane A. Anything at or just below eye level is considered part of a visual or movement obstacle, i.e., walls, furnishings, or other objects. Anything near the ground is classified as such and thus assessed as passable. The procedure is as follows: First, those surface elements F whose surface normals run horizontally or vertically are determined, i.e., surface elements F whose orientation is vertical or horizontal. Subsequently, a horizontal layering is performed between the floor plane B of the building's interior and the assumed eye-level plane A, using a plurality of layer layers S i (i=1,...N) of a specified layer thickness.In the example shown the . Fig. 5 about six layers S i (N=6) with a layer thickness of 30cm each are used.

[0027] Firstly, vertical surface elements Fv j (j=1, ...M) are identified as visual or movement obstacles. These visual or movement obstacles usually represent walls, room dividers or tall pieces of furniture such as cupboards and the like. Secondly, horizontal surface elements Fh k (k=1,...P) or a set of adjacent horizontal surface elements Fh k in the same layer position are identified as visual or movement obstacles if horizontal secondary surface elements Fn can be found in their peripheral area that are located more than one layer below the horizontal surface element Fh k or the set of adjacent horizontal surface elements Fh k. These visual or movement obstacles usually represent low pieces of furniture such as tables and the like, in the example of the Fig. 5 A table is indicated. The entirety of adjacent horizontal surface elements Fh k identified as a visual or movement obstacle can have a height distance of approximately 90-120 cm from the floor level B, as in the example of the Fig. 5 . This layering also allows horizontal surface elements Fh 1 (l=1, ...Q) located in vertically successive layer positions S i to be identified as steps, as also in the Fig. 5 is indicated.

[0028] Horizontal faces are thus categorized as seating or table surfaces, floors, and the like. Vertical faces, on the other hand, are categorized as walls and thus as impassable visual or movement obstacles. By projecting the vertices and faces onto a 2D surface, a plan of the scanned environment can be created. An example of such a plan is shown in Figur 3Based on the height and orientation of the faces' normal vectors, the visualization of the faces can be refined to highlight height differences between visual or movement obstacles.

[0029] The invention thus provides a method for displaying a computer-generated, virtual object in the field of vision of a real building interior perceived by a person, in which the virtual object can be correctly displayed according to the person's current line of sight, taking into account obscuring elements such as walls, doors, room dividers or even furniture as visual or movement obstacles.

Claims

1. Method for displaying a computer-generated virtual object in the field of view of a real building interior as perceived by a person, in particular in the field of view of a transparent display unit (1), wherein the virtual object is displayed depending on the current position and line of sight of the person, and the virtual object is additionally displayed depending on the presence of an obstacle to the person's vision or movement, characterized in that the obstacle to vision or movement is determined on the basis of a digital image of surfaces of the interior of the building formed from planar surface elements (F) by means of horizontal layering between a floor level (B) of the interior of the building at an assumed eye level plane (A) with a plurality of layer positions (Si) of a predetermined layer thickness, and horizontal surface elements (Fhk) or a plurality of mutually adjoining horizontal surface elements (Fhk) in the same layer are determined as obstacles to vision or movement if horizontal secondary surface elements (Fn) are found within their peripheral area which are located more than one layer position below the horizontal surface element (Fhk) or the entirety of mutually adjoining horizontal surface elements (Fhk), and vertical surface elements (FVj) are determined as obstacles to vision or movement.

2. Method according to claim 1, characterized in that the layer thickness is specified as having a value of 5-30 cm.

3. Method according to claim 1 or 2, characterized in that the digital image is determined as a two-dimensional image with the surface elements (F) identified as obstacles to vision or movement.

4. Method according to one of claims 1 to 3, characterized in that connecting lines between an eye point of the person corresponding to the current position and direction of view and image points of the virtual object are calculated and, within the distance between the eye point and the image points of the virtual object, the presence of intersection points of the connecting lines with the obstacle to vision or movement present as planar surface element (F) is determined, wherein, if there is an intersection point for an image point of the virtual object, the image point in question is not displayed or is not visibly displayed in the person's field of view, and if there is no intersection point for an image point of the virtual object, the image point in question is visibly displayed.

5. Method according to one of claims 1 to 4, characterized in that a sequence of camera images of the real environment perceived by the person is produced by means of a camera at a predetermined time interval, and a displacement vector is determined on the basis of the image features using known methods for tracking image features between a captured camera image and a reference image assigned to a known recording position, wherein, if the displacement vector falls below a predetermined displacement value, the known recording position of the person assigned to the corresponding reference image is assigned as the current position.

6. Method according to one of claims 1 to 5, characterized in that 3D images corresponding to the current position and direction of view of the person are produced by means of a camera capable of measuring depth information, and an updated digital image is determined from the 3D camera images obtained in this way using known mathematical methods.

7. Device comprising a portable, transparent display unit (1) for displaying a computer-generated virtual object into the field of view of a real building interior perceived by a person, in particular into the field of view of the transparent display unit (1), and a computer unit (2) connected to the display unit (1) via a bidirectional data connection, which is designed to carry out the method according to one of claims 1 to 6.