METHOD AND SYSTEM FOR CONTROLLING A DISPLAY OF AN AUGMENTED REALITY DEVICE AND COMPUTER PROGRAM WITH INSTRUCTIONS FOR PERFORMING THE METHOD

Description

[0001] The present invention relates to a method and a system for controlling a display of an augmented reality device. The invention further relates to a computer program with instructions for carrying out the method.

[0002] For years, the increasing prevalence of digital technologies in society has been evident. It is expected that this digitalization of society will increase significantly in everyday life. Virtual and augmented reality technologies and applications represent a particular form of digitalization. Augmented reality (AR) involves enriching the real world with virtual elements that are accurately registered in three-dimensional space and allow for real-time interaction. Since the term "augmented reality" has become more common in German-speaking professional circles than "erweiterte Realität," the former will be used in the following discussion. The term "mixed reality" is also used synonymously. Augmented reality glasses, for example, can be used to display AR content.Augmented reality glasses are worn like regular glasses, but feature one or more projection units or displays that project information in front of the wearer's eyes or directly onto their retina. The glasses are designed so that the wearer can still perceive their surroundings. New technologies for augmented reality glasses, such as light field technology, advances in battery technology, and the entry of large companies into this market suggest that these devices will become increasingly popular.

[0003] Alternatively, AR displays can also be generated using head-up displays (HUDs), particularly in the automotive sector. In a vehicle's head-up display, the light beams from a display integrated into the dashboard are folded by several mirrors and lenses and reflected onto a projection surface, projecting a virtual image into the driver's field of vision. In automotive applications, the windshield often serves as the projection surface, and its curved shape must be taken into account during the display process. As an alternative, an additional glass or plastic panel is sometimes used, positioned on the dashboard between the driver and the windshield.

[0004] Overall, it is foreseeable that augmented reality glasses, augmented reality smartphones, and augmented reality head-up displays will be increasingly used by end customers. Against this backdrop, the question arises as to how multiple users can experience augmented reality together, for example, how they can view augmented information simultaneously.

[0005] In this context, US Patent 2013 / 0135180 A1 describes a system for distributed collaboration among multiple users. An active user wears smart glasses connected to a central computer. The central computer receives environmental information from the smart glasses and uses it to generate a spatial model. The central computer also receives information about an object to be worked on, as well as input from the active user and other users. Taking the user input into account, a virtual representation of the object is inserted into the spatial model and displayed to the active user.

[0006] US Patent 2014 / 0098131 A1 describes a method for using augmentations for augmented reality displays. According to the method, augmentation data is received that is associated with a first augmentation used by a remote augmented reality device to create an augmented reality display of a first real-world situation. A second augmentation is then used within the context of an augmented reality display of a second real-world situation. The display of the second augmentation depends, at least in part, on the received augmentation data. Finally, information about the use or non-use of the augmentation data is transmitted to the remote augmented reality device.

[0007] US Patent 2015 / 0178990 A1 describes a method for interaction within a vehicle. The vehicle has a head-up display for the driver and a simple display for the front passenger. In response to a command from the driver, an augmentation object is generated based on the driver's current viewpoint and displayed on the front passenger's display. The front passenger can interact with this display. In response to such interaction, augmentation objects are generated for the head-up display and shown to the driver.

[0008] WO 2017 / 115365 A1 describes a method for using a user's line of sight or gaze direction to enrich displayed data and control various displays and instruments. The user's line of sight is tracked, and relevant elements in a scene the user is looking at are identified. The corresponding displayed data is then enriched or otherwise manipulated with respect to these identified elements.

[0009] It is often unclear to a user of an augmented reality device which objects are currently being viewed by other people. For example, the user can only guess which object another user is referring to or pointing at.

[0010] In this context, US 2018 / 0005429 A1 describes a method for allowing a user of a head-mounted display (HMD) to join virtual reality (VR) scenes. The method involves providing an initial perspective of a VR scene to a first user's HMD. When a notification is received that a second user with a second HMD requests to join the provided VR scene, real-world position and orientation data of the second HMD relative to the first HMD are determined. Based on this data, a second perspective of the VR scene is provided. Within the VR scenes, users can...

[0011] The viewpoints of different users are displayed.

[0012] US Patent 2018 / 0039076 A1 describes a method for facilitating communication between users who share a visual cue. The viewpoint of a first user is determined as an approximation in three-dimensional space and transmitted to a second user. Based on this viewpoint, a gaze focus of the first user is displayed to the second user.

[0013] US Patent 2014 / 0306994 A1 describes a method for creating and displaying personalized virtual scoreboards in an augmented reality environment. These personalized scoreboards allow for the exchange of personalized information between individuals in an environment who may have varying levels of acquaintance.

[0014] One of the aims of the invention is to provide solutions that enable a shared experience of the environment by a large number of users.

[0015] This problem is solved by a method having the features of claim 1, by a computer program with instructions according to claim 6, and by a system having the features of claim 7. Preferred embodiments of the invention are the subject of the dependent claims.

[0016] According to a first aspect of the invention, a method for controlling a display of an augmented reality device of a first user comprises the following steps: Receiving information about the viewpoint of at least a second user by a backend; transmitting the viewpoint of the second user to the augmented reality device of the first user by the backend; and displaying the viewpoint of the second user in the display of the augmented reality device of the first user, wherein the viewpoint of the second user is only transmitted by the backend to the augmented reality device of the first user if the augmented reality device of the first user is in close proximity to the second user, wherein the distance up to which a device is considered to be in close proximity can be individually controlled by the second user.

[0017] Accordingly, a computer program contains instructions which, when executed by a computer, cause the computer to perform the following steps to control a first user's augmented reality device display: Receiving information about the viewpoint of at least a second user by a backend; transmitting the viewpoint of the second user to the augmented reality device of the first user by the backend; and displaying the viewpoint of the second user in the display of the augmented reality device of the first user, wherein the viewpoint of the second user is only transmitted by the backend to the augmented reality device of the first user if the augmented reality device of the first user is in close proximity to the second user, wherein the distance up to which a device is considered to be in close proximity can be individually controlled by the second user.

[0018] Similarly, a system for controlling the display of an augmented reality device of a first user exhibits: A backend configured to receive information about the viewpoint of at least one second user and to transmit the viewpoint of the second user to the augmented reality device of the first user; and a device for controlling the display of the augmented reality device of the first user, with an input for receiving the viewpoint of the second user from the backend and a graphics generator for displaying the viewpoint of the second user in the display of the augmented reality device of the first user, wherein the viewpoint of the second user is only transmitted from the backend to the augmented reality device of the first user if the augmented reality device of the first user is in close proximity to the second user, wherein the distance at which a device is considered to be in close proximity can be individually adjusted by the second user.

[0019] The term "computer" used above is to be understood broadly. In particular, it also includes control units and other processor-based data processing devices.

[0020] The solutions according to the invention are based on the approach of sharing users' focus points or viewpoints. On the one hand, users can be those operating an augmented reality device, which facilitates the capture of a viewpoint. On the other hand, viewpoint determination can also be performed for users who do not use an augmented reality device, provided their respective viewpoints can be determined by eye-tracking. The shared information can then serve as the basis for a range of applications and enable the realization of a shared augmented reality.

[0021] According to one aspect of the invention, an object in the user's field of vision is identified. Information about the object is then transmitted to the backend. The continuously increasing computing power available in motor vehicles and smartphones allows for the implementation of increasingly complex object recognition approaches. The information about objects detected and classified using object recognition can be combined with information about the user's gaze direction. This fusion enables the unambiguous identification of dynamic objects, such as other road users, and allows for the optional enrichment of this information with digital data.

[0022] According to one aspect of the invention, a gaze direction is detected by capturing head orientation or by means of eye tracking. The gaze point is then determined from the gaze direction. Numerous solutions already exist for capturing a gaze direction, so that determining the gaze point can be implemented cost-effectively. For example, driver observation cameras or interior cameras can be used in motor vehicles to monitor head orientation. Currently available smart glasses also already provide sensor technology for calculating the head position and orientation of a user in space. The head orientation then corresponds at least approximately to the gaze direction. The gaze point can then be determined, for example, as the point located in the center along the visual axis or an intersection of the visual axis with an object encountered along the visual axis.

[0023] Transmitting information to a backend enables versatile use of the data, for example, for company-specific analysis or for cross-user information exchange. Depending on the use case, it is not always necessary to make the perspective of all relevant users available to all users. It may also be desirable to determine the perspective only and evaluate it for the mobility provider's purposes, without sharing it with individual users.

[0024] According to one aspect of the invention, the transmitted information is evaluated in a time-based manner in the backend. For example, the time of capture of the gaze point can be evaluated. This type of time-based evaluation makes it possible, for instance, to determine how users' interest in certain objects develops over time. It can also be used to detect whether several users focus their gaze on the same location at the same time or at least within a specific period. Alternatively or additionally, the period during which a user's gaze remains on a particular area can also be evaluated. If this period exceeds a defined minimum duration, this can be used as a trigger for further steps.

[0025] According to one aspect of the invention, the transmission of information to an augmented reality device or to the backend is subject to user privacy settings. User-adjustable privacy settings allow for data protection and / or give the user the ability to control the distribution of the collected information. For example, the user can determine whether and which information is collected or transmitted, and in what form and to what extent it is evaluated or shared with other users.

[0026] According to one aspect of the invention, the viewpoint of the second user is displayed saliently in the display of the first user's augmented reality device. This salient display makes it immediately apparent to a user where another user is looking, even if the viewpoint is only within the latter's peripheral field of vision. Alternatively or additionally, elements in the display of the first user's augmented reality device are marked when they are also displayed to the second user. This marking of elements in the display clearly indicates to the user that information about their viewpoint is being shared.

[0027] A method according to the invention is particularly advantageous when used in an augmented reality device, for example, in augmented reality glasses, an augmented reality smartphone, or an augmented reality head-up display. Such an augmented reality device can be used, for example, in a vehicle, especially a motor vehicle. However, its use is fundamentally possible with any form of mobility, including non-motorized transport. In the case of motorized transport, use is not limited to the driver; passengers, other people not directly involved in the driving task, or other persons can also be users of such an augmented reality device.

[0028] Further features of the present invention will become apparent from the following description and the attached claims in conjunction with the figures. Fig. 1 schematically shows an augmented reality headset; Fig. 2 schematically shows a general structure of a head-up display device for a motor vehicle; Fig. 3 schematically shows a method for providing information by an augmented reality device; Fig. 4 shows a first embodiment of a device for providing information by an augmented reality device; Fig. 5 schematically shows a method for providing information to control a display of an augmented reality device; Fig. 6 shows a first embodiment of a device for providing information to control a display of an augmented reality device; Fig. 7 schematically shows a method for controlling a display of an augmented reality device of a first user; Fig. 8 shows a first embodiment of a device for controlling a display of an augmented reality device of a first user; Fig.Figure 9 shows a second embodiment of the devices. Fig. 4 , Fig. 6 and Fig. 8 Fig. 10 schematically represents a motor vehicle in which a solution according to the invention is implemented; Fig. 11 schematically shows a system diagram of the solution according to the invention; and Fig. 12 schematically shows an application example of the solution according to the invention.

[0029] To better understand the principles of the present invention, embodiments of the invention are explained in more detail below with reference to the figures. It is understood that the invention is not limited to these embodiments and that the described features can also be combined or modified without leaving the scope of protection of the invention as defined in the appended claims.

[0030] Fig. 1 Figure 1 schematically shows an augmented reality headset 1 as an example of an augmented reality display device. The augmented reality headset 1 has a camera 2 for capturing environmental data from the surroundings of the headset. An interface 3 connects it to an external pose detection device 4 for determining the pose of the augmented reality headset 1 based on the environmental data. The pose detection device 4 can, for example, detect and evaluate edges or prominent points in the image data. The pose detection device 4 can, of course, also be integrated into the augmented reality headset 1. A graphics unit 5 allows the display of the augmented reality headset 1 to be adjusted according to the determined pose. For this purpose, the pose detection device 4 and the graphics unit 5 can exchange data with each other.As shown in the example, the graphics unit 5 can be part of the augmented reality glasses 1 or connected to the augmented reality glasses 1 via interface 3. The display generated by the graphics unit 5 is projected by a projection unit 6. In the example, this is shown in . Fig. 1 A monocular display is used, meaning that a display is positioned in front of only one of the user's eyes. The solution described here can, of course, also be implemented with augmented reality glasses 1 with a binocular display, where one display is used for each eye, or with a binocular display where a single display is used for both eyes.

[0031] Fig. 2 Figure 10 schematically shows a head-up display 10 as an example of an augmented reality display device for a motor vehicle 90. With the aid of the head-up display 10, content can be displayed on a projection surface 13 of the motor vehicle 90, for example, on the windshield or on an additional glass or plastic panel located on the dashboard between the driver and the windshield. The displayed content is generated by an imaging unit 11 and projected onto the projection surface 13 by means of an optical module 12. Typically, the projection is onto an area of ​​the windshield above the steering wheel. The imaging unit 11 can, for example, be an LCD-TFT display. The head-up display 10 is usually installed in the dashboard of the motor vehicle 90.

[0032] Fig. 3 Figure 1 schematically illustrates a method for providing information via an augmented reality device. In this method, a user's gaze point is determined.21 This can be achieved, for example, by first capturing the user's head orientation or using eye tracking.20 The gaze point can then be determined from this gaze direction. Finally, information about the user's gaze point is transmitted to a device connected to the augmented reality device.23 The connected device could, for example, be a backend connected via a network. Additionally, an object in the user's field of view can be identified.22 Information about the object can then also be transmitted to the connected device. The transmission of information can be subject to the user's privacy settings.

[0033] Fig. 4 Figure 1 shows a simplified schematic representation of a first embodiment of a device 30 for providing information via an augmented reality device 1, 10. The device 30 has an input 31 through which, for example, data about the gaze direction of a user of the augmented reality device 1, 10 can be received. The device 30 also has a gaze point determination module 32 for determining the user's gaze point. The gaze point can be determined, for example, from the gaze direction. Information about at least the user's gaze point is output via an output 36 of the device 30 to a connected device. The connected device can be, for example, a backend 100 connected via a network or another user's augmented reality device. The transmission of information can be subject to the user's privacy settings, which are implemented by a data protection module 33.

[0034] The focus point determination module 32 and the data protection module 33 can be controlled by a control unit 34. Settings of the focus point determination module 32, the data protection module 33, or the control unit 34 can be changed via a user interface 37. The data generated in the device 30 can be stored in a memory 35 of the device 30 as needed, for example, for later evaluation or for use by the components of the device 30. The focus point determination module 32, the data protection module 33, and the control unit 34 can be implemented as dedicated hardware, for example, as integrated circuits. Of course, they can also be partially or fully combined or implemented as software running on a suitable processor, such as a GPU or a CPU.Input 31 and output 36 can be implemented as separate interfaces or as a combined bidirectional interface. In the described example, device 30 is a standalone component. However, it can also be integrated into the augmented reality display device 1, 10 or into a control unit of the augmented reality display device 1, 10.

[0035] Fig. 5 Figure 41 schematically illustrates a method for providing information to control the display of an augmented reality device used by a first user. In this method, the gaze point of at least one second user is determined. For example, data about the second user's gaze direction can be received. The gaze point can then be determined from this gaze direction. Subsequently, information about the second user's gaze point is transmitted to the first user's augmented reality device. The transmission of this information can be subject to the second user's privacy settings. The second user does not necessarily need to use an augmented reality device to implement this method.

[0036] Fig. 6 Figure 50 shows a simplified schematic representation of a first embodiment of a device 50 for providing information to control the display of an augmented reality device 1, 10 of a first user. The device 50 has an input 51 through which, for example, data on the gaze direction of a second user can be received. The device 50 also has a gaze point determination module 52 for determining the gaze point of the second user. The gaze point can be determined, for example, from the gaze direction. Information about at least the gaze point of the second user is transmitted to the augmented reality device 1, 10 of the first user via an output 56 of the device 50. The transmission of information can be subject to privacy settings of the second user, which are implemented by a data protection module 53.

[0037] The focus point detection module 52 and the data protection module 53 can be controlled by a control unit 54. Settings of the focus point detection module 52, the data protection module 53, or the control unit 54 can be changed via a user interface 57. The data generated in the device 50 can be stored in a memory 55 of the device 50 as needed, for example, for later evaluation or for use by the components of the device 50. The focus point detection module 52, the data protection module 53, and the control unit 54 can be implemented as dedicated hardware, for example, as integrated circuits. Of course, they can also be partially or fully combined or implemented as software running on a suitable processor, such as a GPU or a CPU.The input 51 and the output 56 can be implemented as separate interfaces or as a combined bidirectional interface.

[0038] Fig. 7 Figure 1 schematically shows a method for controlling the display of an augmented reality device used by a first user. In a first step, information about the viewpoint of at least a second user is received. From this information, the viewpoint of the second user is determined and then displayed on the first user's augmented reality device. The viewpoint of the second user can be displayed saliently on the first user's augmented reality device. Similarly, elements on the first user's augmented reality device can be highlighted if they are also displayed to the second user.

[0039] Fig. 8 Figure 70 shows a simplified schematic representation of a first embodiment of a device 70 for controlling the display of an augmented reality device 1, 10 of a first user. The device 70 has an input 71 through which information about the viewpoint of at least a second user can be received. The device 70 also has a graphics generator 72 for displaying the viewpoint of the second user in the display of the augmented reality device 1, 10 of the first user. The viewpoint of the second user can be displayed saliently in the display of the augmented reality device of the first user. Similarly, elements in the display of the augmented reality device of the first user can be highlighted if they are also displayed to the second user. The display generated by the graphics generator 72 is output to the augmented reality device 1, 10 of the first user via an output 75 of the device 70.Alternatively, you can simply issue an instruction to generate a corresponding display.

[0040] The graphics generator 72 can be controlled by a control unit 73. Settings of the graphics generator 72 or the control unit 73 can be changed via a user interface 76. The data generated in the device 70 can be stored in a memory 74 of the device 70 as needed, for example, for later evaluation or for use by the components of the device 70. The graphics generator 72 and the control unit 73 can be implemented as dedicated hardware, for example, as integrated circuits. Of course, they can also be partially or completely combined or implemented as software running on a suitable processor, for example, a GPU or a CPU. The input 71 and the output 75 can be implemented as separate interfaces or as a combined bidirectional interface.In the described example, the device 70 is an independent component. However, it can also be integrated into the augmented reality display device 1, 10 of the first user or into a control unit of the augmented reality display device 1, 10.

[0041] Fig. 9 shows a simplified schematic representation of a second embodiment of the devices. Fig. 4 , Fig. 6 and Fig. 8 The device 80 comprises a processor 82 and a memory 81. For example, the device 80 is a computer or a control unit. Instructions are stored in the memory 81 that, when executed by the processor 82, cause the device 80 to perform the steps according to one of the described methods. The instructions stored in the memory 81 thus embody a program executable by the processor 82, which implements the method. The device 80 has an input 83 for receiving information. Data generated by the processor 82 is provided via an output 84. Furthermore, it can be stored in the memory 81. The input 83 and the output 84 can be combined into a bidirectional interface.

[0042] The processor 82 can comprise one or more processor units, such as microprocessors, digital signal processors, or combinations thereof.

[0043] The memory elements 35, 55, 74, 81 of the described embodiments can have both volatile and non-volatile memory areas and can include a wide variety of storage devices and storage media, for example hard disks, optical storage media or semiconductor memory.

[0044] Fig. 10 Figure 1 schematically depicts a motor vehicle 90 in which a solution according to the invention is implemented. The motor vehicle 90 has an augmented reality display device 1, 10 with an associated control unit 91. Furthermore, the motor vehicle 90 has a device 30 for providing information via the augmented reality device 1, 10, and a device 70 for controlling a display of the augmented reality device 1, 10. The devices 30, 70 can, of course, also be integrated into the augmented reality display device 1, 10 or into the control unit 91 of the augmented reality display device 1, 10. Further components of the motor vehicle 90 are a camera 92 for interior monitoring and a sensor system 93 for environmental detection, a navigation system 94, and a data transmission unit 95. A connection to a backend can be established via the data transmission unit 95. Memory 96 is available for storing data.Data exchange between the various components of the motor vehicle 90 takes place via a network 97.

[0045] A preferred embodiment of the invention will be described below with reference to the Figuren 11 und 12 be described.

[0046] Fig. 11 Figure 1 schematically shows a system diagram of the solution according to the invention. Three users 101, 102, 107 are connected to a backend 100 via a network 106. Both the first user 101 and the second user 102 use an augmented reality device 1, 10. Accordingly, each of these two users 101, 102 has a device 30 for providing information via the augmented reality device 1, 10 and a device 70 for controlling the display of the augmented reality device 1, 10. The devices 30 for providing information acquire data on the direction or focus of the two users 101, 102, and optionally data on objects in the users' field of vision, and transmit the acquired data to the backend 100 via the network 106.User 107 does not use an augmented reality device himself; however, suitable sensors collect information about his gaze direction and transmit it to the backend 100 via the network 106. The backend 100 evaluates and processes the received information. A device 50, which provides information for controlling the displays of the augmented reality devices 1 and 10, transmits information about the gaze positions of the various users 101, 102, and 107 to the relevant users 101 and 102. The devices 70, which control the displays of the augmented reality devices 1 and 10, then ensure that the gaze positions of the various users 101, 102, and 107 are displayed on the augmented reality devices 1 and 10.

[0047] Fig. 12Figure 1 schematically shows an application example that serves to illustrate the solution according to the invention. A first user 101 and a second user 102 are in a motor vehicle 90. Both users are wearing augmented reality glasses 1. In front of the motor vehicle 90, within the field of vision of both users 101 and 102, are two objects 105. In this example, these are a pedestrian and another motor vehicle. The viewpoint 103 of the first user 103, the driver of the motor vehicle 90, is on the motor vehicle in front. The viewpoint 104 of the second user 102, however, is on the pedestrian.

[0048] In vehicle 90, the gaze points 103, 104 of the two users 101, 102 are determined. For this purpose, for example, surveillance cameras (not shown) installed in vehicle 90 can be used to detect the head orientation of the vehicle occupants. This corresponds at least approximately to the respective gaze direction. Alternatively or additionally, sensors for determining the pose of the augmented reality glasses 1 can be used for this purpose. These sensors can be part of the augmented reality glasses 1 or installed in vehicle 90. Unlike the claimed solution, the respective gaze points determined from the gaze direction are transmitted in the local network to the connected augmented reality devices. In this example, these are the augmented reality glasses 1 of the vehicle occupants.Each of the connected augmented reality devices receives this information and overlays the respective viewpoints correctly into the field of vision of users 101 and 102. For example, the object 105 that is illuminated by the gaze of the other user 101 or 102 is highlighted. Options for this include, for example, color highlighting using hue, saturation, brightness, etc., a direct display of the viewpoint 103 or 104 on the surface of the illuminated object 105, a marker adjacent to object 105, or the display of a text message or annotation.

[0049] Depending on the use case, it is not always necessary to display the viewpoint of all users connected to the local network to every user. It may also be preferable to determine the viewpoint only and evaluate it for the mobility provider's purposes, without disclosing it to individual users. The way it is displayed can differentiate between elements visible to multiple users and elements visible to only one user.

[0050] Instead of being limited to a local network, the invention utilizes a backend functionality for sharing individual viewpoints. Users can preferably configure their respective augmented reality application's privacy settings to determine whether or not they wish to share their viewpoint. If they consent to sharing, the associated augmented reality device continuously transmits the viewing direction or viewpoint to the backend. The backend then shares the current viewpoint with authorized augmented reality devices in the user's vicinity. The distance at which a device is considered "nearby" can be individually configured by the users.

[0051] Analyzing gaze points in the backend enables numerous applications. While riding in an (autonomous) taxi, the gaze points of other passengers can be displayed. This can be used to create a game that can only be won by simultaneously focusing or aligning one's gaze on the same point. The point could, for example, be a moving object. Another application records whether an object is being viewed by one person or multiple people. If a viewer is looking at the object alone, e.g., a landmark, only details relevant to them are displayed. If two or more viewers are looking at the object, suggestions for a joint visit are presented. In yet another application, when a user looks at an (empty, unbooked) taxi, the system displays how many other people in the immediate vicinity are also interested in that vehicle.

[0052] The evaluation of received information in the backend can include a temporal component. For example, if a user's gaze lingers on a specific area for at least a defined period, this can be stored in the backend. This information can be used for company-specific analyses, such as determining a user's interests. A user's interests are particularly relevant for advertising purposes. The information can also be used as input for augmented reality applications. For instance, a user can be notified that a certain number of people viewed a specific object the previous day. Similarly, when viewing a landmark, the most frequently viewed points by other users over a defined period can be highlighted. Reference symbol list

[0053] 1 Augmented reality glasses 2 Camera 3 Interface 4 Pose determination device 5 Graphics unit 6 Projection unit 10 Head-up display 11 Imaging unit 12 Optical module 13 Projection surface 20 Determining a gaze direction 21 Determining a gaze point 22 Determining an object in the field of view 23 Transmitting information to a connected device 30 Device 31 Input 32 Gaze point determination module 33 Data protection module 34 Control unit 35 Memory 36 Output 37 User interface 40 Receiving data on a gaze direction of a second user 41 Determining a gaze point of the second user 42 Transmitting information to an augmented reality device of a first user 50 Device 51 Input 52 Gaze point determination module 53 Data protection module 54 Control unit 55 Memory 56 Output 57 User interface 60 Receiving information about a second user's viewpoint 61 Determining the second user's viewpoint 62 Displaying the second user's viewpoint70 Device 71 Input 72 Graphics Generator 73 Control Unit 74 Memory 75 Output 76 User Interface 80 Device 81 Memory 82 Processor 83 Input 84 Output 90 Motor Vehicle 91 Augmented Reality Display Device Control Unit 92 Camera 93 Sensor System 94 Navigation System 95 Data Transmission Unit 96 Memory 97 Network 100 Backend 101 First User 102 Second User 103 First User's Viewpoint 104 Second User's Viewpoint 105 Object 106 Network 107 Additional User

Claims

1. Method for controlling a display of an augmented reality device (1, 10) of a first user (101), comprising the steps of: - a backend (100) receiving (60) information about a viewpoint (104) of at least a second user (102); - the backend (100) transmitting the viewpoint (104) of the second user (102) to the augmented reality device (1, 10) of the first user (101); and - displaying (62) the viewpoint (104) of the second user (102) in the display of the augmented reality device (1, 10) of the first user (101), characterized in that the viewpoint (104) of the second user (102) is only transmitted from the backend (100) to the augmented reality device (1, 10) of the first user (101) when the augmented reality device (1, 10) of the first user (101) is in the vicinity of the second user (102), it being possible for the second user (102) to individually control the distance up to which a device is considered to be in the vicinity.

2. Method according to claim 1, wherein a viewing direction of the first user (101) is detected (20) by detecting a head orientation or by means of eye tracking and the viewpoint (103) is determined (21) from the viewing direction.

3. Method according to claim 1, wherein a time-related evaluation of the transmitted information takes place in the backend (100).

4. Method according to any one of the preceding claims, wherein the transmission of information to the augmented reality device (1, 10) of the first user (101) or the transmission (23) of information to the backend (100) is subject to privacy settings of the users (101, 102).

5. Method according to any one of the preceding claims, wherein the viewpoint (104) of the second user (102) is displayed saliently in the display of the augmented reality device (1, 10) of the first user (101) or elements are marked in the display of the augmented reality device (1, 10) of the first user (101) when they are also displayed to the second user (102).

6. Computer program having instructions that, when executed by a computer, cause the computer to carry out the steps of a method according to any one of claims 1 to 5.

7. System for controlling a display of an augmented reality device (1, 10) of a first user (101), comprising: - a backend (100) configured to receive (60) information about a viewpoint (104) of at least a second user (102) and to transmit the viewpoint (104) of the second user (102) to the augmented reality device (1, 10) of the first user (101); and - an apparatus (70) for controlling the display of the augmented reality device (1, 10) of the first user (101), having an input (71) for receiving (60) the viewpoint (104) of the second user (102) from the backend (100) and a graphics generator (72) for displaying the viewpoint (104) of the second user (102) in the display of the augmented reality device (1, 10) of the first user (101), characterized in that the viewpoint (104) of the second user (102) is only transmitted from the backend (100) to the augmented reality device (1, 10) of the first user (101) when the augmented reality device (1, 10) of the first user (101) is in the vicinity of the second user (102), it being possible for the second user (102) to individually control the distance up to which a device is considered to be in the vicinity.