Method and system for controlling the display of an augmented reality device of a first user

By collecting and sharing viewpoint information from augmented reality device users, the problem of multiple users being unable to share their environmental experience is solved, enabling secure and clear information sharing and interaction among augmented reality devices.

CN111902792BActive Publication Date: 2024-11-26VOLKSWAGEN AG
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN201980017582.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2018-03-14
Filing Date
2019-02-18
Publication Date
2024-11-26
Estimated Expiration
2039-02-18

AI Technical Summary

Technical Problem

In existing technologies, multiple users cannot simultaneously share the environmental experience of augmented reality devices, especially since it is impossible to determine and display the viewpoints of other users, resulting in unclear information sharing.

Method used

By determining the viewpoint of augmented reality device users and transmitting relevant information to connected devices, viewpoint sharing and display are achieved. Head orientation or eye tracking technology is used to collect the direction of gaze, and combined with object recognition and privacy settings, the security of information transmission and privacy protection are ensured.

Benefits of technology

It enables the sharing of environmental experiences among multiple users, improves the clarity and security of information, and enhances the interactivity of augmented reality devices.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN111902792B_ABST
    Figure CN111902792B_ABST
Patent Text Reader

Abstract

Information about the viewpoints (103, 104) of users (101, 102) is shared. The users (101, 102) may be users of an augmented reality device (1), but the viewpoints (103, 104) may also be determined for users (101, 102) who do not use an augmented reality device. The shared information may then be used as a basis for a range of applications and may enable the realization of a shared extended reality. For example, the viewpoint (104) of the second user (102) may be faded into the augmented reality display of the first user (101).
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to a method and a device for providing information by means of an augmented reality device, a method and a device for providing information for controlling the display of an augmented reality device, and a method and a device for controlling the display of an augmented reality device. The invention also relates to a computer-readable storage medium with instructions for executing the method and an augmented reality device using the method according to the invention or the device according to the invention. In addition, the invention relates to a motor vehicle having such an augmented reality device. Background Art

[0002] Over the years, we have seen an increasing penetration of digital technologies in society. It is foreseeable that this digitization of society will increase significantly in everyday life. Virtual reality and augmented reality and their applications represent a special form of digitalization. "Extended reality" is the process of enriching the real world with virtual elements, correctly registering these virtual elements in three-dimensional space and allowing real-time interaction. In the German-speaking academic community, the term "extended reality" is often used to describe the real world. )" prefers the expression "Augmented Reality", which will therefore be used below. The expression "Mixed Reality" is also used as a synonym. For example, augmented reality glasses can be used to represent an AR display. Augmented reality glasses are worn like ordinary glasses, but have one or more projection units or displays, with the help of which information can be projected in front of the eyes of the wearer of the glasses or directly onto the retina. In this case, the glasses are designed so that the wearer can also perceive the environment. New technologies for augmented reality glasses, such as light field technology, progress in the battery technology used and the involvement of large companies mean that an increasing acceptance of the devices can be expected.

[0003] Alternatively, in particular in the automotive sector, AR displays can also be generated with the aid of a head-up display (HUD). In a head-up display of a motor vehicle, the light beam of a display mounted on the dashboard is folded via a plurality of mirrors and lenses and reflected into the driver's eyes via a projection surface, so that the driver perceives a virtual image outside the vehicle. In the automotive sector, the windshield is usually used as a projection surface, the curvature of which must be taken into account in the display. As an alternative, additional sheets made of glass or plastic are also used in some cases, which are arranged on the dashboard between the driver and the front windshield.

[0004] In general, it is foreseeable that augmented reality glasses, augmented reality smartphones and augmented reality head-up displays will be increasingly used by end customers. Against this background, the following question arises: in what way can multiple users experience extended reality together, for example, how can multiple users see augmented information at the same time.

[0005] Here, US2013 / 0135180 A1 describes a system for distributed collaboration of multiple users. Active users wear data glasses connected to a central computer. The central computer receives environmental information from the data glasses and generates a spatial model based on this. In addition, the central computer receives information about the objects that should be processed and inputs from the active user and other users. Taking into account the user's input, a virtual image of the object is inserted into the spatial model and displayed to the active user.

[0006] US2014 / 0098131 A1 describes a method for using augmentation for augmented reality display. According to the method, augmentation data is received, which augmentation data is associated with a first augmentation, which has been used by a remotely located augmented reality device for an augmented reality display of a first actual situation. In addition, a second augmentation is used within the scope of an augmented reality display of a second actual situation. In this case, the display of the second augmentation depends at least in part on the received augmentation data. Finally, information about the use or non-use of the augmentation data is transmitted to the remotely located augmented reality device.

[0007] US2015 / 0178990 A1 describes a method for interaction in a vehicle. The vehicle has a head-up display for the driver and a simple display for the co-driver. In response to the driver's commands, augmented objects are generated based on the driver's current viewpoint and displayed on the co-driver's display. The co-driver can interact with the display. In response to this interaction, augmented objects for the head-up display are generated and displayed to the driver.

[0008] For the user of the augmented reality device, it is usually unclear which objects are being observed by the people around him. For example, the user can only guess which object the other user is referring to or what he is pointing to. Summary of the invention

[0009] The technical problem to be solved by the present invention is to disclose a solution capable of realizing environmental experience sharing by a large number of users.

[0010] This object is achieved by a method having the features of claims 1, 6 and 8, by a computer-readable storage medium having instructions according to claim 10, and by an apparatus having the features of claims 11, 12 and 13. Preferred embodiments of the invention are the subject matter of the dependent claims.

[0011] According to a first aspect of the present invention, a method for providing information through an augmented reality device comprises the steps of:

[0012] - determining the viewpoint of a user of the augmented reality device; and

[0013] - transmitting at least information about the user's viewpoint to an apparatus connected to the augmented reality device.

[0014] Correspondingly, the computer-readable storage medium contains instructions, which, when implemented by a computer, cause the computer to implement the following steps to provide information through an augmented reality device:

[0015] - determining the viewpoint of a user of the augmented reality device; and

[0016] - transmitting at least information about the user's viewpoint to an apparatus connected to the augmented reality device.

[0017] Similarly, the apparatus for providing information via an augmented reality device has:

[0018] - a viewpoint determination module for determining a viewpoint of a user of the augmented reality device; and

[0019] - an output for transmitting at least information about the user's viewpoint to a device connected to the augmented reality device.

[0020] According to a further aspect of the invention, a method for providing information for controlling a display of an augmented reality device of a first user comprises the steps of:

[0021] - determining the viewpoint of at least one second user; and

[0022] - Transmitting information about the second user's viewpoint to the first user's augmented reality device.

[0023] Correspondingly, the computer-readable storage medium includes instructions that, when implemented by a computer, cause the computer to implement the following steps to provide information for controlling the display of an augmented reality device of the first user:

[0024] - determining the viewpoint of at least one second user; and

[0025] - Transmitting information about the second user's viewpoint to the first user's augmented reality device.

[0026] Similarly, the apparatus for providing information for controlling the display of an augmented reality device of a first user has:

[0027] - a viewpoint ascertaining module for determining a viewpoint of at least one second user; and

[0028] - an output for transmitting information about the second user's viewpoint to the augmented reality device of the first user.

[0029] According to a further aspect of the present invention, a method for controlling the display of an augmented reality device of a first user comprises the steps of:

[0030] - receiving information about the viewpoint of at least one second user; and

[0031] -Fading in the second user's viewpoint in the display of the first user's augmented reality device.

[0032] Correspondingly, the computer-readable storage medium includes instructions, which, when implemented by a computer, cause the computer to implement the following steps to control the display of the augmented reality device of the first user:

[0033] - receiving information about the viewpoint of at least one second user; and

[0034] -Fading in the second user's viewpoint in the display of the first user's augmented reality device.

[0035] Similarly, the apparatus for controlling the display of the augmented reality device of the first user has:

[0036] - an input for receiving information about a viewpoint of at least one second user; and

[0037] - A graphics generator for fading in a second user's viewpoint in a display of the first user's augmented reality device.

[0038] The term "computer" used above should be understood in a broad sense. Computers also include control devices and other processor-based data processing devices.

[0039] The solution according to the invention is based on a method of sharing the focus or viewpoint of a user. On the one hand, the user can be the user of the augmented reality device, which makes the acquisition of the viewpoint easier. On the other hand, the determination of the viewpoint can also be performed for users who do not use an augmented reality device, as long as the corresponding viewpoint of the user can be determined by line of sight observation. The shared information can then be used as a basis for a series of applications and enable the realization of a shared extended reality.

[0040] According to one aspect of the invention, objects in the user's field of vision are determined. The information about the objects is then transmitted to a device connected to the augmented reality device. Due to the increasing computing power available in motor vehicles or smartphones, increasingly complex methods for object recognition can be implemented. The information about the objects recognized and classified by means of object recognition can be combined with the information about the user's viewing direction. By means of this fusion, dynamic objects (such as other traffic participants) can also be clearly identified and, if necessary, enriched with digital information.

[0041] According to one aspect of the invention, the line of sight direction is acquired by acquiring the head orientation or by means of eye tracking. The viewpoint is then determined based on the line of sight direction. There are already numerous solutions for acquiring the line of sight direction, so that the determination of the viewpoint can be achieved at low cost. For example, a driver observation camera or an interior space camera can be used in a motor vehicle to monitor the head orientation. Currently available data glasses also already provide sensor technology for calculating the head position and head orientation of the user in space. The head orientation then corresponds at least approximately to the line of sight direction. Then, for example, a point located in the center along the visual axis or the intersection of the sighting axis and the object hit along the sighting axis can be determined as the viewpoint.

[0042] According to one aspect of the invention, the connected device is a backend connected via a network. The transmission of information to the backend enables a multifaceted use of the information, for example for enterprise-specific analysis or for information exchange across users. Depending on the application, it is not mandatory to provide all relevant users with the viewpoints separately. It may also be desirable to determine the viewpoints separately and evaluate them for the purpose of the mobile service provider without sharing them with the individual users.

[0043] According to one aspect of the invention, a time-dependent evaluation of the transmitted information is performed in the backend. In this case, for example, the time point of the acquisition of the viewpoint can be evaluated. This type of time-dependent evaluation allows, for example, to determine how the interest of a user in a specific object develops over time. In the same way, it can be detected in this way whether multiple users focus their gaze on the same location at the same time or at least in a specific time period. Alternatively or additionally, the time period during which the user's gaze remains stationary in a specific area can also be evaluated. If this time period exceeds a defined minimum duration, this can be used as a trigger for the continued step.

[0044] According to one aspect of the invention, the transmission of information to the augmented reality device or to an apparatus connected to the augmented reality device is subject to the privacy settings of the user. Privacy settings that can be influenced by the user allow ensuring data protection or provide the user with the possibility of controlling the distribution of the collected information. For example, the user can determine whether and which information is collected or transmitted, and in what form and to what extent the information is evaluated or forwarded to another user.

[0045] According to one aspect of the invention, the viewpoint of the second user is highlighted in the display of the augmented reality device of the first user. By highlighting, it is immediately visible to the user where the other user is looking, even if the viewpoint is only in the user's external field of view. Alternatively or additionally to this, the element in the display of the augmented reality device of the first user is marked if it is also displayed to the second user. By such marking of the element in the display, it is clearly indicated to the user that information about the user's viewpoint is shared.

[0046] Particularly advantageously, the method according to the invention or the device according to the invention is used in an augmented reality device, for example in augmented reality glasses, in an augmented reality smartphone or in an augmented reality head-up display. Such an augmented reality device can be used, for example, in a vehicle, in particular in a motor vehicle. In principle, however, it can be used in all forms of transportation, even in non-motorized vehicles. In motorized vehicles, the use is not limited to the driver; the co-driver, passengers or other persons not directly involved in the driving task can also be users of the corresponding augmented reality device. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Further features of the present invention are apparent from the following description and appended claims taken in conjunction with the accompanying drawings.

[0048] Figure 1 Augmented reality glasses are schematically shown;

[0049] Figure 2 Schematically illustrates the general structure of a head-up display device for a motor vehicle;

[0050] Figure 3 A method for providing information via an augmented reality device is schematically illustrated;

[0051] Figure 4 A first embodiment of an apparatus for providing information via an augmented reality device is shown;

[0052] Figure 5 Schematically illustrated is a method for providing information for controlling a display of an augmented reality device;

[0053] Figure 6 A first embodiment of an apparatus for providing information for controlling a display of an augmented reality device is shown;

[0054] Figure 7 A method for controlling the display of an augmented reality device of a first user is schematically illustrated;

[0055] Figure 8 A first embodiment of an apparatus for controlling a display of an augmented reality device of a first user is shown;

[0056] Fig. 9 Shown according to Figure 4 , Figure 6 and Figure 8 A second embodiment of the device;

[0057] Fig.10 A motor vehicle is schematically shown in which the solution according to the invention is implemented;

[0058] Fig.11 A system diagram schematically showing a solution according to the invention; and

[0059] Fig.12 An example of application of the solution according to the invention is schematically shown. DETAILED DESCRIPTION

[0060] In order to better understand the principle of the present invention, the embodiments of the present invention are described in more detail below with reference to the accompanying drawings. It goes without saying that the present invention is not limited to these embodiments, and the described features can also be combined or modified without departing from the protection scope of the present invention as defined in the attached claims.

[0061] Figure 1 Augmented reality glasses 1 are schematically shown as an example for an augmented reality display device. The augmented reality glasses 1 have a camera 2 for acquiring environmental data of the environment of the augmented reality glasses 1. By means of an interface 3, there is a connection to an external posture determination device 4, which is used to determine the posture of the augmented reality glasses 1 based on the environmental data. For this purpose, the posture determination device 4 can, for example, detect and evaluate edges or mark significant points in the image data. The posture determination device 4 can of course also be a component of the augmented reality glasses 1. With the help of a graphics unit 5, the display of the augmented reality glasses 1 can be adjusted corresponding to the determined posture. For this purpose, the posture determination device 4 and the graphics unit 5 can exchange data with each other. As shown in the example, the graphics unit 5 can be a component of the augmented reality glasses 1 or can be connected to the augmented reality glasses 1 via the interface 3. Via the projection unit 6, the display generated by the graphics unit 5 is faded in. In Figure 1In the example in , a monocular display is used, i.e. the display is arranged in front of only one eye of the user. Of course, the currently described solution can also be implemented with augmented reality glasses 1 having a binocular display, in which one display is used for each eye, or with augmented reality glasses 1 having a binocular display, in which a common display is used for both eyes.

[0062] Figure 2 A head-up display 10 is schematically shown as an example for an augmented reality display device for a motor vehicle 90. With the help of the head-up display 10, content can be displayed on a projection surface 13 of the motor vehicle 90, for example on the windshield or on an additional sheet made of glass or plastic, which is arranged on the dashboard between the driver and the windshield. The displayed content is generated by an imaging unit 11 and projected onto the projection surface 13 with the help of an optical module 12. In this case, the projection is usually realized in the area of ​​the windshield above the steering wheel. The imaging unit 11 can be, for example, an LCD-TFT display. The head-up display 10 is usually installed in the dashboard of the motor vehicle 90.

[0063] Figure 3 A method for providing information via an augmented reality device is schematically shown. According to the invention, the viewpoint of a user of the augmented reality device is determined 21. To this end, the direction of the user's gaze can first be acquired 20, for example, by acquiring a head orientation or by means of eye tracking. The viewpoint can then be determined based on the gaze direction. Finally, the information about the user's viewpoint is transmitted 23 to a device connected to the augmented reality device. The connected device can, for example, be a backend connected via a network. In addition, objects in the user's field of vision can be determined 22. Information about the objects can then also be transmitted to the connected device. In this case, the transmission 23 of the information can be subject to the user's privacy settings.

[0064] Figure 4 A simplified schematic diagram of a first embodiment of an apparatus 30 for providing information via an augmented reality device 1, 10 is shown. The apparatus 30 has an input 31, via which, for example, data about the direction of sight of a user of the augmented reality device 1, 10 can be received. In addition, the apparatus 30 has a viewpoint determination module 32 for determining the viewpoint of the user. For example, the viewpoint can be determined based on the direction of sight. Via an output 36 of the apparatus 30, at least information about the viewpoint of the user is output to a connected device. The connected device can be, for example, a backend 100 connected via a network or an augmented reality device of another user. In this case, the transmission of information can be subject to the privacy settings of the user, which are implemented by a data protection module 33.

[0065] The viewpoint determination module 32 and the data protection module 33 can be controlled by the control unit 34. If necessary, the settings of the viewpoint determination module 32, the data protection module 33 or the control unit 34 can be changed through the user interface 37. When necessary, the data accumulated in the device 30 can be stored in the memory 35 of the device 30, for example, for later evaluation or for use by the components of the device 30. The viewpoint determination module 32, the data protection module 33 and the control unit 34 can be implemented as dedicated hardware, for example, as an integrated circuit. Of course, they can also be partially or completely combined or implemented as software, which runs on a suitable processor, such as a GPU or a CPU. The input terminal 31 and the output terminal 36 can be implemented as separate interfaces or as a combined bidirectional interface. In the described example, the device 30 is an independent component. However, the device can also be integrated in the augmented reality display device 1, 10 or the control device of the augmented reality display device 1, 10.

[0066] Figure 5 A method for providing information for controlling the display of an augmented reality device of a first user is schematically shown. According to the invention, a viewpoint of at least one second user is determined 41. For this purpose, for example, data about the direction of a line of sight of the second user can be received 40. The viewpoint can then be determined based on the line of sight. Subsequently, the information about the viewpoint of the second user is transmitted 42 to the augmented reality device of the first user. In this case, the transmission 42 of the information can be subject to the privacy settings of the second user. In order to carry out the method, the second user does not necessarily also have to use an augmented reality device.

[0067] Figure 6 A simplified schematic diagram of a first embodiment of an apparatus 50 for providing information for controlling the display of an augmented reality device 1, 10 of a first user is shown. The apparatus 50 has an input 51, via which data about the direction of sight of a second user can be received, for example. In addition, the apparatus 50 has a viewpoint ascertainment module 52 for determining the viewpoint of the second user. For example, the viewpoint can be determined based on the direction of sight. Via an output 56 of the apparatus 50, at least information about the viewpoint of the second user is transmitted to the augmented reality device 1, 10 of the first user. In this case, the transmission of the information can be subject to the privacy settings of the second user, which are implemented by a data protection module 53.

[0068] The viewpoint ascertainment module 52 and the data protection module 53 can be controlled by a control unit 54. If necessary, the settings of the viewpoint ascertainment module 52, the data protection module 53 or the control unit 54 can be changed through the user interface 57. When necessary, the data accumulated in the device 50 can be stored in the memory 55 of the device 50, for example, for later evaluation or for use by components of the device 50. The viewpoint ascertainment module 52, the data protection module 53 and the control unit 54 can be implemented as dedicated hardware, for example, as an integrated circuit. Of course, they can also be partially or completely combined or implemented as software, which runs on a suitable processor, such as a GPU or a CPU. The input terminal 51 and the output terminal 56 can be implemented as separate interfaces or as a combined bidirectional interface.

[0069] Figure 7 A method for controlling the display of an augmented reality device of a first user is schematically shown. In a first step, information about the viewpoint of at least one second user is received 60. Based on this information, the viewpoint of the second user is determined 61 and then faded in 62 into the display of the augmented reality device of the first user. In this case, the viewpoint of the second user can be prominently displayed in the display of the augmented reality device of the first user. If an element in the display of the augmented reality device of the first user is also displayed to the second user, the element in the display of the augmented reality device of the first user can also be marked.

[0070] Figure 8 A simplified schematic diagram of a first embodiment of an apparatus 70 for controlling the display of an augmented reality device 1, 10 of a first user is shown. The apparatus 70 has an input 71, via which information about the viewpoint of at least one second user can be received. In addition, the apparatus 70 has a graphics generator 72, which is used to fade in the viewpoint of the second user in the display of the augmented reality device 1, 10 of the first user. Here, the viewpoint of the second user can be prominently displayed in the display of the augmented reality device of the first user. If an element in the display of the augmented reality device of the first user is also displayed to the second user, the element in the display of the augmented reality device of the first user can also be marked. The display generated by the graphics generator 72 is output to the augmented reality device 1, 10 of the first user via the output 75 of the apparatus 70. Alternatively, only an indication of the generation of the corresponding display can also be output.

[0071] The graphics generator 72 can be controlled by the control unit 73. If necessary, the settings of the graphics generator 72 or the control unit 73 can be changed through the user interface 76. When necessary, the data accumulated in the device 70 can be stored in the memory 74 of the device 70, for example, for later evaluation or for use by the components of the device 70. The graphics generator 72 and the control unit 73 can be implemented as dedicated hardware, for example, as an integrated circuit. Of course, they can also be partially or completely combined or implemented as software, which runs on a suitable processor, such as a GPU or a CPU. The input end 71 and the output end 75 can be implemented as separate interfaces or as a combined bidirectional interface. In the described example, the device 70 is an independent component. However, the device can also be integrated in the augmented reality display device 1, 10 of the first user or the control device of the augmented reality display device 1, 10.

[0072] Fig. 9 Shown according to Figure 4 , Figure 6 and Figure 8 80 is a simplified schematic diagram of a second embodiment of an apparatus according to the present invention. The apparatus 80 has a processor 82 and a memory 81. For example, the apparatus 80 is a computer or a control device. In the memory 81, instructions are stored, which, when executed by the processor 82, cause the apparatus 80 to perform steps according to one of the described methods. Thus, the instructions stored in the memory 81 embody a program executable by the processor 82, which program implements the method according to the present invention. The apparatus 80 has an input 83 for receiving information. The data generated by the processor 82 are provided via an output 84. In addition, the data generated by the processor can be stored in the memory 81. The input 83 and the output 84 can be combined into a bidirectional interface.

[0073] Processor 82 may include one or more processor units, such as a microprocessor, a digital signal processor, or a combination thereof.

[0074] The memory 35 , 55 , 74 , 81 of the described embodiments may have volatile and non-volatile memory areas and may include different storage devices and storage media, such as a hard disk, an optical storage medium or a semiconductor memory.

[0075] Fig.10A motor vehicle 90 is schematically shown in which the solution according to the invention is implemented. The motor vehicle 90 has an augmented reality display device 1, 10 with an associated control device 91. In addition, the motor vehicle 90 has a device 30 for providing information via the augmented reality device 1, 10 and a device 70 for controlling the display of the augmented reality device 1, 10. Of course, the devices 30, 70 can also be integrated into the augmented reality display device 1, 10 or the control device 91 of the augmented reality display device 1, 10. Other components of the motor vehicle 90 are a camera 92 for monitoring the interior space and a sensor system 93 for environmental acquisition, a navigation system 94, and a data transmission unit 95. With the help of the data transmission unit 95, a connection to the back end can be established. There is a memory 96 for storing data. The data exchange between the different components of the motor vehicle 90 is carried out via a network 97.

[0076] The following reference Fig.11 and Fig.12 To describe the preferred embodiments of the present invention.

[0077] Fig.11 A system diagram of a solution according to the invention is schematically shown. Exemplarily, three users 101, 102, 107 are connected to a backend 100 via a network 106. Both the first user 101 and the second user 102 use augmented reality devices 1, 10. Accordingly, on the two users 101, 102 side, a device 30 for providing information via the augmented reality devices 1, 10 and a device 70 for controlling the display of the augmented reality devices 1, 10 are respectively provided. The device 30 for providing information collects data about the sight direction or viewpoint of the two users 101, 102, and if necessary, collects data about objects in the field of vision of the users 101, 102, and transmits the collected data to the backend 100 via the network 106. The other user 107 does not use the augmented reality device itself, but the information about the sight direction of the other user is also collected by a suitable sensor mechanism and transmitted to the backend 100 via the network 106. The evaluation and processing of the received information is carried out in the backend 100. By means of the device 50 for providing information for controlling the display of the augmented reality device 1, 10, information about the viewpoints of the different users 101, 102, 107 is transmitted to the relevant user 101, 102. The device 70 for controlling the display of the augmented reality device 1, 10 then ensures that the viewpoints of the different users 101, 102, 107 are faded into the display of the augmented reality device 1, 10.

[0078] Fig.12An example of application of the solution according to the invention is schematically shown. A first user 101 and a second user 102 are located in a motor vehicle 90. Both users use augmented reality glasses 1. In front of the motor vehicle 90, in the field of vision of the two users 101, 102, there are two objects 105. In the example, these are pedestrians and another motor vehicle. The viewpoint 103 of the first user 103, i.e. the driver of the motor vehicle 90, is located on the motor vehicle traveling in front. The viewpoint 104 of the second user 102 is located on the pedestrian.

[0079] The viewpoints 103, 104 of the two users 101, 102 are determined in the motor vehicle 90. For this purpose, for example, an observation camera (not shown) installed in the motor vehicle 90 can be used, with which the head orientation of the vehicle passengers is captured. This head orientation corresponds at least approximately to the corresponding line of sight direction. Alternatively or additionally, the sensor system for posture determination of the augmented reality glasses 1 can also be used for this purpose. The sensor system can be a component of the augmented reality glasses 1 or can also be installed in the motor vehicle 90. The corresponding viewpoint determined according to the line of sight is forwarded in the local network to the connected augmented reality device. In the example, this is the augmented reality glasses 1 of the vehicle passenger. Each of the connected augmented reality devices receives this information and correctly fades the corresponding viewpoint position into the field of view of the users 101, 102. For example, the object 105 hit by the line of sight of each other user 101, 102 is highlighted. Options for this are, for example, color highlighting by means of hue, saturation, brightness, etc.; direct display of viewpoint 103 , 104 on the surface of the hit object 105 ; marking adjacent to object 105 ; or display of textual hints or comments.

[0080] Depending on the application, it is not mandatory to display the viewpoints of all users connected to the local network for all users. It may also be desirable to determine the viewpoints only individually and evaluate them for the purposes of the mobile service provider without sharing them with the individual users. In this case, a distinction can be made between elements visible to multiple users and elements visible to only one user by the type of display.

[0081] Instead of being limited to the local network, the backend functionality can also be used to share the respective sight directions or viewpoints. Here, the user can preferably determine in the privacy settings of the user's corresponding augmented reality application whether the user needs to share the user's viewpoint. If the user agrees to forwarding, the associated augmented reality device continuously transmits the sight direction or viewpoint to the backend. The backend evaluates the received information or shares the current viewpoint with authorized augmented reality devices near the user. Preferably, the user can individually adjust at what distance a device is considered "nearby" or which devices are "authorized".

[0082] The evaluation of viewpoints in the backend enables numerous applications. The viewpoints of other passengers can be displayed while driving an (automatic) taxi. Games can be built on this basis, which can only be won by focusing at the same time or directing the line of sight to the same point. The point can be a moving object, for example. In other applications, it is collected whether the object is observed by one person or multiple people. If the observer is only looking at the object, such as a place of interest, only the relevant details are displayed to this observer separately. On the other hand, if two or more observers are looking at the object, a suggestion for a joint visit is made. Corresponding to another application, when the user looks at a (empty, unreserved) taxi, it is displayed how many people from the surrounding environment are still interested in the vehicle.

[0083] The evaluation of the information received in the backend can include a time component. For example, if the sight remains stationary in a specific area for at least a defined time period, this can be stored in the backend. In addition, this information can be used for enterprise-specific analysis, for example, in order to determine where the user is interested. The interests of the user are particularly relevant for advertising purposes. The information can also be used as input values ​​for augmented reality applications. For example, the user can be informed that a certain number of people observed a specific object the day before. Similarly, when looking at places of interest, the locations that were observed most by other people in a defined past time period can be marked.

[0084] Reference numerals list

[0085] 1. Augmented reality glasses

[0086] 2 Camera

[0087] 3 Interface

[0088] 4 Posture determination device

[0089] 5 Graphics Unit

[0090] 6 Projection unit

[0091] 10 Heads-up display

[0092] 11 Imaging unit

[0093] 12 Optical Module

[0094] 13 Projection surface

[0095] 20 Determine the direction of sight

[0096] 21 Determine the viewpoint

[0097] 22 Identify objects in the field of view

[0098] 23 Transmit information to connected devices

[0099] 30 Devices

[0100] 31 Input

[0101] 32 Viewpoint determination module

[0102] 33 Data Protection Module

[0103] 34 Control Unit

[0104] 35 Memory

[0105] 36 Output

[0106] 37 User Interface

[0107] 40 Receive data about the second user's line of sight

[0108] 41 Determining the Second User’s Viewpoint

[0109] 42 Transmitting information to the augmented reality device of the first user

[0110] 50 Devices

[0111] 51 Input

[0112] 52 Viewpoint Identification Module

[0113] 53 Data Protection Module

[0114] 54 Control Unit

[0115] 55 Memory

[0116] 56 Output

[0117] 57 User Interface

[0118] 60 Receiving information about the second user's viewpoint

[0119] 61 Determining the Second User's Viewpoint

[0120] 62 Fade in the second user's viewpoint

[0121] 70 Devices

[0122] 71 Input

[0123] 72 Graph Generator

[0124] 73 Control Unit

[0125] 74 Memory

[0126] 75 Output

[0127] 76 User Interface

[0128] 80 Devices

[0129] 81 Memory

[0130] 82 Processors

[0131] 83 Input

[0132] 84 Output

[0133] 90 Motor Vehicles

[0134] 91. Control device for augmented reality display device

[0135] 92 Camera

[0136] 93 Sensor System

[0137] 94 Navigation System

[0138] 95 Data Transmission Unit

[0139] 96 Memory

[0140] 97 Network

[0141] 100 Backend

[0142] 101 First User

[0143] 102 Second User

[0144] 103 First User's Viewpoint

[0145] 104 Second User's Viewpoint

[0146] 105 Objects

[0147] 106 Network

[0148] 107 other users

Claims

1. A method for controlling the display of an augmented reality device of a first user, comprising the steps of: - receiving, by the backend, information about the viewpoint of at least one second user; - transmitting the viewpoint of the second user to the augmented reality device of the first user via the backend; and - displaying the second user's viewpoint in the display of the first user's augmented reality device; It is characterized in that Only when the augmented reality device of the first user is in the vicinity of the second user is the second user's viewpoint transmitted from the backend to the augmented reality device of the first user, wherein the second user is individually set at what distance the device is considered to be in the vicinity.

2. The method according to claim 1, wherein: The sight direction of the first user is acquired by acquiring the head orientation or by means of eye tracking, and the viewpoint is determined according to the sight direction.

3. The method according to claim 1, wherein: In the backend, a time-dependent evaluation of the transmitted information takes place.

4. The method according to any one of claims 1 to 3, wherein: The receipt of information by the augmented reality device or the transmission of information to the backend is subject to the user's privacy settings.

5. The method according to any one of claims 1 to 3, wherein: The second user's viewpoint is prominently displayed in the display of the first user's augmented reality device, or the element in the display of the first user's augmented reality device is marked if the element in the display of the first user's augmented reality device is also displayed to the second user. 6 . A computer-readable storage medium having instructions which, when executed by a computer, cause the computer to execute the steps of the method according to claim 1 .

7. A system for controlling a display of an augmented reality device of a first user, comprising: a backend designed to receive information about a viewpoint of at least one second user and to transmit the viewpoint of the second user to an augmented reality device of the first user; and - an apparatus for controlling the display of an augmented reality device of a first user, the apparatus having an input for receiving a viewpoint of a second user by a backend and a graphics generator for displaying the viewpoint of the second user in the display of the augmented reality device of the first user; It is characterized in that Only when the augmented reality device of the first user is in the vicinity of the second user is the second user's viewpoint transmitted from the backend to the augmented reality device of the first user, wherein the second user is individually set at what distance the device is considered to be in the vicinity.

8. An augmented reality device, characterized in that: The augmented reality device is configured to implement the method according to any one of claims 1 to 5.

9. A motor vehicle, characterized in that: The motor vehicle has an augmented reality device according to claim 8 .