Vehicle, apparatus and method for adjusting depth of detail of augmented reality user interface of vehicle

Abstract

The invention relates to a vehicle (10), a device (1) and a method for adjusting the depth of detail of an augmented reality user interface of a vehicle (10). The method comprises the steps of determining a predefined position (2, 3, 4, 5, 6) of a finger (7) of the user (9) on a translucent surface (11) in an interior space of the vehicle (10), in response thereto, adjusting a depth of detail of the augmented reality display (12).

Description

Field of the Invention

[0001] The present invention relates to a vehicle, a device and a method for adjusting the depth of detail of an augmented reality user interface of a vehicle. In particular, the present invention relates to providing a hidden interaction possibility for a user when configuring an augmented reality (AR) blend in a vehicle. Background Art

[0002] Generally, a user operates their vehicle via hardware buttons / sliders and a touchscreen or other hardware-designed touch-sensitive and / or pressure-sensitive user interfaces. A particularly hidden interaction possibility is now known as "Shy Tech". Operating elements are displayed semi-transparently on a surface (which may have, for example, a fabric and / or wood veneer, etc.). The display of the user interface is performed by rear projection (rear projection), while the determination of the position of the user's finger on the user interface is performed by sensors integrated in the material (capacitive, resistive, etc.).

[0003] In addition, AR user interfaces are also becoming increasingly popular in vehicles. This is understood as an artificial supplement to the vehicle environment directly visible to the passengers. For example, after determining the position of the user's eyes and the position of environmental objects, contact simulation prompts regarding the environmental objects are also output to the user here. The AR content is now displayed at the intersection of the optical axis between the above points and the display surface. For selecting AR content or AR functions that can be configured if necessary, there is currently no setting possibility on the user side. In other words, the user always depends on the content of the AR user interface provided and displayed by the developer side. Summary of the Invention

[0004] Based on the above prior art, the task of the present invention is to enable convenient user-acceptable AR-related user interface configuration.

[0005] According to the present invention, the above task is solved by a method for adjusting the depth of detail of an augmented reality user interface of a vehicle.

[0006] The dependent claims give preferred extensions of the present invention.

[0007] This method can be implemented in a vehicle (car, transporter, motorcycle, truck, airplane, and / or ship). The augmented reality user interface can use the vehicle's display device and / or a display device carried by the user. In a first step, the predefined position of the user's finger on a translucent surface within the interior space of the vehicle is detected. In other words, when the user's finger approaches or touches the translucent surface, the position of the user's finger is located. This can be achieved, for example, by means of an infrared camera that continuously detects the translucent surface. The user's finger can be determined based on its infrared signature. In response to determining the finger at the predefined position, the level of detail of the augmented reality display is adjusted via the augmented reality user interface. This adjustment can be performed automatically. In other words, the predefined position represents a specific level of detail that can be predefined step by step and / or continuously. The level of detail can relate to different contents / features of the augmented reality display. For example, in a first level, environmental objects of a specific category may not be highlighted by AR content, while in a second level of detail, environmental objects of these categories are visually highlighted for the user of the interface according to the invention. However, when the level of detail is increased, multiple user interfaces / display devices / enlarged display surfaces or the like can also be used. Further examples of the level of detail are given in the following parts of this specification.

[0008] Preferably, the position of the finger is determined by means of an optical sensor. The optical sensor can be arranged, for example, in the roof lining and / or in the A-pillar and / or in the roof operating module / function center "roof". The optical sensor can include, for example, an infrared camera, which can also be used for other functions in the vehicle (such as seat occupancy recognition, fatigue recognition, vital sign detection, or the like). Thus, the hardware costs for implementing the device according to the invention are reduced.

[0009] The translucent surface can be arranged, for example, in the door trim of the vehicle and / or on the dashboard. It can have fibers and / or pores or the like through which the light projected from the back can emerge from the surface of the translucent surface with a relatively clear contour. In this way, switch elements and / or their operating states (activated / deactivated) or the like can be displayed.

[0010] The preferred translucent surface may have a fabric and / or a wood veneer and / or a fiber material. The holes and / or the light conductor elements may be embedded in the surface in such a way that the optical appearance corresponds to a natural material, in particular an organic or inorganic material, in a designed rear projection. In other words, when the display device is turned off, in particular the classic appearance of the matrix display cannot be seen. Only by activating the rear projection can the dynamic appearance of the user interface according to the invention be activated by means of the translucent surface. This does not exclude the possibility of visually perceiving predefined positions on the translucent surface even without rear projection. For example, the material may be structured and / or colored in other ways so that the predefined positions can still be recognized without rear projection. In particular, in response to recognizing that a user's finger forms a predefined spatial relationship with a predefined position or another predefined position, the rear projection can be activated and thus more visual details can be added to the user interface. By activating the rear projection, for example, one predefined position or a plurality of (in particular adjacent) predefined positions, preferably all predefined positions, of the user interface can be marked and / or visually highlighted. For example, the predefined positions can also be visually integrated by adding an edge line, a border or the like around the predefined positions (which are arranged in a circular structure shape or in a linear shape in particular). The depth of detail of the AR display may have other accompanying parameters that are not necessarily based on AR technology, and these accompanying parameters are output, for example, by means of a vibration exciter and / or a ventilation device and / or a heating / air conditioning device and / or an aroma and / or a conventional display device. For example, a vibration exciter can be used at a first level of detail in order to present a specific event / environmental object, while a vibration exciter should not be used at another (in particular lower) level of detail. This also applies to the use of an aroma bottle and / or a ventilation device setting and / or a heating / air conditioning device temperature. If a dynamically dimmable window glass is provided in a vehicle, the window glass can be dimmed at a first (higher) level of detail, while this dimming is not provided at a lower level of detail. This also applies to the number of projectors to be used, the category and position of the environmental objects indicated during the AR display, and the actuators to be used and their parameters in response to predefined events.

[0011] The levels of detail that can be activated or switched via the above user interface can for example relate to the following: In the first level (level 0), all AR and accompanying signal notifications are switched off. In the second level (level 1), a conventional head-up display with known content is used. Lane departure warnings are signaled, if necessary, by means of seat vibration. In the third level (level 2), the head-up display is used, via three-dimensional elements (depth head-up display), optionally also via seat vibration patterns, smartwatch binding, and a ventilation device for ventilation and scent diffusion, to provide an AR user experience. In the final level (level 4), the vehicle's window glass is additionally dimmed (for example based on electrochromic substances), and in response, a fully virtual environmental image is projected onto the now dimmed window glass. Here, if necessary, environmental objects and other environmental features can in principle be represented arbitrarily by alternative display objects, such that the user experiences environmental entertainment alienation when needed. If the window glass is only partially opaque in this state, the display corresponds to an AR environment. If the window glass is (almost) completely opaque, the display can become virtual reality (VR).

[0012] Regardless of the current AR / VR detail depth selected by the user, automatically locally releasing the display device used therefor can be ergonomically and driving safety-technologically advantageous, especially when determining environmental objects with predefined characteristics in the environment of a vehicle by means of sensor technology. The environmental object can, for example, be located in the collision-related environment of the vehicle with the aid of a camera, with the aid of a lidar sensor, and / or with the aid of an ultrasonic sensor. In response thereto, a local reduction or clearing of the display content in the AR / VR display can be effected, especially at the location in the line of sight of the user to the environmental object. In other words, regardless of the currently displayed display content, such an area is switched completely or partially transparent, and the area switched transparent is to ensure that the user's line of sight to the environmental object is at least partially, especially completely unobstructed. Here, a predefined border around the environmental object can be displayed or its content can be released in a simplified or completely content-cleared manner in order to better classify the situation of the environmental object into the actual vehicle environment. Preferably, the line-of-sight trajectory of the display device can also be predicted, so that a larger area is retained in the visual movement direction from the user's perspective, where the display content is reduced or completely cleared. In this way, the user can, in particular, unobstructedly and naturally see the environmental area where the environmental object will be located based on the relative trajectory between their own vehicle and the environmental object. Alternatively or additionally, depending on the position of the environmental object, a predefined display area (e.g., the left half, right half, lower half, upper half, etc. of the display) can also be switched completely transparent or the AR / VR content can be reduced. In this way, an area or corona is formed around the environmental object, within which the user can see the environment substantially or completely unobstructed. Of course, for this purpose, in an advantageous embodiment, the position of the user's eyes is first determined by means of sensor technology in order to be able to determine the line of sight and keep it unobstructed as described above. This method is in principle known in the prior art.

[0013] The predefined characteristics of the environmental object can in particular determine to what extent the user prefers to see the environmental object unobstructedly rather than having to observe it through the AR / VR display. In particular, reference can be made to collision relevance or information associated therewith. Such information is, for example, distance, speed, relative movement direction, predicted trajectory, mass, the material of the environmental object, or other information. The environmental object can also be identified (e.g., wirelessly, information-technology-wise, electromagnetically, etc.). In particular, the environmental object can also be classified for this purpose, and predefined characteristics are assigned to the found class, which can be used for the transparent switching of the display as described above.

[0014] According to a second aspect of the present invention, there is provided an apparatus for adjusting the depth of detail of an augmented reality user interface of a vehicle. The user interface may also be understood as a driver assistance system of a vehicle (such as a car, a transport vehicle, a motorcycle, a truck, an airplane, and / or a ship). The apparatus includes a data input terminal, an evaluation unit (such as an electronic controller), and a data output terminal. The evaluation unit is thus configured to determine, via the data input terminal, a predefined position of a user's finger on a translucent surface in the interior space of the vehicle and, in response thereto, adjust, if necessary via the data output terminal, the depth of detail of the augmented reality display. This step can be performed automatically. In other words, the apparatus according to the present invention is apparently configured to implement features, feature combinations, and advantages resulting therefrom in a corresponding manner. For the sake of avoiding repetition, reference is made to the above description in connection with the method according to the present invention.

[0015] According to a third aspect of the present invention, there is provided a vehicle having the apparatus according to the second aspect of the invention. In this regard, features, feature combinations, and advantages are also produced in a corresponding manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Other details, features, and advantages of the present invention can be derived from the following description and the drawings. The drawings are as follows: [[ID=--]] [[ID=--]]

[0017] [[ID=--]] Figure 1 A perspective view showing the interior space of a vehicle according to the present invention, having an embodiment of the apparatus according to the present invention when implementing the method according to the present invention; and [[ID=--]] [[ID=--]]

[0018] [[ID=--]] Figure 2 A flowchart showing the steps of an embodiment of a method for adjusting the depth of detail of an augmented reality user interface of a vehicle. [[ID=--]] DETAILED DESCRIPTION [[ID=--]] [[ID=--]]

[0019] [[ID=--]] Figure 1Shows the driver's driving position of an embodiment of a vehicle 10 in the form of a sedan according to the present invention. The user 9 is sitting in the driver's seat and wishes to adjust the visual highlighting of the cyclist 15 located on the right side of the vehicle 10. To this end, he slides his finger 7 at a predefined position 3 on the translucent surface 11 located in the instrument panel. The predefined position is permanently marked invisibly. This also applies to the predefined positions 2, 4, 5, and 6 adjacent to the predefined position 3. The infrared camera 22 in the A-pillar area detects the finger 7 on the predefined position 3 and reports this data via the data input 19 to the electronic controller serving as the evaluation unit 20. The evaluation unit 20 recognizes the finger 7 of the user 9 and uses the data output 21 to provide a rear-projection projector 8 to display the edges around the predefined positions 2 to 6. By moving his finger 7, the user 9 can now select the level of detail associated with one of the predefined positions 2 to 6 in order to adjust the augmented reality display 12 around the cyclist 15 according to his preferences. In the process, the operating parameters of the ventilation device 16 (which can release the fragrance 13 into the vehicle interior) and the vibration exciter 14 can also be adjusted via the data output 21. The higher the level of detail of the AR display, the more the above-mentioned actuators and the features of the vehicle 10 are used to provide the AR user experience. In the process, reference should also be made to the projector 17 of the head-up display, the display surface 18 of which can be enlarged or reduced according to the selected predefined positions 2 to 6. With the device 1 shown, the method discussed in conjunction with Figure 2 can be implemented.

[0020] Figure 2 Shows the steps of an embodiment of the method according to the present invention. In step 100, the predefined position of the user's finger on the translucent surface in the interior space of the vehicle is determined. The translucent surface can be, for example, a fabric on a transparent carrier layer, behind which the projector is located. Matrix displays can also be used for this, as long as they are bright enough to produce a visible visual appearance for the user behind the translucent surface. In response to the fact that the user's finger is determined to be located on the predefined position, in step 200, the depth of detail of the augmented reality display is adjusted. To this end, in one example, the category of the environmental objects highlighted by the artificial visual cues via the augmented reality user interface is changed. In step 300, as a response to the determined position of the user's finger, the rear projection through the translucent surface is started. Thereby, the user obtains feedback on the successful operation and the user interface (since it is clearly currently being operated by the user) is visually appealingly presented.

[0021] List of reference numerals

[0022] 1 Device

[0023] 2, 3, 4, 5, 6 Predefined positions

[0024] 7 Finger

[0025] 8 Projector

[0026] 9 User

[0027] 10 Vehicle

[0028] 11 Translucent surface

[0029] 12 Augmented reality display

[0030] 13 Fragrance

[0031] 14 Vibration exciter

[0032] 15 Cyclist

[0033] 16 Ventilation device

[0034] 17 Projector

[0035] 18 Display surface

[0036] 19 Data input terminal

[0037] 20 Evaluation unit

[0038] 2I Data output terminal

[0039] 22 Infrared camera

[0040] 100 to 300 Method steps

Claims

1. A method for adjusting the depth of detail of an augmented reality user interface of a vehicle (10), the method comprising the following steps: Determining (100) a predefined position (2, 3, 4, 5, 6) of a finger (7) of a user (9) on a translucent surface (11) in an interior space of a vehicle (10), in response thereto: Adjusting (200) the depth of detail of the augmented reality display (12).

2. The method according to claim 1, wherein, The position of the finger (7) is determined with the aid of an optical sensor (22), in particular an infrared camera.

3. The method according to any one of the preceding claims, wherein, The translucent surface (11) is provided in a door trim and / or a dashboard of a vehicle (10).

4. The method according to any one of the preceding claims, wherein, The translucent surface (11) comprises a fabric and / or wood veneer.

5. The method according to any one of the preceding claims, wherein, The predefined position (3), in particular also the adjacent predefined positions (2, 4, 5, 6), are marked and / or visually highlighted by means of a back-space projection.

6. The method according to any one of the preceding claims, further comprising: In response to determining a predefined position (3) of a finger (7), back projection through a translucent surface (11) is initiated (300).

7. The method according to any one of the preceding claims, wherein, Adjusting the depth of detail of an augmented reality display (12) involves: increasing / decreasing the number of environmental objects (15) that are highlighted or replaced in the augmented reality display (12), and / or Increase / decrease the events that are responded to by means of the vibration actuator (14) and / or by means of the ventilation device (16) and / or by means of fragrance, and / or Enlarging / shrinking the display surface (18) of the head-up display, and / or dimming the windows of the vehicle (10), and / or Increase / decrease the number of projectors (17) used for augmented reality display.

8. The method according to any one of the preceding claims, further comprising the steps of: The vehicle (10) determines an environmental object having predefined characteristics using sensor technology and responds thereto: Reduces or eliminates the display content that lies in the user's view axis to objects in the environment.

9. The method according to claim 8, wherein, The predefined properties of the environment object include: distance and / or Size and / or Quality and / or Collision correlation and / or trajectory.

10. The method according to claim 8 or 9, wherein In particular, according to the visual axis trajectory of the user to the environmental object, the display content in the predefined area around the environmental object is further reduced or cleared.

11. An apparatus (1) for adjusting the depth of detail of an augmented reality user interface of a vehicle (10), the apparatus comprising: Data input terminal (19), Assessment Unit (20) and Data output terminal (21), where The evaluation unit (20) is configured to: A predefined position (3) of a finger (7) of a user (9) on a translucent surface (11) in the interior of a vehicle (10) is determined by means of a data input (19) and in response thereto: Adjust the depth of detail of the augmented reality display (12). 12 . The device according to claim 11 , which is configured to carry out the method according to claim 1 .

13. Vehicle comprising a device (1) according to any one of the preceding claims 11 or 12.

Images