Means of locomotion, device and method for adjusting a depth of detail of an augmented-reality user interface of a means of locomotion

Abstract

The invention relates to a means of locomotion (10), a device (1) and a method for adjusting a depth of detail of an augmented-reality user interface of a means of locomotion (10). The method comprises the steps of: determining a predefined position (2, 3, 4, 5, 6) of a finger (7) of a user (9) on a translucent surface (11) in the interior of a means of locomotion (10), in response thereto adjusting the depth of detail of an augmented-reality display (12).

Description

[0001] Means of transport, device and method for adapting a level of detail of an augmented reality user interface of a means of transport

[0002] Description

[0003] The present invention relates to a means of transportation, a device, and a method for adapting the level of detail of an augmented reality user interface of a means of transportation. In particular, the present invention relates to a discreet interaction option for a user when configuring an AR mix in the means of transportation.

[0004] Users typically operate their vehicles via hardware buttons / sliders and touchscreens or other hardware-based touch-sensitive and / or pressure-sensitive user interfaces. A particularly discreet form of interaction has become known as "shy tech." This involves a translucent display of control elements on a surface, which may, for example, have a fabric and / or wood look or similar. The user interface is displayed via rear projection, while the position of a user's finger on this user interface is determined using sensors (capacitive, electroresistive, or similar) integrated into the material.

[0005] Furthermore, AR user interfaces in means of transport are on the rise. This refers to the artificial augmentation of the vehicle environment directly perceived by an occupant. For example, contact-analog references to surrounding objects are also provided to the user after the user's eye position and the position of a surrounding object have been determined. The AR content is now displayed at the intersection of the optical axis between the aforementioned points and the display surface. Until now, there were no user-side settings options for selecting the potentially configurable AR content or AR functions. In other words, the user was always dependent on the AR user interface content offered and displayed by the developer being satisfactory.

[0006] Based on the aforementioned prior art, it is an object of the present invention to be able to conveniently carry out an AR-related configuration of a user interface that is acceptable to a user.

[0007] The above-mentioned object is achieved according to the invention by a method for adapting a level of detail of an augmented reality user interface of a means of transport.

[0008] The subclaims show preferred developments of the invention.

[0009] The method can be carried out in a means of transport (car, van, motorcycle, truck, aircraft, and / or watercraft). The augmented reality user interface can utilize a display device of the means of transport and / or a display device worn by the user. In a first step, a predefined position of a user's finger on a translucent surface in the interior of a means of transport is detected. In other words, the user's finger is located when it approaches or touches a translucent surface. This can be done, for example, using an infrared camera that continuously records the translucent surface. The user's finger can be identified based on its infrared signature.In response to detecting the finger at the predefined position, the level of detail of the augmented reality representation is adjusted by the augmented reality user interface. This adjustment can occur automatically. In other words, the predefined position represents a specific level of detail, which can be predefined in stages and / or continuously. The level of detail can refer to different content / features of the augmented reality representation. For example, certain categories of environmental objects may not be highlighted by AR content in a first stage, while in a second level of detail, these categories of environmental objects are visually highlighted for the user of the interface according to the invention. However, a plurality of user interfaces / display devices / enlarged display areas, or the like, can also be used if the level of detail is increased.Further examples of the level of detail are given later in this disclosure.

[0010] Preferably, the position of the finger can be determined using an optical sensor. The optical sensor can be arranged, for example, in a roof lining and / or in an A-pillar and / or in a roof control module / roof function center. The optical sensor can, for example, comprise an infrared camera, which can also be used for other functions in the vehicle (e.g., seat occupancy detection, fatigue detection, recording of vital signs, or similar). This reduces the hardware complexity for implementing the device according to the invention.

[0011] A translucent surface can be arranged, for example, in a door panel and / or on a dashboard of the vehicle. It can have fibers and / or holes or similar elements through which rear-projected light can emerge from the surface of the translucent surface with relatively sharp contours. This makes it possible to display switching elements and / or visualize their operating state (activated / deactivated) or similar.

[0012] The translucent surface can preferably comprise a fabric and / or a wood paneling and / or a fiber material. Drill holes and / or light guide elements can be embedded in the surface in such a way that the optical appearance, when rear projection is activated, corresponds to a natural, in particular organic or inorganic material. In other words, in particular, no classic shape of a matrix display is recognizable when the display device is switched off. Only when rear projection is switched on is a dynamic appearance of the user interface according to the invention activated by means of the translucent surface. This does not preclude predefined positions from being optically perceptible on the translucent surface even without rear projection. For example, the material can be structured differently and / or colored differently in order to allow predefined positions to be recognized even without rear projection.In particular, in response to the detection of a user's finger in a predefined spatial relationship to the predefined position or another predefined position, the back projection can be activated, thus enriching the user interface with further optical details. By activating the back projection, for example, a predefined position or several (in particular adjacent) predefined positions, preferably all predefined positions of the user interface, can be marked and / or optically highlighted. For example, the predefined positions can also be optically summarized by placing a border line, a frame, or the like around the predefined positions, which are arranged in particular in the form of a circular structure or in the form of a line (linear).The level of detail of the AR representation can include additional accompanying parameters, not necessarily based on AR technology, which can be output, for example, using vibration exciters and / or fans and / or heating / air conditioning systems and / or fragrances and / or conventional display devices. For example, a vibration exciter can be used at a first level of detail to visualize certain events / environmental objects, while the vibration exciter should not be used at another (particularly lower) level of detail. The same applies to the use of a fragrance bottle and / or a fan level and / or heating / air conditioning temperature. If dynamically dimmable windows are provided in the means of transport, the dimming of the windows can be provided at a (higher) first level of detail, while such dimming is not provided at a lower level of detail.

[0013] The same applies to the number of projectors to be used, the category and position of environmental objects to be pointed out during the AR presentation, as well as the actuators to be used and their parameters when responding to predefined events.

[0014] The detail levels that can be activated or changed via the user interface described above can, for example, concern the following content: In a first level (level 0), all AR and accompanying signaling are switched off. In a second level (level 1), the classic head-up display with the familiar content is used. If necessary, a lane departure warning or similar is signaled via seat vibration. In a third level (level 2), the head-up display is supplemented by a three-dimensional component (depth head-up display), optionally also seat vibration patterns, smartwatch integration and fans for airflow and fragrance to provide the AR user experience. In the last level (level 4), the darkening of the windows of the vehicle (e.g.This is triggered (due to electrochromic substances), in response to which a completely virtual image of the environment is projected onto the now darkened panes. If desired, environmental objects and other environmental features can be freely represented by alternative display objects, allowing the user to experience an entertaining alienation of the environment if necessary. If the panes are only partially opaque in this state, this representation corresponds to an AR environment. This can lead to a virtual reality if the panes are (almost) completely opaque.

[0015] Regardless of the current AR / VR level of detail selected by the user, an automatic partial release of a display device used for this purpose can be advantageous from an ergonomic and driving safety perspective, particularly if an environmental object with predefined properties has been detected in the vicinity of the vehicle using sensors. For example, the environmental object can be located in the collision-relevant environment of the vehicle using a camera, a LiDAR sensor, and / or an ultrasonic sensor. In response, a partial reduction or removal of display content from the AR / VR display can occur, particularly at those positions that are located in the user's line of sight to the environmental object.In other words, regardless of the currently displayed display content, the area that needs to be made transparent for at least a partial, in particular completely unobstructed, view of the surrounding object by the user is made completely or partially transparent. A predefined frame can be displayed around the surrounding object, or it can also be kept free with reduced content or completely empty of content in order to better integrate the situation of the surrounding object into the actual vehicle environment. Preferably, a trajectory of the viewing axis can also be predicted on the display device, so that a larger area with reduced or completely removed display content is provided in the optical direction of movement from the user's perspective.In this way, the user can, particularly based on the relative trajectory between the ego-based vehicle and the environment object, unobstructed and naturally view the environmental areas in which the environment object will soon be located. Alternatively or additionally, depending on the position of the environment object, predefined display areas (e.g., left half of the display, right half of the display, lower half of the display, upper half of the display, etc.) can be switched to completely transparent or operated with AR / VR content reduced. This creates an area or corona around the environment object, within which the environment is also largely or completely recognizable to the user.Of course, in an advantageous embodiment, the position of the user's eyes must first be determined using sensors in order to establish the line of sight and release it as described above. Such methods are generally known in the prior art.

[0016] The predefined property of the environmental object can, in particular, determine the probability that the user would prefer an unobstructed view of the environmental object rather than having to look through a display subject to ARA / R. In particular, a collision relevance or information associated with it can be taken into account. Such information includes, for example, the distance, the speed, the relative direction of movement, a predicted trajectory, a mass, a material of the environmental object, or other information. The environmental object can also be identified (e.g., wireless, information technology, electromagnetic, or similar). In particular, the environmental object can also be categorized for this purpose, with predefined properties being assigned to the identified category, which can be used to switch the display device to transparency, as described above.

[0017] According to a second aspect of the present invention, a device for adapting the level of detail of an augmented reality user interface of a means of transport is proposed. The user interface could also be understood as a driver assistance system of the means of transport (e.g., a car, van, motorcycle, truck, aircraft, and / or watercraft). The device comprises a data input, an evaluation unit (e.g., an electronic control unit), and a data output. The evaluation unit is thus configured to use the data input to determine a predefined position of a user's finger on a translucent surface in the interior of the means of transport and, in response thereto, to adapt the level of detail of the augmented reality representation, if necessary, using the data output. This step can occur automatically.In other words, the device according to the invention is clearly designed to implement the features, combinations of features and the advantages resulting from them in a corresponding manner, so that in order to avoid repetition, reference is made to the above statements in connection with the method according to the invention.

[0018] According to a third aspect of the present invention, a means of transportation is proposed which comprises a device according to the second aspect of the invention. In this regard, the features, combinations of features, and advantages also arise in a corresponding manner.

[0019] Further details, features, and advantages of the invention will become apparent from the following description and the figures. They show:

[0020] Figure 1 is a perspective view of an interior of a means of transport according to the invention with an embodiment of a device according to the invention during the execution of a method according to the invention; and

[0021] Figure 2 is a flowchart illustrating steps of an embodiment of a method according to the invention for adapting a level of detail of an augmented reality user interface of a means of transport.

[0022] Figure 1 shows a driver's workstation of an exemplary embodiment of a means of transport 10 according to the invention in the form of a passenger car. The user 9 is seated in the driver's seat and wishes to adjust the manner in which a cyclist 15 located to the right of the means of transport 10 is visually highlighted. To do so, the user swipes his finger 7 across a translucent surface 11 located in the dashboard at a predefined position 3. The predefined position 3 is discreetly and permanently marked. The same applies to predefined positions 2, 4, 5, and 6 adjacent to the predefined position 3. An infrared camera 22 in the area of ​​an A-pillar detects the finger 7 at the predefined position 3 and reports this data via a data input 19 to an electronic control unit acting as an evaluation unit 20.The evaluation unit 20 identifies the finger 7 of the user 9 and, via a data output 21, causes a projector 8 to provide a rear projection, visualizing a border around the predefined positions 2 to 6. By moving their finger 7, the user 9 can now select a level of detail associated with one of the predefined positions 2 to 6 in order to adapt the augmented reality representation 12 around the cyclist 15 according to their preferences. In the process, the operating parameters of a fan 16, which can deliver fragrances 13 into the interior, and of a vibration exciter 14 are also adjusted via the data output 21. The higher the level of detail of the AR representation, the more likely the aforementioned actuators and features of the means of transport 10 are used to provide an AR user experience.In this context, reference should also be made to a projector 17 of a head-up display, whose display area 18 can be enlarged or reduced depending on the selected predefined position 2 to 6. The method discussed in connection with Figure 2 can be carried out using the device 1 shown.

[0023] Figure 2 shows steps of an embodiment of a method according to the invention. In step 100, a predefined position of a user's finger on a translucent surface in the interior of a means of transport is determined. The translucent surface can, for example, be a fabric on a transparent carrier layer, behind which a projector is located. Matrix displays could also be used for this purpose, provided they are bright enough to create visible optical phenomena for the user behind the translucent surface. In response to the fact that the user's finger has been determined at the predefined position, the level of detail of the augmented reality representation is adjusted in step 200. For this purpose, in one example, the category of environmental objects that are highlighted by artificial optical cues using the augmented reality user interface is changed.In step 300, a backprojection through the translucent surface is initiated to acknowledge the determined position of the user's finger. This provides the user with feedback about the successful operation, and the user interface is rendered visually appealing (since it is obviously currently being operated by the user).

[0024] List of reference symbols:

[0025] 1 device

[0026] 2, 3, 4, 5, 6 predefined positions

[0027] 7 fingers

[0028] 8 projector

[0029] 9 users

[0030] 10 means of transport

[0031] 11 translucent surface

[0032] 12 Augmented Reality Display

[0033] 13 Fragrance

[0034] 14 vibration exciters

[0035] 15 cyclists

[0036] 16 fans

[0037] 17 Projector

[0038] 18 display area

[0039] 19 Data input

[0040] 20 Evaluation unit

[0041] 21 Data output

[0042] 22 infrared camera

[0043] 100 to 300 process steps

Claims

Patent claims:

1. A method for adapting a level of detail of an augmented reality user interface of a means of transport (10) comprising the steps • Determining (100) a predefined position (2, 3, 4, 5, 6) of a finger (7) of a user (9) on a translucent surface (11) in the interior of a means of transport (10), in response thereto • Adjusting (200) the level of detail of an augmented reality display (12).

2. Method according to claim 1, wherein the position of the finger (7) is determined by means of an optical sensor (22), in particular by means of an infrared camera.

3. Method according to one of the preceding claims, wherein the translucent surface (11) is arranged in a door panel and / or in a dashboard of the means of transport (10).

4. Method according to one of the preceding claims, wherein the translucent surface (11) comprises a fabric and / or a wood paneling.

5. Method according to one of the preceding claims, wherein the predefined position (3), in particular also adjacent predefined positions (2, 4, 5, 6), is marked and / or optically highlighted by means of a backspace projection.

6. Method according to one of the preceding claims, further comprising - Starting (300) a back projection through the translucent surface (11) in response to determining the predefined position (3) of the finger (7).

7. Method according to one of the preceding claims, wherein the adjustment of the level of detail of the augmented reality representation (12) - an increase / decrease of environmental objects (15) which are highlighted or replaced in the augmented reality representation (12), and / or - an increase / decrease in events which are acknowledged by means of a vibration exciter (14) and / or by means of a fan (16) and / or by means of a fragrance note and / or - an extension / reduction of a display area (18) of a head-up display and / or - a darkening of the windows of the means of transport (10) and / or - an increase / decrease in the number of projectors used to display augmented reality (17).

8. Method according to one of the preceding claims further comprising the steps - sensor-based detection of an environmental object by the means of transport (10) with a predefined property, and in response thereto - Reducing or removing display content located in the user's line of sight to the surrounding object.

9. The method according to claim 8, wherein the predefined property of the environment object is a - Distance and / or - Size and / or - Mass and / or - Collision relevance and / or - T rajectory defined.

10. The method according to claim 8 or 9, wherein the display contents are further reduced or removed in a predefined area around the environmental object, in particular depending on a trajectory of the user's line of sight to the environmental object.

11. Device (1) for adapting a level of detail of an augmented reality user interface of a means of transport (10) comprising - a data input (19), - an evaluation unit (20) and - a data output (21), wherein the evaluation unit (20) is arranged, - to determine a predefined position (3) of a finger (7) of a user (9) on a translucent surface (11) in the interior of the means of transport (10) by means of the data input (19) and in response thereto -to adjust the level of detail of the augmented reality display (12).

12. Apparatus according to claim 11, which is arranged to carry out a method according to one of the preceding claims 1 to 10.

13. Means of locomotion comprising a device (1) according to one of the preceding claims 11 or 12.

Images