Method and device for using a viewing device in a motor vehicle, and motor vehicle

Abstract

The invention relates to a method and a device for using a viewing device (10) in a motor vehicle (1), wherein the method has the steps of providing an optoelectronically readable code (2) in a motor vehicle (1), reading the optoelectronically readable code (2) using a viewing device (10) of a user, and modifying the display contents (11) of the viewing device (10) on the basis of the optoelectronically readable code (2).

Description

[0001] Method and device for using a viewing device in a motor vehicle and motor vehicle

[0002] DESCRIPTION:

[0003] The invention relates to a method and a device for using a viewing device in a motor vehicle and to a motor vehicle.

[0004] Visual devices such as AR glasses are known in principle. However, integrating them into a motor vehicle, especially a ferry operation, is difficult.

[0005] It is therefore an object of the present invention to provide an improved method and an improved device for using a viewing device in a motor vehicle.

[0006] This problem is solved by the subject matter of the independent claims. Advantageous further developments are specified in the dependent claims.

[0007] According to a first aspect, a method for using a viewing device in a motor vehicle is provided.

[0008] The method according to the invention comprises the steps of providing an optoelectronically readable code in a motor vehicle, reading the optoelectronically readable code with a user's visual device, and changing the display content of the visual device depending on the optoelectronically readable code. The method serves, in particular, to simplify the use of a visual device in a motor vehicle and, at the same time, to make it safer. In particular, the method serves to direct the driver's attention to the ferry operation in situations where this is necessary, particularly regardless of whether a visual device is connected to the motor vehicle.

[0009] First, an optoelectronically readable code is provided in the motor vehicle.

[0010] An optoelectronically readable code is a sequence of symbols, characters or pixels that can be read out using an image capture device, whereby the code contains information that is typically not directly recognizable or decipherable by humans, but in particular through evaluation by a processor unit, and through which a specific function or action can be triggered.

[0011] The optoelectronically readable code is designed, for example, as a one- or two-dimensional code, in particular as a barcode, QR code and / or Sema code, and usually consists of sections with a lighter and a darker color tone or wavelength, resulting in a pattern of lines or dots that can be read optoelectronically.

[0012] The optoelectronically readable code can be provided in particular in the interior of the motor vehicle, for example, in or on a dashboard and / or in or on a center console. The optoelectronically readable code can be provided both analogously, for example, via a sticker, embossing, and / or texture, or digitally, for example, on a display device, as will be explained below.

[0013] In particular, more than one optoelectronically readable code can be provided in the interior of the motor vehicle, for example on one or more control elements of a comfort function of the dashboard and / or the center console, as will be described below.

[0014] In a further step, the optoelectronically readable code is read using a user's viewing device.

[0015] A viewing device, which can also be referred to as a viewing device, is designed in particular as a device that is configured to present the user or wearer thereof, in particular the driver, with augmented reality (AR). In this case, augmented reality means mixing or displaying a visual representation of digital content together with actual elements of the real world, in particular the user's environment, as it occurs around the motor vehicle in the exterior space as well as in the interior of the motor vehicle. The reality can also be referred to as virtual reality (VR), augmented virtuality (AV) and / or mixed reality (MR).The method is designed to represent both virtual reality and augmented reality, whereby both terms are used synonymously below unless otherwise indicated.

[0016] In particular, the viewing device is a pair of glasses worn on the nose and / or in front of the user's eyes. Alternatively, it can also be a device attached to the rider's headgear, such as a visor, particularly a helmet, such as an open-face or full-face helmet. When worn, at least part of the viewing device is positioned in front of the rider's eyes in such a way that the rider can simultaneously see or perceive both the elements and / or information displayed thereon and the actual, real world.

[0017] For this purpose, the viewing device can have one or more

[0018] Include image display devices that are designed to provide one or more digital contents and / or information to the user, such as one or more projectors and / or screen devices that are in particular transparently integrated into a viewing element such as a spectacle lens through which the user looks.

[0019] The viewing device may also comprise one or more image capturing devices that capture an area that corresponds at least partially to the user's field of vision in an image, in particular continuously, regularly and / or periodically.

[0020] In particular, the one or more image capture devices can be designed to capture and read the previously provided optoelectronically readable code or to decode or evaluate the information contained therein, in particular with the aid of a processor device.

[0021] The display device can be coupled to the motor vehicle or an infotainment system of the motor vehicle. Alternatively or additionally, the display device can be coupled to another electronic device, such as a user terminal, in particular a mobile phone, a smartphone, a tablet, a phablet, and / or a wearable device, such as a smartwatch. This can be done, in particular, via wireless near-field communication such as Bluetooth and / or Wi-Fi.

[0022] Alternatively, the display device may not be coupled to the motor vehicle and / or another electronic device and may in particular be designed to be self-sufficient and, for example, have a mobile radio communication system via which content is provided to the display device.

[0023] In a further step, the display content of the display device is changed depending on the optoelectronically readable code. Thus, in response to the reading of the optoelectronically readable code or its content, the display content of the display device is changed. In this case, changing includes displaying the display content for the first time, turning it off, and switching it over.

[0024] For example, in response to the reading of the optoelectronically readable code, content can be displayed even though no content was previously displayed.

[0025] Likewise, for example, in response to the reading of the optoelectronically readable code, a previously or previously displayed content can be switched off.

[0026] Again, for example, in response to the reading of the optoelectronically readable code, switching can be made from a first, previously or up to then displayed content to a second content.

[0027] In this case, switching does not mean merely changing one or more pixels or image points of a moving and / or animated digital image, but rather switching to a completely different content.

[0028] For example, social media content is initially played and, in response to the reading of the optoelectronically readable code, the social media content is switched off and a warning relating to the motor vehicle is displayed instead, such as a collision warning and / or a distance warning.

[0029] Likewise, for example, a video stream can first be played back, for example a film or television content, and in response to the reading of the optoelectronically readable code, the video stream is switched off or stopped and vehicle information is displayed instead, such as an outside temperature and / or a navigation instruction.

[0030] The method according to the invention makes it possible to simplify the use of a viewing device in a motor vehicle and at the same time make it safer.

[0031] In particular, the method serves to draw the driver's attention to ferry operations in situations where this is necessary, particularly regardless of whether a visual device is connected to the motor vehicle.

[0032] In particular, by reading the optoelectronically readable code, the visual device can interact or communicate with the motor vehicle, even if they are not coupled to each other.

[0033] Even if these are coupled together, the method according to the invention represents a redundancy or a fallback position in case communication between the motor vehicle and the visual device is disrupted.

[0034] According to a further development, the method further comprises the steps of detecting a user gesture and changing a vehicle function of the motor vehicle depending on the user gesture.

[0035] A user gesture is a gesture performed by the user of the visual device, in particular the driver of the motor vehicle, and includes a body movement. The gesture may include a movement of a head and / or one or more limbs, in particular one or more forearms, hands, and / or fingers.

[0036] For example, the user gesture may be a nod, a turn, a shake, and / or a head rotation. Alternatively or additionally, the user gesture may be a rotation, tilt, extension, and / or movement of an arm in a predetermined direction.

[0037] Alternatively or additionally, the user gesture is a raising, lowering, extending and / or reaching of a hand in a predetermined direction, in particular to an object in the interior of the motor vehicle, such as a steering wheel, a gear selector lever and / or to another predetermined location, such as a button or a lever.

[0038] Alternatively or additionally, the user gesture involves raising, lowering and / or extending one or more fingers, such as raising an index finger or forming a thumbs-up gesture.

[0039] In particular, the user gesture is a simple gesture that is not composed of multiple gestures and / or does not require long, wide, and / or complex movement sequences. For example, the user gesture is a maximum of 5 s, 2 s, or 1 s long. Furthermore, the user gesture is a gesture that involves or requires only a single body part.

[0040] The user gesture can be recorded in particular by one or more interior cameras of the motor vehicle, which are coupled to a processor device with the aid of which the user gesture can then be evaluated, for example by using an image recognition algorithm.

[0041] Alternatively or additionally, the user gesture can be detected using a proximity and / or touch sensor on the steering wheel, gear selector, and / or another predetermined object, such as a button or lever. The user gesture can also be detected by the visual device, as will be described below.

[0042] The user gesture can be recorded in response to the change in the display content. In particular, the user gesture includes confirmation and / or acknowledgement of the changed display content, as will also be explained below.

[0043] In response to the detected or received user gesture, a vehicle function of the motor vehicle is changed.

[0044] A vehicle function can in particular include a so-called comfort function, such as interior air conditioning, a seat heating function, a massage function and / or an infotainment system.

[0045] In particular, a glance or a head movement of the user of the display device towards the optoelectronically readable code located on an object, in particular a control element, of the respective comfort function can trigger the process, whereupon a request to change the comfort function is displayed on the display device, which is then confirmed or carried out by a user gesture, whereupon the comfort function is changed as desired.

[0046] For example, the user focuses on a control element for the interior climate control, whereupon an optoelectronically readable code located there is read, and the currently set interior temperature is displayed on the display. With a simple thumbs-up or thumbs-down gesture, the user can indicate a desired change to the interior temperature, which is then implemented. The vehicle function can alternatively or additionally be a driving mode, as will be described later.

[0047] Conversely, if no user gesture has been detected, in particular not within a predetermined time such as less than 30 s, 20 s, 10 s, 5 s or 2 s, a vehicle function cannot be changed or in particular can be changed alternatively, as will be described later.

[0048] This further development enables the user of the display device to communicate his wishes and needs, particularly with regard to the changed display contents, to the display device and / or the motor vehicle and thus to interact with it.

[0049] According to a further development, the detection of the user gesture is carried out using the visual device

[0050] As described above, the viewing device can comprise one or more image capturing devices that capture an area that corresponds at least partially to the user's field of vision in one or more images.

[0051] These image capture devices can then be coupled together with a processor device, by means of which the user gesture can then be evaluated, for example by using an image recognition algorithm, as also described above.

[0052] This training makes it possible to grasp the user gesture particularly quickly and intuitively.

[0053] According to a further development, the vehicle function includes a driving mode. A driving mode can also be referred to as a driving mode and can, for example, refer to a driving program in which various damping, steering, braking, and / or acceleration characteristics are used for the driving operation of the motor vehicle.

[0054] For example, a comfortable driving program can be provided in which braking and acceleration are carried out particularly proactively and damping and steering are gentle, a dynamic driving program in which acceleration and braking are carried out particularly sharply and steering and damping are harsh, and an ecological driving program in which acceleration and braking are carried out particularly gently.

[0055] For example, the user can look at a specific optoelectronically readable code on the gear selector lever, whereupon the option of changing the driving program is displayed on the display device, which is then selected by performing a gesture.

[0056] In particular, the driving mode may also include one of several driving assistance modes and / or autonomous driving modes, which is changed depending on or in response to the user gesture.

[0057] For example, the user can look at a specific optoelectronically readable code on the dashboard, whereupon the option of changing from a manual to an autonomous driving mode and / or a next higher or lower autonomous driving mode is displayed on the display device.

[0058] This advanced training makes it possible to set or change a driving mode particularly conveniently.

[0059] According to a further development, changing the display content includes providing driving information. Driving information includes one or more items of information relating to the operation of the motor vehicle, for example, the current speed, the current engine speed, the mileage, an activated turn signal and / or hazard warning lights, an activated low beam and / or high beam, a selected gear, and / or the fill level of an energy storage device.

[0060] In particular, the driving information includes information that is necessary for safe, particularly active, ferry operation and that may have previously been hidden, as will be described below.

[0061] This training will enable the user to enjoy particularly safe ferry operations.

[0062] In particular, driving information that may have previously been provided on another display device in the interior of the motor vehicle may be hidden in response to the display content being changed to driving information.

[0063] It is therefore possible that one or more display devices can be used to provide other content, in particular content that is of interest to a passenger.

[0064] According to a further development, the optoelectronically readable code is provided in the infrared range.

[0065] In particular, the optoelectronically readable code is provided exclusively in the infrared range. This means that the optoelectronically readable code cannot be perceived by the human eye. This further development ensures that the user is not distracted by the optoelectronically readable code or that it does not interfere with their field of vision.

[0066] According to a further development, the optoelectronically readable code is provided on a display device of the motor vehicle.

[0067] A display device can comprise one or more displays or screens arranged in the interior of the motor vehicle.

[0068] For example, the optoelectronically readable code is displayed on a screen of a central infotainment system, on a screen behind the steering wheel and / or on a so-called head-up display, which is projected from behind the dashboard onto the windscreen from the inside.

[0069] Alternatively or additionally, one or more small, for example 1 cm x 1 cm, dedicated display devices can be provided in the interior for certain functions and / or interaction options, such as on or in one or more control elements of a comfort function, which then display an optoelectronically readable code as needed or permanently.

[0070] In particular, the optoelectronically readable code is only displayed when needed and / or when appropriate and serves as the trigger for the procedure, as will be described later.

[0071] This training makes it possible to provide the optoelectronically readable code as needed.

[0072] In particular, a need to change a driving mode, in particular an autonomous driving mode, can initially be detected, for example as a triggering event of the method according to the invention. For example, the motor vehicle can initially be in an autonomous driving mode, for example, an SAE Level 3 or higher, in particular 4 or 5, in which the driver does not have to continuously concentrate on the road or ferry operation and can attend to other things, which they can also do, for example, by watching a film on the display.

[0073] An autonomous driving device can detect that manual takeover by the driver will soon be necessary, for example because one or more of the vehicle's camera devices detects a construction site and, in particular, a downgrade of the autonomous driving mode to level 2 or less is necessary.

[0074] In response to this finding, an optoelectronically readable code, in particular exclusively in the infrared range, can then be displayed on one or more display devices, in particular all display devices of the motor vehicle, such as at least the display device behind the steering wheel, which code is detected and read out by the display device.

[0075] In response to the readout, the previously played film can be faded out and, if necessary, stopped, and in return one or more driving information, such as current speed, can be displayed on the display.

[0076] This may in particular be accompanied by a request to accept the impending takeover or downgrade and / or to activate manual driving mode.

[0077] This can be done, in particular, through textual and / or visual representations on the display and / or accompanied by an acoustic warning tone and / or a haptic signal. Alternatively, the user can perform the user gesture described below based on this change in content, without any further information.

[0078] Alternatively, the display content can first be changed to driving information and, if no user gesture has been detected after a waiting time of a maximum of 5 s, 4 s, 3 s, 2 s or 1 s, the request can be displayed.

[0079] In response to this, a user gesture, in particular a predetermined and / or unambiguous one, which indicates that the user or driver is ready to take over control of the vehicle or which announces and / or confirms the takeover of control of the vehicle, can then be detected and, in response to this detection, the autonomous driving mode can be changed, in particular exited or downgraded.

[0080] However, if no user gesture can be detected, especially not within a predetermined time and / or period, such as less than 30 s, 20 s, 10 s, 5 s, or 2 s, another action can be performed instead of downgrading to a less autonomous driving mode. For example, an acoustic warning tone and / or a haptic signal can be emitted. Additionally or alternatively, an emergency program can be initiated, such as emergency braking and / or autonomously driving to the hard shoulder.

[0081] The previously described embodiments make it possible to announce and execute a particularly safe and, at the same time, convenient transition to manual driving mode. This is particularly convenient in chauffeur-driven vehicles where the passenger does not want to be disturbed. The described redundancy and fail-safety also ensures the driver's attention. This results in a particularly safe and, in particular, traffic-compliant use of a visual device such as AR glasses in a motor vehicle.

[0082] For use cases or application situations that may arise during the method and which are not explicitly described here, it may be provided that, in accordance with the method, an error message and / or a request to enter user feedback is issued and / or a default setting and / or a predetermined initial state is set.

[0083] According to a further aspect, a device is provided. The device may comprise a data processing device or a processor device configured to carry out an embodiment of the method according to the invention.

[0084] In particular, the device is configured to communicate and / or cooperate with the viewing device, as described above, in order to carry out or at least effect the above-described embodiments of the method. In particular, the device can also comprise the viewing device or be configured as such, or together with such a mobile viewing device, form a system for carrying out embodiments of the method according to the invention.

[0085] For this purpose, the processor device can have at least one microprocessor and / or at least one microcontroller and / or at least one FPGA (Field Programmable Gate Array) and / or at least one DSP (Digital Signal Processor). In particular, a CPU (Central Processing Unit), a GPU (Graphical Processing Unit) or an NPU (Neural Processing Unit) can be used as the microprocessor. Furthermore, the processor device can have program code which, when executed by the processor device, is configured to carry out the embodiment of the method according to the invention. The program code can be stored in a data memory of the processor device. The processor device can, for example, be based on at least one circuit board and / or on at least one SoC (System on Chip).

[0086] According to a further aspect, a motor vehicle is provided which comprises such a device.

[0087] The motor vehicle according to the invention is preferably designed as a motor vehicle, in particular as a passenger car or truck, or as a passenger bus or motorcycle. In particular, the motor vehicle is a vehicle configured to drive in an at least partially autonomous driving mode, in particular an autonomous driving mode with an SAE level 3 or higher, in particular 4 or 5, in which the driver does not have to permanently concentrate on the road or ferry operation and can attend to other matters.

[0088] As a further solution, the invention also encompasses a computer-readable storage medium comprising program code which, when executed by a computer or computer network, causes the computer to carry out an embodiment of the method according to the invention. The storage medium can be provided at least partially as a non-volatile data memory (e.g. as a flash memory and / or as an SSD - solid state drive) and / or at least partially as a volatile data memory (e.g. as a RAM - random access memory). The storage medium can be arranged in the computer or computer network. However, the storage medium can also be operated, for example, as a so-called app store server and / or cloud server on the Internet. The computer or computer network can provide a processor circuit with, for example, at least one microprocessor.The program code may be provided as binary code and / or as assembly code and / or as source code of a programming language (e.g. C) and / or as a program script (e.g. Python).

[0089] The invention also encompasses combinations of the features of the described embodiments. The invention therefore also encompasses implementations that each comprise a combination of the features of several of the described embodiments, unless the embodiments are described as mutually exclusive.

[0090] With regard to the embodiments of the device, the motor vehicle and the storage medium as well as the associated advantages, reference is made to the previously described embodiments of the method and the associated advantages.

[0091] Exemplary embodiments of the invention are described below. Shown are:

[0092] Fig. 1 is a schematic side view of an embodiment of a method for using a viewing device in a motor vehicle and a device therefor.

[0093] The exemplary embodiments explained below are preferred embodiments of the invention. In the exemplary embodiments, the described components of the embodiments each represent individual features of the invention that can be considered independently of one another, each of which also develops the invention independently of one another. Therefore, the disclosure is intended to encompass combinations of the features of the embodiments other than those shown. Furthermore, the described embodiments can also be supplemented by further features of the invention already described.

[0094] In the figures, the same reference symbols designate elements with the same function.

[0095] Fig. 1 shows a schematic view of an embodiment of a method for using a viewing device 10 in a motor vehicle 1 and a device for this purpose. The device, which is not shown in detail, is comprised by the motor vehicle 1 and is designed to carry out or at least bring about the steps of the method described below, in particular in cooperation with the viewing device 10. The device can in particular be an infotainment system or can be used to carry out individual method steps.

[0096] First, an optoelectronically readable code 2 is displayed on a display device 12 in the motor vehicle 1, which is shown here as an example as the display device 12 behind the steering wheel.

[0097] The optoelectronically readable Code 2 is provided in the infrared range and is not visible to a user.

[0098] This optoelectronically readable code 2 is captured and read out using a driver's viewing device 10, which is shown here as an example of AR glasses.

[0099] In response to the detection of the optoelectronically readable code, a display content 11 of the display device 10 is changed.

[0100] This example shows how driving information is displayed, including engine speed, current speed, selected gear, time of day and outside temperature.

[0101] This happens especially after other content or no content at all has been displayed.

[0102] In response to the change in display content, the driver can perform a user gesture that is detected by the display device 10. A vehicle function is then changed in response to this user gesture.

[0103] The vehicle function includes an autonomous driving mode, which is changed, in particular exited, in response to the user gesture.

[0104] Overall, the examples show how a level of safety can be provided for the use of AR glasses in a motor vehicle.

Claims

PATENT CLAIMS: 1 . Method for using a viewing device (10) in a motor vehicle (1), comprising: Providing an optoelectronically readable code (2) in a motor vehicle (1); Reading the optoelectronically readable code (2) with a viewing device (10) of a user; and Changing a display content (11) of the display device (10) depending on the optoelectronically readable code (2).

2. The method according to claim 1, further comprising: Detecting a user gesture; and changing a vehicle function of the motor vehicle (1) depending on the user gesture.

3. The method according to claim 2, wherein the detection of the user gesture is carried out using the viewing device.

4. The method according to claim 2 or 3, wherein the vehicle function comprises a driving mode.

5. The method according to any one of the preceding claims, wherein changing the display content comprises providing driving information.

6. Method according to one of the preceding claims, wherein the optoelectronically readable code (2) is provided in the infrared range.

7. Method according to one of the preceding claims, wherein the optoelectronically readable code (2) is provided on a display device (12) of the motor vehicle.

8. Device for using a viewing device (10) in a motor vehicle (1), wherein the device is designed to carry out a method according to one of the preceding claims.

9. Motor vehicle comprising a device according to claim 8.

Images