Method for positioning a symbol on a display device in a motor vehicle, computer program product and motor vehicle

Abstract

A method is described for positioning a symbol on a display device in a motor vehicle, wherein an activating sound wave is detected, an origin and a direction of the activating sound wave are determined, wherein an intersection point of the activating sound wave with the display device is determined, wherein the symbol is positioned in the region of the intersection point.

Description

[0001] A method for positioning a symbol on a display device in a motor vehicle, a computer program product and a motor vehicle are described.

[0002] Methods for positioning a symbol on a display device in a motor vehicle, computer program products and motor vehicles of the type mentioned at the beginning are known in the prior art.

[0003] Motor vehicles are often equipped with voice-controlled assistants. These assistants help passengers and the driver configure certain functions and activate or deactivate them, such as the navigation system, media playback, telephone, text messaging, climate control settings, driver assistance systems, ambience settings, and other applications.

[0004] Often, a corresponding symbol is shown on a display device in the vehicle that is part of the human-machine interface when the voice assistant is currently active.

[0005] From US 2014 / 0 337 741 A1, a method is known which comprises determining one or more prominent sound sources based on one or more sound sources using signals acquired by two or more microphones configured to acquire an acoustic signal from one or more sound sources. The method further comprises determining one or more directions relative to a position of one or more of the two or more microphones for one or more prominent sound sources. The method comprises modifying one or more user interface elements displayed on a user interface of a display to display, at least in part, one or more directions, relative to the position of at least one microphone, of the one or more prominent sound sources.In addition, DE 10 2010 040 818 A1 shows an emotional advisory system for use by one or more occupants of a motor vehicle, and JP 2021 123 215 A relates to the provision of a display device that enables a user to grasp the processing state of a system and a control method for the display device.

[0006] The degree of use of voice control systems, which can increase road safety, varies greatly from user to user. It proves difficult to convince occasional or passive users of the benefits of voice control functions. In addition, such users often forget over time that their vehicle is equipped with a voice control system or assistant and therefore do not try out such functions.

[0007] The task therefore arises of further developing methods for positioning a symbol on a display device in a motor vehicle, computer program products and motor vehicles of the type mentioned at the beginning in such a way that voice control systems are more present for drivers and passengers of motor vehicles with such systems.

[0008] The object is achieved by a method for positioning a symbol on a display device in a motor vehicle according to claim 1, a computer program product according to the independent claim 9 and a motor vehicle according to the independent claim 10. Further embodiments and developments are the subject of the dependent claims.

[0009] A method is described for positioning a symbol on a display device in a motor vehicle, wherein an activating sound wave is detected, an origin and a direction of the activating sound wave are determined, wherein an intersection point of the activating sound wave with the display device is determined, wherein the symbol is positioned in the region of the intersection point.

[0010] The method involves evaluating a sound source—in the case of speech, usually the mouth of a passenger or driver—and a direction of speech, as it has been shown that people often look in the same direction they are speaking. This allows the symbol to always be placed in the speaker's field of vision.

[0011] The symbol can be positioned at the intersection point or in a defined area around the intersection point, for example, in order not to obscure other displays on the display device.

[0012] Sound typically propagates within a more or less wide sound cone, but within the sound cone, the sound exhibits different amplitudes in different directions, one of which has a maximum amplitude. This direction is described here as the direction of the activating sound wave.

[0013] An activating sound wave is a sound wave that represents a corresponding interaction with the motor vehicle. This distinguishes between normal noises and normal conversations in the vehicle, including noises generated by a multimedia system, and targeted, planned interactions between passengers or the driver and the vehicle. Such a sound wave can be a complex sound wave and, for example, represent speech.

[0014] The symbol can, in particular, symbolize a voice assistant. The symbol can be dynamic or animated, meaning it can change over time.

[0015] In this way, it is possible to give the voice assistant a dedicated space that is always in the user's field of vision, thereby making the omnipresence of the voice assistant clear to the user and making the voice assistant a central part of the human-machine interface of the vehicle.

[0016] In a first further embodiment, it is provided that the origin and direction of the activating sound wave are determined by means of a microphone arrangement with several microphones distributed in a motor vehicle interior.

[0017] With the help of such a microphone arrangement with several distributed microphones with known positions relative to a reference in the vehicle, both the respective volume and the respective phase of the sound wave can be recorded by the microphones involved, whereby it is possible to determine the origin and direction of the sound by triangulation.

[0018] In a further refinement, it is provided that the seat or bench from which the activating sound wave originates is determined.

[0019] This helps to localize the speaker.

[0020] In a further refinement, it is provided that the symbol is positioned at an exit point in the motor vehicle without an activating sound wave.

[0021] The symbol may be present outside the display device, particularly when not in use.

[0022] A specific embodiment may provide for the display element to be arranged within a control panel, which may, for example, contain a ferrofluid display used to display the corresponding voice assistant. Thus, the voice assistant can always be present, even when not in use.

[0023] The ferrofluid display makes it possible for the display to change, so that the voice assistant can, for example, move towards the corresponding screen and then is no longer visible on the ferrofluid display.

[0024] Furthermore, the ferrofluid display can be used to represent various states of the voice assistant, such as active, inactive, listening, processing, speaking, and the like. The dynamics can also help draw attention to the voice assistant.

[0025] In a further refinement, it is provided that the starting point is positioned on a pressure-sensitive surface.

[0026] If the voice assistant is located at the starting point outside the display device, it can be combined with a pressure-sensitive surface.

[0027] In alternative embodiments or additionally, the voice assistant can also be provided on one of the display devices, which can also be touch-sensitive or pressure-sensitive.

[0028] In a further refinement, it is provided that a sound wave is analyzed, wherein the symbol is only positioned in the area of ​​the intersection point if the sound wave has a signature of an activating sound wave.

[0029] This prevents the symbol from jumping back and forth between passengers in the vehicle during normal conversations. The symbol is only positioned at the intersection point or in the area of ​​the intersection point when the voice assistant is being used.

[0030] In a further further embodiment, it is provided that a sound wave is analyzed by means of a speech recognition algorithm, wherein the symbol is only positioned in the area of ​​the intersection point if at least one specific activation word is recognized.

[0031] In a further refinement, the symbol indicates the status of an assistance system of the motor vehicle.

[0032] A first independent subject matter relates to a computer program product comprising a computer-readable storage medium on which instructions are embedded which, when executed by at least one computing unit, cause the at least one computing unit to be configured to carry out the method of the aforementioned type.

[0033] The method can be carried out in a distributed manner on one or more computing units, so that certain method steps are carried out on one computing unit and other method steps are carried out on at least one further computing unit, whereby calculated data can be transmitted between the computing units if necessary.

[0034] Another independent subject matter relates to a motor vehicle with a computer program product of the type described above.

[0035] Further advantages, features, and details will become apparent from the following description, which – where appropriate with reference to the drawings – describes at least one embodiment in detail. Identical, similar, and / or functionally equivalent parts are provided with the same reference numerals.

[0036] They show schematically: Fig. 1 a plan view of a motor vehicle; Fig. 2 a human-machine interface of the motor vehicle Fig. 1, and Fig. 3 a flowchart of the procedure.

[0037] Fig. 1 shows a plan view of a motor vehicle 2.

[0038] The motor vehicle 2 has a vehicle interior 2.1 in which a row of seats is provided. A driver's seat 4.1 and a passenger seat 4.2 are arranged in the front area. A rear bench seat 6 with a left seat 6.1 and a right seat 6.2 is provided in the rear area.

[0039] A dashboard 7.1 and a center console 7.2 are located in front of the front seats 4.1 and 4.2. A display device 8 consisting of several screens, a driver screen 8.1, a central screen 8.2, and a passenger screen 8.3, is provided on the dashboard 7.1. A rear screen 8.4 is located in front of the rear seat bench 6.

[0040] The motor vehicle 2 is further equipped with a microphone array 10 comprising several microphones 10.1 to 10.5 distributed throughout the vehicle interior 2.1. Using the microphones 10.1 to 10.5, it is possible to identify sound sources and main sound axes emitted by the passengers of the motor vehicle 2.

[0041] With the aid of a controller 12, which is connected to both the display device 8 and the microphone arrangement 10, the source of a sound wave, for example speech, can be located and the screens 8.1 to 8.4 can be controlled.

[0042] In Fig. 1 shows a person 14 on the passenger seat 4.2 who is speaking and emitting a sound wave 16, of which Fig. 1, the main axis is shown for clarity. Using the microphones 10.1 to 10.5, it can be determined from the phases, i.e., the relative propagation time differences, as well as the volume levels, that the source of the sound or sound wave 16 is located on the passenger seat 4.2, and an intersection point 18 with the display device 8 can be determined, which in this case is located on the passenger screen 8.3. For this purpose, a main sound axis can be determined from the sound actually emitted by the person 14 as a sound cone. And this main axis can be correlated with known positions of the display device 8.

[0043] Fig. 2 shows a view of the motor vehicle interior 2.1 with a view of the dashboard 7.1 and center console 7.2.

[0044] Arranged in the center console 7.2 is a control panel 30 having a ferrofluid button 32 configured to change its shape to indicate a status of the human-machine interface of the motor vehicle 2. The control panel 30 can, for example, be a ventilation system with a row of air nozzles, between which the ferrofluid button 32 is arranged.

[0045] The ferrofluid button 32 is also touch- or pressure-sensitive, so that the voice assistant of the motor vehicle 2 can be activated by pressing the button 32.

[0046] The ferrofluid button 32 is shown below with a symbol intended to represent the voice assistant. By activating the voice assistant, e.g., with an activation phrase, the symbol 34 can be moved along a trajectory 36 from the button 32 to the intersection point 18, so that the symbol 34 is immediately visible to the person speaking.

[0047] Fig. 3 shows a flow of the procedure.

[0048] First, the voice assistant monitors for an activation phrase using microphones 10.1 to 10.5 until the activation phrase is spoken. Then, the location and direction of the sound source, i.e., the speaking person 14, are determined via microphones 10.1 to 10.5, and an intersection point 18 of the main axis of the sound wave 16 with one of the screens 8.1 to 8.4 is determined. An animation of the movement of the symbol 34 toward the intersection point 18 is then generated.

[0049] Although the invention has been illustrated and explained in detail by means of preferred embodiments, the invention is not limited by the disclosed examples, and other variations may be derived therefrom by those skilled in the art without departing from the scope of the invention. It is therefore clear that a multitude of possible variations exist. It is also clear that the embodiments mentioned by way of example are truly only examples and should not be construed as limiting the scope, possible applications, or configuration of the invention in any way.Rather, the preceding description and the description of the figures enable the person skilled in the art to implement the exemplary embodiments in concrete terms, whereby the person skilled in the art, with knowledge of the disclosed inventive concept, can make various changes, for example with regard to the function or the arrangement of individual elements mentioned in an exemplary embodiment, without departing from the scope of protection defined by the claims and their legal equivalents, such as a further explanation in the description. List of reference symbols 2 motor vehicles 2.1 Vehicle interior 4.1 Driver's seat 4.2 Passenger seat 6 Rear seat bench 6.1 left seat 6.2 right seat 7.1 Dashboard 7.2 Center console 8 Display device 8.1 Driver screen 8.2 Central screen 8.3 Passenger screen 8.4 rear screen 10 Microphone arrangement 10.1 - 10.5 Microphone 12 Control 14 people 16 sound wave 18 Intersection 30 control panel 32 Ferrofluid probes 34 Symbol 36 Trajectory

Claims

[1] Method for positioning a symbol (34) on a display device (8) in a motor vehicle (2), characterized by that an activating sound wave (16) is detected, an origin and a direction of the activating sound wave (16) are determined, wherein an intersection point (18) of the activating sound wave (16) with the display device (8) is determined, wherein the symbol (34) is positioned in the region of the intersection point (18). [2] Method according to claim 1, characterized by that the origin and direction of the activating sound wave (16) are determined by means of a microphone arrangement (10) with several microphones (10.1 - 10.5) distributed in a motor vehicle interior (2.1). [3] Method according to claim 1 or 2, characterized by that it is determined from which seat (4.1, 4.2, 6.1, 6.2) or which bench (6) the activating sound wave (16) originates. [4] Method according to one of the preceding claims, characterized bythat the symbol (34) is positioned at an output location (32) in the motor vehicle (2) without an activating sound wave (16). [5] Method according to claim 4, characterized by that the starting location (32) is positioned on a pressure-sensitive surface. [6] Method according to one of the preceding claims, characterized by that a sound wave (16) is analyzed, wherein the symbol (34) is positioned in the region of the intersection point (18) only if the sound wave (16) has a signature of an activating sound wave (16). [7] Method according to one of the preceding claims, characterized by that a sound wave (16) is analyzed by means of a speech recognition algorithm, wherein the symbol (34) is only positioned in the region of the intersection point (18) if at least one specific activation word is recognized. [8] Method according to one of the preceding claims, characterized bythat the symbol (34) indicates a status of an assistance system of the motor vehicle (2). [9] Computer program product, comprising a computer-readable storage medium on which instructions are embedded which, when executed by at least one computing unit, cause the at least one computing unit to be adapted to carry out the method according to one of the preceding claims. [10] Motor vehicle (2) with a computer program product according to claim 9.

Images