System and procedure for information

Abstract

Method (300) for providing reassuring information to a user (60, 60', 60'') of a vehicle (80), the method comprising: - detecting a facial expression of the user (60, 60', 60''), wherein the facial expression can be associated with a worried or anxious state of mind; - determining (308) one or more operating parameters that characterize operation of the vehicle (80); - determining (310), based on the determined one or more operating parameters, that no hazardous situation exists; and - Outputting (314) corresponding information to the user (60, 60', 60'').

Description

[0001] The disclosure relates to a system and method for informing users of a vehicle. In particular, the disclosure relates to a system and method for providing reassuring information to users of a vehicle, including occupants of automated or autonomous motor vehicles. State of the art

[0002] Various systems are known in the prior art that can be used to detect the driver of a vehicle. Typically, various sensors, such as cameras or microphones, are used to detect the driver or their behavior, such as the driver's operation of the vehicle. Sensors can also be used to detect a physiological characteristic of the driver, such as body temperature or heart rate.

[0003] In addition, special systems can detect one or more of the following: the driver's line of sight, objects in the area surrounding the vehicle, or the driver's willingness or ability to take control of the vehicle. There are also display systems that project information into the driver's field of vision (e.g., so-called "head-up" displays).

[0004] The collection of driver-related data can be used to determine the driver's state of mind and, if necessary, to influence it.

[0005] US 2010 / 0134302 A1 describes a system and method for controlling a driver's emotion. The system includes a detection unit, a control unit, and an emotion control unit. The detection unit detects emotion information, including the driver's voice, expression, gesture, heart rate, and body temperature. The control unit compares the detected emotion information with previously stored reference emotion information of the driver to determine whether the driver's emotion should be controlled and outputs a control signal for controlling the driver's emotion based on the determination result. The emotion control unit controls the driver's emotion according to the control signal from the control unit.

[0006] US 2018 / 061415 A1 describes methods and systems for determining an emotion of a human driver of a vehicle and using the emotion to generate a vehicle response. The method includes capturing a face of the human driver through a camera of the vehicle. The capture is configured to capture a plurality of images over a period of time, and analyzing the plurality of images to identify a facial expression and changes in the facial expression of the human driver over the period of time. The method further includes capturing voice input from the human driver through a microphone of the vehicle. The voice input is captured over the period of time. The voice input is analyzed to identify a voice profile and changes in the voice profile of the human driver over the period of time.The method processes, by a processor of the vehicle, a combination of the facial expression and the voice profile captured during the period to predict the emotion of the human driver. The method generates the vehicle response responsive to the emotion of the human driver. The vehicle response is configured to make at least one adjustment to a setting of the vehicle. The setting is selected based on the emotion of the human driver. The vehicle response may be used to calm the driver and / or help reduce distracted driving. The prediction of the emotion may be further enhanced by capturing and analyzing touch and / or gesture characteristics of the human driver when interfacing with a graphical user interface or surfaces of the vehicle or systems of the vehicle.

[0007] The collection of user-related data can be used to determine the mood of the user of an electronic device and, based on this, improve communication with the user. Publication WO 2002 / 063599 A1 describes a method for performing focus detection, ambiguity resolution, and mood classification in accordance with multimodal input data under varying operating conditions to provide an effective conversational computing environment for one or more users.

[0008] As mobility evolves, it is becoming increasingly important to focus not only on the driver of a vehicle, but also on all users and occupants, regardless of whether they are a driver, co-driver, passenger, or passenger. In this disclosure, the terms "user" and "occupant" are used synonymously. Particularly in the case of automated or autonomous vehicles, it can be assumed that an occupant will only assume the role of a traditional driver in the vehicle in rare situations or temporarily, or that a driver will become completely obsolete.

[0009] However, known systems are often focused exclusively on the driver of a vehicle and, in particular, on whether the driver is able to maintain or take control of the vehicle.

[0010] There is therefore a need for systems and procedures that take into account not only the driver but all occupants of a vehicle when recording.

[0011] The less involved a passenger is in the driving process, the less information is available to the passenger to assess the current driving situation or is consciously perceived by the passenger. This applies to conventional front or rear passengers as well as to passengers of automated or autonomous vehicles. This lack of consciously perceived information and / or the reduced or complete lack of opportunity to intervene in the driving process in certain situations can impair the well-being of the passengers. To ensure the well-being of all passengers, it would therefore be necessary to determine the emotional state of the passengers and to intervene if any disturbance or impairment is detected.

[0012] This effect can be amplified by the fact that, with increasing automation and the introduction of autonomous driving, occupants increasingly devote themselves to other activities while driving, such as consuming infotainment content, particularly in the form of music, films, or text. Perception of the current driving situation thus increasingly fades into the background, whereby the occurrence of an unusual situation—or a situation that could at least subjectively be perceived as unusual—can easily lead to the aforementioned impairment of the occupant's well-being.

[0013] There is therefore a need for systems and methods that can reliably detect the emotional state of one or more occupants and determine whether it is necessary to influence the emotional state, for example with the aim of calming the occupant(s).

[0014] In particular, there is a need for systems and methods that can influence the emotional state of one or more occupants in a vehicle in a multimodal manner, for example by visual, audiovisual, acoustic, haptic, thermal, or olfactory means, or a combination thereof. Disclosure of the invention

[0015] It is an object of the present disclosure to provide methods and systems that avoid one or more of the described disadvantages and / or enable one or more of the described advantages.

[0016] This object is achieved by the subject matter of the independent claims. Advantageous embodiments are specified in the subclaims.

[0017] In a first aspect, according to embodiments of the present disclosure, there is a method for informing one or more users of a vehicle. The method comprises detecting, for each of the one or more users, one or more state parameters that characterize a state of the respective user; determining, for each of the one or more users, a state of the respective user based on the detected one or more state parameters; determining, for each of the one or more users, a respective deviation of the determined respective state from a normal state; determining one or more operating parameters that characterize an operation of the vehicle; determining, based on the determined one or more operating parameters and on the determined one or more deviations, whether information should be output to the one or more users;and, if information is to be output: determining the information to be output based on the determined one or more operating parameters and on the determined one or more deviations; and outputting the information to be output to the one or more users.;

[0018] In a second aspect according to aspect 1, the one or more state parameters are detected based on a sensor system of the vehicle.The sensor system preferably comprises one or more of: a first sensor device configured to detect a facial expression of one or more users, the first sensor device preferably comprising one or more cameras or optical sensors; a second sensor device configured to detect a viewing direction of one or more users, the second sensor device preferably comprising one or more cameras or optical sensors; a third sensor device configured to detect operation of one or more control elements of the vehicle by one or more users; and a fifth sensor device configured to detect noises in the vehicle and / or voice inputs of one or more users; the fifth sensor device preferably comprising one or more microphones.

[0019] In a third aspect according to one of aspects 1 or 2, the one or more operating parameters are determined based on one or more sensors of the vehicle. Preferably, the sensor system comprises a fourth sensor device configured to detect one or more operating parameters of the vehicle; wherein the determination of the one or more operating parameters preferably takes place indirectly via a control unit of the vehicle.

[0020] In a fourth aspect according to any one of aspects 1 to 3, determining the information to be output comprises one or more of: determining audiovisual information; determining acoustic information; and determining visual information.

[0021] In a fifth aspect according to any one of aspects 1 to 4, outputting the information to be output to the one or more users comprises one or more of: displaying visual information on a window surface of the vehicle; displaying a visual part of audiovisual information on a window surface of the vehicle; reproducing acoustic information via one or more acoustic output units of the vehicle; reproducing an acoustic part of audiovisual information via one or more acoustic output units of the vehicle; and displaying at least part of the information to be output on one or more display units of the vehicle.

[0022] In a sixth aspect according to any one of aspects 1 to 5, the method further comprises determining one or more context parameters configured to characterize a context of use of the vehicle by the one or more users. The steps of determining whether information should be output to the one or more users and / or determining the information to be output are further based on the determined one or more context parameters.

[0023] In a seventh aspect according to the preceding aspect 6, the normal state comprises a respective normal state for each of the one or more users. Additionally or alternatively, the normal state comprises a respective context-dependent normal state for each of the one or more users, wherein the respective context-dependent normal state depends on the one or more context parameters.

[0024] In an eighth aspect according to one of aspects 1 to 7, the method further comprises determining, for each of the one or more users, the normal state based on determining the one or more state parameters of the respective user over a predetermined period of time; and / or determining, for each of the one or more users, the normal state based on one or more state parameters of the respective user, wherein the one or more state parameters are stored in a memory, preferably wherein the memory is implemented in a control unit of the vehicle and / or in one or more back-end components.

[0025] A ninth aspect provides a system for informing one or more users of a vehicle. The system comprises a control unit and / or one or more back-end components; wherein the control unit and / or the one or more back-end components are configured to execute the method according to any one of the preceding aspects 1 to 8.

[0026] In a tenth aspect, a vehicle comprising a control unit is specified, wherein the control unit is configured to carry out one or more steps of the method according to one of the preceding aspects 1 to 8

[0027] According to another aspect, a software program is described. The software program may be configured to be executed on a processor and thereby to perform methods disclosed herein.

[0028] According to another aspect, a storage medium is described. The storage medium may include a software program configured to be executed on a processor and thereby to perform methods disclosed herein.

[0029] The systems and methods disclosed herein provide one or more of the following advantages.

[0030] Devices, methods, and systems according to the present disclosure enable the detection of parameters that characterize a condition, in particular the state of mind, not only of a driver, but of all users or passengers in a vehicle. This can enable the improvement of a variety of functions that process such parameters.

[0031] Devices, methods, and systems according to the present disclosure further enable the state or emotional state of one or more users or occupants to be reliably detected and to determine whether influencing the emotional state is necessary or advisable, for example, with the aim of calming the user or occupants. This can increase the acceptance of automated and / or autonomous systems and methods by the users or occupants and / or improve the well-being of the users or occupants.

[0032] Devices, methods, and systems according to the present disclosure further enable multimodal influencing of the mood of one or more users or occupants in a vehicle, for example, through visual, audiovisual, acoustic, haptic, thermal, or olfactory means, or a combination thereof. This can enable individually tailored influencing of the mood or well-being of the users or occupants. Short description of the drawings

[0033] Embodiments of the disclosure are illustrated in the figures and are described in more detail below. Unless otherwise noted, the same reference numerals are used below for identical and equivalent elements. Fig. 1 schematically illustrates the interaction of individual components of a system for informing users of a vehicle according to embodiments of the present disclosure; Fig. 2 schematically illustrates the structure of a system for informing users of a vehicle according to embodiments of the present disclosure; and Fig. 3 shows a flowchart of a method for informing users of a vehicle according to embodiments of the present disclosure. Embodiments of the disclosure

[0034] In the following, unless otherwise stated, the same reference symbols are used for identical and equivalent elements.

[0035] Fig. 1 schematically illustrates the interaction of individual components of a system 100 for informing users 60, 60', 60'' of a vehicle 80 according to embodiments of the present disclosure. When reference is made to a user 60 in the present disclosure, this generally includes all users 60, 60', 60'', including a user 60 who can act as a driver, a user 60' who can act as a passenger (e.g., located next to the driver), or a user 60'' who can act as a passenger (e.g., on a seat of the vehicle 80). Unless a different treatment of individual user types is expressly discussed below, the term user 60 is understood to mean any user 60, 60', 60''.

[0036] The term "vehicle" encompasses cars, trucks, buses, mobile homes, motorcycles, etc., used to transport people, goods, etc. In particular, the term includes motor vehicles used to transport people. Electric vehicles (especially hybrid or electric vehicles) are powered by an electric motor and include one or more electrical energy storage devices (e.g., batteries, high-voltage storage devices) that can be connected to a charging station and charged via a charging device on the vehicle. Various charging technologies can be used to charge the electrical storage devices of such hybrid or electric vehicles.

[0037] Preferably, the vehicle comprises an environmental sensor system (not included in Fig. 1), which is configured to capture environmental or surrounding data that characterize an environment and / or objects in the surroundings of the vehicle 80. Preferably, the environmental sensor system comprises at least one LiDAR system and / or at least one radar system and / or at least one camera and / or at least one ultrasound system. The environmental sensor system can provide environmental parameters based on environmental data that characterize an area surrounding the vehicle.

[0038] In the context of this disclosure, the term "automated driving" can be understood as driving with automated longitudinal or lateral guidance, or autonomous driving with automated longitudinal and lateral guidance. Automated driving can, for example, involve extended driving on the highway or limited-time driving for parking or maneuvering. The term "automated driving" encompasses automated driving with any degree of automation. Examples of levels of automation include assisted, partially automated, highly automated, or fully automated driving. These levels of automation were defined by the Federal Highway Research Institute (BASt) (see BASt publication "Research Compact," issue 11 / 2012). In assisted driving, the driver continuously performs longitudinal or lateral guidance, while the system assumes the other function within certain limits.In partially automated driving (TAF), the system assumes longitudinal and lateral guidance for a certain period of time and / or in specific situations, whereby the driver must continuously monitor the system, as in assisted driving. In highly automated driving (HAF), the system assumes longitudinal and lateral guidance for a certain period of time without the driver having to continuously monitor the system; however, the driver must be able to assume control of the vehicle after a certain time. In fully automated driving (VAF), the system can automatically handle driving in all situations for a specific application; for this application, a driver is no longer required. The four levels of automation mentioned above correspond to SAE Levels 1 to 4 of the SAE J3016 standard (SAE - Society of Automotive Engineering). For example, highly automated driving (HAF) corresponds to Level 3 of the SAE J3016 standard.Furthermore, SAE J3016 also specifies SAE Level 5 as the highest level of automation, which is not included in the BASt definition. SAE Level 5 corresponds to driverless driving, where the system can automatically handle all situations like a human driver throughout the entire journey; a driver is generally no longer required.

[0039] The system 100 can essentially be implemented on a control unit 120 of the vehicle 80 and / or on one or more backend components 150 (e.g., servers, services).

[0040] In addition to the control unit 120, the vehicle 80 further comprises a communication unit 130, which is configured for data communication 140 with components external to the vehicle 80 (e.g. backend 150), and a user interface 110, which can be implemented, for example, as a touchscreen in the vehicle 80 (e.g. on or in the instrument panel area of ​​the vehicle 80 or as part of a rear-seat entertainment system in the rear of the vehicle 80; not shown). The communication unit 130 can further be configured to provide a data connection to mobile end devices, in particular to those that are located in the vehicle 80 and / or in the vicinity of the vehicle 80 and / or are used by the user 60. Individual components of the system 100 (e.g. control unit 120, communication unit 130, user interface 110) are shown only schematically, so that the representation is not necessarily or necessarilymust allow compelling conclusions about the placement of the respective component, its arrangement in a specific part of the vehicle 80, or other properties of the respective component. Likewise, it is not absolutely necessary for a system 100 to always include all of the information contained in . Fig. 1 shown components. The representation in Fig. 1 thus shows an exemplary embodiment.

[0041] The system 100 can be partially or completely operated via the user interface 110 in the vehicle 80. The user interface 110 can comprise one or more multimodal user interfaces, in particular user interfaces that are configured for operating the vehicle 80 (e.g., communication, infotainment, climate control, seat adjustment, vehicle settings). The user interface 110 enables the multimodal recording of inputs from a user 60, for example, via a graphical user interface (e.g., touchscreen), via conventional operating elements of the vehicle 80 (e.g., buttons, switches, iDrive controller), via voice control (e.g., using sensor 168), via scanner (e.g., for fingerprints, retina), and the like. The user interface 110 further enables the multimodal output of information to a user 60, for example, via a graphical display element (e.g.,Touchscreen, head-up display 171, instrument cluster, central information display or CID), via tactile elements (e.g. vibration of the steering wheel or parts of the seat), via voice output via a loudspeaker system present in the vehicle (e.g. infotainment system) or acoustic signal generators (e.g. gong, beeper) and the like. Based on corresponding configuration data, the user interface 110 can implement a graphical user interface in which display elements and operating elements are presented that can be used by the user 60 to operate the vehicle 80. Additionally or alternatively, the user interface can include (further) display and operating elements, for example switches, buttons and displays.

[0042] Via the communication unit 130, the control unit 120 can enter into data communication 140 with (external) backend components 150 (e.g., servers or services) and thus, for example, communicate with backend servers and / or services. Alternatively or additionally, the control unit 120 can enter into data communication 140 with the mobile terminal of a user 60 via the communication unit 130 and use data and / or sensors provided by the mobile terminal (as described above).

[0043] The control unit 120 is preferably in data communication with a plurality of components via a data bus (not shown). Via the data bus (e.g., CAN bus, FlexRay), the control unit 120 can, for example, be in data communication with the user interface 110, the communication unit 130, the charging device 88, and / or a plurality of other components. Furthermore, the control unit 120 can be configured to receive signals or data from the data bus and / or to send signals or data via the data bus.

[0044] Furthermore, the system 100 can have one or more backend components 150 external to the vehicle 80 and / or an infrastructure that provides one or more resources (e.g., servers, services). The one or more backend components 150 can be in data communication 140 temporarily or permanently with the control unit 120 of the vehicle 80 and / or with the mobile device. Resource-intensive processing steps (e.g., processing of natural language and / or image or video data, speech recognition, processing of large external data volumes) can preferably be outsourced to the external backend component 150, which would be difficult or impossible to perform by the control unit 120 in the vehicle 80 and / or by the mobile device. Possible requirements regarding computing power, storage capacity, available bandwidth, connection to external data sources, and the like can also be taken into account.

[0045] Fig. 2 schematically illustrates the structure of a system 100 for informing users 60, 60', 60'' of a vehicle 80 according to embodiments of the present disclosure. Fig. 2 shows schematic connections between individual components of the system 100. For reasons of clarity, not all possible components of the system 100 are shown in Fig. 2 shown.

[0046] Regarding the Fig. 1 and Fig. 2, the system 100 further comprises a sensor system 160 including one or more sensors 161, 162, 163, 164, 165, etc. Typically, the sensor system 160 or one or more of the sensors or sensor devices 161, 162, 163, 164, 165 is connected to the control unit 120. For reasons of clarity, individual connections are not shown in Fig. 1 shown. In a preferred embodiment, the sensor system 160 includes one or more of the following sensor devices: a sensor device 161 configured to detect a facial expression of a user 60, 60', 60'' (e.g., comprising one or more cameras), a sensor device 162 configured to detect a viewing direction of a user 60, 60', 60'' (e.g., comprising one or more cameras), a sensor device 163 configured to detect an operation of a user 60, 60', 60'' of one or more operating elements of the vehicle 80 (e.g., indirectly via one or more control units in the vehicle 80), a sensor device 164 configured to detect one or more operating parameters of the vehicle 80 (e.g., indirectly via one or more control units in the vehicle 80), and a sensor device 165 configured to detect noises in the vehicle 80 and / or voice inputs of a user 60, 60', 60'' (e.g.B. comprising one or more microphones). One or more further sensors or sensor devices can be configured to detect sounds or acoustic signals outside the vehicle, for example sounds from other vehicles (e.g. horns, engine or rolling noises), signal horns from emergency vehicles (e.g. rescue services, police, fire department), acoustic signals from other road users (e.g. bicycle bells, verbal statements from pedestrians), acoustic signals from objects of the traffic infrastructure (e.g. signals from traffic lights, noises when rolling over markings or bumps), and the like.

[0047] The system 100 further comprises an output system 170 including one or more displays 171, 172, 173, 174, etc. and / or one or more acoustic output units 178. Typically, the output system 170 or one or more of the displays 171, 172, 173, 174 or output units 178 is connected to the control unit 120. For reasons of clarity, individual connections are not shown in Fig. 1. In a preferred embodiment, the output system 170 includes one or more of the following output devices: an output device 171 configured to output a display on the windshield of the vehicle 80 (e.g., comprising a head-up display), one or more output devices 172, 174 configured to output a display on one or more side windows of the vehicle 80, an output device 173 configured to output a display on the rear window of the vehicle 80, one or more acoustic output units 178 configured to output acoustic outputs in the vehicle 80 (e.g., comprising one or more speakers, amplifiers).

[0048] The system 100 further comprises one or more subsystems 121, 122, 123, 124, ..., preferably wherein the one or more subsystems are implemented by the control unit 120. In some embodiments, at least a portion of the one or more subsystems 121, 122, 123, 124, etc. can alternatively or additionally be implemented by the one or more back-end components 150. In this case, an implementation can optionally be implemented both in the control unit 120 and in one or more back-end components 150, wherein the overall functionality of a subsystem is partially realized in the control unit 120 and partially in one or more back-end components 150, or exclusively either in the control unit 120 or in one or more back-end components 150, wherein the overall functionality is then completely realized in the control unit 120 or completely in one or more back-end components 150.An example of an implementation in both the control unit 120 and one or more back-end components 150 relates to the processing of video or audio data, where pre-processing on the control unit 120 can significantly reduce or minimize a data volume, while post-processing on the one or more back-end components 150 then provides the desired results based on the reduced or minimized data volume.

[0049] In a preferred embodiment, the one or more subsystems 121, 122, 123, 124, ..., provide one or more of the following functions: detecting a change in behavior of one or more users 60, 60', 60'' (e.g., focusing on one or more objects outside the vehicle 80), detecting a facial expression and / or a change in a facial expression of one or more users 60, 60', 60'' (e.g., detecting a facial expression that can be associated with a worried or anxious state of mind), detecting a direction of gaze and / or a change in the direction of gaze of one or more users 60, 60', 60'' (e.g., looking outwards or towards one or more objects outside the vehicle 80), detecting an object in a direction of gaze of one or more users 60, 60', 60'' (e.g.,other road users), detecting a significant change in one or more operating parameters of the vehicle 80, detecting environmental data that characterizes one or more events that are not associated with regular or normal operation of the vehicle 80, determining whether one or more significant changes in one or more operating parameters of the vehicle 80 indicate abnormal or abnormal operation of the vehicle 80, identifying one or more objects outside the vehicle 80 as corresponding to regular or normal operation of the vehicle 80, and outputting an acoustic message to one or more users 60, 60', 60'' of the vehicle 80 regarding one or more operating parameters of the vehicle 80 and / or regarding one or more objects in the environment of the vehicle 80.

[0050] According to the present disclosure, the system 100 is configured to detect a state (in particular a state of mind) of one or more users 60, 60', 60'' of the vehicle 80 and to determine any impairment thereof (e.g., in the form of a deviation from a state classified as regular, normal, or harmless). A regular state or a normal state is configured to identify a state of a user 60, 60', 60'' without impairment and / or a state in which the respective user does not show any distress. The normal state can include a respective context-dependent normal state for each of the one or more users 60, 60', 60'', wherein the respective context-dependent normal state depends on one or more context parameters. For example, the respective normal state can depend on a time of day, a location, a usage context, or other parameters.A respective normal state of a user 60, 60', 60'' can be stored in the vehicle 80 or externally to the vehicle 80. Furthermore, a respective normal state of a user 60, 60', 60'' can be recorded as such over a period of time, wherein the period preferably comprises at least a few minutes, preferably a few tens of minutes, or an hour. The recording can be carried out continuously or, if necessary, over one or more periods of time. Furthermore, the recording can be repeated or updated at regular intervals in order to record or map changes in the behavior of a respective user 60, 60', 60''.

[0051] Based on this, the system 100 can determine whether information should be output to the one or more users 60, 60', 60'' of the vehicle 80 in order to reduce or minimize the impairment of the condition, or to restore a condition classified as regular, normal, or harmless. The information preferably comprises visual, audiovisual, and / or acoustic signals. The condition of the user 60, 60', 60'' is preferably determined quantitatively and / or qualitatively by the system 100. Further preferably, the system 100 is configured to determine and / or keep up-to-date a condition classified as regular, normal, or harmless, preferably for each user 60, 60', 60'' from a plurality of users 60, 60', 60''.All conditions, including those currently existing or currently determined, as well as conditions classified as regular, normal or harmless, are preferably determined, kept up to date and / or stored in a context-based manner so that an assessment can be made in a context-based manner (e.g. depending on time, date, location, time of day, day of the week, weather, temperature and the like).

[0052] In a first application example, the system 100 according to the present disclosure detects a change in the facial expression of a user 60, 60', 60'' and, based on a plurality of facial expressions of the user 60, 60', 60'', determines a state of mind of the user 60, 60', 60'' that expresses concern. Furthermore, the system detects a gaze direction of the user 60, 60', 60'' directed toward a third-party vehicle located in the vicinity of the vehicle 80. The system 100 detects one or more current operating parameters of the vehicle 80 (e.g., direction of travel and speed) and operating parameters of the third-party vehicle (e.g., direction of travel and speed) and determines whether a hazard exists (e.g., contact, collision) based on the detected operating parameters of the vehicle 80 and the third-party vehicle.In the first application example, system 100 determines that, based on the detected operating parameters of vehicle 80 and the other vehicle, no dangerous situation exists and outputs a corresponding output to user 60, 60', 60''. The output comprises a projection of a green-colored, transparent surface onto the window located in the detected viewing direction of user 60, 60', 60'', wherein the projected surface is essentially superimposed on the other vehicle with respect to the detected viewing direction of user 60, 60', 60''. Thus, user 60, 60', 60'' is subtly suggested that the other vehicle targeted by user 60, 60', 60'' poses no danger.

[0053] In a second application example, the system 100 according to the present disclosure detects a change in the facial expression of two users 60, 60', 60'' and, based on a plurality of facial expressions of the respective user 60, 60', 60'', determines a facial expression of the users 60, 60', 60'' that is classified as regular or normal. The system 100 further detects one or more current operating parameters of the vehicle 80 and, based on the detected one or more current operating parameters of the vehicle 80 (e.g., tire pressures determined by means of tire pressure sensors), determines a defect in the vehicle 80 in the form of a sudden tire pressure loss. The system 100 further determines, based on the detected one or more operating parameters of the vehicle 80 (e.g.,concerning equipment features of the vehicle 80) that the vehicle 80 is equipped with run-flat tires, and the vehicle 80 can therefore continue to be operated at a reduced speed. The system 100 thus determines in the second application example that, based on the recorded operating parameters of the vehicle 80, there is no dangerous situation and issues a corresponding output to the user 60, 60', 60''. In the second application example, the output comprises a voice output by means of which the users 60, 60', 60'' are informed about the defect and that further travel is possible. In a preferred embodiment, the system 100 is configured to take into account data from the navigation system. Based on this data, the system 100 can determine a remaining distance and compare it with a recommended maximum remaining distance due to the tire defect. Based on the comparison, the system 100 caninform users 60, 60', 60'' that a destination entered into the navigation system can still be reached, or alternatively that a workshop should be headed for to repair the defect.

[0054] In a third application example, the system 100 according to the present disclosure considers a plurality of context parameters that characterize a usage context and individual thresholds for each of the users 60, 60', 60'' in the vehicle. The context parameters are configured to characterize a usage context, for example, including time, date, location, time of day, day of the week, weather, and / or temperature. In the third application example, a first user 60, 60', 60'' and a second user 60, 60', 60'' have different thresholds, wherein the first user 60, 60', 60'' is less likely to be alarmed and the second user 60, 60', 60'' is more likely to be alarmed, for example, due to a bad experience. The context parameters indicate a trip in bad weather and high traffic density.In this case, the system 100 detects that the second user 60, 60', 60'' is alarmed, while detection of the first user 60, 60', 60'' does not provide any indication of this. Accordingly, the system 100 outputs information to the second user 60, 60', 60'' that the vehicle's operating parameters are within the normal range and that, despite the higher traffic density, a punctual arrival at the destination is expected. This can be done via a visual display (e.g., in the rear-seat infotainment system) exclusively for the second user 60, 60', 60'', or alternatively or additionally via voice output for all users 60, 60', 60'', regardless of any alarm detected for the respective user(s) 60, 60', 60''.

[0055] The threshold values ​​can optionally be determined based on or dependent on the context parameters. Thus, the system 100 is configured to evaluate the specific driving situation and, if necessary, inform the users 60, 60', 60'' depending on the context parameters and / or the individual threshold values. For example, it is possible for individual users 60, 60', 60'' to have relatively high threshold values ​​(i.e., not tend to cause alarm) in a first context (e.g., during the day, in city traffic, at low speeds) and relatively low threshold values ​​(i.e., more tend to cause alarm) in a second context (e.g., in the evening or at night, on highways or country roads, at medium or higher speeds).

[0056] In a fourth application example, the system 100 according to the present disclosure detects a change in the facial expression of one or more users 60, 60', 60'' and, based on a plurality of facial expressions of the respective user 60, 60', 60'', determines a facial expression of the users 60, 60', 60'' that differs from a respective regular or normal facial expression. Furthermore, the system 100 detects one or more environmental parameters that characterize the operation of the vehicle 80 and, based on the detected one or more environmental parameters, determines an unusual noise development outside the vehicle 80 caused by an emergency vehicle with a horn. The system 100 further determines, based on the detected one or more environmental parameters, a probable trajectory of the emergency vehicle and relates this to a planned or alternative trajectory of the vehicle 80 itself.In this case, the system 100 can take into account a possible reaction to the presence of the emergency vehicle and, if necessary, deviate from the planned trajectory (e.g., forming an emergency lane, swerving onto a hard shoulder or grass verge). In the fourth application example, the system 100 thus determines that, based on the one or more detected environmental parameters, no dangerous situation exists and issues a corresponding output to the user 60, 60', 60''. In the fourth application example, the output comprises a voice output by means of which the users 60, 60', 60'' are informed of the presence of the emergency vehicle. In a preferred embodiment, the system 100 is configured to inform the user(s) 60, 60', 60'' of a possibly necessary temporary deviation from a planned trajectory (e.g., forming an emergency lane, swerving onto a hard shoulder or grass verge) and that further travel will be possible shortly.

[0057] In a fifth application example, the system 100 according to the present disclosure detects one or more environmental parameters that characterize operation of the vehicle 80 and, based on the detected one or more environmental parameters, determines an unusual noise development outside the vehicle 80, which is caused by a noise development emanating from another road user (e.g., horn of another vehicle; alternatively, bicycle bell, verbal utterance of a pedestrian). The system 100 can in particular be configured to become active before detecting a change in the facial expression of one or more users 60, 60', 60'' and a comparison based on a plurality of facial expressions of the respective user 60, 60', 60'' with a respective regular or classified as normal facial expression of the users 60, 60', 60''.Based on the detected one or more environmental parameters, system 100 further determines that the noise or acoustic signal of the other road user is or was directed at a third road user and that no dangerous situation exists. The system therefore issues a corresponding output to the user(s) 60, 60', 60''. In the fifth application example, the output includes a voice output, by means of which the user(s) 60, 60', 60'' is / are informed about the traffic situation.

[0058] The system 100 can be further configured to determine the effectiveness of outputting information to one or more users 60, 60', 60''. Effectiveness can be determined, for example, based on a quantifiable and / or qualitatively detectable change in the state of a user 60, 60', 60''. The system can be configured to detect a change (or the lack of a change) within predetermined intervals or triggered by a detected change in the state of one or more users 60, 60', 60'' and to determine the effectiveness accordingly. If the effectiveness is lacking or too low, the system 100 can again output information to the user(s) 60, 60', 60''. The absence of a change after detecting a state of a user 60, 60', 60'' and outputting information to the user 60, 60', 60'' can be associated with low effectiveness.In such a case, the system 100 may be configured to change a modality of the output to increase effectiveness (e.g., speech output instead of or in addition to visual output).

[0059] Fig.3 shows a flowchart of a method 300 for informing users 60, 60', 60'' of a vehicle 80 according to embodiments of the present disclosure. The method begins at step 301. In step 302, for each of the one or more users 60, 60', 60'', one or more state parameters are detected, which characterize a state of the respective user 60, 60', 60''. In step 304, for each of the one or more users 60, 60', 60'', a state of the respective user 60, 60', 60'' is determined based on the detected one or more state parameters. In step 306, for each of the one or more users 60, 60', 60'', a respective deviation of the determined respective state from a normal state is determined. In step 308, one or more operating parameters that characterize operation of the vehicle 80 are determined.In step 310, based on the determined one or more operating parameters and the determined one or more deviations, it is determined whether information should be output to the one or more users 60, 60', 60''. If information should be output, in step 312 the information to be output is determined based on the determined one or more operating parameters and the determined one or more deviations; and in step 314 the information to be output is output to the one or more users 60, 60', 60''.

[0060] Although the invention has been illustrated and explained in detail by means of preferred embodiments, the invention is not limited by the disclosed examples, and other variations may be derived therefrom by those skilled in the art without departing from the scope of the invention. It is therefore clear that a multitude of possible variations exist. It is also clear that the embodiments mentioned by way of example are truly only examples and should not be construed as limiting the scope, possible applications, or configuration of the invention in any way.Rather, the preceding description and the description of the figures enable the person skilled in the art to implement the exemplary embodiments in concrete terms, whereby the person skilled in the art, with knowledge of the disclosed inventive concept, can make various changes, for example with regard to the function or the arrangement of individual elements mentioned in an exemplary embodiment, without departing from the scope of protection defined by the claims and their legal equivalents, such as further explanations in the description.

Claims

[1] Method (300) for providing reassuring information to a user (60, 60', 60'') of a vehicle (80), the method comprising: - detecting a facial expression of the user (60, 60', 60''), wherein the facial expression can be associated with a worried or anxious state of mind; - determining (308) one or more operating parameters that characterize operation of the vehicle (80); - determining (310), based on the determined one or more operating parameters, that no hazardous situation exists; and - Outputting (314) corresponding information to the user (60, 60', 60''). [2] Method (300) according to claim 1, wherein the dangerous situation concerns a contact or collision with a foreign vehicle in the vicinity of the motor vehicle (80). [3] Method (300) according to claim 1 or 2, wherein the operating parameters comprise a direction of travel and a speed of travel. [4] Method (300) according to one of the preceding claims 1 to 3, wherein the information to be output comprises one or more of: audiovisual information; an acoustic information; and visual information. [5] Method (300) according to one of the preceding claims 1 to 4, wherein outputting (314) the information to be output to the user (60, 60', 60'') comprises one or more of: Displaying visual information on a window surface (170; 171, 172, 173, 174) of the vehicle (80); Displaying a visual part of audiovisual information on a window surface (170; 171, 172, 173, 174) of the vehicle (80); Reproducing acoustic information via one or more acoustic output units (178) of the vehicle (80); Reproducing an acoustic part of audiovisual information via one or more acoustic output units (178) of the vehicle (80); and Displaying at least part of the information to be output on one or several display units (110) of the vehicle (80). [6] Method (300) according to one of the preceding claims, further comprising determining one or more context parameters configured to characterize a context of use of the vehicle (80) by the user (60, 60', 60''); wherein the steps of determining (310) whether information should be output to the user (60, 60', 60'') and / or determining (312) the information to be output are further based on the determined one or more context parameters. [7] System (100) for providing reassuring information to a user (60, 60', 60'') of a vehicle (80), the system (100) comprising: a control device (120); and / or one or more back-end components (150); wherein the control unit (120) and / or the one or more back-end components (150) are configured to execute the method (300) according to one of claims 1 to 6. [8] Vehicle (80) comprising a control unit (120), wherein the control unit (120) is configured to carry out one or more steps of the method (300) according to one of claims 1 to 6.

Images