System for providing information to a vehicle occupant

Abstract

A system for providing information to a vehicle occupant may comprise one or more occupant sensors, an infotainment system, and a controller in electrical communication with the one or more occupant sensors and the infotainment system. The controller is programmed to determine one or more inputs using at least the one or more occupant sensors. The controller is further programmed to determine a layout of each of a multitude of user interface (UI) elements of the infotainment system based at least partially on the one or more inputs. The controller is further programmed to provide information to the vehicle occupant by displaying the multitude of UI elements based at least partially on the layout of each of the multitude of UI elements.

Description

Technical field

[0001] The present disclosure relates to systems and methods for providing information to an occupant of a vehicle and in particular to vehicle infotainment system user interface functionality and design. Introduction

[0002] To facilitate user interaction with infotainment systems in vehicle applications, various user interface technologies can be employed. Vehicle infotainment systems can include display screens, touch-sensitive controls, physical buttons, voice recognition modules, and / or similar devices to enable interaction with navigation, media playback, climate control, communication systems, and other vehicle functions. In some examples, user interfaces may include configurable layouts or menu structures that adapt based on user preferences. Additionally, infotainment system interfaces can be integrated with external devices via wireless communication protocols, allowing users to access mobile applications, cloud services, or real-time traffic and weather updates.Some vehicle infotainment interfaces also provide multimodal interaction capabilities, such as combining voice commands with haptic feedback or gesture recognition, to improve usability while minimizing driver distraction.

[0003] DE 10 2014 118 957 A1 describes a method for operating a control arrangement for a motor vehicle, in which a touch of at least one button displayed on the control element is detected as an operating action by means of a control element which is arranged on a steering wheel of the motor vehicle and which has a touch-sensitive screen, and in which at least one button is selected from a plurality of buttons for display on the control element by means of a control device and / or in which the display of the at least one button on the control element is adapted, wherein a change of state relating to the motor vehicle and / or a driver of the motor vehicle is detected by means of a detection device and in which the at least one button is selected and / or the display of the at least one button is adapted by means of the control device depending on the detected change of state.

[0004] DE 10 2016 202 251 A1 describes an arrangement, a user terminal device, a means of locomotion, and a method for assisting a user in operating a touch-sensitive display device. The method comprises the following steps: - Displaying content visible from a first position and a second position using the display device, - Detecting the user's approach to the display device, - Detecting that the user is located away from the second position with respect to the display device, - Changing the operating state of the display device from a display mode to an operating mode, wherein at least one display element of the content shown on the display device, which is associated with the second position with respect to the display device and for which a display mode and an operating mode are provided, remains in a display mode.

[0005] DE 10 2007 015 878 A1 describes an operating unit, particularly for a vehicle, with a display for showing selectable functions. The display is designed such that a first display image can be shown in a first viewing direction, and a second display image different from the first can be shown in a second viewing direction. The two display images differ with respect to at least one selectable function shown in each display image. The operating unit is characterized by an interface to a detection device for determining the viewing direction from which the at least one control device is operated and for assigning the operation performed to a displayed selectable function in the display image visible from the determined viewing direction.

[0006] DE 10 2023 123 205 A1 describes a method for adapting a graphical user interface of an operating device in a motor vehicle to a driving state of the motor vehicle, whereby the driving state is differentiated between a stationary motor vehicle and a motor vehicle in motion.The procedure involves repeatedly acquiring geopositional data at each acquisition time, which indicates a GNSS-based geoposition and / or driving speed of the motor vehicle, and calculating an average speed value from the geopositional data of at least three different acquisition times. If it is detected that the average speed value is less than a predefined threshold, the driving state "stationary" is signaled; otherwise, the driving state "driving" is signaled. A state-specific interface layout and / or functional scope of the user interfaces, assigned to the respective signaled driving state, is then set depending on the signaled vehicle status.

[0007] While systems and methods for vehicle infotainment user interfaces fulfill their intended purpose, the object of the invention is to provide new and improved systems and methods for vehicle infotainment system user interface functionality and design for providing information to vehicle occupants. Description of the invention

[0008] The invention is defined by the claims.

[0009] According to the invention, a system for providing information to a vehicle occupant is provided. The system comprises one or more occupant sensors, an infotainment system, and a controller in electrical communication with the one or more occupant sensors and the infotainment system. The controller is programmed to determine one or more inputs using at least one or more occupant sensors. The controller is further programmed to determine a layout of each of a plurality of user interface (UI) elements of the infotainment system based at least partially on the one or more inputs. The controller is further programmed to provide information to the vehicle occupant by displaying the plurality of UI elements based at least partially on the layout of each of the plurality of UI elements.To determine the layout of each of the multitude of UI elements, the controller is further programmed to determine an importance rating for each of the multitude of UI elements based at least partially on one or more inputs and one or more UI element properties of each of the multitude of UI elements. The layout includes at least one position of each of the multitude of UI elements on the infotainment system. A distance between the position of each of the multitude of UI elements and an occupant's hand position or viewing position is negatively correlated with the importance rating of each of the multitude of UI elements.

[0010] According to one embodiment, the one or more occupant sensors further comprise an occupant position tracking device. To determine the one or more inputs, the controller is further programmed to determine the occupant's position relative to the infotainment system using the occupant position tracking device.

[0011] According to a further embodiment, the controller for determining the importance rating of each of the multitude of UI elements is further programmed to determine the importance rating of each of the multitude of UI elements based at least partially on one or more inputs. The one or more inputs include the position of the occupant relative to the infotainment system. The importance rating of each of the multitude of UI elements is negatively correlated with a distance between a hand position of the occupant and each of the multitude of UI elements.

[0012] According to a further embodiment, the one or more occupant sensors further comprise an occupant position tracking device. To determine the one or more inputs, the controller is further programmed to determine the occupant's viewing position relative to the infotainment system using the occupant position tracking device.

[0013] According to a further embodiment, the control system for determining the importance rating of each of the multitude of UI elements is further programmed to determine the importance rating of each of the multitude of UI elements based at least partially on one or more inputs. The one or more inputs include the occupant's gaze position relative to the infotainment system. The importance rating of each of the multitude of UI elements is negatively correlated with the distance between the gaze position and each of the multitude of UI elements.

[0014] According to another embodiment, the controller for determining one or more inputs is further programmed to determine measurements of vehicle operating conditions. The vehicle operating conditions include one or more of the following: vehicle speed, vehicle acceleration, and vehicle vibration.

[0015] According to another embodiment, the controller is further programmed to determine one or more inputs in order to ascertain information about external environmental conditions. The external environmental conditions include one or more of the following: a weather condition in the environment surrounding the vehicle, and a traffic dust condition in the environment surrounding the vehicle. To determine one or more inputs, the controller is further programmed to ascertain information about internal environmental conditions. The internal environmental conditions include one or more of the following: a number of occupants in the vehicle, a type of occupant in the vehicle, and a noise level inside the vehicle.

[0016] According to another embodiment, the control for determining the layout of each of the multitude of UI elements is further programmed to determine the layout of each of the multitude of UI elements based at least partially on the importance rating of each of the multitude of UI elements. The layout includes at least one size of each of the multitude of UI elements. The size of each of the multitude of UI elements is positively correlated with the importance rating of each of the multitude of UI elements.

[0017] Further areas of application will become apparent from the description provided herein. It is understood that the description and specific examples serve only for illustrative purposes and are not intended to limit the scope of this disclosure. Brief description of the drawings

[0018] The drawings described herein serve only for illustrative purposes and are not intended to limit the scope of the present disclosure in any way. Fig. Figure 1 is a schematic representation of a system for providing information to an occupant of a vehicle according to an exemplary embodiment; Fig. Figure 2 is a schematic representation of the interior of the vehicle according to an exemplary embodiment; Fig. Figure 3 is a schematic representation of a first exemplary user interface for an infotainment system according to an exemplary embodiment; Fig. Figure 4 is a flowchart of a method for providing information to an occupant of a vehicle according to an exemplary embodiment; Fig. Figure 5 is a schematic representation of a second exemplary user interface for an infotainment system according to an exemplary embodiment; and Fig. Figure 6 is a schematic representation of a third exemplary user interface for an infotainment system according to an exemplary embodiment. Detailed description

[0019] The following description is merely exemplary and is not intended to limit the present disclosure, application or uses.

[0020] In aspects of the present disclosure, it is advantageous to adapt / adjust the layout and / or functionality of a vehicle infotainment system's user interface based on vehicle conditions, environmental conditions, and / or occupant behavior. Accordingly, the present disclosure provides a new and improved system and method for providing information to a vehicle occupant that takes into account current conditions and occupant behavior when generating user interfaces.

[0021] With reference to Fig. Figure 1 illustrates a system for providing information to a vehicle occupant and is generally specified by reference numeral 10. The system 10 is shown with an exemplary vehicle 12. While a passenger car is illustrated, it is understood that the vehicle 12 can be any type of vehicle without derogating from the scope of this disclosure. The system 10 generally comprises a control unit 14, one or more vehicle sensors 16, and an infotainment system 18.

[0022] The controller 14 is used to implement a method 100 for providing information to a vehicle occupant, as described below. The controller 14 includes at least one processor 20 and one non-volatile, computer-readable storage device or storage medium 22. The processor 20 can be a custom or commercially available processor, a central processing unit (CPU), a graphics processing unit (GPU), an auxiliary processor among several processors associated with the controller 14, a microprocessor-based semiconductor (in the form of a microchip or chipset), a macroprocessor, a combination thereof, or generally, a device for executing instructions.

[0023] The computer-readable storage device or computer-readable storage media 22 may include volatile and non-volatile storage in read-only memory (ROM), random-access memory (RAM), and keep-alive memory (KAM). KAM is persistent or non-volatile memory that can be used to store various operating variables while the processor 20 is powered off. The computer-readable storage device or computer-readable storage media 22 may be implemented using a variety of storage devices, such as PROMs (programmable read-only memory), EPROMs (electrical PROMs), EEPROMs (electrically erasable PROMs), flash memory, or other electrical, magnetic, optical, or combination storage devices capable of storing data, some of which may be executable instructions used by the controller 14 to control various systems of the vehicle 12.

[0024] The control unit 14 can also comprise multiple control units that communicate electrically with each other. The control unit 14 can be connected to additional systems and / or controls of the vehicle 12, which allows the control unit 14 to access data such as the vehicle 12's speed, acceleration, braking, and steering angle.

[0025] The controller 14 communicates electrically with the one or more vehicle sensors 16 and the infotainment system 18. In an exemplary embodiment, the electrical communication is established, for example, using a CAN network, a FLEXRAY network, a local area network (e.g., WiFi, Ethernet, and the like), a network with a serial peripheral interface (SPI), or the like. It is understood that various additional wired and wireless techniques and communication protocols for communicating with the controller 14 are within the scope of this disclosure. It is further understood that, within the scope of this disclosure, the electrical communication also includes the transfer of power and / or energy between electrical devices (e.g., using conductive wires and / or wireless power transmission techniques).

[0026] The one or more vehicle sensors 16 are used to acquire information relevant to the vehicle 12. In an exemplary embodiment, the one or more vehicle sensors 16 comprise sensors to determine performance data and / or operating conditions of the vehicle 12. In a non-limiting example, the one or more vehicle sensors 16 comprise at least one electric motor speed sensor, an electric motor torque sensor, an electric drive motor voltage and / or current sensor, an accelerator pedal position sensor, a brake position sensor, a coolant temperature sensor, a cooling fan speed sensor, and a transmission oil temperature sensor.

[0027] In an exemplary embodiment, the one or more vehicle sensors 16 further comprise a vehicle speed sensor 24. The vehicle speed sensor 24 is used to measure the speed of the vehicle 12. In an exemplary embodiment, the vehicle speed sensor 24 comprises a toothed ferromagnetic ring or a toothed ferromagnetic disk (also known as a reluctor or tone ring) that is mechanically connected to a wheel of the vehicle 12 such that the toothed ferromagnetic ring rotates at the same speed as the wheel of the vehicle 12.The vehicle speed sensor 24 further comprises a sensor (for example, a wire coil) that is attached near the toothed ferromagnetic ring and is stationary relative to the toothed ferromagnetic ring, so that the sensor experiences an induced alternating voltage due to the changing magnetic field caused by the rotation of the toothed ferromagnetic ring.

[0028] In a non-restrictive example, the induced AC voltage is sinusoidal, and a magnitude and / or frequency of the induced AC voltage is proportional to a rotational speed of the toothed ferromagnetic ring and thus to the vehicle speed. In one exemplary embodiment, the induced AC voltage is transmitted directly to the controller 14 for further processing, for example, using an analog-to-digital converter of the controller 14. In another exemplary embodiment, the vehicle speed sensor 24 further comprises a signal conditioning circuit that processes the induced AC voltage before transmission to the controller 14 (for example, amplifying the induced AC voltage, digitizing the induced AC voltage, encoding the vehicle speed value into a communication protocol, and / or the like).

[0029] It is understood that various additional devices and methods for determining vehicle speed, such as global navigation satellite systems (for example, GPS), sensors that measure other powertrain components (for example, transmission speed sensor, electric motor speed sensor, and / or the like), and / or the like, are within the scope of this disclosure. The vehicle speed sensor 24 is in electrical communication with the controller 14, as discussed above.

[0030] In an exemplary embodiment, the one or more vehicle sensors 16 further comprise an inertial measurement unit (IMU) 26. The IMU 26 is used to determine the orientation, velocity, and gravitational forces acting on the vehicle 12. In an exemplary embodiment, the IMU 26 comprises multiple sensors, including accelerometers, gyroscopes, and / or magnetometers. In a non-limiting example, the IMU 26 comprises tri-axis accelerometers and tri-axis gyroscopes integrated into a single unit. The accelerometers measure the linear acceleration along each axis, while the gyroscopes measure the angular velocity about each axis. The IMU 26 processes data from the sensors to calculate the current orientation, velocity, direction, yaw rate (that is, rate of change of direction), and acceleration of the vehicle 12 in three-dimensional space.The IMU 26 is in electrical communication with the controller 14, as discussed above.

[0031] In another exemplary embodiment, the one or more vehicle sensors 16 further comprise sensors for determining information about external environmental conditions in an environment 28 surrounding the vehicle 12. In a non-limiting example, the one or more vehicle sensors 16 further comprise at least one of an ambient air temperature sensor, an air pressure sensor, and / or a global navigation satellite system (GNSS).

[0032] In an exemplary embodiment, the one or more vehicle sensors 16 further comprise one or more perception sensors capable of perceiving objects and / or measuring distances in the environment 28 surrounding the vehicle 12, such as a camera system 30 and / or a LiDAR sensor 32. It is understood that various additional types of perception sensors, such as ultrasonic distance sensors, radar sensors and / or time-of-flight sensors, are within the scope of this disclosure.

[0033] The camera system 30 is a perception sensor used to capture images and / or videos of the environment 28 surrounding the vehicle 12. In one exemplary embodiment, the camera system 30 comprises a photo and / or video camera positioned to view the environment 28 surrounding the vehicle 12. In a non-limiting example, the camera system 30 comprises a camera mounted inside the vehicle 12, for example, in the headliner of the vehicle 12, with a view through the windshield. In another non-limiting example, the camera system 30 comprises a camera mounted outside the vehicle 12, for example, on the roof of the vehicle 12, with a view of the environment 28 in front of the vehicle 12.

[0034] In another exemplary embodiment, the camera system 30 is a surround-view camera system comprising a plurality of cameras (also known as satellite cameras) arranged to provide a view of the surroundings 28 on all sides of the vehicle 12. In a non-limiting example, the camera system 30 comprises a forward-facing camera (mounted, for example, in a front grille of the vehicle 12), a rear-facing camera (mounted, for example, on a tailgate of the vehicle 12), and two side-facing cameras (mounted, for example, below each of two side mirrors of the vehicle 12). In another non-limiting example, the camera system 30 further comprises an additional reversing camera mounted near a centrally mounted high-mounted brake light of the vehicle 12.

[0035] It is understood that camera systems with additional cameras and / or additional mounting points are within the scope of this disclosure. It is further understood that cameras with different sensor types, including, for example, CCD sensors (CCD = charge-coupled device), CMOS sensors (CMOS = complementary metal oxide semiconductor), and / or HDR sensors (HDR = high dynamic range), are within the scope of this disclosure. Furthermore, cameras with different lens types, including, for example, wide-angle lenses and / or narrow-angle lenses, are also within the scope of this disclosure.

[0036] The LiDAR sensor 32 is used for remote sensing and environmental mapping by emitting laser pulses and measuring the time it takes for the laser pulses to return to the LiDAR sensor 32 after striking objects. In an exemplary embodiment, the LiDAR sensor 32 comprises a LiDAR laser source, a LiDAR scanner or mirror, a LiDAR photodetector, and a LiDAR time-of-flight measurement system. In a non-restrictive example, the LiDAR laser source emits laser pulses that travel toward the target area, and the LiDAR scanner directs these pulses in various directions. The emitted laser pulses interact with objects in the environment, and their reflections are detected by the LiDAR photodetector. The LiDAR time-of-flight measurement system calculates the distance to the objects based on the time between the emission of the laser pulses by the LiDAR laser source and the reception of the reflected laser pulses by the LiDAR photodetector.The LiDAR sensor 32 is in electrical communication with the controller 14, as discussed above.

[0037] In another exemplary embodiment, the one or more vehicle sensors 16 further comprise a vehicle communication system 34 for receiving information from external systems. The vehicle communication system 34 is used by the controller 14 to communicate with other systems outside the vehicle 12. For example, the vehicle communication system 34 includes capabilities for communicating with vehicles (“V2V” communication), infrastructure (“V2I” communication), remote systems in a remote call center (for example, GENERAL MOTORS ON-STAR), and / or personal devices. In general, the term vehicle-to-everything communication (“V2X” communication) refers to communication between the vehicle 12 and any remote system (for example, vehicles, infrastructure, and / or remote systems).

[0038] In certain embodiments, the vehicle communication system 34 is a wireless communication system configured to communicate over a wireless local area network (WLAN) using IEEE 802.11 standards or using cellular data communication (for example, using GSMA standards such as SGP.02, SGP.22, SGP.32, and the like). Accordingly, the vehicle communication system 34 may further comprise an embedded universal integrated circuit (eUICC) configured to store at least one cellular connection configuration profile, for example, an embedded subscriber identity module (eSIM) profile.

[0039] The vehicle communication system 34 is further configured to communicate via a personal network (for example, BLUETOOTH), near-field communication (NFC), and / or any additional type of radio frequency communication. However, additional or alternative communication methods, such as a dedicated short-range wireless communication (DSRC) channel and / or mobile telecommunications protocols based on the standards of the 3rd Generation Partnership Project (3GPP), are also considered within the scope of this disclosure. DSRC channels refer to one-way or two-way short- to medium-range wireless communication channels specifically designed for automotive use and a corresponding set of protocols and standards. The 3GPP refers to a partnership between several standards organizations that develop protocols and standards for mobile telecommunications.3GPP standards are structured as "releases". Therefore, communication procedures based on 3GPP Releases 14, 15, 16 and / or future 3GPP releases are considered within the scope of this disclosure.

[0040] Accordingly, the vehicle communication system 34 may include one or more antennas and / or communication transceivers for receiving and / or transmitting signals, such as cooperative sensing messages (CSMs). The vehicle communication system 34 is configured to wirelessly communicate information between the vehicle 12 and another vehicle. Furthermore, the vehicle communication system 34 is configured to wirelessly communicate information between the vehicle 12 and infrastructure or other vehicles. It is understood that the vehicle communication system 34 may be integrated into the controller 14 (for example, on the same circuit board as the controller 14 or otherwise be a part of the controller 14) without deviating from the scope of this disclosure.

[0041] In another exemplary embodiment, the one or more vehicle sensors 16 further comprise sensors for determining information about internal environmental conditions of an environment within the vehicle 12. In a non-limiting example, the one or more vehicle sensors 16 further comprise at least one of seat occupancy sensors, a cabin air temperature sensor, a cabin motion detection sensor, a cabin camera, a cabin microphone, and / or the like.

[0042] In another exemplary embodiment, the one or more vehicle sensors 16 further comprise one or more occupant sensors 36. In a non-limiting example, the one or more occupant sensors comprise an occupant position tracking device.

[0043] The occupant position tracking device is used to determine the position of an occupant in the vehicle 12. For the purposes of this disclosure, in a non-limiting example, the occupant includes a driver, a passenger, and / or any additional persons in the vehicle 12. For example, the occupant position tracking device can track the position of the occupant's head, appendages (for example, arms, hands, legs, feet, and so on), and / or eyes. In an exemplary embodiment, the occupant position tracking device includes one or more cameras arranged in the vehicle 12, for example, as part of a driver monitoring system (DMS) used to monitor the occupant's attention when automated driving or driver assistance features are used.

[0044] In a non-limiting example, the occupant position tracking device is configured to determine the position of the occupant relative to the infotainment system 18 by processing data from one or more cameras using computer vision algorithms. For example, the occupant position tracking device is configured to determine the hand position of the occupant relative to the infotainment system 18. Within the scope of this disclosure, the hand position of the occupant is the position of a hand and / or fingertip of the occupant within the vehicle 12.

[0045] In another non-restrictive example, the occupant position tracking device is configured to determine the occupant's gaze position relative to the infotainment system 18. Within the scope of this disclosure, the occupant's gaze position is a position within the vehicle 12 toward which the occupant is looking (that is, a position at a center point in the occupant's field of vision). In an exemplary embodiment, the occupant's gaze position is determined by processing data from the one or more cameras using computer vision algorithms to identify a pupil center point, compute a gaze vector based on the position of the pupil center point, and determine the gaze position based on an intersection point with the gaze vector and the vehicle 12 and / or the infotainment system 18.

[0046] It is understood that various additional methods for determining hand position and / or gaze position are within the scope of this disclosure. The occupant position tracking device is in electrical communication with the controller 14, as described above.

[0047] It is understood that the one or more vehicle sensors 16 discussed above are merely exemplary and that the one or more vehicle sensors 16 may further comprise additional and / or alternative sensors without deviating from the scope of the present disclosure.

[0048] The infotainment system 18 is used to provide information to the occupant of the vehicle 12. In one exemplary embodiment, the infotainment system 18 comprises a human-machine interface (HMI) located within the occupant's field of vision and capable of displaying text, graphics, and / or images. It is understood that HMI display systems, including LCD displays, LED displays, and the like, are within the scope of this disclosure. In some embodiments, the HMI is located relatively centrally in the vehicle 12, as discussed below. Fig. Figure 2 shows that in some embodiments, the HMI occupies a substantial portion of the entire width of the vehicle 12's dashboard, extends behind the vehicle 12's steering wheel, and provides additional functions such as an instrument cluster display. In other embodiments, the HMI comprises multiple displays arranged across the vehicle 12's dashboard, some of which are primarily configured for viewing by the driver of the vehicle 12 and some of which are primarily configured for viewing by a passenger of the vehicle 12. In still other embodiments, the HMI comprises display(s) arranged in an additional passenger compartment(s) of the vehicle 12 (for example, in a rear passenger compartment).It is understood that the HMI and thus the infotainment system can include any 18 displays within the vehicle 12 that are configured to provide information to any occupants of the vehicle 12 and / or receive input from them.

[0049] In another exemplary embodiment, the infotainment system 18 includes a head-up display (HUD) configured to provide information to the occupant by projecting text, graphics, and / or images onto the windshield of the vehicle 12. The text, graphics, and / or images are reflected off the windshield of the vehicle 12 and are visible to the occupant without them having to look away from the road ahead. In a further exemplary embodiment, the infotainment system 18 includes an augmented reality head-up display (AR-HUD). The AR-HUD is a type of HUD configured to enhance the occupant's view of the road ahead by overlaying text, graphics, and / or images onto physical objects in the environment surrounding the vehicle 12 within the occupant's field of vision.

[0050] In one exemplary embodiment, the occupant can interact with the infotainment system 18 using a human-interface device (HID) comprising, for example, a touchscreen, an electromechanical switch, a capacitive switch, a rotary knob, and the like. It is understood that additional systems for displaying information to the occupant of the vehicle 12 are also within the scope of this disclosure.

[0051] With reference to Fig. Figure 2 shows a schematic representation of the interior of the vehicle 12, which includes the infotainment system 18. In an exemplary embodiment, the infotainment system 18 comprises a touchscreen that is arranged essentially centrally within a passenger compartment of the vehicle 12, for example within a center console / dashboard of the vehicle 12, as shown in Figure 2. Fig. 2 shown. In a non-restrictive example, one or more edges of the infotainment system 18 are defined as edges of the touchscreen, comprising, for example, a left edge 40a, a top edge 40b, a right edge 40c, and a bottom edge 40d. In a non-restrictive example, in a left-hand drive vehicle, the left edge 40a is closest to a driver of the vehicle 12. In a right-hand drive vehicle, the right edge 40c is closest to a passenger of the vehicle 12. It is understood that the shape, size, position, and configuration of the Fig. The infotainment systems 18 shown are merely exemplary and that larger infotainment systems 18 are also fully within the scope of this disclosure.

[0052] With reference to Fig. Figure 3 shows a first exemplary user interface (UI) 50a for the infotainment system 18. The first exemplary UI 50a comprises a variety of UI elements 52 within a first content area 54a and a second content area 54b. The variety of UI elements 52 are shapes, text, images, and / or the like, intended for interaction with the vehicle occupant 12 (for example, by selecting, clicking, touching, tapping, dragging, and / or the like). The first content area 54a is used to display one or more currently running applications and / or a start screen. Fig. 3 The home screen is displayed. In a non-restrictive example, the multitude of UI elements 52 comprises a multitude of application icons 56a, which are displayed on the home screen. The multitude of application icons 56a is configured to launch various applications, such as a weather information application, a mobile phone application, a navigation application, a settings application, and / or the like. When the vehicle occupant 12 interacts with one or more of the multitude of application icons 56a, an application can be opened and replace the home screen in the first content area 54a.

[0053] The second content area 54b is used to provide quick access to launch important applications and to provide the vehicle occupant 12 with relevant information. In one exemplary embodiment, the second content area 54b is always displayed on the screen, regardless of which application is running or displayed in the first content area 54a. In a non-restrictive example, the plurality of UI elements 52 further includes a plurality of control bar icons 56b that are displayed in the second content area 54b. In a non-restrictive example, the plurality of control bar icons 56b includes a music application icon 58a, a telephone application icon 58b, a navigation application icon 58c, a settings application icon 58d, and a signal strength indicator icon 58e.

[0054] In one exemplary embodiment, each of the plurality of UI elements 52 has one or more UI element properties. In a non-restrictive example, the one or more UI element properties include a UI element type (for example, button, slider, text, image, and so on), a UI element purpose (for example, an application type with which the UI element starts, such as a weather purpose, a phone purpose, a navigation purpose, and so on), and / or the like. The UI element purpose may further include information about an inherent importance of each of the plurality of UI elements 52. For example, a UI element for a navigation application is inherently more important to the driving task than a UI element for a music application.In a non-restrictive example, the one or more UI element properties further include dynamic and / or statistical information about each of the multitude of UI elements 52, for example, a frequency of use or selection by the occupant.

[0055] Furthermore, each of the plurality of UI elements 52 has a layout. In a non-restrictive example, the layout includes a size, a shape, a rotation angle, a color, a position within the first content area 54a and / or the second content area 54b, and / or the like for each of the plurality of UI elements 52. The determination of the layout for each of the plurality of UI elements 52 is discussed in more detail below with reference to Procedure 100. It is understood that the above discussed and in Fig. 3. The first exemplary user interface (UI) shown in 50a is merely exemplary and additional and / or alternative designs, layouts, and functionality are within the scope of this disclosure. Furthermore, the Fig. The first exemplary UI shown in Figure 3 (50a) is not necessarily to scale, except that it relates to the relative size of the multitude of UI elements (52) between those discussed below. Fig. 3 and Fig. 5 refers to.

[0056] With reference to Fig. Figure 4 shows a flowchart of procedure 100 for providing information to a vehicle occupant. Procedure 100 begins at block 102 and proceeds to block 104. At block 104, the controller 14 determines one or more inputs. In a non-restrictive example, the one or more inputs are determined using the one or more vehicle sensors 16.In an exemplary embodiment, the one or more inputs include measurements of vehicle operating conditions, such as, for example, the speed of the vehicle 12 (determined, for example, using the vehicle speed sensor 24), the acceleration of the vehicle 12 (determined, for example, using the IMU 26), the vibration of the vehicle 12 (determined, for example, using the IMU 26), the activation status of the traction control system of the vehicle 12 (that is, whether the traction control system is acting to prevent skidding, indicating slippery road conditions), a time-averaged frequency of accelerator / brake pedal actuations, and / or the like.

[0057] In an exemplary embodiment, the one or more inputs further include information about external environmental conditions, such as, for example, a weather condition in the environment 28 surrounding the vehicle 12 (determined, for example, using information received by the vehicle communication system 34 and / or the camera system 30), a traffic dust condition and / or traffic density in the environment 28 surrounding the vehicle 12 (determined, for example, using information received by the vehicle communication system 34 and / or the camera system 30), visibility in the environment 28 surrounding the vehicle 12 (determined, for example, using the camera system 30), a distance between the vehicle 12 and the nearest distant vehicle (determined, for example, using the LiDAR sensor 32 and / or the camera system 30), and / or the like.

[0058] In an exemplary embodiment, the one or more inputs further include information about internal environmental conditions, such as, for example, a number of occupants in the vehicle 12 (determined, for example, using the one or more occupant sensors 36), a type of occupant in the vehicle 12 (determined, for example, using the cabin camera), and a noise level inside the vehicle 12 (determined, for example, using the cabin microphone), and / or the like.

[0059] In an exemplary embodiment, the one or more inputs further include information about the position of the occupant within the vehicle 12, such as, for example, the position of the occupant relative to the infotainment system 18, the hand position of the occupant relative to the infotainment system 18, a distance between the hand position and each of the plurality of UI elements 52, the gaze position of the occupant relative to the infotainment system 18, a distance between the gaze position and each of the plurality of UI elements 52 and / or the like.

[0060] It is understood that the one or more inputs discussed above are merely examples and that various additional and / or alternative inputs may be determined in Block 104. After Block 104, Procedure 100 proceeds to Block 106.

[0061] At block 106, the controller 14 determines an importance rating for each of the plurality of UI elements 52. Within the scope of this disclosure, the importance rating indicates the importance of each of the plurality of UI elements 52 for the occupant of the vehicle 12. In an exemplary embodiment, the controller 14 determines the importance rating for each of the plurality of UI elements 52 based at least partially on the one or more inputs determined at block 104 and the one or more UI element properties of each of the plurality of UI elements 52. In a non-limiting example, the importance rating for each of the plurality of UI elements 52 is determined using a machine learning model trained to receive the one or more inputs and the one or more UI element properties of each of the plurality of UI elements 52 and to provide an importance rating as output.

[0062] In another non-restrictive example, the importance rating of each of the set of UI elements 52 is determined using a deterministic algorithm. For example, the importance rating is determined as a normalized weighted average of the one or more inputs determined at block 104, where the weights corresponding to each of the one or more inputs are determined based on the one or more UI element properties (for example, inherent importance). For example, the importance rating of each of the set of UI elements 52 is positively correlated with the inherent importance of each of the set of UI elements 52.

[0063] In another example, the importance rating of each of the 52 UI elements is positively correlated with the speed of vehicle 12. In another example, the importance rating of each of the 52 UI elements is positively correlated with the traffic density in the environment 28 surrounding vehicle 12. In another example, the importance rating of each of the 52 UI elements is negatively correlated with the distance between vehicle 12 and the nearest distant vehicle. In yet another example, the importance rating of each of the 52 UI elements is increased by a predetermined amount when the traction control system is active.

[0064] In another example, the importance rating of each of the multitude of UI elements 52 is positively correlated with the time-averaged frequency of accelerator / brake pedal actuations in the vehicle 12. In yet another example, the importance rating of each of the multitude of UI elements 52 is negatively correlated with the distance between the occupant's hand position and each of the multitude of UI elements 52. In other words, UI elements 52 located near the occupant's hand position have a higher importance rating. Therefore, the importance rating of each of the multitude of UI elements 52 will change dynamically as the occupant moves their hand in the vehicle 12 near the infotainment system 18 (for example, in preparation for selecting one of the UI elements 52).

[0065] In another example, the importance rating of each of the multitude of UI elements 52 is negatively correlated with the distance between the gaze position and each of the multitude of UI elements 52. In other words, UI elements 52 located near the occupant's gaze position have a higher importance rating. Therefore, the importance rating of each of the multitude of UI elements 52 will dynamically change as the occupant moves their gaze in the vehicle 12 near the infotainment system 18 (for example, in preparation for selecting one of the UI elements 52).

[0066] It is understood that the above-discussed methods for determining the importance rating are merely exemplary and that any machine learning-based or non-machine learning-based methods (for example, mathematical functions, lookup tables, and / or the like) may be used without deviating from the scope of this disclosure. Following Block 106, Method 100 transitions to Block 108.

[0067] In block 108, the controller 14 determines the layout of each of the plurality of UI elements 52. In an exemplary embodiment, the controller 14 determines the layout of each of the plurality of UI elements 52 based at least partially on the importance rating of each of the one or more UI elements determined in block 106. In an exemplary embodiment, the layout of each of the plurality of UI elements 52 is determined using a rule-based algorithm. For example, if the importance rating of a UI element is greater than a first predetermined threshold, the size of the UI element is increased by a predetermined amount (for example, ten percent) larger than a standard size of the UI element. In another example, if the importance rating of a UI element is greater than a second predetermined threshold, the position of the UI element is moved to a predetermined area of ​​the infotainment system 18.

[0068] In another example, if the importance rating of a UI element is less than a third predefined threshold, the UI element is not displayed on the infotainment system 18. In another example, if the importance rating of a UI element is greater than a fourth predefined threshold, the color of the UI element is set to a predefined color indicating high importance (for example, red). In yet another example, if the importance rating of a UI element is less than a fifth predefined threshold, the color of the UI element is set to a predefined color indicating low importance (for example, gray).

[0069] In another example, the size of each of the multitude of UI elements 52 is correlated with the importance rating of each of the multitude of UI elements 52. In a non-restrictive example, the size of each of the multitude of UI elements 52 is positively correlated with the importance rating of each of the multitude of UI elements 52 (for example, proportionally). In another example, the position of each of the multitude of UI elements 52 is determined based on the importance rating of each of the multitude of UI elements 52. In a non-restrictive example, a distance between each of the multitude of UI elements 52 and an edge of the infotainment system 18 (for example, the left edge 40a, the top edge 40b, the right edge 40c, and / or the bottom edge 40d) is negatively correlated with the importance rating of each of the multitude of UI elements 52.For example, UI elements with a higher importance rating are located closer to an edge that is closest to the occupant (for example, the left edge 40a in a left-hand drive vehicle or the right edge 40c in a right-hand drive vehicle).

[0070] In another non-restrictive example, the distance between each of the multitude of UI elements 52 and the occupant's hand position is negatively correlated with the importance rating of each of the multitude of UI elements 52. In other words, UI elements 52 with a higher importance rating are positioned closer to the occupant's hand position. In another non-restrictive example, the distance between each of the multitude of UI elements 52 and the occupant's gaze position is negatively correlated with the importance rating of each of the multitude of UI elements 52. In other words, UI elements 52 with a higher importance rating are positioned closer to the occupant's gaze position.

[0071] In another non-restrictive example, based on comparing the importance rating of each of the multitude of UI elements 52 with one or more thresholds, it is determined that the position of one or more of the multitude of UI elements 52 is located within one or more predefined areas of the infotainment system 18. For example, the one or more predefined areas may include those that are easily accessible to the occupant (for example, areas near the occupant's seating position and / or areas within the occupant's unobstructed line of sight). The one or more predefined areas may be determined based on the specific shape, size, and position of the infotainment system 18.

[0072] In another exemplary embodiment, the layout of each of the plurality of UI elements 52 is determined using a machine learning algorithm that is trained to determine a layout for each of the plurality of UI elements 52 based on the importance rating of each of the plurality of UI elements 52.

[0073] In a non-restrictive example, the machine learning algorithm comprises several layers, including an input layer and an output layer, as well as one or more hidden layers. The input layer receives the importance rating for each of the 52 UI elements as input. The inputs are then passed to the hidden layers. Each hidden layer applies a transformation (for example, a nonlinear transformation) to the data and passes the result to the next hidden layer, up to the final hidden layer. The output layer generates the layout (for example, size, position, color, and so on) for each of the 52 UI elements on the infotainment system.

[0074] To train the machine learning algorithm, a dataset of inputs and the corresponding layout (for example, size, position, color, and so on) is used for each of the numerous UI elements. The dataset can be refined manually, crowdsourced from data on how occupants prefer to use the infotainment system, and / or using reinforcement learning based on feedback from test subjects. The algorithm is trained by adjusting internal weights between nodes in each hidden layer to minimize the prediction error. During training, an optimization technique (for example, gradient descent) is used to adjust the internal weights to reduce the prediction error. The training process is repeated with the entire dataset until the prediction error is minimized, and the resulting trained model is then used to process new input data.

[0075] After sufficient training of the machine learning algorithm, the algorithm is able to determine an optimal layout (e.g., size, position, color, etc.) for each of the 52 UI elements based on the importance rating for each of the 52 UI elements. By adjusting the weights between the nodes in each hidden layer during training, the algorithm "learns" to recognize patterns in the data that indicate the optimal layout (e.g., size, position, color, etc.) for each of the 52 UI elements.

[0076] In some embodiments, the machine learning algorithm can be adapted and further trained on-site based on feedback from the occupant and / or data collected on how the occupant uses the infotainment system 18.

[0077] It is understood that the above-provided examples of rule-based and machine learning algorithms are merely illustrative and that various additional and / or alternative algorithms, functions and / or procedures for determining the layout for each of the multitude of UI elements 52 are within the scope of this disclosure.

[0078] With reference to Fig. Figure 5 shows a second exemplary user interface (UI) 50b for the infotainment system 18. With reference to Fig. Figures 3-5 illustrate the second exemplary UI 50b, illustrating exemplary layout changes to the multitude of UI elements 52 compared to that in Fig. 3. First exemplary UI 50a shown. In an exemplary embodiment, as shown in the second exemplary UI 50b compared to the first exemplary UI 50a, some of the plurality of application symbols 56a are not shown in the first content area 54a (for example, because the importance rating of some of the plurality of application symbols 56a is less than the third predetermined threshold), and the remaining plurality of application symbols 56a are moved towards the left edge 40a and / or resized. In a non-restrictive example, the remaining plurality of application symbols 56a are moved towards the left edge 40a to make it easier for a driver of a left-hand drive vehicle to reach the remaining plurality of application symbols 56a.It is understood that in a right-hand drive vehicle, the remaining of the multitude of application symbols 56a can be moved towards the right edge 40c.

[0079] In an exemplary embodiment, as shown in the second exemplary UI 50b compared to the first exemplary UI 50a, some of the plurality of control bar icons 56b are not shown in the second content area 54b (for example, because the importance rating of some of the plurality of control bar icons 56b is less than the third predetermined threshold), and the remaining plurality of control bar icons 56b are rearranged so that icons with a higher importance rating (for example, due to higher inherent importance) are closer to the left edge 40a. For example, the navigation application icon 58c is placed closest to the left edge 40a, followed by the phone application icon 58b, and then the music application icon 58a.Furthermore, as shown in the second exemplary UI 50b compared to the first exemplary UI 50a, the multitude of control bar symbols 56b are enlarged in size (for example, because the importance rating of each of the multitude of control bar symbols 56b is greater than the first predetermined threshold).

[0080] It goes without saying that the above-discussed and in Fig. The second exemplary user interface (UI) shown in 50b is merely exemplary and additional and / or alternative designs, layouts and functionality are within the scope of this disclosure. Furthermore, the Fig. The second exemplary UI shown in 50b is not necessarily to scale, except that it relates to the relative size of the multitude of UI elements 52 between the Fig. 3 and Fig. 5 refers. With renewed reference to Fig. 4. Procedure 100 proceeds from block 108 to block 110.

[0081] With reference to Fig. Figure 6 shows a third exemplary user interface (UI) 50c for the infotainment system 18 with an exemplary occupant hand 60. With reference to Fig. 3-4 and Fig. Figure 6 illustrates the third exemplary UI 50c, showing exemplary layout changes to the multitude of UI elements 52 compared to that in Fig. The first exemplary UI 50a is shown in Figure 3. In an exemplary embodiment, as shown in the third exemplary UI 50c in comparison to the first exemplary UI 50a, the size of the plurality of application symbols 56a is changed based on their proximity to the exemplary inmate hand 60. For example, some of the plurality of application symbols 56a near the exemplary inmate hand 60 are enlarged, and some of the plurality of application symbols 56a farther away from the exemplary inmate hand 60 are reduced in size (for example, due to the negative correlation between importance rating and distance to hand position). It is understood that the layout shown in the third exemplary UI 50c is merely exemplary and not necessarily to scale.Furthermore, the foregoing description is also applicable to embodiments that use the viewing position, without deviating from the scope of the present disclosure.

[0082] It goes without saying that the above-discussed and in Fig. 6. The third exemplary user interface (UI) shown in 50c is merely exemplary and additional and / or alternative designs, layouts, and functionality are within the scope of this disclosure. Furthermore, the Fig. The third exemplary UI 50c shown in Figure 6 is not necessarily to scale, except that it relates to the relative size of the multitude of UI elements 52 between the Fig. 3 and Fig. 6 refers. With renewed reference to Fig. 4. Procedure 100 proceeds from block 108 to block 110.

[0083] At Block 110, the controller 14 uses the infotainment system 18 to display the multitude of UI elements 52 to provide information to the vehicle occupant 12. In one exemplary embodiment, the controller 14 displays the multitude of UI elements 52 based at least partially on the layout of each of the multitude of UI elements 52 determined at Block 108 (that is, size, position, color, and so on). In a non-restrictive example, adjustments to the layout of each of the multitude of UI elements 52 are displayed smoothly (for example, using animations or the like) to help the occupant understand and adapt to changes in the layout of each of the multitude of UI elements 52. In another non-restrictive example, the controller 14 uses hysteresis rules or algorithms to limit the frequency and / or speed of changes to the layout of each of the multitude of UI elements 52.After block 110, procedure 100 transitions to block 112 to enter a standby state.

[0084] In one exemplary embodiment, the controller 14 repeatedly exits the standby state 112 and restarts the procedure 100 at block 102. In a non-restrictive example, the controller 14 exits the standby state 112 and restarts the procedure 100, for example, every three hundred milliseconds on a timer. Therefore, the layout of the multitude of UI elements 52 is dynamically adjusted and / or updated based on one or more inputs, including, for example, the occupant's position.

[0085] System 10 and Method 100 of this disclosure offer several advantages. By adapting the layout of UI elements based on the occupant's position, vehicle state characteristics, and / or external environment characteristics, the user interface of the infotainment system 18 adapts to current conditions to enhance occupant comfort. For example, by increasing the size of UI elements near the occupant's hand or viewing position, occupants can more easily select UI elements and control the infotainment system 18. Furthermore, the use of System 10 and Method 100 enables dynamic adaptation of the UI layout based on the occupant's position.

Claims

[1] System (10) for providing information to an occupant of a vehicle (12), the system (10) comprising: one or more occupant sensors (36); an infotainment system (18); and a control unit (14) in electrical communication with the one or more occupant sensors (36) and the infotainment system (18), wherein the control unit (14) is programmed to: Determine (104) one or more inputs using at least one or more occupant sensors (36); Determine (108) a layout of each of a multitude of user interface, UI, elements (52) of the infotainment system (18) based at least partially on one or more inputs; and Providing (110) information to the occupant of the vehicle (12) by displaying (110) the multitude of UI elements (52) based at least partially on the layout of each of the multitude of UI elements (52); wherein, to determine (108) the layout of each of the multitude of UI elements (52), the control (14) is further programmed to: Determine (106) an importance rating of each of the multitude of UI elements (52) based at least partially on the one or more inputs and one or more UI element properties of each of the multitude of UI elements (52); and Determining (108) the layout of each of the multitude of UI elements (52) based at least partially on the importance rating of each of the multitude of UI elements (52); wherein the layout includes at least one position of each of the plurality of UI elements (52) on the infotainment system (18); and where a distance between the position of each of the multitude of UI elements (52) and a hand position of the occupant or a gaze position of the occupant is negatively correlated with the importance rating of each of the multitude of UI elements (52). [2] System (10) according to claim 1, wherein the one or more occupant sensors (36) further comprise an occupant position tracking device, and wherein, for determining (104) the one or more inputs, the controller (14) is further programmed to: Determine (104) the position of the occupant relative to the infotainment system (18) using the occupant position tracking device. [3] System (10) according to claim 2, wherein, for determining (106) the importance rating of each of the plurality of UI elements (52), the controller (14) is further programmed to: Determine (106) the importance rating of each of the multitude of UI elements (52) based at least partially on the one or more inputs, wherein the one or more inputs comprise the position of the occupant relative to the infotainment system (18); and where the importance rating of each of the multitude of UI elements (52) is negatively correlated with a distance between a hand position of the occupant and each of the multitude of UI elements (52). [4] System (10) according to claim 1, wherein the one or more occupant sensors (36) further comprise an occupant position tracking device; and wherein, to determine (104) one or more inputs, the control (14) is further programmed to: Determine (104) an occupant's viewing position relative to the infotainment system (18) using the occupant position tracking device. [5] System (10) according to claim 4, wherein, for determining (106) the importance rating of each of the plurality of UI elements (52), the control (14) is further programmed to: Determine (106) the importance rating of each of the multitude of UI elements (52) based at least partially on the one or more inputs; wherein the one or more inputs comprise the occupant's gaze position relative to the infotainment system (18), and wherein the importance rating of each of the multitude of UI elements (52) is negatively correlated with a distance between the gaze position and each of the multitude of UI elements (52). [6] System (10) according to claim 1, wherein the controller (14) is further programmed to determine (104) one or more inputs: Determining (104) measurements of vehicle operating conditions, wherein the vehicle operating conditions include one or more of the following: a speed of the vehicle (12), an acceleration of the vehicle (12) and a vibration of the vehicle (12). [7] System (10) according to claim 1, wherein the controller (14) is further programmed to determine (104) one or more inputs: Determining (104) information about external environmental conditions, wherein the external environmental conditions comprise one or more of the following: a weather condition in an environment (28) surrounding the vehicle (12), and a traffic dust condition in the environment (28) surrounding the vehicle (12); and Determining (104) information about internal environmental conditions, wherein the internal environmental conditions include one or more of the following: a quantity of occupants in the vehicle (12), a type of occupants in the vehicle (12) and a noise level inside the vehicle (12). [8] System (10) according to claim 1, wherein the control (14) is further programmed to determine (108) the layout of each of the plurality of UI elements (52): Determining (108) the layout of each of the plurality of UI elements (52) based at least partially on the importance rating of each of the plurality of UI elements (52), wherein the layout includes at least one size of each of the plurality of UI elements (52), wherein the size of each of the plurality of UI elements (52) is positively correlated with the importance rating of each of the plurality of UI elements (52).

Images