Adaptive user interface based on vehicle occupant position

Abstract

A system for providing information to occupants of a vehicle may include one or more occupant sensors, an infotainment system, and a controller electrically communicating with the one or more occupant sensors and the infotainment system. The controller is programmed to determine one or more inputs using at least one or more occupant sensors. The controller is also programmed to determine the layout of each of a plurality of UI elements of the infotainment system, at least in part, based on the layout of each of the plurality of UI elements, thereby providing information to the occupants of the vehicle by displaying the plurality of UI elements.

Description

Technical Field

[0001] This disclosure relates to systems and methods for providing information to occupants of a vehicle, and more specifically, to systems and methods for the design and functionality of a user interface for a vehicle infotainment system. Background Technology

[0002] To facilitate user interaction with the infotainment system in a vehicle application, various user interface technologies can be utilized. Vehicle infotainment systems may include displays, touch-sensitive controls, physical buttons, voice recognition modules, and / or similar devices to enable interaction with navigation, media playback, climate control, communication systems, and other vehicle functions. In some examples, the user interface may include a configurable layout or menu structure that can be tailored to user preferences. Additionally, the infotainment system interface can be integrated with external devices via wireless communication protocols, allowing users to access mobile applications, cloud services, or real-time traffic and weather updates. Some vehicle infotainment system interfaces also offer multimodal interaction capabilities, such as combining voice commands with haptic feedback or gesture recognition to improve usability while minimizing driver distraction.

[0003] While the systems and methods used for vehicle infotainment user interfaces have achieved their intended purpose, new and improved systems and methods are still needed for the functionality and design of vehicle infotainment system user interfaces to provide information to vehicle occupants. Summary of the Invention

[0004] According to several aspects, a system for providing information to occupants of a vehicle is provided. The system may include one or more occupant sensors, an infotainment system, and a controller electrically communicating with the one or more occupant sensors and the infotainment system. The controller is programmed to determine one or more inputs using at least one or more occupant sensors. The controller is also programmed to determine the layout of each of a plurality of user interface (UI) elements of the infotainment system, at least in part, based on the one or more inputs. The controller is further programmed to provide information to the occupants of the vehicle by displaying these plurality of UI elements, at least in part, based on the layout of each of the plurality of UI elements.

[0005] In another aspect of this disclosure, in order to determine the layout of each of a plurality of UI elements, the controller is also programmed to determine an importance score for each of the plurality of UI elements based at least in part on one or more inputs and one or more UI element features of each of the plurality of UI elements. To determine the layout of each of the plurality of UI elements, the controller is also programmed to determine the layout of each of the plurality of UI elements based at least in part on the importance score of each of the plurality of UI elements.

[0006] In another aspect of this disclosure, one or more occupant sensors may further include an occupant position tracking device. To determine one or more inputs, the controller is also programmed to use the occupant position tracking device to determine the occupant's position relative to the infotainment system.

[0007] In another aspect of this disclosure, to determine the importance score of each of the plurality of UI elements, the controller is also programmed to determine the importance score of each of the plurality of UI elements based at least in part on one or more inputs. The one or more inputs include the occupant's position relative to the infotainment system. The importance score of each of the plurality of UI elements is negatively correlated with the occupant's hand position and the distance between each of the plurality of UI elements.

[0008] In another aspect of this disclosure, one or more occupant sensors may also include an occupant position tracking device. To determine one or more inputs, the controller is also programmed to use the occupant position tracking device to determine the occupant's gaze position relative to the infotainment system.

[0009] In another aspect of this disclosure, to determine the importance score of each of the plurality of UI elements, the controller is also programmed to determine the importance score of each of the plurality of UI elements based at least in part on one or more inputs. The one or more inputs include the occupant's gaze position relative to the infotainment system. The importance score of each of the plurality of UI elements is negatively correlated with the gaze position and the distance between each of the plurality of UI elements.

[0010] In another aspect of this disclosure, in order to determine one or more inputs, the controller is also programmed to determine measurements of vehicle operating conditions. Vehicle operating conditions include one or more of the following: vehicle speed, vehicle acceleration, and vehicle vibration.

[0011] In another aspect of this disclosure, in order to determine one or more inputs, the controller is also programmed to determine information about external environmental conditions. External environmental conditions include one or more of the following: weather conditions in the environment surrounding the vehicle and traffic congestion conditions in the environment surrounding the vehicle. In order to determine one or more inputs, the controller is also programmed to determine information about internal environmental conditions. Internal environmental conditions include one or more of the following: the number of occupants in the vehicle, the type of occupants in the vehicle, and the sound level inside the vehicle.

[0012] In another aspect of this disclosure, to determine the layout of each of the plurality of UI elements, the controller is also programmed to determine the layout of each of the plurality of UI elements at least in part based on the importance score of each of the plurality of UI elements. The layout includes at least the size of each of the plurality of UI elements. The size of each of the plurality of UI elements is positively correlated with the importance score of each of the plurality of UI elements.

[0013] In another aspect of this disclosure, to determine the layout of each of the plurality of UI elements, the controller is also programmed to determine the layout of each of the plurality of UI elements at least in part based on the importance score of each of the plurality of UI elements. The layout includes at least the position of each of the plurality of UI elements on the infotainment system. The distance between the position of each of the plurality of UI elements and the position of the occupant's hand or gaze position is negatively correlated with the importance score of each of the plurality of UI elements.

[0014] According to several aspects, a method for providing information to occupants of a vehicle is provided. The method may include determining one or more inputs using one or more occupant sensors. The method may also include determining the layout of each of a plurality of user interface (UI) elements of an infotainment system, at least in part based on the one or more inputs. The method may further include providing information to vehicle occupants by displaying the plurality of UI elements, at least in part based on the layout of each of the plurality of UI elements.

[0015] In another aspect of this disclosure, determining the layout of each of the plurality of UI elements may further include: determining an importance score for each of the plurality of UI elements based at least in part on one or more inputs and one or more UI element features of each of the plurality of UI elements. Determining the layout of each of the plurality of UI elements may further include determining the layout of each of the plurality of UI elements based at least in part on the importance score of each of the plurality of UI elements.

[0016] In another aspect of this disclosure, determining one or more inputs may also include using an occupant position tracking device in one or more occupant sensors to determine the distance between the occupant's hand position and each of a plurality of UI elements of the infotainment system.

[0017] In another aspect of this disclosure, determining the importance score of each of the plurality of UI elements may further include determining the importance score of each of the plurality of UI elements based at least in part on one or more inputs. The importance score of each of the plurality of UI elements is negatively correlated with the occupant's hand position and the distance between each of the plurality of UI elements.

[0018] In another aspect of this disclosure, determining one or more inputs may also include using an occupant position tracking device with one or more occupant sensors to determine the occupant's gaze position relative to the infotainment system.

[0019] In another aspect of this disclosure, determining the importance score of each of the plurality of UI elements may further include determining the importance score of each of the plurality of UI elements based at least in part on one or more inputs. The one or more inputs include the occupant's gaze position. The importance score of each of the plurality of UI elements is negatively correlated with the gaze position and the distance between each of the plurality of UI elements.

[0020] In another aspect of this disclosure, determining the layout of each of the plurality of UI elements may further include using a trained machine learning algorithm to determine the layout of each of the plurality of UI elements, the machine learning algorithm determining the layout at least in part based on the importance score of each of the plurality of UI elements.

[0021] According to several aspects, a system for providing information to occupants of a vehicle is provided. The system may include an occupant position tracking device, an infotainment system, and a controller electrically communicating with the occupant position tracking device and the infotainment system. The controller is programmed to use the occupant position tracking device to determine the position of an occupant relative to the infotainment system. The controller is also programmed to determine an importance score for each of a plurality of user interface (UI) elements of the infotainment system, at least in part based on the occupant's position. The importance score for each of the plurality of UI elements is negatively correlated with the occupant's hand position and the distance between each of the plurality of UI elements. The controller is also programmed to determine the layout of each of the plurality of UI elements, at least in part based on the importance score of each of the plurality of UI elements. The controller is further programmed to provide information to the vehicle occupants by displaying these plurality of UI elements, at least in part based on the layout of each of the plurality of UI elements.

[0022] In another aspect of this disclosure, to determine the importance score of each of the plurality of UI elements, the controller is also programmed to determine the importance score of each of the plurality of UI elements at least in part based on the inherent importance of each of the plurality of UI elements. To determine the importance score of each of the plurality of UI elements, the controller is also programmed to determine the importance score of each of the plurality of UI elements at least in part based on measurements of vehicle operating conditions. Vehicle operating conditions include one or more of the following: vehicle speed, vehicle acceleration, and vehicle vibration.

[0023] In another aspect of this disclosure, to determine the layout of each of the plurality of UI elements, the controller is also programmed to determine the layout of each of the plurality of UI elements at least in part based on the importance score of each of the plurality of UI elements. The layout includes at least the position of each of the plurality of UI elements on the infotainment system. The distance between the position of each of the plurality of UI elements and the position of the occupant's hands is negatively correlated with the importance score of each of the plurality of UI elements. The size of each of the plurality of UI elements is positively correlated with the importance score of each of the plurality of UI elements.

[0024] Further applicability will become apparent from the description provided herein. It should be understood that the descriptions and specific examples are for illustrative purposes only and are not intended to limit the scope of this disclosure. Attached Figure Description

[0025] The accompanying drawings described herein are for illustrative purposes only and are not intended to limit the scope of this disclosure in any way.

[0026] Figure 1 This is a schematic diagram of a system for providing information to occupants of a vehicle, according to an exemplary embodiment.

[0027] Figure 2 This is a schematic diagram of the interior of a vehicle according to an exemplary embodiment;

[0028] Figure 3 This is a schematic diagram of a first exemplary user interface for an infotainment system according to an exemplary embodiment;

[0029] Figure 4 This is a flowchart of a method for providing information to occupants of a vehicle according to an exemplary embodiment;

[0030] Figure 5 This is a schematic diagram of a second exemplary user interface for an infotainment system according to an exemplary embodiment; and

[0031] Figure 6 This is a schematic diagram of a third exemplary user interface for an infotainment system according to an exemplary embodiment. Detailed Implementation

[0032] The following description is merely illustrative in nature and is not intended to limit the content, application, or purpose of this disclosure.

[0033] In various aspects of this disclosure, it is advantageous to adapt / adjust the layout and / or functionality of the user interface of a vehicle infotainment system based on vehicle conditions, environmental conditions, and / or occupant behavior. Therefore, this disclosure provides a novel and improved system and method for providing information to vehicle occupants by taking into account current conditions and occupant behavior when generating the user interface.

[0034] refer to Figure 1 The figure illustrates a system for providing information to the occupants of a vehicle, and the system is generally indicated by reference numeral 10. System 10 is shown together with an exemplary vehicle 12. Although a passenger car is shown in the figure, it should be understood that vehicle 12 can be any type of vehicle without departing from the scope of this disclosure. System 10 generally includes a controller 14, one or more vehicle sensors 16, and an infotainment system 18.

[0035] As described below, controller 14 is used to implement a method 100 for providing information to the occupants of a vehicle. Controller 14 includes at least one processor 20 and a non-transitory computer-readable storage device or medium 22. Processor 20 may be a custom or commercially available processor, a central processing unit (CPU), a graphics processing unit (GPU), an auxiliary processor among a plurality of processors associated with controller 14, a semiconductor-based microprocessor (in the form of a microchip or chipset), a macroprocessor, a combination thereof, or a device typically used for executing instructions.

[0036] For example, the computer-readable storage device or medium 22 may include volatile and non-volatile storage devices in the form of read-only memory (ROM), random access memory (RAM), and keep-alive memory (KAM). KAM is a persistent or non-volatile memory that can be used to store various operational variables when the processor 20 is powered off. The computer-readable storage device or medium 22 may be implemented using a variety of storage devices, such as PROM (programmable read-only memory), EPROM (electrically programmable read-only memory), EEPROM (electrically erasable programmable read-only memory), flash memory, or another electrical, magnetic, optical, or combined storage device capable of storing data, some of which represents executable instructions used by the controller 14 to control various systems of the vehicle 12.

[0037] Controller 14 may also include multiple controllers that communicate electrically with each other. Controller 14 may interconnect with other systems and / or controllers of vehicle 12, thereby allowing controller 14 to acquire data such as the speed, acceleration, braking, and steering angle of vehicle 12.

[0038] The controller 14 communicates electrically with one or more vehicle sensors 16 and the infotainment system 18. In exemplary embodiments, electrical communication is established using, for example, a CAN network, a FLEXRAY network, a local area network (e.g., WiFi, Ethernet, etc.), a Serial Peripheral Interface (SPI) network, etc. It should be understood that various other wired and wireless technologies and communication protocols used for communicating with the controller 14 are within the scope of this disclosure. It should also be understood that, within the scope of this disclosure, electrical communication also includes power and / or energy transfer between electrical devices (e.g., using conductive wires and / or wireless power transfer technologies).

[0039] One or more vehicle sensors 16 are used to acquire information related to the vehicle 12. In an exemplary embodiment, the one or more vehicle sensors 16 include sensors for determining performance data and / or operating conditions of the vehicle 12. In a non-limiting example, the one or more vehicle sensors 16 include at least one of a motor speed sensor, a motor torque sensor, an electric drive motor voltage and / or current sensor, an accelerator pedal position sensor, a brake position sensor, a coolant temperature sensor, a cooling fan speed sensor, and a transmission oil temperature sensor.

[0040] In an exemplary embodiment, one or more vehicle sensors 16 further include a vehicle speed sensor 24. The vehicle speed sensor 24 is used to measure the speed of the vehicle 12. In an exemplary embodiment, the vehicle speed sensor 24 includes a toothed ferromagnetic ring or disk (also referred to as a magnetic reluctance wheel or toothed ring) mechanically linked to a wheel of the vehicle 12, such that the toothed ferromagnetic ring rotates at the same speed as the wheel of the vehicle 12. The vehicle speed sensor 24 also includes a sensor (e.g., a wire coil) attached near the toothed ferromagnetic ring and stationary relative to the toothed ferromagnetic ring, such that the sensor experiences an induced alternating voltage due to changes in the magnetic field caused by the rotation of the toothed ferromagnetic ring.

[0041] In a non-limiting example, the induced alternating voltage is a sine wave, and the amplitude and / or frequency of the induced alternating voltage is proportional to the rotational speed of the toothed ferromagnetic ring, and therefore proportional to the vehicle speed. In an exemplary embodiment, the induced alternating voltage is directly transmitted to the controller 14 for further processing, such as using an analog-to-digital converter of the controller 14. In another exemplary embodiment, the vehicle speed sensor 24 also includes a signal conditioning circuitry system that further processes the induced alternating voltage (e.g., amplifies the induced alternating voltage, digitizes the induced alternating voltage, encodes the vehicle speed value into a communication protocol, and / or similar operations) before transmitting the induced alternating voltage to the controller 14.

[0042] It should be understood that various additional devices and methods for determining vehicle speed (e.g., global navigation satellite systems (e.g., GPS), sensors for measuring other drivetrain components (e.g., transmission speed sensors, electric motor speed sensors, etc.) and / or similar devices) are within the scope of this disclosure. As discussed above, vehicle speed sensor 24 is in electrical communication with controller 14.

[0043] In an exemplary embodiment, one or more vehicle sensors 16 further include an inertial measurement unit (IMU) 26. The IMU 26 is used to determine the direction, velocity, and gravity acting on the vehicle 12. In an exemplary embodiment, the IMU 26 includes multiple sensors, including an accelerometer, a gyroscope, and / or a magnetometer. In a non-limiting example, the IMU 26 includes a three-axis accelerometer and a three-axis gyroscope integrated into a single unit. The accelerometer measures linear acceleration along each axis, while the gyroscope measures angular velocity about each axis. The IMU 26 processes data from the sensors to calculate the vehicle 12's current direction, velocity, heading, yaw rate (i.e., rate of change of heading), and acceleration in three-dimensional space. As discussed above, the IMU 26 is in electrical communication with the controller 14.

[0044] In another exemplary embodiment, one or more vehicle sensors 16 further include sensors for determining information about external environmental conditions in the environment 28 surrounding the vehicle 12. In a non-limiting example, one or more vehicle sensors 16 further include at least one of an ambient air temperature sensor, an atmospheric pressure sensor, and / or a Global Navigation Satellite System (GNSS).

[0045] In an exemplary embodiment, one or more vehicle sensors 16 may also include one or more sensing sensors, such as camera system 30 and / or lidar (LiDAR) sensor 32, capable of sensing objects in the environment 28 surrounding the vehicle 12 and / or measuring distances. It should be understood that various other types of sensing sensors, such as ultrasonic ranging sensors, radar sensors, and / or time-of-flight sensors, are within the scope of this disclosure.

[0046] The camera system 30 is a sensing sensor for capturing images and / or video of the environment 28 surrounding the vehicle 12. In an exemplary embodiment, the camera system 30 includes a camera and / or video camera positioned to observe the environment 28 surrounding the vehicle 12. In a non-limiting example, the camera system 30 includes a camera attached to the interior of the vehicle 12 (e.g., the headliner of the vehicle 12) having a field of view through the windshield. In another non-limiting example, the camera system 30 includes a camera attached to the exterior of the vehicle 12 (e.g., on the roof of the vehicle 12) having a field of view of the environment 28 in front of the vehicle 12.

[0047] In another exemplary embodiment, the camera system 30 is a surround-view camera system comprising a plurality of cameras (also referred to as satellite cameras) arranged to provide a view of the environment 28 adjacent to all sides of the vehicle 12. In a non-limiting example, the camera system 30 includes a front camera (e.g., mounted in the front grille of the vehicle 12), a rear camera (e.g., mounted on the tailgate of the vehicle 12), and two side cameras (e.g., mounted below each of the two side mirrors of the vehicle 12). In another non-limiting example, the camera system 30 also includes an additional rear-view camera mounted near the high-mounted brake light of the vehicle 12.

[0048] It should be understood that camera systems with additional cameras and / or additional mounting locations are within the scope of this disclosure. It should also be understood that cameras with various sensor types (including, for example, charge-coupled device (CCD) sensors, complementary metal-oxide-semiconductor (CMOS) sensors, and / or high dynamic range (HDR) sensors) are all within the scope of this disclosure. Additionally, cameras with various lens types (including, for example, wide-angle lenses and / or narrow-angle lenses) are also within the scope of this disclosure.

[0049] The lidar sensor 32 performs remote sensing and environmental mapping by emitting laser pulses and measuring the time required for the laser pulses to return to the lidar sensor 32 after hitting an object. In an exemplary embodiment, the lidar sensor 32 includes a lidar laser source, a lidar scanner or reflector, a lidar photodetector, and a lidar time-of-flight measurement system. In a non-limiting example, the lidar laser source emits laser pulses that travel to a target area, and the lidar scanner guides these pulses in different directions. The emitted laser pulses interact with objects in the environment, and their reflections are captured by the lidar photodetector. The lidar time-of-flight measurement system calculates the distance to the object based on the time between the lidar laser source emitting the laser pulse and the lidar photodetector receiving the reflected laser pulse. As discussed above, the lidar sensor 32 is in electrical communication with the controller 14.

[0050] In another exemplary embodiment, one or more vehicle sensors 16 further include a vehicle communication system 34 for receiving information from external systems. The controller 14 utilizes the vehicle communication system 34 to communicate with other systems outside the vehicle 12. For example, the vehicle communication system 34 includes the ability to communicate with other vehicles (“V2V” communication), with infrastructure (“V2I” communication), with remote systems at remote call centers (e.g., General Motors’ OnStar), and / or personal devices. Generally, the term “vehicle-to-everything communication” (“V2X” communication) refers to communication between the vehicle 12 and any remote system (e.g., vehicles, infrastructure, and / or remote systems).

[0051] In some embodiments, the vehicle communication system 34 is a wireless communication system configured to communicate using a wireless local area network (WLAN) based on the IEEE 802.11 standard, or using cellular data communication (e.g., using GSMA standards such as SGP.02, SGP.22, SGP.32, etc.). Therefore, the vehicle communication system 34 may also include an embedded universal integrated circuit card (eUICC) configured to store at least one cellular connectivity profile, such as an embedded subscriber identity module (eSIM) profile.

[0052] The vehicle communication system 34 is also configured to communicate via a personal area network (e.g., Bluetooth), near field communication (NFC), and / or any other type of radio frequency communication. However, other or alternative communication methods, such as Dedicated Short Range Communication (DSRC) channels and / or mobile telecommunications protocols based on 3GPP standards, are also considered within the scope of this disclosure. A DSRC channel refers to a unidirectional or bidirectional short- to medium-range wireless communication channel specifically designed for automotive use, along with a corresponding set of protocols and standards. 3GPP refers to collaboration among several standards organizations that develop protocols and standards for mobile communications. 3GPP standards are architected by "release". Therefore, communication methods based on 3GPP releases 14, 15, 16, and / or future 3GPP releases are considered within the scope of this disclosure.

[0053] Therefore, the vehicle communication system 34 may include one or more antennas and / or communication transceivers for receiving and / or transmitting signals such as Cooperative Sensing Messages (CSM) . The vehicle communication system 34 is configured to wirelessly transmit information between vehicle 12 and another vehicle. Furthermore, the vehicle communication system 34 is configured to wirelessly transmit information between vehicle 12 and infrastructure or other vehicles. It should be understood that, without departing from the scope of this disclosure, the vehicle communication system 34 may be integrated with the controller 14 (e.g., integrated on the same circuit board as the controller 14, or otherwise as part of the controller 14).

[0054] In another exemplary embodiment, one or more vehicle sensors 16 further include sensors for determining information about the interior environmental conditions of the vehicle 12. In a non-limiting example, one or more vehicle sensors 16 further include at least one of a seat occupancy sensor, a cabin air temperature sensor, a cabin motion detection sensor, a cabin camera, a cabin microphone, etc.

[0055] In another exemplary embodiment, the one or more vehicle sensors 16 further include one or more occupant sensors 36. In a non-limiting example, the one or more occupant sensors include an occupant position tracking device.

[0056] Occupant position tracking devices are used to determine the positions of occupants in vehicle 12. Within the scope of this disclosure, in a non-limiting example, occupants include the driver, passengers, and / or any other person in vehicle 12. For example, the occupant position tracking device may track the position of the occupant's head, appendages (e.g., arms, hands, legs, feet, etc.), and / or eyes. In an exemplary embodiment, the occupant position tracking device includes one or more cameras disposed within vehicle 12, for example, as part of a driver monitoring system (DMS) for monitoring occupant attention when using autonomous driving or driver assistance functions.

[0057] In a non-limiting example, the occupant position tracking device is configured to determine the position of an occupant relative to the infotainment system 18 by processing data from one or more cameras using a computer vision algorithm. For example, the occupant position tracking device is configured to determine the position of an occupant's hands relative to the infotainment system 18. Within the scope of this disclosure, the occupant's hand position refers to the position of the occupant's hands and / or fingertips within the vehicle 12.

[0058] In another non-limiting example, the occupant position tracking device is configured to determine the occupant's gaze position relative to the infotainment system 18. Within the scope of this disclosure, the occupant's gaze position is the position within the vehicle 12 where the occupant is looking (i.e., the position at the center of the occupant's field of vision). In an exemplary embodiment, the occupant's gaze position is determined by processing data from one or more cameras using a computer vision algorithm that identifies the pupil center, calculates a gaze vector based on the position of the pupil center, and determines the gaze position based on the intersection of the gaze vector with the vehicle 12 and / or the infotainment system 18.

[0059] It should be understood that various additional methods for determining hand position and / or gaze position are within the scope of this disclosure. As described above, the occupant position tracking device is in electrical communication with the controller 14.

[0060] It should be understood that the one or more vehicle sensors 16 discussed above are merely exemplary in nature, and without departing from the scope of this disclosure, one or more vehicle sensors 16 may also include additional and / or alternative sensors.

[0061] The infotainment system 18 is used to provide information to the occupants of the vehicle 12. In an exemplary embodiment, the infotainment system 18 includes a human-machine interface (HMI) located within the occupants' field of vision and capable of displaying text, graphics, and / or images. It should be understood that HMI display systems, including LCD displays, LED displays, etc., are within the scope of this disclosure. In some embodiments, the HMI is relatively centrally located within the vehicle 12, as described below. Figure 2 As discussed in [the previous section]. In some embodiments, the HMI occupies a significant portion of the entire width of the dashboard of vehicle 12, extending behind the steering wheel of vehicle 12 and additionally providing functionality such as an instrument panel display. In other embodiments, the HMI includes multiple displays disposed on the instrument panel of vehicle 12, some of which are primarily configured for viewing by the driver of vehicle 12, and some of which are primarily configured for viewing by the passengers of vehicle 12. In other embodiments, the HMI includes displays disposed in an additional passenger compartment of vehicle 12 (e.g., a rear passenger compartment). It should be understood that the HMI, as well as the infotainment system 18, may include any displays within vehicle 12 configured to provide information to any occupant of vehicle 12 and / or receive input from any occupant of vehicle 12.

[0062] In another exemplary embodiment, the infotainment system 18 includes a head-up display (HUD) configured to provide information to occupants by projecting text, graphics, and / or images onto the windshield of the vehicle 12. The text, graphics, and / or images are reflected by the windshield of the vehicle 12, and the occupants can see these text, graphics, and / or images without taking their eyes off the road in front of the vehicle 12. In another exemplary embodiment, the infotainment system 18 includes an augmented reality head-up display (AR-HUD). An AR-HUD is a type of HUD configured to enhance occupant vision of the road ahead of the vehicle 12 by overlaying text, graphics, and / or images onto physical objects in the environment surrounding the vehicle 12 within the occupants' field of vision.

[0063] In an exemplary embodiment, occupants can interact with the infotainment system 18 using a human-machine interface (HID) device, which includes, for example, a touchscreen, electromechanical switches, capacitive switches, knobs, etc. It should be understood that additional systems for displaying information to the occupants of the vehicle 12 are also within the scope of this disclosure.

[0064] refer to Figure 2 This illustration shows a schematic diagram of the interior of a vehicle 12, including an infotainment system 18. In an exemplary embodiment, the infotainment system 18 includes a touchscreen that is positioned substantially centrally within the passenger compartment of the vehicle 12, for example, within the center console / dashboard of the vehicle 12. Figure 2As shown. In a non-limiting example, one or more edges of the infotainment system 18 are defined as edges of the touchscreen, such as the left edge 40a, top edge 40b, right edge 40c, and bottom edge 40d. In a non-limiting example, in a left-hand drive vehicle, the left edge 40a is closest to the driver of vehicle 12. In a right-hand drive vehicle, the right edge 40c is closest to the passenger of vehicle 12. It should be understood that... Figure 2 The shape, size, location, and configuration of the infotainment system 18 shown are merely exemplary in nature, and a larger full-width infotainment system 18 is also within the scope of this disclosure.

[0065] refer to Figure 3 This illustrates a first exemplary user interface (UI) 50a for an infotainment system 18. The first exemplary UI 50a includes multiple UI elements 52 within a first content area 54a and a second content area 54b. The multiple UI elements 52 are shapes, text, images, etc., designed for interaction with the occupants of the vehicle 12 (e.g., through selection, click, touch, tap, drag, etc.). The first content area 54a is used to display one or more currently running applications and / or the home screen. Figure 3 The main screen is displayed in the first content area 54a. In a non-limiting example, multiple UI elements 52 include multiple application icons 56a displayed on the main screen. The multiple application icons 56a are configured to launch various applications, such as a weather information application, a mobile application, a navigation application, a settings application, etc. When the occupants of vehicle 12 interact with one or more of the multiple application icons 56a, the application can be opened and replace the main screen in the first content area 54a.

[0066] The second content area 54b is used to provide quick access to launch important applications and to provide relevant information to the occupants of vehicle 12. In an exemplary embodiment, the second content area 54b is always displayed on the screen, regardless of what application is running or displayed in the first content area 54a. In a non-limiting example, the plurality of UI elements 52 also include a plurality of control bar icons 56b displayed in the second content area 54b. In a non-limiting example, the plurality of control bar icons 56b includes a music application icon 58a, a phone application icon 58b, a navigation application icon 58c, a settings application icon 58d, and a signal strength indicator icon 58e.

[0067] In an exemplary embodiment, each of the plurality of UI elements 52 has one or more UI element features. In a non-limiting example, the one or more UI element features include UI element type (e.g., button, slider, text, image, etc.), UI element purpose (e.g., the type of application the UI element activates, such as weather, phone, navigation, etc.), etc. The UI element purpose may also include information about the inherent importance of each of the plurality of UI elements 52. For example, for a driving task, a UI element for a navigation application is inherently more important than a UI element for a music application. In a non-limiting example, the one or more UI element features also include dynamic and / or statistical information about each of the plurality of UI elements 52, such as the frequency of occupant use or selection.

[0068] Additionally, each of the multiple UI elements 52 has a layout. In a non-limiting example, the layout includes the size, shape, rotation angle, color, position within the first content area 54a and / or the second content area 54b, etc., of each of the multiple UI elements 52. The determination of the layout for each of the multiple UI elements 52 will be discussed in more detail below in conjunction with method 100. It should be understood that the above discussion and... Figure 3 The first exemplary user interface (UI) 50a shown is merely exemplary in nature, and additional and / or alternative designs, layouts, and functions are within the scope of this disclosure. Furthermore, Figure 3 The first exemplary UI 50a shown is not necessarily drawn to scale unless it involves Figure 3 And discussed below Figure 5 The relative size relationship of multiple UI elements 52.

[0069] refer to Figure 4 A flowchart of a method 100 for providing information to the occupants of a vehicle is shown. Method 100 begins at step 102 and proceeds to block 104. In block 104, controller 14 determines one or more inputs. In a non-limiting example, one or more vehicle sensors 16 are used to determine one or more inputs. In an exemplary embodiment, one or more inputs include measurements of vehicle operating conditions, such as the speed of vehicle 12 (e.g., determined using vehicle speed sensor 24), the acceleration of vehicle 12 (e.g., determined using inertial measurement unit 26), the vibration of vehicle 12 (e.g., determined using inertial measurement unit 26), the activation state of the traction control system of vehicle 12 (i.e., whether the traction control system is working to prevent slippage, which indicates slippery road conditions), the time-averaged frequency of accelerator / brake pedal actuation, etc.

[0070] In an exemplary embodiment, one or more inputs may also include information about external environmental conditions, such as weather conditions in the environment 28 surrounding the vehicle 12 (e.g., determined using information received using the vehicle communication system 34 and / or camera system 30), traffic congestion and / or traffic density in the environment 28 surrounding the vehicle 12 (e.g., determined using information received using the vehicle communication system 34 and / or camera system 30), visibility in the environment 28 surrounding the vehicle 12 (e.g., determined using the camera system 30), and the distance between the vehicle 12 and the nearest remote vehicle (e.g., determined using the lidar sensor 32 and / or camera system 30), etc.

[0071] In an exemplary embodiment, one or more inputs may also include information about the internal environmental conditions, such as the number of occupants in the vehicle 12 (e.g., determined using one or more passenger sensors 36), the type of occupants in the vehicle 12 (e.g., the age of the passengers, human or animal passengers, etc., determined, for example, using a cabin camera), and the sound level in the vehicle 12 (e.g., determined using a cabin microphone), etc.

[0072] In an exemplary embodiment, one or more inputs may also include information about the position of an occupant within the vehicle 12, such as the occupant's position relative to the infotainment system 18, the position of the occupant's hand relative to the infotainment system 18, the distance between the hand position and each of the plurality of UI elements 52, the occupant's gaze position relative to the infotainment system 18, and the distance between the gaze position and each of the plurality of UI elements 52, etc.

[0073] It should be understood that the one or more inputs discussed above are merely exemplary in nature, and various additional and / or alternative inputs may be identified at box 104. Following box 104, method 100 proceeds to box 106.

[0074] In block 106, controller 14 determines an importance score for each of the plurality of UI elements 52. Within the scope of this disclosure, the importance score indicates the importance of each of the plurality of UI elements 52 to the occupants of vehicle 12. In an exemplary embodiment, controller 14 determines the importance score for each of the plurality of UI elements 52 based at least in part on one or more inputs determined at block 104 and one or more UI element features of each of the plurality of UI elements 52. In a non-limiting example, a trained machine learning model is used to determine the importance score for each of the plurality of UI elements 52, the machine learning model receiving one or more inputs and one or more UI element features of each of the plurality of UI elements 52 and providing an importance score as output.

[0075] In another non-limiting example, a deterministic algorithm is used to determine the importance score of each of the plurality of UI elements 52. For example, the importance score is determined as a normalized weighted average of one or more inputs determined at box 104, wherein the weight corresponding to each of the one or more inputs is determined based on one or more UI element features (e.g., inherent importance). For example, the importance score of each of the plurality of UI elements 52 is positively correlated with the inherent importance of each of the plurality of UI elements 52.

[0076] In another example, the importance score of each of the multiple UI elements 52 is positively correlated with the speed of vehicle 12. In another example, the importance score of each of the multiple UI elements 52 is positively correlated with the traffic density in the surrounding environment 28 of vehicle 12. In another example, the importance score of each of the multiple UI elements 52 is negatively correlated with the distance between vehicle 12 and the nearest remote vehicle. In another example, if the traction control system is active, the importance score of each of the multiple UI elements 52 increases by a predetermined amount.

[0077] In another example, the importance score of each of the multiple UI elements 52 is positively correlated with the average frequency of accelerator / brake pedal actuation in vehicle 12. In another example, the importance score of each of the multiple UI elements 52 is negatively correlated with the occupant's hand position and the distance between each of the multiple UI elements 52. In other words, UI elements 52 near the occupant's hand position will have a higher importance score. Therefore, the importance score of each of the multiple UI elements 52 will dynamically change as the occupant in vehicle 12 moves their hand near the infotainment system 18 (e.g., preparing to select one of the UI elements 52).

[0078] In another example, the importance score of each of the multiple UI elements 52 is negatively correlated with the gaze position and the distance between each of the multiple UI elements 52. In other words, UI elements 52 near the occupant's gaze position will have a higher importance score. Therefore, the importance score of each of the multiple UI elements 52 will dynamically change as the passenger moves their gaze near the infotainment system 18 in the vehicle 12 (e.g., preparing to select one of the UI elements 52).

[0079] It should be understood that the methods discussed above for determining importance scores are merely exemplary in nature, and any machine learning-based or non-machine learning-based methods (e.g., mathematical functions, lookup tables, etc.) may be used without departing from the scope of this disclosure. Following box 106, method 100 proceeds to box 108.

[0080] In box 108, controller 14 determines the layout of each of a plurality of UI elements 52. In an exemplary embodiment, controller 14 determines the layout of each of the plurality of UI elements 52 based at least in part on an importance score of each of one or more UI elements determined at box 106. In an exemplary embodiment, a rule-based algorithm is used to determine the layout of each of the plurality of UI elements 52. For example, if the importance score of a UI element is greater than a first predetermined threshold, the size of the UI element is increased by a predetermined amount (e.g., 10%), thus exceeding the default size of the UI element. In another example, if the importance score of a UI element is greater than a second predetermined threshold, the position of the UI element is moved to a predetermined area within the infotainment system 18.

[0081] In another example, if the importance score of a UI element is below a third predetermined threshold, the UI element is not displayed on the infotainment system 18. In another example, if the importance score of a UI element is above a fourth predetermined threshold, the color of the UI element is set to a predetermined color indicating high importance (e.g., red). In another example, if the importance score of a UI element is below a fifth predetermined threshold, the color of the UI element is set to a predetermined color indicating low importance (e.g., gray).

[0082] In another example, the size of each of the multiple UI elements 52 is correlated with the importance score of each of the multiple UI elements 52. In a non-limiting example, the size of each of the multiple UI elements 52 is positively correlated (e.g., proportional) with the importance score of each of the multiple UI elements 52. In another example, the position of each of the multiple UI elements 52 is determined based on the importance score of each of the multiple UI elements 52. In a non-limiting example, the distance between each of the multiple UI elements 52 and the edges of the infotainment system 18 (e.g., left edge 40a, top edge 40b, right edge 40c, and / or bottom edge 40d) is negatively correlated with the importance score of each of the multiple UI elements 52. For example, a UI element with a high importance score is located closer to the edge closest to the occupant (e.g., left edge 40a in a left-hand drive vehicle, or right edge 40c in a right-hand drive vehicle).

[0083] In another non-limiting example, the distance between each of the multiple UI elements 52 and the occupant's hand position is negatively correlated with the importance score of each of the multiple UI elements 52. In other words, UI elements 52 with high importance scores will be positioned closer to the occupant's hand position. In another non-limiting example, the distance between each of the multiple UI elements 52 and the occupant's gaze position is negatively correlated with the importance score of each of the multiple UI elements 52. In other words, UI elements 52 with high importance scores will be positioned closer to the occupant's gaze position.

[0084] In another non-limiting example, the location of one or more UI elements 52 within one or more predefined areas of the infotainment system 18 is determined based on a comparison of the importance score of each of the UI elements 52 with one or more thresholds. For example, one or more predefined areas may include areas easily accessible to occupants (e.g., areas near the occupant's seat and / or areas within the occupant's clear line of sight). One or more predefined areas may be determined based on the specific shape, size, and location of the infotainment system 18.

[0085] In another exemplary embodiment, the layout of each of the plurality of UI elements 52 is determined using a trained machine learning algorithm that determines the layout of each of the plurality of UI elements 52 based on an importance score of each of the plurality of UI elements 52.

[0086] In a non-limiting example, the machine learning algorithm includes multiple layers, including an input layer and an output layer, and one or more hidden layers. The input layer receives an importance score for each of the multiple UI elements 52 as input. The input is then passed to the hidden layers. Each hidden layer transforms the data (e.g., a non-linear transformation) and passes the result to the next hidden layer, until the last hidden layer. The output layer produces the layout (e.g., size, position, color, etc.) of each of the multiple UI elements 52 on the infotainment system 18.

[0087] To train the machine learning algorithm, a dataset was used, consisting of the input and the corresponding layouts (e.g., size, position, color, etc.) of each of the multiple UI elements 52. The dataset could be generated manually, crowdsourced from data on passenger preferences for how the infotainment system 18 is used, and / or improved using reinforcement learning based on feedback from test subjects. The algorithm was trained by adjusting the internal weights between nodes in each hidden layer to minimize prediction error. During training, optimization techniques (e.g., gradient descent) were used to adjust the internal weights to reduce prediction error. The training process was repeated using the entire dataset until the prediction error was minimized, and then the resulting trained model was used to process new input data.

[0088] After being adequately trained on the machine learning algorithm, the algorithm is able to determine the optimal layout (e.g., size, position, color, etc.) of each of the multiple UI elements 52 based on the importance score of each of them. By adjusting the weights between nodes in each hidden layer during the training process, the algorithm "learns" to recognize patterns in the data that indicate the optimal layout (e.g., size, position, color, etc.) of each of the multiple UI elements 52.

[0089] In some embodiments, the machine learning algorithm may be adapted and further trained on-site based on feedback from occupants and / or data collected on how occupants use the infotainment system 18.

[0090] It should be understood that the examples of rule-based algorithms and machine learning algorithms provided above are merely illustrative in nature, and various additional and / or alternative algorithms, functions, and / or methods for determining the layout of each of the multiple UI elements 52 are within the scope of this disclosure.

[0091] refer to Figure 5 This illustrates a second exemplary user interface (UI) 50b for the infotainment system 18. (See reference...) Figures 3 to 5 ,and Figure 3 Compared to the first exemplary UI 50a shown, the second exemplary UI 50b illustrates an exemplary layout change of a plurality of UI elements 52. In an exemplary embodiment, as shown in the second exemplary UI 50b, compared to the first exemplary UI 50a, some of the plurality of application icons 56a are not shown in the first content area 54a (e.g., because some of the plurality of application icons 56a have an importance score below a third predetermined threshold), and the remaining portions of the plurality of application icons 56a are shifted and / or resized toward the left edge 40a. In a non-limiting example, the remaining portions of the plurality of application icons 56a are shifted toward the left edge 40a to make the remaining portions of the plurality of application icons 56a more easily accessible to the driver of a left-hand drive vehicle. It should be understood that in a right-hand drive vehicle, the remaining portions of the plurality of application icons 56a may be shifted toward the right edge 40c.

[0092] In an exemplary embodiment, as shown in the second exemplary UI 50b, some of the multiple control bar icons 56b are not shown in the second content area 54b compared to the first exemplary UI 50a (e.g., because some of the multiple control bar icons 56b have an importance score below a third predetermined threshold), and the remaining multiple control bar icons 56b are rearranged such that icons with high importance scores (e.g., due to high inherent importance) are closer to the left edge 40a. For example, the navigation app icon 58c is placed closest to the left edge 40a, followed by the phone app icon 58b, and then the music app icon 58a. Additionally, as shown in the second exemplary UI 50b, the multiple control bar icons 56b are enlarged in size compared to the first exemplary UI 50a (e.g., because the importance score of each of the multiple control bar icons 56b is greater than the first predetermined threshold).

[0093] It should be understood that the above discussion and Figure 5 The second exemplary user interface (UI) 50b shown is merely exemplary in nature, and additional and / or alternative designs, layouts, and functions are within the scope of this disclosure. Furthermore, Figure 5 The second exemplary UI 50b shown is not necessarily drawn to scale unless it involves Figure 3 and Figure 5 The relative size relationships of multiple UI elements (52). Refer again. Figure 4 After box 108, method 100 proceeds to box 110.

[0094] refer to Figure 6 The third exemplary user interface (UI) 50c of the infotainment system 18 is shown as having an exemplary occupant hand 60. (See reference...) Figures 3 to 4 and Figure 6 ,and Figure 3 Compared to the first exemplary UI 50a shown, the third exemplary UI 50c illustrates an exemplary layout variation of multiple UI elements 52. In an exemplary embodiment, as shown in the third exemplary UI 50c, multiple application icons 56a are resized based on their proximity to the exemplary occupant's hand 60 compared to the first exemplary UI 50a. For example, some of the multiple application icons 56a near the exemplary occupant's hand 60 are enlarged, and some of the multiple application icons 56a farther from the exemplary occupant's hand 60 are reduced in size (e.g., due to the negative correlation between importance score and distance to the hand position). It should be understood that the layout depicted in the third exemplary UI 50c is merely exemplary and not necessarily drawn to scale. Furthermore, the above description also applies to embodiments utilizing gaze position without departing from the scope of this disclosure.

[0095] It should be understood that the above discussion and Figure 6 The third exemplary user interface (UI) 50c shown is merely exemplary in nature, and additional and / or alternative designs, layouts, and functions are within the scope of this disclosure. Furthermore, Figure 6 The third exemplary UI 50c shown is not necessarily drawn to scale unless it involves Figure 3 and Figure 6 The relative size relationships of multiple UI elements (52). Refer again. Figure 4 After box 108, method 100 proceeds to box 110.

[0096] At block 110, controller 14 uses infotainment system 18 to display multiple UI elements 52 to provide information to the occupants of vehicle 12. In an exemplary embodiment, controller 14 displays the multiple UI elements 52 based at least in part on the layout (i.e., size, position, color, etc.) of each of the multiple UI elements 52 determined at block 108. In a non-limiting example, adjustments to the layout of each of the multiple UI elements 52 are displayed smoothly (e.g., using animation, etc.) to help the occupants understand and adapt to changes in the layout of each of the multiple UI elements 52. In another non-limiting example, controller 14 uses hysteresis rules or algorithms to limit the frequency and / or speed of changes in the layout of each of the multiple UI elements 52. After block 110, method 100 proceeds to block 112 to enter a standby state.

[0097] In an exemplary embodiment, controller 14 repeatedly exits standby state 112 and restarts method 100 at block 102. In a non-limiting example, controller 14 exits standby state 112 and restarts method 100 under the control of a timer, for example, once every 300 milliseconds. Thus, the layout of multiple UI elements 52 is dynamically adjusted and / or updated based on one or more inputs (e.g., including the location of occupants).

[0098] The system 10 and method 100 of this disclosure offer several advantages. By adjusting the layout of UI elements based on occupant position, vehicle state characteristics, and / or external environment characteristics, the user interface of the infotainment system 18 adapts to the current situation to improve occupant convenience. For example, by increasing the size of UI elements near the occupant's hand or gaze position, the occupant can more easily select UI elements and control the infotainment system 18. Furthermore, the use of system 10 and method 100 allows for dynamic changes to the UI layout based on occupant position.

[0099] The description in this disclosure is exemplary in nature only, and variations thereof that do not depart from the spirit and scope of this disclosure are intended to fall within its scope. Such variations should not be considered as departing from the spirit and scope of this disclosure.

Claims

1. A system for providing information to occupants of a vehicle, the system comprising: One or more occupant sensors; Infotainment systems; as well as A controller, which is electrically in communication with the one or more occupant sensors and the infotainment system, wherein the controller is programmed to: One or more inputs are determined using at least one or more of the occupant sensors; Based at least in part on the one or more inputs, determine the layout of each of the plurality of user interface (UI) elements of the infotainment system; and Information is provided to the occupants of the vehicle by displaying the plurality of UI elements, based at least in part on the layout of each of the plurality of UI elements.

2. The system according to claim 1, wherein, To determine the layout of each of the plurality of UI elements, the controller is also programmed to: An importance score for each of the plurality of UI elements is determined, at least in part, based on the one or more inputs and one or more UI element features of each of the plurality of UI elements. as well as The layout of each of the plurality of UI elements is determined at least in part based on the importance score of each of the plurality of UI elements.

3. The system according to claim 2, wherein, The one or more occupant sensors further include an occupant position tracking device, and wherein, in order to determine the one or more inputs, the controller is further programmed to: The occupant location tracking device is used to determine the occupant's position relative to the infotainment system.

4. The system according to claim 3, wherein, To determine the importance score of each of the plurality of UI elements, the controller is also programmed to: The importance score of each of the plurality of UI elements is determined at least in part based on the one or more inputs, wherein the one or more inputs include the position of the occupant relative to the infotainment system, and wherein the importance score of each of the plurality of UI elements is negatively correlated with the position of the occupant's hands and the distance between each of the plurality of UI elements.

5. The system according to claim 2, wherein, The one or more occupant sensors further include an occupant position tracking device, and wherein, in order to determine the one or more inputs, the controller is further programmed to: The occupant position tracking device is used to determine the occupant's gaze position relative to the infotainment system.

6. The system according to claim 5, wherein, To determine the importance score of each of the plurality of UI elements, the controller is also programmed to: The importance score of each of the plurality of UI elements is determined at least in part based on the one or more inputs, wherein the one or more inputs include the occupant's gaze position relative to the infotainment system, and wherein the importance score of each of the plurality of UI elements is negatively correlated with the gaze position and the distance between each of the plurality of UI elements.

7. The system according to claim 2, wherein, To determine the one or more inputs, the controller is also programmed to: Determine the measured values ​​of the vehicle's operating condition, wherein the vehicle's operating condition includes one or more of the following: the vehicle's speed, the vehicle's acceleration, and the vehicle's vibration.

8. The system according to claim 2, wherein, To determine the one or more inputs, the controller is also programmed to: Information regarding external environmental conditions, wherein the external environmental conditions include one or more of the following: weather conditions in the environment surrounding the vehicle and traffic congestion conditions in the environment surrounding the vehicle; and Information determining the internal environmental conditions, wherein the internal environmental conditions include one or more of the following: the number of occupants in the vehicle, the type of occupants in the vehicle, and the sound level inside the vehicle.

9. The system according to claim 2, wherein, To determine the layout of each of the plurality of UI elements, the controller is also programmed to: The layout of each of the plurality of UI elements is determined at least in part based on the importance score of each of the plurality of UI elements, wherein the layout includes at least the size of each of the plurality of UI elements, and wherein the size of each of the plurality of UI elements is positively correlated with the importance score of each of the plurality of UI elements.

10. The system according to claim 2, wherein, To determine the layout of each of the plurality of UI elements, the controller is also programmed to: The layout of each of the plurality of UI elements is determined at least in part based on the importance score of each of the plurality of UI elements, wherein the layout includes at least the position of each of the plurality of UI elements on the infotainment system, wherein the distance between the position of each of the plurality of UI elements and the position of the occupant's hand or the position of the occupant's gaze is negatively correlated with the importance score of each of the plurality of UI elements.

Images