Method and system for notifying and facilitating multiple choices of upcoming actions of assisted driving

Abstract

A method and system are provided for presenting a plurality of choices to a host vehicle operator during autonomous driving of the host vehicle. The method includes receiving information based at least in part on an environment of the host vehicle and determining a plurality of first actions when the host vehicle is a certain distance from a location for which a benefit-gaining action is estimated. The plurality of first actions includes at least a first choice and a second choice. The method further includes presenting the plurality of first actions to the vehicle operator, receiving an input from the vehicle operator, and determining a second action based on the input from the vehicle operator. The second action includes at least one of the following items: providing no further system assistance to the host vehicle operator, providing system assistance to the host vehicle operator corresponding to the first choice, or providing system assistance to the host vehicle operator corresponding to the second choice.

Description

Technical Field

[0001] This disclosure relates to systems and methods for providing information to vehicle operators. Background Technology

[0002] To enhance occupant alertness and convenience, vehicles can be equipped with Advanced Driver Assistance Systems (ADAS) and / or Automated Driving Systems (ADS). ADAS systems use various sensors, such as cameras, radar, and LiDAR (Light Detection and Ranging), to detect and identify objects around the vehicle, including other vehicles, pedestrians, road configurations, traffic signs, and road markings. ADAS systems can take actions based on environmental conditions, such as applying the brakes or alerting vehicle occupants. ADS systems use various sensors to detect objects in the vehicle's environment and control the vehicle to navigate through the environment to a predetermined destination. However, current ADAS and ADS systems may not effectively facilitate smooth driving and safe passage through green lights when the goal-achieving system's actions are deemed too aggressive and unexpected for the vehicle operator.

[0003] Therefore, while ADAS and ADS systems and methods have achieved their intended purpose, there is still a need for new and improved systems and methods to avoid driver accidents and facilitate user interaction to achieve advantageous actions. Summary of the Invention

[0004] According to several aspects of this disclosure, a method is provided for presenting a plurality of options to a primary vehicle operator to maintain smooth driving during autonomous or driver-assisted driving of the primary vehicle. The method includes receiving information, at least in part based on the primary vehicle environment, and determining a plurality of first actions based on the information when the primary vehicle is at a distance from a location used to estimate an advantage-gaining action. The plurality of first actions includes at least a first choice and a second choice. The method further includes presenting the plurality of first actions to the primary vehicle operator, receiving input from the primary vehicle operator based on the plurality of first actions, and determining a second action based on the input from the primary vehicle operator. The second action includes at least one of the following: not providing further system assistance to the primary vehicle operator, providing system assistance to the primary vehicle operator corresponding to a first choice, or providing system assistance to the primary vehicle operator corresponding to a second choice.

[0005] According to another aspect of this disclosure, the primary vehicle environment includes at least one of the surrounding traffic modes or the primary vehicle steering selection.

[0006] According to another aspect of this disclosure, the information includes V2X messages, traffic light signals, traffic patterns, vehicle perception data, primary vehicle route planning, primary vehicle recent speed distribution, or primary vehicle operator preference models related to key information affecting traffic participants.

[0007] According to another aspect of this disclosure, determining the first and second options includes determining a plurality of master vehicle control options.

[0008] According to another aspect of this disclosure, determining a plurality of first actions includes determining a third option.

[0009] According to another aspect of this disclosure, presenting a number of first actions to the main vehicle operator includes: presenting a first option that has a greater advantage than a second option and presenting a second option that has a higher probability of success than the first option.

[0010] According to another aspect of this disclosure, presenting multiple first actions to the master vehicle operator includes presenting the master vehicle operator with a preview stage having suggestions, the suggestions including at least one of estimated location, action type, feasible space and time range, expected challenges, or major impacts of each lane.

[0011] According to another aspect of this disclosure, the presentation includes refreshing and re-determining the first and second selection actions until input is received from the main vehicle operator.

[0012] According to another aspect of this disclosure, presenting multiple first actions to the main vehicle operator includes presenting notifications of multiple first actions to the human-machine interface (HMI).

[0013] According to another aspect of this disclosure, receiving input includes receiving an instruction that the main vehicle operator performs a veto of all the plurality of first actions, vetoes one of the plurality of first actions, switches the order of the plurality of first actions, or no input is made.

[0014] According to another aspect of this disclosure, the method also includes using learning-based optimization based on input received from the master vehicle operator.

[0015] According to another aspect of this disclosure, the method also includes providing system assistance by driving the master vehicle based on input from the master vehicle operator.

[0016] According to several aspects of this disclosure, a system is provided for presenting a plurality of options to a primary vehicle operator to maintain smooth driving during autonomous or driver-assisted driving of the primary vehicle. The system includes a vehicle communication system, a display, and a controller in electrical communication with the vehicle communication system and the display. The controller is programmed to receive information at least partially based on the primary vehicle environment and to determine a plurality of first actions based on that information when the primary vehicle is at a distance to estimate a position for taking action. The plurality of first actions includes at least a first choice and a second choice. The controller is also programmed to present the plurality of first actions to the primary vehicle operator, receive input from the primary vehicle operator based on the plurality of first actions, and determine a second action based on the input from the primary vehicle operator. The second action includes at least one of the following: not providing further system assistance to the primary vehicle operator, providing system assistance to the primary vehicle operator corresponding to a first choice, or providing system assistance to the primary vehicle operator corresponding to a second choice.

[0017] According to another aspect of this disclosure, the primary vehicle environment includes at least one of the surrounding traffic modes or the primary vehicle steering selection.

[0018] According to another aspect of this disclosure, the information includes at least one of the following: V2X messages, traffic light signals, traffic patterns, vehicle perception data, primary vehicle route planning, primary vehicle recent speed distribution, or primary vehicle operator preference model related to key information affecting traffic participants.

[0019] According to another aspect of this disclosure, determining the first and second options includes determining a plurality of master vehicle control options.

[0020] According to another aspect of this disclosure, presenting the main vehicle operator with a number of first actions includes presenting a first option that provides a greater advantage and presenting a second option that has a higher probability of success.

[0021] According to another aspect of this disclosure, the presentation step includes refreshing the redefined first and second selection actions until input is received from the master vehicle operator.

[0022] According to another aspect of this disclosure, receiving input includes receiving an instruction that the main vehicle operator performs a veto of all the plurality of first actions, vetoes one of the plurality of first actions, switches the order of the plurality of first actions, or no input is made.

[0023] According to several aspects of this disclosure, a method is provided to present a host vehicle operator with multiple options to maintain smooth driving during autonomous or driver-assisted driving of the host vehicle. The method includes receiving information at least partially based on the host vehicle's environment, and determining multiple first actions based on that information when the host vehicle is at a distance to estimate a position for taking action. The multiple first actions include at least a first choice and a second choice. The method further includes presenting the multiple first actions to the host vehicle operator and receiving input from the host vehicle operator based on the multiple first actions, the input including receiving an instruction to at least one of the following: the host vehicle operator performs a rejection of all the multiple first actions, rejects one of the multiple first actions, switches the order of the multiple first actions, or does not provide input. The method further includes determining a second action based on the input from the host vehicle operator. The second action includes at least one of the following: not providing further system assistance to the host vehicle operator, providing system assistance to the host vehicle operator corresponding to a first choice, or providing system assistance to the host vehicle operator corresponding to a second choice. Furthermore, the method includes providing system assistance by driving the host vehicle based on input from the host vehicle operator.

[0024] Further areas of application will become apparent from the description provided herein. It should be understood that these descriptions and specific examples are for illustrative purposes only and are not intended to limit the scope of this disclosure. Attached Figure Description

[0025] The accompanying drawings described herein are for illustrative purposes only and are not intended to limit the scope of this disclosure in any way.

[0026] Figure 1 This is a schematic diagram of a system according to the present disclosure for presenting multiple options to the main vehicle operator to maintain smooth driving during autonomous or driver-assisted driving.

[0027] Figure 2 This is a flowchart of a method for presenting multiple options to the main vehicle operator to maintain smooth driving during autonomous or driver-assisted driving, according to the present disclosure.

[0028] Figure 3 This is an exemplary view of the vehicle interior illustrating an exemplary HMI notification, according to this disclosure.

[0029] Figure 4 A schematic environmental view of an example scenario illustrating the use of this disclosure of a method for presenting multiple options to a primary vehicle operator to maintain smooth driving during autonomous or driver-assisted driving is shown.

[0030] Figure 5A schematic environmental view of an example scenario illustrating the use of this disclosure of a method for presenting multiple options to a primary vehicle operator to maintain smooth driving during autonomous or driver-assisted driving is shown.

[0031] Figure 6 A schematic environmental view of an example scenario illustrating the use of this disclosure of a method for presenting multiple options to a primary vehicle operator to maintain smooth driving during autonomous or driver-assisted driving is shown. Detailed Implementation

[0032] The following description is merely exemplary in nature and is not intended to limit this disclosure, its application, or its uses. Furthermore, it is not intended to be bound by any express or implied theory presented in the foregoing technical field, background art, summary of the invention, or the following detailed description. It should be understood that throughout the drawings, corresponding reference numerals denote similar or corresponding parts and features.

[0033] While drivers are often aware that the key objective of the system is to facilitate green light passage (i.e., proceeding without coming to a complete stop) and / or smooth driving, they desire to avoid surprises caused by actions perceived as overly aggressive or lacking in merit. It is also desirable to avoid surprises and to facilitate subjectively desired interactions that better align with driver preferences and foster trust in the system. Therefore, this disclosure provides a novel and improved system and method for avoiding surprises and facilitating driver interaction and advantage-gaining actions while ensuring driver satisfaction. The methods and systems disclosed herein are designed to provide and assist smooth driving and / or green light passage toward a certain intermediate distance (typically the remaining distance to the traffic light or checkpoint ahead), rather than aiming at several intersections or a destination ahead. The design of interaction choices depends on driver comfort and readiness, utilizing the driver's observations of the surrounding driving environment, adapting to the driver's anticipation of satisfactory advantage-gaining actions, and applying learning-based optimization to a driver preference model to improve satisfaction.

[0034] To facilitate green light passage and / or smooth driving, the system provides the driver with multiple (usually two, but sometimes three or more) upcoming advantage-gaining actions (executed by the system / controller and / or the driver) to choose from, such that the first choice (e.g., the default) action will have a higher advantage, while the second choice (e.g., a favorable alternative) action will have a higher probability of success (or conversely, depending on surrounding traffic, driver preferences, etc.). The driver can make or re-make choices within a certain time and / or feasible space range until reaching the final stage (i.e., the range has been exceeded or the driver has rejected at least one choice).

[0035] Furthermore, to accurately incorporate the driver's own judgment into the impending action and support learning-based optimization schemes, the system / controller disclosed herein (during the interaction phase) accepts a choice made or re-made by the driver, which can be either rejecting both options (when the driver feels ready to manipulate or take over control based on their own judgment), rejecting one option (when the driver has sufficient confidence to follow / accept the remaining options), or switching options. In some cases, no input is required (in selected scenarios / situations, if the driver prefers to prioritize one option, but keeps both options available within a certain range).

[0036] To facilitate comfortable driving choices without unexpected events, a preview phase precedes the interaction phase, providing the driver with useful information that will be reduced and / or eliminated during the interaction phase. For example, the driver may be provided with estimated location, action type, other informational attributes of the action choice (e.g., feasible space and / or time range, anticipated challenges), and / or the primary impact of each lane on the main vehicle (“HV”) (regarding green light passage or smooth driving).

[0037] refer to Figure 1 A schematic diagram of a system 10 for providing information to vehicle occupants is provided. System 10 is shown together with a main vehicle 12. Although a passenger vehicle is shown, it should be understood that the main vehicle 12 can be any type of vehicle without departing from the scope of this disclosure. System 10 generally includes a controller 20, multiple vehicle sensors 22, a human-machine interface (HMI) 24, and a head-up display (HUD) 26.

[0038] The controller 20 is used to implement the method 100 for providing information to vehicle occupants, as described below. The controller 20 includes at least one processor 28 and a non-transitory computer-readable storage device or medium 30. The processor 28 may be a custom or commercially available processor, a central processing unit (CPU), a graphics processing unit (GPU), an auxiliary processor among several processors associated with the controller 20, a semiconductor-based microprocessor (in the form of a microchip or chipset), a macroprocessor, a combination of the above, or generally a device for executing instructions.

[0039] Computer-readable storage device or medium 30 may include volatile and non-volatile storage devices such as read-only memory (ROM), random access memory (RAM), and keep-alive memory (KAM). KAM is a persistent or non-volatile memory that can be used to store various operational variables when processor 28 is off. Computer-readable storage device or medium 30 may be implemented using multiple memory devices, such as PROM (programmable read-only memory), EPROM (electrical PROM), EEPROM (electrically erasable PROM), flash memory, or other electrical, magnetic, optical, or combinations thereof memory devices capable of storing data (some of which represents executable instructions) used by controller 20 to control various systems of the master vehicle 12.

[0040] Controller 20 may also consist of multiple controllers that are electrically communicating with each other. Controller 20 may interconnect with additional systems and / or controllers of the main vehicle 12, thereby allowing controller 20 to access data such as the speed, acceleration, braking, and steering angle of the main vehicle 12.

[0041] The controller 20 communicates electrically with multiple vehicle sensors 22, HMI 24, and HUD 26. In one exemplary embodiment, electrical communication is established using, for example, a CAN network, a FLEXRAY network, a local area network (e.g., WiFi, Ethernet, etc.), a Serial Peripheral Interface (SPI) network, etc. It should be understood that various other wired and wireless technologies and communication protocols used for communicating with the controller 20 are within the scope of this disclosure. It should be understood that one or more of the multiple vehicle sensors 22, HMI 24, and HUD 26 may be integrated with the controller 20 (e.g., integrated on the same circuit board as the controller 20 or otherwise incorporated into the controller 20) without departing from the scope of this disclosure. It should also be understood that, within the scope of this disclosure, electrical communication also includes the transfer of power and / or energy between electrical devices (e.g., using wired and / or wireless power transmission technologies).

[0042] Multiple vehicle sensors 22 are used to acquire information about the host vehicle and / or one or more remote vehicles. In one exemplary embodiment, the multiple vehicle sensors 22 include one or more sensing sensors 32, a Global Navigation Satellite System (GNSS) 34, and a vehicle communication system 36.

[0043] One or more sensing sensors 32 are used to sense objects in the environment surrounding the main vehicle 12 and / or measure distances. In one exemplary embodiment, the one or more sensing sensors 32 include at least one of a camera 38, a radar sensor 40, and a light detection and ranging (LiDAR) sensor 42.

[0044] Camera 38 is a sensing sensor for capturing images and / or video of the environment surrounding the main vehicle 12. In one exemplary embodiment, camera 38 includes a photographic camera and / or a video camera positioned to observe the environment surrounding the main vehicle 12. In a non-limiting example, camera 38 includes a camera fixed inside the main vehicle 12 (e.g., in the headliner of the main vehicle 12) having a field of view through the windshield 44 of the main vehicle 12. In another non-limiting example, camera 38 includes a camera fixed outside the main vehicle 12 (e.g., fixed to the roof of the main vehicle 12) having a field of view of the environment in front of the main vehicle 12.

[0045] In another exemplary embodiment, camera 38 is a surround-view camera system including multiple cameras (also referred to as satellite cameras) arranged to provide a field of view of the environment adjacent to all sides of the host vehicle 12. In one non-limiting example, camera 38 includes a front camera (e.g., mounted in the front grille of the host vehicle 12), a rear camera (e.g., mounted on the tailgate of the host vehicle 12), and two side cameras (e.g., mounted below each of the two side mirrors of the host vehicle 12). In another non-limiting example, camera 38 also includes an additional rear-view camera mounted near the center high-mounted brake light of the host vehicle 12.

[0046] It should be understood that camera systems with additional cameras and / or additional mounting locations are within the scope of this disclosure. It should also be understood that cameras with various sensor types are within the scope of this disclosure, including, for example, charge-coupled device (CCD) sensors, complementary metal-oxide-semiconductor (CMOS) sensors, and / or high dynamic range (HDR) sensors. Furthermore, cameras with various lens types, including, for example, wide-angle lenses and / or narrow-angle lenses, are also within the scope of this disclosure. Camera 38 is in electrical communication with controller 20, as described above.

[0047] Radar sensor 40 is used to detect and measure the distance, speed, and direction of an object (e.g., other vehicles around a primary vehicle) by emitting radio waves and analyzing their reflection. In one exemplary embodiment, radar sensor 40 includes a radar transmitter (not shown), a radar antenna (not shown), a radar receiver (not shown), and a radar signal processing unit (not shown). In a non-limiting example, the radar transmitter uses the radar antenna to emit a radio frequency (RF) signal that travels through space until it encounters an object. The RF signal is reflected from the object's surface and returns to radar sensor 40. The radar receiver uses the radar antenna to capture the reflected signal, and the radar signal processing unit analyzes the time delay, frequency shift, and amplitude of the returned RF signal to determine the distance, speed, and direction of the detected object. Radar sensor 40 is in electrical communication with controller 20, as described above.

[0048] LiDAR sensor 42 performs remote sensing and environmental mapping by emitting laser pulses and measuring the time it takes for the laser pulses to return to LiDAR sensor 42 after hitting an object. In one exemplary embodiment, LiDAR sensor 42 includes a LiDAR laser source (not shown), a LiDAR scanner or reflector (not shown), a LiDAR photodetector (not shown), and a LiDAR time-of-flight measurement system (not shown). In a non-limiting example, the LiDAR laser source emits laser pulses that travel to a target area, and the LiDAR scanner directs these pulses in different directions. The emitted laser pulses interact with objects in the environment, and their reflections are captured by the LiDAR photodetector. The LiDAR time-of-flight measurement system calculates the distance to the object based on the time between the LiDAR laser source emitting the laser pulse and the LiDAR photodetector receiving the reflected laser pulse. LiDAR sensor 42 is in electrical communication with controller 20, as described above.

[0049] In one exemplary embodiment, one or more sensing sensors 32 are fixed inside the host vehicle 12 (e.g., fixed in the headliner of the host vehicle 12) and have a field of view through the windshield 44 of the host vehicle 12. In another example, one or more sensing sensors 32 are fixed outside the host vehicle 12 (e.g., fixed on the roof of the host vehicle 12) and have a field of view of the environment surrounding the host vehicle 12. It should be understood that various other types of sensing sensors are within the scope of this disclosure, such as stereo cameras with distance measurement capabilities, ultrasonic ranging sensors, and time-of-flight sensors. One or more sensing sensors 32 are in electrical communication with the controller 20, as described above.

[0050] GNSS 34 is used to determine the geographic location of the main vehicle 12. In one exemplary embodiment, GNSS 34 is a Global Positioning System (GPS). In a non-limiting example, GPS includes a GPS receiver antenna (not shown) and a GPS controller (not shown) in electrical communication with the GPS receiver antenna. The GPS receiver antenna receives signals from multiple satellites, and the GPS controller calculates the geographic location of the main vehicle 12 based on the signals received by the GPS receiver antenna.

[0051] In one exemplary embodiment, the GNSS 34 also includes a map. This map includes information about infrastructure, such as municipal boundaries, roads, railways, sidewalks, buildings, etc. Therefore, the map information is used to consider the geographic location of the primary vehicle 12 within a context. In one non-limiting example, the map is acquired from a remote source using a wireless connection. In another non-limiting example, the map is stored in a database or memory of the GNSS 34.

[0052] It should be understood that various other types of satellite-based radio navigation systems are within the scope of this disclosure, such as the Global Positioning System (GPS), Galileo, GLONASS, and the BeiDou Navigation Satellite System (BDS). GNSS 34 communicates electrically with controller 20 as described above.

[0053] The vehicle communication system 36 is used by the controller 20 to communicate with other systems outside the main vehicle 12. For example, the vehicle communication system 36 has the capability to communicate with other vehicles (“V2V” communication), with infrastructure (“V2I” communication), with remote systems at remote call centers (e.g., General Motors’ ON-STAR), and / or with personal devices. Generally, the term vehicle-to-everything (“V2X” communication) refers to communication between the main vehicle 12 and any remote system (e.g., vehicles, infrastructure, and / or remote systems). These messages enable communication between vehicles (V2V), vehicles and infrastructure (V2I), and vehicles and pedestrians (V2P). Some examples of V2X messages include basic safety messages (BSMs) (e.g., providing information about the location, speed, and direction of travel of the primary vehicle relative to nearby vehicles), information related to key information affecting traffic participants, signal phase and timing (SPaT) messages (e.g., communicating traffic signal status and traffic light signals to vehicles), traveler information messages (TIMs) (e.g., sharing information about road conditions, hazards, and other relevant data), traffic patterns, onboard perception data, primary vehicle route planning, recent speed distribution of the primary vehicle, primary vehicle operator preference models, and / or personal safety messages (PSMs) (e.g., used to enhance pedestrian safety by communicating the presence of pedestrians to nearby vehicles).

[0054] In some embodiments, the vehicle communication system 36 is a wireless communication system configured to communicate via a wireless local area network (WLAN) using the IEEE 802.11 standard or by using cellular data communication (e.g., using GSMA standards such as SGP.02, SGP.22, SGP.32, etc.). Therefore, the vehicle communication system 36 may also include an embedded universal integrated circuit card (eUICC) configured to store at least one cellular connectivity profile, such as an embedded subscriber identity module (eSIM) profile.

[0055] The vehicle communication system 36 is also configured to communicate via a personal area network (e.g., Bluetooth), near field communication (NFC), and / or any other type of radio frequency communication. However, additional or alternative communication methods, such as Dedicated Short Range Communication (DSRC) channels and / or mobile telecommunications protocols based on 3GPP standards, are also considered within the scope of this disclosure. A DSRC channel refers to a unidirectional or bidirectional short-to-medium range wireless communication channel designed specifically for automotive use, along with a corresponding set of protocols and standards. 3GPP refers to a partnership among multiple standards organizations that develop mobile telecommunications protocols and standards. 3GPP standards are structured as “releases.” Therefore, communication methods based on 3GPP releases 14, 15, 16, and / or future 3GPP releases are considered within the scope of this disclosure.

[0056] Therefore, the vehicle communication system 36 may include one or more antennas and / or communication transceivers (not shown) for receiving and / or transmitting signals, such as cooperative sensing messages (CSM). The vehicle communication system 36 is configured to wirelessly transmit information between the host vehicle 12 and another vehicle. Furthermore, the vehicle communication system 36 is configured to wirelessly transmit information between the host vehicle 12 and infrastructure or other vehicles.

[0057] In another exemplary embodiment, the plurality of vehicle sensors 22 further include sensors for determining performance data regarding the main vehicle 12. In a non-limiting example, the plurality of vehicle sensors 22 further include at least one of a motor speed sensor, a motor torque sensor, an electric drive motor voltage and / or current sensor, an accelerator pedal position sensor, a brake position sensor, a coolant temperature sensor, a cooling fan speed sensor, and a transmission oil temperature sensor.

[0058] In another exemplary embodiment, the plurality of vehicle sensors 22 further include additional sensors to determine information about the environment within the main vehicle 12. In a non-limiting example, the plurality of vehicle sensors 22 also include at least one of a seat occupancy sensor, a cabin air temperature sensor, a cabin motion detection sensor, a cabin camera, a cabin microphone, an occupant eye tracker, etc.

[0059] In another exemplary embodiment, the plurality of vehicle sensors 22 further include additional sensors for determining information about the environment surrounding the main vehicle 12. In a non-limiting example, the plurality of vehicle sensors 22 further include at least one of an ambient air temperature sensor, an atmospheric pressure sensor, etc. The plurality of vehicle sensors 22 are in electrical communication with the controller 20, as described above.

[0060] HMI 24 is used to provide information to the occupants of the main vehicle 12. Within the scope of this disclosure, occupants include the driver and / or passengers of the main vehicle 12. Figure 2 In the exemplary embodiment shown, HMI 24 is a display located within the occupant's field of vision and capable of displaying text, graphics, and / or images (e.g., part of the infotainment system of the main vehicle 12). It should be understood that HMI display systems including LCD displays, LED displays, etc., are within the scope of this disclosure. Further exemplary embodiments in which HMI 24 is disposed in a rearview mirror are also within the scope of this disclosure. In one exemplary embodiment, the occupant can interact with HMI 24 using a human-machine interface device (HID), which includes, for example, a touchscreen, electromechanical switches, capacitive switches, knobs, microphones for receiving voice commands, etc. It should be understood that other systems for displaying information to the occupants of the main vehicle 12 are also within the scope of this disclosure. HMI 24 is in electrical communication with controller 20, as described above.

[0061] HUD 26 is used to provide information to the occupants of the host vehicle 12. In one exemplary embodiment, HUD 26 is configured to provide information to the occupants by projecting text, graphics, and / or images onto the windshield 44 of the host vehicle 12. In a non-limiting example, HUD 26 includes a projector (not shown) used by controller 20 to project text, graphics, and / or images onto the windshield 44 of the host vehicle 12. The text, graphics, and / or images are reflected by the windshield 44 of the host vehicle 12 and can be seen by the occupants without having to look away from the road 14 in front of the host vehicle 12. It should be understood that various types of head-up display devices, including, for example, augmented reality head-up display (AR-HUD) devices, are within the scope of this disclosure. In one exemplary embodiment, the occupants can interact with HUD 26 using a human-machine interface device (HID), which includes, for example, a touchscreen, electromechanical switch, capacitive switch, knob, microphone for receiving voice commands, etc. It should be understood that other systems for displaying information to the occupants of the host vehicle 12 are also within the scope of this disclosure. HUD 26 communicates electrically with controller 20 as described above.

[0062] refer to Figure 2 A flowchart of a method 100 for presenting a variety of options to a primary vehicle operator to maintain smooth driving during autonomous or driver-assisted driving is shown. In this disclosure, the term "operator" may be used interchangeably with the terms "driver" or "occupant." Method 100 begins at box 102.

[0063] Box 102 depicts receiving information at least in part based on the primary vehicle's environment. At box 102, controller 20 receives information about the primary vehicle 12 and surrounding traffic and road patterns. This information may include, for example, traffic participants, traffic light signals, primary vehicle route planning, the primary vehicle's recent speed distribution, driver preference models, etc. The driver preference model can be configured to tailor route recommendations based on individual driver preferences. A concrete example could be a Driver Preference-Based Route Planning (DPRP) model that collects data on driver preferences and uses those preferences to recommend optimal routes. DPRP integrates multiple objectives and attributes, such as traffic conditions, road type, and even driver-specific preferences, such as avoiding toll roads or preferring scenic routes. This information may also include at least the primary vehicle's speed, primary vehicle location, primary vehicle identifier (e.g., Vehicle Identification Number (VIN) and / or other temporary identifiers based on the Society of Automotive Engineers (SAE) messaging standard), or message time.

[0064] In a specific example, controller 20 may receive V2X (vehicle-to-everything) messages. In an additional example, controller 20 may receive onboard perception data from perception sensor 32 or other sensors. In one exemplary embodiment, the perception data includes the speed of the master vehicle 12 and position measurements of each vehicle within a 100-foot radius. In a non-limiting example, controller 20 uses camera 38 to perform one or more perception measurements to obtain perception data. In a non-limiting example, controller 20 uses radar sensor 40 to perform one or more perception measurements. In a non-limiting example, controller 20 uses LiDAR sensor 42 to perform one or more perception measurements.

[0065] In one exemplary embodiment, one or more messages are basic safety messages (BSMs) from each of one or more surrounding vehicles, including information such as location, speed, speed history or speed profile over time, acceleration, direction of travel, vehicle type, vehicle identifier, vehicle size, vehicle status, driver intent, message time, etc. In a non-limiting example, vehicle-to-vehicle (V2V) communication technologies (e.g., Dedicated Short Range Communication (DSRC)) are used to transmit one or more messages. In another non-limiting example, messages are relayed via one or more central servers (e.g., transmitted over the Internet or using cellular data communication). Following box 102, method 100 proceeds to box 104.

[0066] Box 104 depicts determining multiple first actions based on the information received in box 102. Controller 20 determines multiple first actions when the master vehicle 12 is at a suitable distance to estimate the position for gaining an advantage in an action. The multiple first actions include at least a first choice and a second choice. In some cases, the multiple first actions may include more than two choices (e.g., a third choice, a fourth choice, a fifth choice, etc.). In some cases, boxes 102 and 104 can be considered a preview phase. During this preview phase, the master vehicle 12 may be at an intermediate distance to estimate at least one position for gaining an advantage in an action, such that system 10 and / or controller 20 have clearly identified the action types, but may not yet have sufficient confidence in their basic properties (e.g., success probability). In one embodiment, determining the first choice and the second choice includes determining multiple master vehicle maneuvering options. Following box 104, method 100 proceeds to box 106.

[0067] Box 106 depicts the presentation of multiple first actions to the main vehicle operator. Controller 20 may use, for example, a human-machine interface 24 and / or a head-up display 26 to present the multiple first actions. Presentation may also include refreshing and re-determining the first and second selected actions until input is received from the main vehicle operator. It should be understood that controller 20 may present the multiple first actions to the main vehicle operator using various other methods, such as verbal presentation via a speaker. Box 106 can be considered an interaction phase. During the interaction phase, system 10 and / or controller 20 have sufficient confidence in the basic properties of the first and second selected actions, and both the first and second selected actions are within feasible spatial and / or temporal ranges.

[0068] In one example, system 10 and / or controller 20 require a driver configuration for a “challenging (but still potentially advantageous) action” with respect to the probability of success (e.g., in the range of 0.15–0.5, where 1 represents complete success and 0 represents failure). In another example, system 10 and / or controller 20 provide two options for an upcoming advantage-gaining action only if there is a certain “challenging action” expected to produce a sufficient advantage, and if this advantage is superior to the expected advantage of a less challenging action. In another example, system 10 and / or controller 20 select the “challenging action” with the best expected advantage (but with a success probability not less than a lower threshold threshold of the driver configuration (e.g., 0.15)) as the first selected action. In yet another example, system 10 and / or controller 20 require a driver configuration for a “comfortable (but still advantageous) action” with respect to the probability of success (e.g., in the range of 0.65–1, or at least the success probability of the first selected action plus 0.35). In another example, system 10 and / or controller 20 selects a "comfort action" with the best expected advantage (the type of which may differ from the type of best expected advantage of the first selected action) as the second selected action. In yet another example, system 10 and / or controller 20 optimizes the comparison between different types of advantages as part of a driver preference model using a learning-based approach.

[0069] refer to Figure 3 An exemplary view 80, observed from the interior of the main vehicle 12, is shown. Exemplary view 80 includes a controller 20 presenting a first and a second selection in the form of an exemplary HMI notification 84 displayed on an HMI 24. The exemplary HMI notification 84 includes a graphical depiction of a set of arrows representing the first and second selections. In this example, the HMI 24 may provide the driver with intuitive illustrations, such as HMI arrow positions indicating the location of the action, different colors for each arrow shape indicating the probability of success, and / or different widths for arrow shapes indicating the expected advantage. Following box 106, method 100 proceeds to box 108.

[0070] Box 108 depicts receiving input from the master vehicle operator based on a plurality of first actions. The master vehicle operator selects one of the plurality of first actions or makes no selection, and this selection is received as input by the controller 20 via, for example, HMI 24. Following box 108, method 100 proceeds to box 110.

[0071] Box 110 depicts determining a second action based on input received from the primary vehicle operator. Controller 20 determines the second action. The second action may include at least one of the following: not providing further system assistance to the primary vehicle operator, providing system assistance to the primary vehicle operator corresponding to the first selection, and / or providing system assistance to the primary vehicle operator corresponding to the second selection. Boxes 108 and 110 can be considered as the final stage. In this final stage, at least the first selection and / or the second selection has been rejected by the primary vehicle operator and / or invalidated by system 10 and / or controller 20 (e.g., beyond the scope of practice).

[0072] Figure 4 A first example scenario 85 of method 100 is shown. In this scenario, lane 1 and lane 2 are shown, with the primary vehicle 12 located in lane 1. To follow a predetermined route, the primary vehicle 12 preferably changes lane 1 to lane 2 and then turns right at the shown traffic light and intersection 86. The primary vehicle 12 is located in lane 1 to the left of the right-turn lane 2, and lane 2 is experiencing a queue of vehicles 88 into which the primary vehicle 12 must maneuver. As shown in box 102, based on received information at least in part based on the primary vehicle's environment (e.g., received V2X and / or onboard perception data, surrounding traffic patterns, primary vehicle steering selection), controller 20 infers (as shown in box 104) that the primary vehicle 12 may choose to change lanes at a first location 90 (e.g., 50 meters before intersection 86 to achieve optimal green light advantage and with sufficient probability of success) or at a second location 92 (e.g., 100 meters before intersection to achieve favorable green light advantage while having a significantly higher probability of success). According to box 104, based on driver preference and the recent speed distribution of the master vehicle 12, two options are designated as a first option and a second option, respectively. As in box 106, the controller 20 presents the first and second options and their comparison to the vehicle operator in advance (e.g., via HMI 24, via HUD 26) (e.g., in the form of visual illustrations of expected advantages, operational convenience, etc.). As in boxes 108 and 110, the controller 20 and / or system 10 receive input from the master vehicle operator and smoothly execute that input toward the preferred option according to subjective needs (e.g., maneuvering the master vehicle 12 into a first position 90, maneuvering the master vehicle 12 into a second position 92, and / or doing nothing).

[0073] Figure 5A second example scenario 93 of method 100 is shown. In this scenario, lanes 0 and 1 are shown, with the primary vehicle 12 located in lane 1. Lane 1 is controlled by traffic light 94, while lane 0 is not controlled by traffic light 94 (and has a more persistent free traffic flow). However, after passing the upcoming intersection 86 or merging point, the primary vehicle 12 needs to be in lane 1 to proceed in the desired direction. As shown in box 102, based on received information at least partially based on the primary vehicle's environment (e.g., received V2X and / or onboard perception data), controller 20 infers or determines (as shown in box 104) that a first option or a second option can be presented to the primary vehicle 12 and / or the primary vehicle operator (in box 106), wherein the first option is to follow the vehicle ahead in the current lane (i.e., lane 1) at a first position 90, and the second option is to change lanes (i.e., change to lane 0) at the nearest feasible position 96. The first choice action is advantageous if the main vehicle 12 will proceed through the intersection at a green light without coming to a complete stop in lane 1, while the second choice action is advantageous in other cases (e.g., if the main vehicle 12 stops at the intersection at a red light). By providing the first and second choices, the controller 20 enables the driver to smoothly become aware of the upcoming advantageous action and to adopt the driver's preferred choice through a confident observation of the driving environment.

[0074] Figure 6A third example scenario 97 of method 100 is shown. In this scenario, lanes 0 and 1 are shown, with the main vehicle 12 located in lane 1. Lanes 1 and 0 are controlled by traffic lights 94. The main vehicle 12 is located in lane 1, and there is a slowly moving truck 98 in front of the main vehicle 12. The main vehicle 12 can confidently pass through the green light in lane 1 at a constant speed (e.g., approximately 30 km / h), but cannot pass through at 60 km / h. The main vehicle 12 can confidently pass through the green light in lane 0 at approximately 60 km / h (but with a smaller time tolerance), and lane 0 can have a major influence on lane changes for incoming vehicles. As shown in box 102, based on received information at least partially based on the host vehicle's environment (e.g., received V2X and / or onboard perception data), controller 20 infers or determines (as shown in box 104) that a first option or a second option can be presented to host vehicle 12 and / or host vehicle operator (in box 106), wherein the first option is to maintain a constant speed (e.g., approximately 30 km / h) in the current lane (lane 1), and the second option is to change lanes from lane 1 to lane 0 at the nearest feasible location 96 and maintain a constant speed (e.g., approximately 60 km / h). If the lane change from lane 1 to lane 0 does not allow host vehicle 12 to pass through the green light at approximately 60 km / h, the first option is advantageous to host vehicle 12, while the second option is advantageous in other cases. By providing the first and second options, controller 20 makes the driver readily aware of the upcoming advantage-gaining action and adopts the driver's preferred choice through a confident observation of the driving environment.

[0075] Box 112 depicts the use of learning-based optimization based on input received from the primary vehicle operator. In this step, system 10 and / or controller 20 can use the vehicle operator's choices or, in the absence of choices, reinforce and optimize the machine learning driver preference model. The machine learning driver preference model can be configured to receive a set of driver preference parameters as input and provide an optimal vehicle handling plan as output.

[0076] Box 114 depicts providing system assistance by driving the master vehicle based on input from the vehicle operator. In this step, the controller 20 can use input from the master vehicle operator to cause the master vehicle 12 to be maneuvered, speed adjusted, or otherwise guided to change position and orientation according to the driver preferences indicated by the input.

[0077] The system 10 and method 100 of this disclosure offer several advantages. First, system 10 and method 100 are designed to avoid driver accidents and facilitate driver interaction and advantage-gaining actions while ensuring driver satisfaction. Method 100 and system 10 are designed to provide and assist smooth driving and / or green light passage toward a certain intermediate distance (typically the remaining distance to the traffic light or checkpoint ahead), rather than aiming for several intersections ahead or even further to the destination. Based on the driver's comfort and readiness, interaction selection is designed to leverage the driver's observations of the surrounding driving environment, adapt to the driver's anticipation and intuition for satisfactory advantage-gaining actions, and apply learning-based optimization to the driver preference model to improve overall satisfaction.

[0078] The descriptions in this disclosure are merely exemplary in nature, and variations thereof that do not depart from the spirit and scope of this disclosure are intended to fall within its scope. Such variations should not be considered as departing from the spirit and scope of this disclosure.

Claims

1. A method for presenting multiple options to a primary vehicle operator to maintain smooth driving during autonomous or driver-assisted driving of the primary vehicle, the method comprising: Receive information, which is at least in part based on the host vehicle environment; When the distance of the main vehicle is used to estimate the position of the advantage-gaining action at a certain distance, a plurality of first actions are determined based on the information, wherein the plurality of first actions includes at least a first choice and a second choice; Present the plurality of first actions to the main vehicle operator; Receive input from the main vehicle operator, the input being based on the plurality of first actions; as well as A second action is determined based on input from the master vehicle operator, wherein the second action includes at least one of the following: not providing further system assistance to the master vehicle operator, providing the master vehicle operator with system assistance corresponding to the first selection, or providing the master vehicle operator with system assistance corresponding to the second selection.

2. The method according to claim 1, wherein, The primary vehicle environment includes at least one of the following: surrounding traffic patterns, or primary vehicle steering selection.

3. The method according to claim 1, wherein, The information includes at least one of the following: V2X messages related to key information affecting traffic participants, traffic light signals, traffic patterns, vehicle perception data, primary vehicle route planning, primary vehicle recent speed profiles, or primary vehicle operator preference models.

4. The method according to claim 1, wherein, Determining the first and second selections includes determining multiple master vehicle control options.

5. The method according to claim 1, wherein, Determining multiple first actions includes determining a third option.

6. The method according to claim 1, wherein, Presenting the plurality of first actions to the main vehicle operator includes presenting a first option that has a greater advantage than the second option and presenting a second option that has a higher probability of success than the first option.

7. The method according to claim 1, wherein, Presenting the plurality of first actions to the master vehicle operator includes presenting the master vehicle operator with a suggested preview phase, the suggestions including at least one of the following: estimated location, action type, feasible space and time range, expected challenges, or major impact on each lane.

8. The method according to claim 1, wherein, The presentation includes refreshing and re-determining the first and second selection actions until input is received from the main vehicle operator.

9. The method according to claim 1, wherein, Presenting the plurality of first actions to the main vehicle operator includes presenting a notification of the plurality of first actions to the human-machine interface (HMI).

10. The method according to claim 1, wherein, Receiving the input includes receiving an instruction from the master vehicle operator, the instruction indicating at least one of the following: rejecting all of the plurality of first actions, rejecting one of the plurality of first actions, switching the order of the plurality of first actions, or no input being made.

Images