Vehicle-mounted multi-mode navigation interaction method, system and equipment and storage medium

Abstract

The invention relates to the technical field of automobile control, in particular to a vehicle-mounted multi-mode navigation interaction method, system and equipment and a storage medium. The vehicle-mounted multi-mode navigation system comprises: a navigation prompt module, which is used for outputting a driving navigation prompt; the safety early warning module is used for outputting a driving safety early warning prompt; the head display module is used for projecting a navigation picture or a safety early warning picture on a vehicle window; the steering wheel tactile feedback module is used for providing navigation prompt vibration feedback or safety early warning vibration feedback on a steering wheel; and the controller is used for adjusting the working modes of the head display module and the steering wheel tactile feedback module according to the driving navigation prompt and the driving safety early warning prompt. According to the embodiment of the invention, by constructing a multi-mode navigation interaction system framework and integrating visual and tactile information transmission channels, the dependence of a driver on a central control screen is remarkably reduced, the sight line transfer frequency is reduced, the distraction driving problem caused by a traditional navigation system is effectively relieved, and the driving safety is improved.

Description

Technical Field

[0001] This application relates to the field of automotive control technology, specifically to an in-vehicle multimodal navigation interaction method, system, device, and storage medium. Background Technology

[0002] With the rapid development of automotive intelligent technology, navigation and in-vehicle infotainment systems have gradually become important factors influencing users' car selection. Traditional in-vehicle navigation systems primarily provide navigation guidance to drivers by combining a central control screen with voice prompts. These systems typically mount the central control screen on the vehicle's dashboard, displaying maps and arrow indicators to guide driving directions, while simultaneously providing voice prompts through the in-vehicle audio system to inform the driver of key information such as driving direction and distance. However, because the central control screen is positioned outside the driver's normal line of sight, the driver must frequently shift their gaze while driving, actively turning their head and eyes to check the screen information, resulting in constant switching of eyes between the road and the central control screen. This frequent shifting of gaze not only increases the driver's cognitive load but also significantly increases the risk of distracted driving, easily leading to traffic accidents, especially at high speeds or in complex road conditions.

[0003] Among related technologies, AR-HUD systems can project navigation information into the driver's field of vision, reducing eye deviation. Meanwhile, some high-end models have begun to adopt haptic feedback technology, transmitting information to the driver through vibration. However, visual and auditory cues are relatively simple and lack differentiated information delivery mechanisms adapted to different driving scenarios. In complex road conditions or noisy environments, drivers may miss crucial cues due to distraction, leading to missed lane changes or other important maneuvers. Summary of the Invention

[0004] In related technologies, the visual and auditory prompts in vehicle navigation systems are limited and lack differentiated multimodal information transmission mechanisms for different driving scenarios. As a result, drivers may easily overlook key navigation prompts in complex road conditions or high ambient noise conditions, thus missing important driving operations such as lane changes.

[0005] In a first aspect, embodiments of this application provide an in-vehicle multimodal navigation interaction system, the in-vehicle multimodal navigation interaction system comprising: The navigation prompt module is used to monitor the vehicle's navigation information and output driving navigation prompts; The safety warning module is used to monitor the vehicle's driving safety status and output driving safety warning prompts; Head-mounted display module, which is used to project navigation images or safety warning images onto the vehicle window; Steering wheel haptic feedback module, which is used to provide navigation prompt vibration feedback or safety warning vibration feedback on the steering wheel; A controller is connected to the head-mounted display module and the steering wheel haptic feedback module via signals. The controller is used to adjust the working mode of the head-mounted display module and the steering wheel haptic feedback module according to driving navigation prompts and driving safety warning prompts.

[0006] In conjunction with the first aspect, in one embodiment, the controller is used to adjust the operating modes of the head-mounted display module and the steering wheel haptic feedback module according to driving navigation prompts and driving safety warning prompts, including: When the controller receives only the driving navigation prompts but not the driving safety warning prompts, the controller drives the head-mounted display module to project the navigation image onto the windshield according to the driving navigation prompts, and simultaneously drives the steering wheel haptic feedback module to perform the corresponding navigation prompt vibration feedback. When the controller receives both navigation prompts and driving safety warnings, it drives the head-mounted display module to project a safety warning image onto the windshield based on the safety warning prompt, and simultaneously drives the steering wheel haptic feedback module to perform the corresponding safety warning vibration feedback action.

[0007] In conjunction with the first aspect, in one embodiment, the steering wheel haptic feedback module includes at least six independently configured linear vibration motors for embedding in the steering wheel, and the six linear vibration motors are evenly arranged around the center point of the steering wheel.

[0008] In conjunction with the first aspect, in one embodiment, the head-mounted display module includes: The projection assembly includes a light guide plate and an LED array, the light guide plate being used in conjunction with the LED array to project an image onto the vehicle windshield; The eye-tracking component is connected to the light guide plate signal. The eye-tracking component is used to capture the position of the driver's pupils in real time and drive the light guide plate to adjust the refraction angle according to the position of the driver's pupils.

[0009] In conjunction with the first aspect, in one embodiment, the head-mounted display module further includes a stepper motor connected to the light guide plate, the stepper motor being used to adjust the refraction angle of the light guide plate.

[0010] In conjunction with the first aspect, in one implementation, the safety warning module includes: a lane departure warning component.

[0011] Secondly, embodiments of this application provide an in-vehicle multimodal navigation interaction method utilizing the aforementioned in-vehicle multimodal navigation interaction system, comprising: The navigation prompt module and the safety warning module collect driving navigation information and driving safety status, respectively. Determine whether to output driving navigation prompts and / or driving safety warnings based on driving navigation information and driving safety status; The controller activates the head-mounted display module and the steering wheel haptic feedback module based on the navigation prompts and / or safety warnings received from the controller.

[0012] In conjunction with the second aspect, in one embodiment, the step of driving the head-mounted display module and the steering wheel haptic feedback module to operate based on the driving navigation prompts and / or driving safety warning prompts received by the controller includes: When the controller receives only the driving navigation prompts but not the driving safety warning prompts, the controller drives the head-mounted display module to project the navigation image onto the windshield according to the driving navigation prompts, and simultaneously drives the steering wheel haptic feedback module to perform the corresponding navigation prompt vibration feedback. When the controller receives both navigation prompts and driving safety warnings, it drives the head-mounted display module to project a safety warning image onto the windshield based on the safety warning prompt, and simultaneously drives the steering wheel haptic feedback module to perform the corresponding safety warning vibration feedback action.

[0013] In conjunction with the second aspect, in one embodiment, the controller, based on a safety warning prompt, drives the head-mounted display module to project a safety warning image onto the vehicle window and simultaneously drives the steering wheel haptic feedback module to execute a corresponding safety warning vibration feedback action, including: The current driving status is classified according to the safety warning prompts to obtain the current warning level; Based on the current warning level, the head-mounted display module projects the corresponding safety warning image onto the windshield, and simultaneously drives the steering wheel haptic feedback module to execute the safety warning vibration feedback corresponding to the current warning level.

[0014] In conjunction with the second aspect, in one implementation, the step of classifying the current driving status according to the safety warning prompt to obtain the current warning level includes: Based on the safety warning prompts, the current warning level is classified into three levels; among them... When a safety warning indicates that a vehicle is approaching a potentially dangerous area, the current warning level will be classified as Level 1 warning. When a safety warning indicates that the vehicle is showing a slight tendency to deviate from its lane, the current warning level will be classified as a Level 2 warning. When a safety warning indicates that a vehicle is about to be involved in an emergency collision, the current warning level will be classified as a Level 3 warning.

[0015] In conjunction with the second aspect, in one implementation, the step of driving the head-mounted display module to project a corresponding safety warning image onto the windshield based on the current warning level, and simultaneously driving the steering wheel haptic feedback module to execute safety warning vibration feedback corresponding to the current warning level, includes: When the current warning level is Level 1, the head-mounted display module projects a yellow warning message onto the windshield, and simultaneously drives the linear vibration motor of the steering wheel haptic feedback module to vibrate at a low frequency. When the current warning level is Level 3, the head-mounted display module projects a red warning message onto the windshield, and simultaneously drives the linear vibration motor of the steering wheel haptic feedback module to vibrate at high frequency.

[0016] In conjunction with the second aspect, in one implementation, the step of driving the head-mounted display module to project a corresponding safety warning image onto the windshield based on the current warning level and simultaneously driving the steering wheel haptic feedback module to execute safety warning vibration feedback corresponding to the current warning level further includes: When the current warning level is Level II, the head-mounted display module projects a yellow warning message onto the windshield, and simultaneously drives the steering wheel haptic feedback module to perform mid-frequency vibration on the linear vibration motor on the side deviating from the direction.

[0017] In conjunction with the second aspect, in one implementation, the use of the navigation prompt module and the safety warning module to collect driving navigation information and driving safety status respectively includes: using a lane departure warning component to monitor lane departure trends.

[0018] Thirdly, embodiments of this application provide an in-vehicle multimodal navigation interaction device, the in-vehicle multimodal navigation interaction device including a processor, a memory, and an in-vehicle multimodal navigation interaction program stored in the memory and executable by the processor, wherein when the in-vehicle multimodal navigation interaction program is executed by the processor, it implements the steps of the in-vehicle multimodal navigation interaction method as described in any of the above claims.

[0019] Fourthly, embodiments of this application provide a computer-readable storage medium storing an in-vehicle multimodal navigation interaction program, wherein when the in-vehicle multimodal navigation interaction program is executed by a processor, it implements the steps of the in-vehicle multimodal navigation interaction method as described in any of the preceding claims.

[0020] The beneficial effects of the technical solutions provided in this application include: This application embodiment constructs a multimodal navigation interaction system framework, integrating two information transmission channels: visual (head-mounted display module) and tactile (steering wheel tactile feedback). This significantly reduces the driver's reliance on the central control screen, lowers the frequency of eye shifts, effectively alleviates the distracted driving problem caused by traditional navigation systems, and improves driving safety. Attached Figure Description

[0021] Figure 1 This is a flowchart illustrating an embodiment of the in-vehicle multimodal navigation interaction method of this application; Figure 2This is a controller control logic diagram of the in-vehicle multimodal navigation interaction system in the embodiments of this application; Figure 3 This is a schematic diagram of the steering wheel haptic feedback module in the installation state in an embodiment of this application; Figure 4 This is a schematic diagram of the hardware structure of the in-vehicle multimodal navigation and interaction device involved in the embodiments of this application. Detailed Implementation

[0022] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0023] In related technologies, the visual and auditory prompts in vehicle navigation systems are limited and lack differentiated multimodal information transmission mechanisms for different driving scenarios. As a result, drivers may easily overlook key navigation prompts in complex road conditions or high ambient noise conditions, thus missing important driving operations such as lane changes.

[0024] It's worth noting that among related technologies, AR-HUD systems can project navigation information into the driver's field of vision, reducing eye deviation. Meanwhile, some high-end models have begun to adopt haptic feedback technology, transmitting information to the driver through vibration. However, visual and auditory cues are relatively simplistic and lack differentiated information delivery mechanisms adapted to different driving scenarios. In complex road conditions or noisy environments, drivers may miss crucial cues due to distraction, leading to missed lane changes or other important maneuvers. Furthermore, these technologies are typically developed and deployed as independent systems, lacking effective inter-system coordination mechanisms. Particularly between vehicle safety systems (such as Lane Departure Warning System (LDWS) and Automatic Emergency Braking (AEB) systems) and navigation systems, there are often issues with inconsistent information transmission standards and incompatible interaction logic, potentially causing drivers to receive conflicting information from different systems in complex driving environments.

[0025] In a first aspect, embodiments of this application provide an in-vehicle multimodal navigation interaction system, the in-vehicle multimodal navigation interaction system comprising: a navigation prompt module, a safety warning module, a head-mounted display module, a steering wheel haptic feedback module, and a controller; wherein... The system includes a navigation prompt module for monitoring vehicle navigation information and outputting navigation prompts; a safety warning module for monitoring vehicle driving safety status and outputting driving safety warnings; a head-mounted display module for projecting navigation or safety warning images onto the windows; a steering wheel haptic feedback module for providing vibration feedback for navigation prompts or safety warnings on the steering wheel; and a controller connected to the head-mounted display module and the steering wheel haptic feedback module, which adjusts the operating modes of the head-mounted display module and the steering wheel haptic feedback module according to the navigation prompts and driving safety warnings.

[0026] Optionally, the safety warning module includes a lane departure warning component (LDWS), which is used to monitor whether the vehicle deviates from its lane.

[0027] It is worth noting that the above embodiments of this application, by constructing a multimodal navigation interaction system framework, integrate two information transmission channels: visual (head-mounted display module) and tactile (steering wheel tactile feedback), significantly reducing the driver's dependence on the central control screen, reducing the frequency of eye shifts, effectively alleviating the distracted driving problem caused by traditional navigation systems, and improving driving safety.

[0028] In some specific embodiments, the steering wheel haptic feedback module includes at least six independently configured linear vibration motors, which are embedded in the steering wheel and are evenly distributed around the center point of the steering wheel. Each linear vibration motor is independently controlled.

[0029] Understandably, the tactile coding of orientation perception is achieved through six evenly distributed linear vibration motors, allowing the driver to accurately identify the navigation direction without shifting their gaze (e.g., vibration of the left motor indicates a left turn), improving the accuracy and understandability of navigation prompts and significantly reducing the risk of distraction caused by shifting gaze.

[0030] Optionally, the six linear vibration motors are located at the 3 o'clock, 6 o'clock, 9 o'clock, 12 o'clock and two intermediate transition positions on the steering wheel.

[0031] It is worth noting that the vibration logic of multiple linear vibration motors is linked with navigation commands and safety warning modules (such as the Lane Departure Warning System (LDWS)). For example, in navigation prompt mode, two consecutive vibrations of the left motor indicate a suggestion to change lanes to the left, while three intermittent vibrations of the right motor indicate a suggestion to change lanes to the right. In safety warning mode, high-frequency synchronous vibrations of both motors indicate a risk of lane departure, while continuous vibrations of all motors indicate an emergency collision warning.

[0032] In some preferred embodiments, the head-mounted display module includes: a projection component and an eye-tracking component; wherein, The projection component includes a light guide plate and an LED array. The light guide plate works with the LED array to project the image onto the vehicle's windshield. The eye-tracking component is connected to the light guide plate. The eye-tracking component is used to capture the driver's pupil position in real time and drive the light guide plate to adjust the refraction angle according to the driver's pupil position.

[0033] It is worth noting that by combining eye tracking with mechanical adjustment of the light guide plate, the AR-HUD projection area dynamically adapts to the driver's field of vision. The driver can clearly obtain information without frequently adjusting their gaze, which significantly reduces the risk of distraction caused by gaze shift and improves driving safety and comfort.

[0034] Furthermore, a wedge-shaped light guide plate is combined with a micro-LED array (RGB three colors). The focal point of the field of vision is calculated based on the pupil position, and a stepper motor drives the light guide plate to adjust the refraction angle, so that the projection area dynamically matches the driver's field of vision. Simultaneously, the micro-LED array has a dynamic color-changing function, switching colors according to lane congestion status. In some optional embodiments, a green arrow indicates a recommended clear lane, a red arrow indicates a congested target lane suggesting an alternative route, and a flashing yellow arrow indicates that the lane is about to close or is under construction.

[0035] Optionally, the controller connects to the vehicle's CAN bus and receives signals such as the LDWS (Lane Departure Warning System) to achieve priority coordination between the AR-HUD, haptic feedback, and LDWS. When a safety signal such as lane departure is received from the LDWS, the safety warning mode is triggered first, and navigation prompts are paused to avoid information conflicts.

[0036] Furthermore, the in-vehicle multimodal navigation and interaction system also includes a voice prompt component, which is connected to the controller signal.

[0037] It is worth noting that the system uses voice commands in conjunction with the head-mounted display and steering wheel haptic feedback module to provide coordinated visual, tactile, and auditory alerts, enabling drivers to receive timely and comprehensive reminders.

[0038] In some preferred embodiments, the controller is used to adjust the operating mode of the head-mounted display module and the steering wheel haptic feedback module according to driving navigation prompts and driving safety warning prompts, including multiple judgment situations: Scenario 1: When the controller only receives navigation prompts but not driving safety warnings, it determines that the vehicle is in navigation mode and drives the head-mounted display module and steering wheel haptic feedback module to perform corresponding operations according to the navigation prompts.

[0039] Specifically, the controller drives the head-mounted display module to project the navigation image onto the car window based on the driving navigation prompts, and simultaneously drives the steering wheel haptic feedback module to perform corresponding navigation prompt vibration feedback.

[0040] Scenario 2: When the controller receives both the navigation prompt and the driving safety warning prompt at the same time, it determines that the vehicle is in warning mode and drives the head-mounted display module and the steering wheel haptic feedback module to perform corresponding functions according to the navigation prompt.

[0041] Specifically, the controller drives the head-mounted display module to project a safety warning image onto the car window based on the safety warning prompt, and simultaneously drives the steering wheel haptic feedback module to perform the corresponding safety warning vibration feedback action.

[0042] In some specific embodiments, the control logic for case two above includes: Step S1: Classify the current driving status according to the safety warning prompts to obtain the current warning level.

[0043] Specifically, based on the security warning prompts, the current warning level is classified into three warning levels; among them, When a safety warning indicates that a vehicle is approaching a potentially dangerous area (e.g., road construction ahead that has not yet affected normal driving), the current warning level is classified as Level 1. When a safety warning indicates that a vehicle is showing a slight tendency to deviate from its lane (but has not yet triggered the strict lane departure warning of the LDWS), the current warning level is classified as Level 2. When a safety warning indicates that a vehicle is about to have an emergency collision or has seriously deviated from its lane and is approaching a dangerous area, the current warning level is classified as Level 3.

[0044] Step S2: Based on the current warning level, drive the head-mounted display module to project the corresponding safety warning image onto the windshield, and simultaneously drive the steering wheel haptic feedback module to execute the safety warning vibration feedback corresponding to the current warning level.

[0045] The above step S2 specifically includes: Scenario A: When the current warning level is Level 1, the head-mounted display module projects a yellow warning message onto the windshield, and simultaneously drives the linear vibration motor of the steering wheel haptic feedback module to vibrate at a low frequency.

[0046] Specifically, a portion of the motor on the steering wheel corresponding to the area in front vibrates at a low frequency to alert the driver that road conditions ahead have changed and that they need to remain vigilant. This vibration is weak and will not interfere with the driver's normal driving operations, but it is enough to attract their attention. Combined with the yellow warning information displayed on the head-up display module (AR-HUD), the driver obtains information from both visual and tactile multimodal perspectives, enhancing their perception of potential hazards.

[0047] Scenario B: When the current warning level is Level 2, the head-mounted display module projects a flashing yellow arrow, and multiple motors on the side of the steering wheel that are deviating from the lane begin to vibrate at a medium frequency. The vibration lasts for a slightly longer duration than that of Level 1 warning, allowing the driver to clearly perceive the deviation in the vehicle's driving direction, adjust the steering wheel in time, avoid further deviation from the lane, and reduce the risk of accidents caused by lane departure.

[0048] Situation C: When the current warning level is Level 3, the head-mounted display module projects a red warning message onto the windshield, and simultaneously drives the linear vibration motor of the steering wheel haptic feedback module to vibrate at high frequency.

[0049] Specifically, all linear vibration motors vibrate continuously at high frequency and intensity, while the head-mounted display module shows a prominent red warning message, and the voice also issues an urgent alarm.

[0050] Understandably, in a Level 3 emergency, comprehensive and high-intensity multimodal warnings enable drivers to react quickly and take measures such as braking or evasive maneuvers, maximizing driving safety. Through precise programming and coordinated operation of multiple independently controlled motors, combined with AR-HUD and voice prompts, an efficient, accurate, and comprehensive in-vehicle multimodal warning and interaction system can be built, enhancing driving safety and the user experience.

[0051] Furthermore, in addition to the aforementioned warning prompts, the human-machine interaction collaboration logic of conventional navigation adopts a tiered prompt strategy: Level 1 prompt (500 meters ahead), the AR-HUD displays a green arrow, and a gentle voice reminder is given; Level 2 prompt (300 meters ahead), the AR arrow turns yellow and flashes, and the motor on the corresponding side of the steering wheel triggers a single vibration; Level 3 prompt (100 meters ahead), the AR arrow turns red, a rapid voice warning is given, and the motor vibrates continuously until the lane change is completed.

[0052] In summary, this application addresses the following aspects: Regarding reducing distraction risk, the system employs eye-tracking and light guide plate mechanical adjustment technology, enabling the AR-HUD projection area to dynamically adapt to the driver's field of vision. This eliminates the need for the driver to frequently shift their gaze to the central control screen, effectively reducing distraction risks caused by eye movement and significantly improving driving safety. In terms of enhancing prompt accuracy, the system encodes navigation commands and safety warnings into clearly distinguishable tactile signals based on the precise positional distribution and vibration timing control of the steering wheel motor. Different vibration patterns with varying positions, frequencies, and intensities correspond to different information types, allowing the driver to quickly and accurately identify command content through touch without shifting their gaze. Regarding enhanced safety, the control module achieves intelligent priority coordination between the AR-HUD, tactile feedback, and the vehicle safety system. When safety risks such as lane departure are detected, the system immediately interrupts non-emergency navigation prompts and prioritizes triggering the safety warning mechanism, ensuring that critical safety information is delivered to the driver immediately. This reduces the driver's response time to safety risks by 30%, significantly improving driving safety. These three technologies work synergistically to construct an efficient, safe, and user-friendly in-vehicle multimodal interaction system.

[0053] Secondly, this application provides a vehicle comprising: an in-vehicle multimodal navigation and interaction system, the in-vehicle multimodal navigation and interaction system comprising: a navigation prompt module, a safety warning module, a head-up display module, a steering wheel haptic feedback module, and a controller; wherein; The system includes a navigation prompt module for monitoring vehicle navigation information and outputting navigation prompts; a safety warning module for monitoring vehicle driving safety status and outputting driving safety warnings; a head-mounted display module for projecting navigation or safety warning images onto the windows; a steering wheel haptic feedback module for providing vibration feedback for navigation prompts or safety warnings on the steering wheel; and a controller connected to the head-mounted display module and the steering wheel haptic feedback module, which adjusts the operating modes of the head-mounted display module and the steering wheel haptic feedback module according to the navigation prompts and driving safety warnings.

[0054] Optionally, the safety warning module includes a lane departure warning component (LDWS), which is used to monitor whether the vehicle deviates from its lane.

[0055] It is worth noting that the above embodiments of this application, by constructing a multimodal navigation interaction system framework, integrate two information transmission channels: visual (head-mounted display module) and tactile (steering wheel tactile feedback), significantly reducing the driver's dependence on the central control screen, reducing the frequency of eye shifts, effectively alleviating the distracted driving problem caused by traditional navigation systems, and improving driving safety.

[0056] In some specific embodiments, the steering wheel haptic feedback module includes at least six independently configured linear vibration motors for embedding in the steering wheel. In some optional embodiments, such as... Figure 3 The six linear vibration motors shown are evenly distributed around the center point of the steering wheel. Each linear vibration motor is controlled independently.

[0057] Understandably, the use of six evenly distributed linear vibration motors to achieve tactile encoding for orientation perception allows drivers to accurately identify navigation directions without shifting their gaze (e.g., vibration of the left motor indicates a left turn), improving the accuracy and understandability of navigation prompts and significantly reducing the risk of distraction caused by shifting gaze. This application's multiple independently controlled motors can further refine commands. In addition to left and right lane change prompts, unique vibration modes can be set for different types of roads (e.g., highways, city streets) and different driving scenarios (e.g., roundabouts, intersections). When the vehicle approaches a roundabout, motors at specific positions simulate the direction the vehicle is about to enter, vibrating sequentially. This allows the driver to anticipate driving actions through touch, reducing reliance on visual information and thus improving driving safety in complex road conditions.

[0058] Optionally, the six linear vibration motors are located at the 3 o'clock, 6 o'clock, 9 o'clock, 12 o'clock and two intermediate transition positions on the steering wheel.

[0059] It is worth noting that the vibration logic of multiple linear vibration motors is linked with navigation commands and safety warning modules (such as the Lane Departure Warning System (LDWS)). For example, in navigation prompt mode, two consecutive vibrations of the left motor indicate a suggestion to change lanes to the left, while three intermittent vibrations of the right motor indicate a suggestion to change lanes to the right. In safety warning mode, high-frequency synchronous vibrations of both motors indicate a risk of lane departure, while continuous vibrations of all motors indicate an emergency collision warning.

[0060] In some preferred embodiments, the head-mounted display module includes: a projection component and an eye-tracking component; wherein, The projection component includes a light guide plate and an LED array. The light guide plate works with the LED array to project the image onto the vehicle's windshield. The eye-tracking component is connected to the light guide plate. The eye-tracking component is used to capture the driver's pupil position in real time and drive the light guide plate to adjust the refraction angle according to the driver's pupil position.

[0061] It is worth noting that by combining eye tracking with mechanical adjustment of the light guide plate, the AR-HUD projection area dynamically adapts to the driver's field of vision. The driver can clearly obtain information without frequently adjusting their gaze, which significantly reduces the risk of distraction caused by gaze shift and improves driving safety and comfort.

[0062] Furthermore, a wedge-shaped light guide plate is combined with a micro-LED array (RGB three colors). The focal point of the field of vision is calculated based on the pupil position, and a stepper motor drives the light guide plate to adjust the refraction angle, so that the projection area dynamically matches the driver's field of vision. Simultaneously, the micro-LED array has a dynamic color-changing function, switching colors according to lane congestion status. In some optional embodiments, a green arrow indicates a recommended clear lane, a red arrow indicates a congested target lane suggesting an alternative route, and a flashing yellow arrow indicates that the lane is about to close or is under construction.

[0063] Optionally, the controller connects to the vehicle's CAN bus and receives signals such as the LDWS (Lane Departure Warning System) to achieve priority coordination between the AR-HUD, haptic feedback, and LDWS. When a safety signal such as lane departure is received from the LDWS, the safety warning mode is triggered first, and navigation prompts are paused to avoid information conflicts.

[0064] Furthermore, the in-vehicle multimodal navigation and interaction system also includes a voice prompt component, which is connected to the controller signal.

[0065] It is worth noting that the system uses voice commands in conjunction with the head-mounted display and steering wheel haptic feedback module to provide coordinated visual, tactile, and auditory alerts, enabling drivers to receive timely and comprehensive reminders.

[0066] In some preferred embodiments, the controller is used to adjust the operating mode of the head-mounted display module and the steering wheel haptic feedback module according to driving navigation prompts and driving safety warning prompts, including multiple judgment situations: Scenario 1: When the controller only receives navigation prompts but not driving safety warnings, it determines that the vehicle is in navigation mode and drives the head-mounted display module and steering wheel haptic feedback module to perform corresponding operations according to the navigation prompts.

[0067] Specifically, the controller drives the head-mounted display module to project the navigation image onto the car window based on the driving navigation prompts, and simultaneously drives the steering wheel haptic feedback module to perform corresponding navigation prompt vibration feedback.

[0068] Scenario 2: When the controller receives both the navigation prompt and the driving safety warning prompt at the same time, it determines that the vehicle is in warning mode and drives the head-mounted display module and the steering wheel haptic feedback module to perform corresponding functions according to the navigation prompt.

[0069] Specifically, the controller drives the head-mounted display module to project a safety warning image onto the car window based on the safety warning prompt, and simultaneously drives the steering wheel haptic feedback module to perform the corresponding safety warning vibration feedback action.

[0070] In some embodiments, when driving on a narrow road, if the right side of the vehicle is close to an obstacle, multiple motors at the corresponding position on the right side can vibrate simultaneously or sequentially, and the intensity and number of vibrations can be dynamically adjusted according to the distance, allowing the driver to clearly perceive the specific location and urgency of the danger coming from the right side of the vehicle.

[0071] In a specific embodiment provided in this application, the control logic for the second situation described above includes: Step S1: Classify the current driving status according to the safety warning prompts to obtain the current warning level.

[0072] Specifically, based on the security warning prompts, the current warning level is classified into three warning levels; among them, When a safety warning indicates that a vehicle is approaching a potentially dangerous area (e.g., road construction ahead that has not yet affected normal driving), the current warning level is classified as Level 1. When a safety warning indicates that a vehicle is showing a slight tendency to deviate from its lane (but has not yet triggered the strict lane departure warning of the LDWS), the current warning level is classified as Level 2. When a safety warning indicates that a vehicle is about to have an emergency collision or has seriously deviated from its lane and is approaching a dangerous area, the current warning level is classified as Level 3.

[0073] Step S2: Based on the current warning level, drive the head-mounted display module to project the corresponding safety warning image onto the windshield, and simultaneously drive the steering wheel haptic feedback module to execute the safety warning vibration feedback corresponding to the current warning level.

[0074] The above step S2 specifically includes: Scenario A: When the current warning level is Level 1, the head-mounted display module projects a yellow warning message onto the windshield, and simultaneously drives the linear vibration motor of the steering wheel haptic feedback module to vibrate at a low frequency.

[0075] Specifically, a portion of the motor on the steering wheel corresponding to the area in front vibrates at a low frequency to alert the driver that road conditions ahead have changed and that they need to remain vigilant. This vibration is weak and will not interfere with the driver's normal driving operations, but it is enough to attract their attention. Combined with the yellow warning information displayed on the head-up display module (AR-HUD), the driver obtains information from both visual and tactile multimodal perspectives, enhancing their perception of potential hazards.

[0076] Scenario B: When the current warning level is Level 2, the head-mounted display module projects a flashing yellow arrow, and multiple motors on the side of the steering wheel that are deviating from the lane begin to vibrate at a medium frequency. The vibration lasts for a slightly longer duration than that of Level 1 warning, allowing the driver to clearly perceive the deviation in the vehicle's driving direction, adjust the steering wheel in time, avoid further deviation from the lane, and reduce the risk of accidents caused by lane departure.

[0077] Situation C: When the current warning level is Level 3, the head-mounted display module projects a red warning message onto the windshield, and simultaneously drives the linear vibration motor of the steering wheel haptic feedback module to vibrate at high frequency.

[0078] Specifically, all linear vibration motors vibrate continuously at high frequency and intensity, while the head-mounted display module shows a prominent red warning message, and the voice also issues an urgent alarm.

[0079] Understandably, in a Level 3 emergency, comprehensive and high-intensity multimodal warnings enable drivers to react quickly and take measures such as braking or evasive maneuvers, maximizing driving safety. Through precise programming and coordinated operation of multiple independently controlled motors, combined with AR-HUD and voice prompts, an efficient, accurate, and comprehensive in-vehicle multimodal warning and interaction system can be built, enhancing driving safety and the user experience.

[0080] Furthermore, in addition to the aforementioned warning prompts, the human-machine interaction collaboration logic of conventional navigation adopts a tiered prompt strategy: Level 1 prompt (500 meters ahead), the AR-HUD displays a green arrow, and a gentle voice reminder is given; Level 2 prompt (300 meters ahead), the AR arrow turns yellow and flashes, and the motor on the corresponding side of the steering wheel triggers a single vibration; Level 3 prompt (100 meters ahead), the AR arrow turns red, a rapid voice warning is given, and the motor vibrates continuously until the lane change is completed.

[0081] Thirdly, this application provides an in-vehicle multimodal navigation interaction method utilizing the aforementioned in-vehicle multimodal navigation interaction system, comprising: Step S1: Use the navigation prompt module and the safety warning module to collect driving navigation information and driving safety status respectively; Step S2: Determine whether to output driving navigation prompts and / or driving safety warnings based on driving navigation information and driving safety status.

[0082] Step S3: Drive the head-mounted display module and steering wheel haptic feedback module to operate according to the driving navigation prompts and / or driving safety warning prompts received by the controller.

[0083] The above step S3 specifically includes: Scenario A: When the current warning level is Level 1, the head-mounted display module projects a yellow warning message onto the windshield, and simultaneously drives the linear vibration motor of the steering wheel haptic feedback module to vibrate at a low frequency.

[0084] Specifically, a portion of the motor on the steering wheel corresponding to the area in front vibrates at a low frequency to alert the driver that road conditions ahead have changed and that they need to remain vigilant. This vibration is weak and will not interfere with the driver's normal driving operations, but it is enough to attract their attention. Combined with the yellow warning information displayed on the head-up display module (AR-HUD), the driver obtains information from both visual and tactile multimodal perspectives, enhancing their perception of potential hazards.

[0085] Scenario B: When the current warning level is Level 2, the head-mounted display module projects a flashing yellow arrow, and multiple motors on the side of the steering wheel that are deviating from the lane begin to vibrate at a medium frequency. The vibration lasts for a slightly longer duration than that of Level 1 warning, allowing the driver to clearly perceive the deviation in the vehicle's driving direction, adjust the steering wheel in time, avoid further deviation from the lane, and reduce the risk of accidents caused by lane departure.

[0086] Situation C: When the current warning level is Level 3, the head-mounted display module projects a red warning message onto the windshield, and simultaneously drives the linear vibration motor of the steering wheel haptic feedback module to vibrate at high frequency.

[0087] Specifically, all linear vibration motors vibrate continuously at high frequency and intensity, while the head-mounted display module shows a prominent red warning message, and the voice also issues an urgent alarm.

[0088] Understandably, in a Level 3 emergency, comprehensive and high-intensity multimodal warnings enable drivers to react quickly and take measures such as braking or evasive maneuvers, maximizing driving safety. Through precise programming and coordinated operation of multiple independently controlled motors, combined with AR-HUD and voice prompts, an efficient, accurate, and comprehensive in-vehicle multimodal warning and interaction system can be built, enhancing driving safety and the user experience.

[0089] Furthermore, in addition to the aforementioned warning prompts, the human-machine interaction collaboration logic of conventional navigation adopts a tiered prompt strategy: Level 1 prompt (500 meters ahead), the AR-HUD displays a green arrow, and a gentle voice reminder is given; Level 2 prompt (300 meters ahead), the AR arrow turns yellow and flashes, and the motor on the corresponding side of the steering wheel triggers a single vibration; Level 3 prompt (100 meters ahead), the AR arrow turns red, a rapid voice warning is given, and the motor vibrates continuously until the lane change is completed.

[0090] Fourthly, embodiments of this application also provide an in-vehicle multimodal navigation interaction device, which includes: an information collection unit, a judgment unit, and a control unit; wherein, The information collection unit collects driving navigation information and driving safety status using the navigation prompt module and safety warning module, respectively. The judgment unit determines whether to output driving navigation prompts and / or driving safety warnings based on the driving navigation information and driving safety status. The control unit drives the head-up display module and steering wheel haptic feedback module to operate based on the driving navigation prompts and / or driving safety warnings received from the controller.

[0091] The functions of each module in the aforementioned in-vehicle multimodal navigation interaction device are used to implement the in-vehicle multimodal navigation interaction method, which includes: Step S1: Use the navigation prompt module and the safety warning module to collect driving navigation information and driving safety status respectively; Step S2: Determine whether to output driving navigation prompts and / or driving safety warnings based on driving navigation information and driving safety status.

[0092] Step S3: Drive the head-mounted display module and steering wheel haptic feedback module to operate according to the driving navigation prompts and / or driving safety warning prompts received by the controller.

[0093] The above step S3 specifically includes: Scenario A: When the current warning level is Level 1, the head-mounted display module projects a yellow warning message onto the windshield, and simultaneously drives the linear vibration motor of the steering wheel haptic feedback module to vibrate at a low frequency.

[0094] Specifically, a portion of the motor on the steering wheel corresponding to the area in front vibrates at a low frequency to alert the driver that road conditions ahead have changed and that they need to remain vigilant. This vibration is weak and will not interfere with the driver's normal driving operations, but it is enough to attract their attention. Combined with the yellow warning information displayed on the head-up display module (AR-HUD), the driver obtains information from both visual and tactile multimodal perspectives, enhancing their perception of potential hazards.

[0095] Scenario B: When the current warning level is Level 2, the head-mounted display module projects a flashing yellow arrow, and multiple motors on the side of the steering wheel that are deviating from the lane begin to vibrate at a medium frequency. The vibration lasts for a slightly longer duration than that of Level 1 warning, allowing the driver to clearly perceive the deviation in the vehicle's driving direction, adjust the steering wheel in time, avoid further deviation from the lane, and reduce the risk of accidents caused by lane departure.

[0096] Situation C: When the current warning level is Level 3, the head-mounted display module projects a red warning message onto the windshield, and simultaneously drives the linear vibration motor of the steering wheel haptic feedback module to vibrate at high frequency.

[0097] Specifically, all linear vibration motors vibrate continuously at high frequency and intensity, while the head-mounted display module shows a prominent red warning message, and the voice also issues an urgent alarm.

[0098] Understandably, in a Level 3 emergency, comprehensive and high-intensity multimodal warnings enable drivers to react quickly and take measures such as braking or evasive maneuvers, maximizing driving safety. Through precise programming and coordinated operation of multiple independently controlled motors, combined with AR-HUD and voice prompts, an efficient, accurate, and comprehensive in-vehicle multimodal warning and interaction system can be built, enhancing driving safety and the user experience.

[0099] Furthermore, in addition to the aforementioned warning prompts, the human-machine interaction collaboration logic of conventional navigation adopts a tiered prompt strategy: Level 1 prompt (500 meters ahead), the AR-HUD displays a green arrow, and a gentle voice reminder is given; Level 2 prompt (300 meters ahead), the AR arrow turns yellow and flashes, and the motor on the corresponding side of the steering wheel triggers a single vibration; Level 3 prompt (100 meters ahead), the AR arrow turns red, a rapid voice warning is given, and the motor vibrates continuously until the lane change is completed.

[0100] Fifthly, embodiments of this application provide an in-vehicle multimodal navigation interaction device, which can be a personal computer (PC), laptop computer, server, or other device with data processing capabilities.

[0101] Reference Figure 4 , Figure 4 This is a schematic diagram of the hardware structure of the in-vehicle multimodal navigation interaction device involved in the embodiments of this application. In the embodiments of this application, the in-vehicle multimodal navigation interaction device may include a processor, a memory, a communication interface, and a communication bus.

[0102] The communication bus can be of any type and is used to interconnect the processor, memory, and communication interface.

[0103] The communication interface includes input / output (I / O) interfaces, physical interfaces, and logical interfaces used for interconnecting internal components of the in-vehicle multimodal navigation interaction device, as well as interfaces used for interconnecting the in-vehicle multimodal navigation interaction device with other devices (such as other computing devices or user equipment). Physical interfaces can be Ethernet interfaces, fiber optic interfaces, ATM interfaces, etc.; user equipment can be displays, keyboards, etc.

[0104] Memory can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical storage, hard disk, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), etc.

[0105] The processor can be a general-purpose processor, which can call the in-vehicle multimodal navigation interaction program stored in the memory and execute the in-vehicle multimodal navigation interaction method provided in the embodiments of this application. For example, the general-purpose processor can be a central processing unit (CPU). The method executed when the in-vehicle multimodal navigation interaction program is called can be referred to the various embodiments of the in-vehicle multimodal navigation interaction method of this application, which will not be repeated here.

[0106] Those skilled in the art will understand that Figure 4 The hardware structure shown does not constitute a limitation of this application and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0107] Sixthly, embodiments of this application also provide a computer-readable storage medium.

[0108] The present application provides a computer-readable storage medium storing an in-vehicle multimodal navigation interaction program, wherein when the in-vehicle multimodal navigation interaction program is executed by a processor, it implements the steps of the in-vehicle multimodal navigation interaction method described above.

[0109] The method implemented when the in-vehicle multimodal navigation interaction program is executed can be referred to in various embodiments of the in-vehicle multimodal navigation interaction method of this application, and will not be repeated here.

[0110] It should be noted that the sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0111] The terms "comprising" and "having," and any variations thereof, in the specification, claims, and accompanying drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus. The terms "first," "second," and "third," etc., are used to distinguish different objects, etc., and do not indicate a sequence, nor do they limit "first," "second," and "third" to different types.

[0112] In the description of the embodiments of this application, terms such as "exemplary," "for example," or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplary," "for example," or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary," "for example," or "for instance" is intended to present the relevant concepts in a concrete manner.

[0113] In the description of the embodiments of this application, unless otherwise stated, " / " means "or". For example, A / B can mean A or B. The "and / or" in the text is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiments of this application, "multiple" means two or more.

[0114] In some processes described in the embodiments of this application, multiple operations or steps are included in a specific order. However, it should be understood that these operations or steps may not be executed in the order they appear in the embodiments of this application, or they may be executed in parallel. The sequence number of the operation is only used to distinguish different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations or steps may be executed sequentially or in parallel, and these operations or steps may be combined.

[0115] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device to execute the methods described in the various embodiments of this application.

[0116] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A vehicle-mounted multimodal navigation and interaction system, characterized in that, The in-vehicle multimodal navigation and interaction system includes: The navigation prompt module is used to monitor the vehicle's navigation information and output driving navigation prompts; The safety warning module is used to monitor the vehicle's driving safety status and output driving safety warning prompts; Head-mounted display module, which is used to project navigation images or safety warning images onto the vehicle window; Steering wheel haptic feedback module, which is used to provide navigation prompt vibration feedback or safety warning vibration feedback on the steering wheel; A controller is connected to the head-mounted display module and the steering wheel haptic feedback module via signals. The controller is used to adjust the working mode of the head-mounted display module and the steering wheel haptic feedback module according to driving navigation prompts and driving safety warning prompts.

2. The in-vehicle multimodal navigation and interaction system as described in claim 1, characterized in that, The controller is used to adjust the operating modes of the head-mounted display module and the steering wheel haptic feedback module according to driving navigation prompts and driving safety warning prompts, including: When the controller receives only the driving navigation prompts but not the driving safety warning prompts, the controller drives the head-mounted display module to project the navigation image onto the windshield according to the driving navigation prompts, and simultaneously drives the steering wheel haptic feedback module to perform the corresponding navigation prompt vibration feedback. When the controller receives both navigation prompts and driving safety warnings, it drives the head-mounted display module to project a safety warning image onto the windshield based on the safety warning prompt, and simultaneously drives the steering wheel haptic feedback module to perform the corresponding safety warning vibration feedback action.

3. The in-vehicle multimodal navigation and interaction system as described in claim 1, characterized in that, The steering wheel haptic feedback module includes at least six independently configured linear vibration motors, which are embedded in the steering wheel and are evenly arranged around the center point of the steering wheel.

4. The in-vehicle multimodal navigation and interaction system as described in claim 1, characterized in that, The head-mounted display module includes: The projection assembly includes a light guide plate and an LED array, the light guide plate being used in conjunction with the LED array to project an image onto the vehicle windshield; The eye-tracking component is connected to the light guide plate signal. The eye-tracking component is used to capture the position of the driver's pupils in real time and drive the light guide plate to adjust the refraction angle according to the position of the driver's pupils.

5. The in-vehicle multimodal navigation and interaction system as described in claim 4, characterized in that, The head-mounted display module also includes a stepper motor connected to the light guide plate, the stepper motor being used to adjust the refraction angle of the light guide plate.

6. The in-vehicle multimodal navigation and interaction system as described in claim 1, characterized in that, The safety warning module includes a lane departure warning component.

7. A vehicle-mounted multimodal navigation interaction method utilizing the vehicle-mounted multimodal navigation interaction system as described in claim 1, characterized in that, include: The navigation prompt module and the safety warning module collect driving navigation information and driving safety status, respectively. Determine whether to output driving navigation prompts and / or driving safety warnings based on driving navigation information and driving safety status; The controller activates the head-mounted display module and the steering wheel haptic feedback module based on the navigation prompts and / or safety warnings received from the controller.

8. The in-vehicle multimodal navigation interaction method as described in claim 7, characterized in that, The step of driving the head-mounted display module and the steering wheel haptic feedback module to operate based on the driving navigation prompts and / or driving safety warning prompts received by the controller includes: When the controller receives only the driving navigation prompts but not the driving safety warning prompts, the controller drives the head-mounted display module to project the navigation image onto the windshield according to the driving navigation prompts, and simultaneously drives the steering wheel haptic feedback module to perform the corresponding navigation prompt vibration feedback. When the controller receives both navigation prompts and driving safety warnings, it drives the head-mounted display module to project a safety warning image onto the windshield based on the safety warning prompt, and simultaneously drives the steering wheel haptic feedback module to perform the corresponding safety warning vibration feedback action.

9. The in-vehicle multimodal navigation interaction method as described in claim 8, characterized in that, The controller, based on the safety warning prompt, drives the head-mounted display module to project a safety warning image onto the vehicle window and simultaneously drives the steering wheel haptic feedback module to execute corresponding safety warning vibration feedback actions, including: The current driving status is classified according to the safety warning prompts to obtain the current warning level; Based on the current warning level, the head-mounted display module projects the corresponding safety warning image onto the windshield, and simultaneously drives the steering wheel haptic feedback module to execute the safety warning vibration feedback corresponding to the current warning level.

10. The in-vehicle multimodal navigation interaction method as described in claim 9, characterized in that, The process of classifying the current driving status based on safety warning prompts to obtain the current warning level includes: Based on the safety warning prompts, the current warning level is classified into three levels; among them... When a safety warning indicates that a vehicle is approaching a potentially dangerous area, the current warning level will be classified as Level 1 warning. When a safety warning indicates that the vehicle is showing a slight tendency to deviate from its lane, the current warning level will be classified as a Level 2 warning. When a safety warning indicates that a vehicle is about to be involved in an emergency collision, the current warning level will be classified as a Level 3 warning.

11. The in-vehicle multimodal navigation interaction method as described in claim 10, characterized in that, The process of driving the head-mounted display module to project a corresponding safety warning image onto the windshield based on the current warning level and simultaneously driving the steering wheel haptic feedback module to execute safety warning vibration feedback corresponding to the current warning level includes: When the current warning level is Level 1, the head-mounted display module projects a yellow warning message onto the windshield, and simultaneously drives the linear vibration motor of the steering wheel haptic feedback module to vibrate at a low frequency. When the current warning level is Level 3, the head-mounted display module projects a red warning message onto the windshield, and simultaneously drives the linear vibration motor of the steering wheel haptic feedback module to vibrate at high frequency.

12. The in-vehicle multimodal navigation interaction method as described in claim 10, characterized in that, The process of driving the head-mounted display module to project a corresponding safety warning image onto the windshield based on the current warning level and simultaneously driving the steering wheel haptic feedback module to execute a safety warning vibration feedback corresponding to the current warning level also includes: When the current warning level is Level II, the head-mounted display module projects a yellow warning message onto the windshield, and simultaneously drives the steering wheel haptic feedback module to perform mid-frequency vibration on the linear vibration motor on the side deviating from the direction.

13. The in-vehicle multimodal navigation interaction method as described in claim 7, characterized in that, The navigation prompt module and safety warning module collect driving navigation information and driving safety status respectively, including: using the lane departure warning component to monitor lane departure trends.

14. A vehicle-mounted multimodal navigation and interaction device, characterized in that, The in-vehicle multimodal navigation interaction device includes a processor, a memory, and an in-vehicle multimodal navigation interaction program stored in the memory and executable by the processor, wherein when the in-vehicle multimodal navigation interaction program is executed by the processor, it implements the steps of the in-vehicle multimodal navigation interaction method as described in any one of claims 7 to 13.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores an in-vehicle multimodal navigation interaction program, wherein when the in-vehicle multimodal navigation interaction program is executed by a processor, it implements the steps of the in-vehicle multimodal navigation interaction method as described in any one of claims 7 to 13.

Images