A control system and method for a cockpit-mounted HUD that projects mobile phone information

Abstract

This invention discloses a control system and method for projecting mobile phone information onto a head-up display (HUD) in a cockpit. The system includes: an in-vehicle host, a driver monitoring camera, a head-up display (HUD), and a steering wheel with integrated multi-function buttons. The driver monitoring camera is used to capture and recognize images of the driver's face. The in-vehicle host is connected to the driver monitoring camera, the HUD, and the steering wheel. The in-vehicle host communicates with a designated smartphone via Bluetooth BLE or UWB and authenticates the user's identity using the driver's facial image. After successful authentication, the in-vehicle host projects the smartphone's information onto the HUD for display, and the user interacts with the application interface on the HUD via the function buttons on the steering wheel. This invention enables users to safely use their mobile phones on the HUD, achieving a three-dimensional safety loop integrating the user, the machine, and the vehicle.

Description

Technical Field

[0001] This invention relates to the technical field of in-vehicle mobile phone information interaction, and more specifically, to a control system and method for a mobile phone information projection HUD in the cockpit. Background Technology

[0002] With the rapid development of smart cockpit and vehicle networking technologies, users expect to safely and conveniently access a wealth of applications and services on their mobile devices (mainly smartphones) while driving, such as instant messaging, financial information, and multimedia entertainment. Traditional solutions involve connecting the phone to the vehicle's central control screen via wired or wireless means, such as Apple CarPlay or Android Auto, or using Bluetooth for simple notification pushes and voice control. However, these solutions have significant shortcomings: screen projection requires the driver to significantly shift their gaze, posing a safety hazard if the driver's hands remain on the steering wheel and their eyes are not strayed from the road for extended periods; simple notification pushes cannot meet users' needs for in-depth application interaction, such as viewing stock charts or operating full app interfaces. Therefore, finding a convenient and efficient way to connect and adapt smartphones to the vehicle's head-up display (HUD) is of great importance. Summary of the Invention

[0003] This invention provides a control system and method for a mobile phone information projection HUD in the cockpit, which solves the problems of inconvenience and safety in the interaction between mobile phones and head-up displays in existing vehicle cockpits. It enables users to use mobile phones safely on the HUD, realizing a three-in-one safety closed loop of "human-machine-vehicle".

[0004] To achieve the above objectives, the present invention provides the following technical solution:

[0005] A control system for a mobile phone information projection HUD in a cockpit includes: an in-vehicle host, a driver monitoring camera, a head-up display, and a steering wheel with integrated multi-function buttons;

[0006] The driver monitoring camera is used to capture and recognize images of the driver's face;

[0007] The vehicle-mounted host is connected to the driver monitoring camera, the head-up display, and the steering wheel signal, respectively.

[0008] The vehicle-mounted host communicates with a designated smartphone via Bluetooth BLE or UWB and authenticates the driver's identity using a facial image.

[0009] After successful authentication, the vehicle-mounted host will project the smartphone information onto the head-up display for display, and interact with the application interface on the head-up display through the function buttons on the steering wheel.

[0010] Preferably, the steering wheel is equipped with a 3D joystick, which has multiple function buttons for interacting with the application interface on the head-up display.

[0011] Preferably, the vehicle-mounted host is connected to the head-up display via LVDS or FPD-Link III.

[0012] The present invention also provides a control method for a mobile phone information projection HUD in a cockpit, using the above-described control system, comprising:

[0013] When the vehicle's onboard unit senses the proximity of a smartphone via the Bluetooth / UWB module, it automatically activates the vehicle-to-vehicle connectivity service and establishes an initial connection via Bluetooth Low Energy, triggering the authentication process.

[0014] The driver monitoring camera (DMS) inside the vehicle captures the driver's facial image, and the on-board unit compares the driver's facial image with the facial features of a pre-registered user who has been bound to a corresponding smartphone for identity verification.

[0015] After identity verification, a high-bandwidth data channel is established for the vehicle's infotainment system using Ultra Wideband (UWB), and corresponding smartphone information is projected onto the screen.

[0016] Preferred options also include:

[0017] When authentication fails, the vehicle-mounted unit maintains only the basic connection with the head-up display, rejects the screen projection request, and displays an "Unauthorized User" message on the head-up display to protect the owner's privacy.

[0018] Preferred options also include:

[0019] After identity verification, the corresponding smartphone transmits the specified application interface to the vehicle host in real time in the form of an application stream, based on user presets or scenario recommendations, and then performs screen projection interaction through the head-up display.

[0020] Preferably, the step of encoding and transmitting the specified application interface to the vehicle host in real time in the form of an application stream includes:

[0021] The specified application interface is captured graphically, intelligently encoded, and transmitted in packets to the vehicle host. Only the application's rendering instructions and metadata are transmitted, without transmitting full-screen pixel data.

[0022] Preferably, the screen projection interaction via the head-up display includes:

[0023] The vehicle-mounted host performs real-time parsing, semantic understanding, and visual reconstruction of the received application interface rendering instructions and metadata to adapt the application interface on the smartphone screen to a form suitable for the three-dimensional spatial display of the head-up display, and redistributes the layer objects to different depth layers of the head-up display.

[0024] Preferred options also include:

[0025] Information is exchanged and responded to via the multi-function buttons on the steering wheel and the application interface on the head-up display.

[0026] Preferably, the entire process from a user approaching the vehicle to completing a full interaction includes:

[0027] When a smartphone enters the vehicle's sensing area, the vehicle's host system detects the paired smartphone via Bluetooth / UWB module;

[0028] The vehicle's onboard unit sends a connection request via Bluetooth / UWB module, and the mobile phone automatically responds to establish a pre-connection.

[0029] Simultaneously, the driver monitoring camera is triggered to verify the driver's identity;

[0030] Once the verification is successful, the vehicle's onboard unit and the smartphone will automatically establish a high-speed Wi-Fi Direct connection.

[0031] The vehicle's infotainment system creates a virtual display interface and notifies the smartphone to start screen mirroring.

[0032] Application projection and interface reconstruction are used to adapt the application interface on the smartphone screen to a form suitable for the three-dimensional spatial display of the head-up display.

[0033] Users interact and respond to the application interface on the head-up display using the multi-function buttons on the steering wheel.

[0034] The driver monitoring camera detects changes in the driver's status in real time, enabling the driver to maintain their status and exit safely.

[0035] This invention provides a control system and method for projecting mobile phone information onto a head-up display (HUD) in the cockpit. The onboard host automatically initiates a connection when an authorized smartphone approaches; it then uses the driver monitoring system (DMS) to acquire the driver's biometrics for authentication; only after successful authentication is a high-bandwidth data channel established between the smartphone and the onboard host for application interface projection; the system receives and controls the projected application interface on the HUD via the steering wheel input device. This solves the inconvenience and safety issues of existing mobile phone-head-up display interactions in vehicle cockpits, enabling users to safely use their phones on the HUD and achieving a three-dimensional safety closed loop involving the user, the machine, and the vehicle. Attached Figure Description

[0036] To more clearly illustrate the specific embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly described below.

[0037] Figure 1 This is a schematic diagram of a control system for a mobile phone information projection HUD in a cockpit, provided by the present invention.

[0038] Figure 2 This is a schematic diagram of a control method for a mobile phone information projection HUD in a cockpit, provided by an embodiment of the present invention. Detailed Implementation

[0039] To enable those skilled in the art to better understand the embodiments of the present invention, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings and implementation methods.

[0040] To address the inconvenience and safety issues of current mobile phone and head-up display (HUD) interaction in vehicle cabins, this invention provides a control system and method for projecting mobile phone information into a HUD in the cabin. This solves the problems of inconvenience and safety in existing mobile phone and HUD interaction in vehicle cabins, enabling users to safely use their mobile phones on the HUD and achieving a three-in-one safety closed loop of "human-machine-vehicle".

[0041] like Figure 1 As shown, a control system for a cockpit-mounted head-up display (HUD) projecting mobile phone information includes: an in-vehicle host, a driver monitoring camera (DMS), a head-up display (HUD), and a steering wheel with integrated multi-function buttons. The driver monitoring camera is used to capture and recognize images of the driver's face. The in-vehicle host is connected to the driver monitoring camera, the HUD, and the steering wheel. The in-vehicle host communicates with a designated smartphone via Bluetooth (BLE) or UWB and authenticates the driver's identity using the facial image. After successful authentication, the in-vehicle host projects the smartphone information onto the HUD for display and allows interaction with the application interface on the HUD via the function buttons on the steering wheel.

[0042] Specifically, the system uses DMS biometric recognition for dynamic screen projection permission control: it creatively uses DMS facial recognition results as a real-time trigger and authorization switch to determine whether mobile phone content can be projected onto the vehicle's HUD system. This is not a simple account login, but rather a dynamic binding of the driver's biometrics, their personal mobile phone, and the content displayed on the vehicle's HUD, achieving a seamless privacy and security loop integrating "human-machine-vehicle".

[0043] Furthermore, the steering wheel is equipped with a 3D joystick, which has multiple function buttons for interacting with the application interface on the head-up display.

[0044] In practical applications, the steering wheel features a control area optimized for HUD interaction, including: ① Directional control buttons: used to move and select the focus within the core interaction area of ​​the HUD. ② Function buttons: including a "Confirm / Open" button and a "Quick Reply / Action" button. This system defines an interaction logic centered on dedicated steering wheel buttons, solidifying high-frequency, critical operations (such as selection, confirmation, and quick reply) into simple physical button actions, and closely linking them with focus feedback on the HUD. In particular, the "one-click triggering of the quick reply menu and sending" process compresses communication interaction steps in driving scenarios to the extreme, achieving a balance between safety and efficiency.

[0045] Furthermore, the vehicle-mounted host is connected to the head-up display via LVDS or FPD-Link III.

[0046] Specifically, the system includes: an in-vehicle host integrating a DMS processing unit and a HUD rendering engine, a driver monitoring camera (DMS), an augmented reality head-up display (AR-HUD), a steering wheel with integrated multi-function buttons, and the user's smartphone. The system adopts a distributed layered architecture, including: a perception layer, a source layer, an in-vehicle infotainment layer, a display layer, and an interaction layer. The perception layer uses the driver monitoring camera for driver facial recognition and continuous verification, connecting to other devices via the in-vehicle CAN bus or Ethernet. The source layer uses application data sources from the smartphone for encoding and streaming, connecting to the in-vehicle host via a high-speed wireless link. The in-vehicle infotainment layer uses the in-vehicle host (which can be a domain controller) for protocol parsing, rendering management, and interactive control. The display layer uses the AR-HUD for 3D spatial presentation, connected via LVDS or FPD-Link III. The interaction layer receives user input commands via a 3D joystick on the steering wheel, connected via a clock spring and a LIN bus, with real-time reporting of the 3D joystick's X / Y / Z axis displacement data.

[0047] Smartphone-vehicle head unit connection link: The control channel is based on Bluetooth BLE or UWB for device discovery, authentication, and control command issuance; the data channel is based on Wi-Fi 6 / 6E or Wi-Fi Direct for high-bandwidth application streaming data transmission.

[0048] Driver Monitoring Camera (DMS) connects to the vehicle's main unit: Physical connection: via vehicle Ethernet or high-speed CAN; Data interaction: DMS transmits facial feature vectors in real time, and the vehicle's main unit performs identity comparison and continuous verification; Triggering mechanism: when DMS detects a change in driver, it immediately sends an interrupt signal to the vehicle's main unit, triggering a privacy protection strategy.

[0049] Connection between the vehicle-mounted head unit and the head-up display: Physical connection: adopts FPD-Link III or GMSL serial deserializer solution, supporting 4K@60Hz transmission; Data transmission: after the GPU of the vehicle-mounted head unit completes rendering, it transmits the layered image data to the HUD optical engine-10 via LVDS; Synchronization mechanism: the HUD returns a VSync synchronization signal to ensure tear-free display.

[0050] As can be seen, this invention provides a control system for a HUD (Head-Up Display) projecting mobile phone information into the cockpit. The onboard host automatically initiates a connection when an authorized smartphone approaches; it then uses the driver monitoring camera (DMS) to acquire the driver's biometrics for authentication; only after successful authentication is a high-bandwidth data channel established between the smartphone and the onboard host for application interface projection; the system receives and controls the projected application interface on the HUD via the steering wheel input device. This solves the inconvenience and safety issues of existing mobile phone-head-up display interactions in vehicle cockpits, enabling users to safely use their phones on the HUD and achieving a three-dimensional safety closed loop integrating "human-machine-vehicle".

[0051] Accordingly, such as Figure 2 As shown, the present invention also provides a control method for a mobile phone information projection HUD in a cockpit, using the above-described control system, comprising:

[0052] S1: When the vehicle host detects the proximity of a smartphone via the Bluetooth / UWB module, it automatically wakes up the vehicle-to-vehicle connectivity service and establishes an initial connection via Bluetooth Low Energy, triggering the authentication process.

[0053] S2: The driver monitoring camera (DMS) inside the vehicle captures the driver's facial image, and the vehicle host compares the driver's facial image with the facial features of a pre-registered user who has been bound to a corresponding smartphone for identity verification.

[0054] S3: After identity verification, a high-bandwidth data channel is established for the vehicle's infotainment system using Ultra Wideband (UWB), and corresponding smartphone information is projected onto the screen.

[0055] Specifically, the method can be divided into five stages: proximity perception and pre-connection stage, identity authentication and channel establishment stage, application projection and interface reconstruction stage, interactive operation and response stage, and state maintenance and secure exit stage.

[0056] Phase 1: Proximity Detection and Pre-Connection, Scenario Trigger: The user, carrying a bound smartphone, approaches the vehicle at a distance of 5-10 meters. Phase Output: A safety control channel is established between the phone and the vehicle's infotainment system; the DMS (Distributed Management System) is ready, awaiting the user's arrival. Specific steps are as follows:

[0057] Beacon Broadcast: Continuously broadcasts BLE beacons in low-power mode via the vehicle's Bluetooth module;

[0058] Device scanning: The system scans for matching vehicle beacons via the mobile app.

[0059] Two-way authentication: Security certificates are exchanged between the mobile phone and the vehicle's infotainment system to complete two-way authentication;

[0060] Pre-connection establishment: The onboard host assigns a virtual connection slot to the identified mobile phone;

[0061] The DMS pre-wake-up system detects that the driver's door is open and wakes up the camera in advance.

[0062] Phase Two: Identity Authentication and High-Speed ​​Channel Establishment. Scenario Trigger: The user sits in the driver's seat and fastens their seatbelt. Phase Output: Identity authentication complete, user type confirmed; high-speed data channel ready to transmit application stream; virtual display buffer created on the vehicle's infotainment system. Specific steps are as follows:

[0063] Face capture: Driver facial images are captured via the Driver Monitoring System (DMS).

[0064] Feature extraction and comparison: Facial feature vectors are extracted through the vehicle-mounted host and compared with pre-registered templates;

[0065] Identity verification: The vehicle's onboard unit determines whether the user is a bound, authorized user.

[0066] Access control decision: The onboard unit determines subsequent actions based on the identity verification result;

[0067] High-speed channel establishment: Automatically establish Wi-Fi Direct connection via mobile phone and vehicle system;

[0068] Capability negotiation: Exchange capability sets between the two parties through mobile phones and vehicle systems.

[0069] Phase 3: Application projection and interface reconstruction. Scenario trigger: Identity verification passed, high-speed channel established.

[0070] The method also includes: when authentication fails, the vehicle-mounted host maintains only the basic connection with the head-up display, rejects the screen projection request, and displays an "Unauthorized User" message on the head-up display to protect the owner's privacy.

[0071] In practical applications, the basic connection only provides signal connectivity and does not transmit image data.

[0072] The method further includes: after authentication is successful, the corresponding smartphone encodes and transmits the specified application interface in real time to the vehicle host in the form of an application stream according to user presets or scenario recommendations, and then performs screen projection interaction through the head-up display.

[0073] Furthermore, the step of encoding and transmitting the specified application interface to the vehicle-mounted host in real time as an application stream includes:

[0074] The specified application interface is captured graphically, intelligently encoded, and transmitted in packets to the vehicle host. Only the application's rendering instructions and metadata are transmitted, without transmitting full-screen pixel data.

[0075] Furthermore, the screen projection interaction via the head-up display includes:

[0076] The vehicle-mounted host performs real-time parsing, semantic understanding, and visual reconstruction of the received application interface rendering instructions and metadata to adapt the application interface on the smartphone screen to a form suitable for the three-dimensional spatial display of the head-up display, and redistributes the layer objects to different depth layers of the head-up display.

[0077] In practical applications, intelligent reconstruction and layered management of mobile application interfaces for HUDs are implemented: A dedicated interface rendering management module was designed specifically for the display characteristics of HUDs. This module can intelligently rearrange, scale, and extract key information from the common interfaces of mobile apps, and adapt them according to a layered strategy of "core interaction area" and "edge state area," thus solving the problem of complex app interfaces displaying chaotically on HUDs.

[0078] The method also includes: interacting and responding with the application interface on the head-up display via the multi-function buttons on the steering wheel.

[0079] Specifically, Phase 4: Interactive Operation and Response, Scene Trigger: Users interact with the application interface on the HUD through the 3D joystick on the steering wheel.

[0080] In one embodiment, taking "viewing stock trends and replying to WeChat messages" as an example, the interaction flow of the stock application is as follows:

[0081] User input: The user presses the 3D joystick forward to switch the focus from the state layer to the core layer;

[0082] Command capture: The joystick displacement data is encapsulated into LIN messages via the steering wheel LIN bus;

[0083] Command parsing: The LIN message is parsed by the onboard host interactive processing module and identified as a "layer switching" command;

[0084] Focus Update: Moves the visual focus from the status layer icons to the core layer application via the HUD rendering management module;

[0085] User input: Users switch between their watchlist by moving the joystick left or right;

[0086] Command mapping: The "left and right swipe" gesture is mapped to the "previous watchlist / next watchlist" operation via the vehicle host;

[0087] Control command transmission: The vehicle host sends a command to the mobile phone to switch the selected stocks via the control channel (Bluetooth BLE);

[0088] Mobile app response: The mobile app executes the switch to a watchlist and updates the interface;

[0089] Interface update flow: The mobile encoding module captures the updated interface, encodes it, and transmits it.

[0090] HUD Update: The HUD rendering management module receives the updated interface and re-renders and displays it.

[0091] Furthermore, the entire process from a user approaching the vehicle to completing a full interaction includes:

[0092] When a smartphone enters the vehicle's sensing area, the vehicle's host system detects the paired smartphone via Bluetooth / UWB module;

[0093] The vehicle's onboard unit sends a connection request via Bluetooth / UWB module, and the mobile phone automatically responds to establish a pre-connection.

[0094] Simultaneously, the driver monitoring camera is triggered to verify the driver's identity;

[0095] Once the verification is successful, the vehicle's onboard unit and the smartphone will automatically establish a high-speed Wi-Fi Direct connection.

[0096] The vehicle's infotainment system creates a virtual display interface and notifies the smartphone to start screen mirroring.

[0097] Application projection and interface reconstruction are used to adapt the application interface on the smartphone screen to a form suitable for the three-dimensional spatial display of the head-up display.

[0098] Users interact and respond to the application interface on the head-up display using the multi-function buttons on the steering wheel.

[0099] The driver monitoring camera detects changes in the driver's status in real time, enabling the driver to maintain their status and exit safely.

[0100] In practical applications, Phase 5: State Maintenance and Safe Exit, triggered by the following scenario: continuous operation during driving, or detection of a driver change / vehicle shutdown. The specific steps are as follows:

[0101] Continuous identity verification: Driver facial images are periodically collected and compared via DMS;

[0102] Identity change detection: The DMS detected the face of a new driver;

[0103] Privacy protection triggered: An identity change event is received via the vehicle's onboard unit;

[0104] Connection suspension: The vehicle's main unit suspends the high-speed data channel with the original mobile phone;

[0105] New user authentication: The vehicle's main unit performs facial recognition on new drivers;

[0106] Restore screen mirroring: The vehicle-mounted unit will re-establish screen mirroring after the new user is authenticated;

[0107] Exit when engine is off: The on-board unit exits control when it detects a vehicle engine off signal;

[0108] State saving: The mobile app records the current application state.

[0109] As can be seen, this invention provides a control method for a mobile phone information projection HUD in the cockpit. When the vehicle's main unit detects an authorized smartphone approaching, it automatically initiates a connection; it then uses the driver's biometrics obtained through the driver monitoring camera (DMS) for identity verification; only after successful verification is a high-bandwidth data channel established from the smartphone to the vehicle's main unit for application interface projection; the system receives and controls the projected application interface on the HUD via the steering wheel input device. This solves the inconvenience and safety issues of existing mobile phone and head-up display interactions in vehicle cockpits, enabling users to safely use their phones on the HUD and achieving a three-dimensional safety closed loop involving the "human-machine-vehicle".

[0110] The structure, features, and effects of the present invention have been described in detail above with reference to the embodiments shown in the figures. The above description is only a preferred embodiment of the present invention, but the present invention is not limited to the scope of implementation shown in the figures. Any changes made in accordance with the concept of the present invention, or equivalent embodiments modified to have equivalent changes, shall be within the protection scope of the present invention as long as they do not exceed the spirit covered by the specification and figures.

Claims

1. A control system for a mobile phone information projection HUD in a cockpit, characterized in that, include: In-vehicle infotainment system, driver monitoring camera, head-up display, and steering wheel with integrated multi-function buttons; The driver monitoring camera is used to capture and recognize images of the driver's face; The vehicle-mounted host is connected to the driver monitoring camera, the head-up display, and the steering wheel signal, respectively. The vehicle-mounted host communicates with a designated smartphone via Bluetooth BLE or UWB and authenticates the driver's identity using a facial image. After successful authentication, the vehicle-mounted host will project the smartphone information onto the head-up display for display, and interact with the application interface on the head-up display through the function buttons on the steering wheel.

2. The control system for a mobile phone information projection HUD in a cockpit according to claim 1, characterized in that, The steering wheel is equipped with a 3D joystick, which has multiple function buttons for interacting with the application interface on the head-up display.

3. The control system for a mobile phone information projection HUD in a cockpit according to claim 2, characterized in that, The vehicle-mounted host is connected to the head-up display via LVDS or FPD-Link III.

4. A control method for a mobile phone information projection HUD in a cockpit, using the control system described in any one of claims 1 to 3, characterized in that, include: When the vehicle's onboard unit senses the proximity of a smartphone via the Bluetooth / UWB module, it automatically activates the vehicle-to-vehicle connectivity service and establishes an initial connection via Bluetooth Low Energy, triggering the authentication process. The driver monitoring camera (DMS) inside the vehicle captures the driver's facial image, and the on-board unit compares the driver's facial image with the facial features of a pre-registered user who has been bound to a corresponding smartphone for identity verification. After identity verification, a high-bandwidth data channel is established for the vehicle's infotainment system using Ultra Wideband (UWB), and corresponding smartphone information is projected onto the screen.

5. The control method for a mobile phone information projection HUD in a cockpit according to claim 4, characterized in that, Also includes: When authentication fails, the vehicle-mounted unit maintains only the basic connection with the head-up display, rejects the screen projection request, and displays an "Unauthorized User" message on the head-up display to protect the owner's privacy.

6. The control method for a mobile phone information projection HUD in a cockpit according to claim 5, characterized in that, Also includes: After identity verification, the corresponding smartphone transmits the specified application interface to the vehicle host in real time in the form of an application stream, based on user presets or scenario recommendations, and then performs screen projection interaction through the head-up display.

7. The control method for a mobile phone information projection HUD in a cockpit according to claim 6, characterized in that, The step of encoding and transmitting the specified application interface to the vehicle host in real time as an application stream includes: The specified application interface is captured graphically, intelligently encoded, and transmitted in packets to the vehicle host. Only the application's rendering instructions and metadata are transmitted, without transmitting full-screen pixel data.

8. The control method for a mobile phone information projection HUD in a cockpit according to claim 7, characterized in that, The screen projection interaction via the head-up display includes: The vehicle-mounted host performs real-time parsing, semantic understanding, and visual reconstruction of the received application interface rendering instructions and metadata to adapt the application interface on the smartphone screen to a form suitable for the three-dimensional spatial display of the head-up display, and redistributes the layer objects to different depth layers of the head-up display.

9. The control method for a mobile phone information projection HUD in a cockpit according to claim 8, characterized in that, Also includes: Information is exchanged and responded to via the multi-function buttons on the steering wheel and the application interface on the head-up display.

10. The control method for a mobile phone information projection HUD in a cockpit according to claim 9, characterized in that, The entire process from a user approaching the vehicle to completing a full interaction includes: When a smartphone enters the vehicle's sensing area, the vehicle's host system detects the paired smartphone via Bluetooth / UWB module; The vehicle's onboard unit sends a connection request via Bluetooth / UWB module, and the mobile phone automatically responds to establish a pre-connection. Simultaneously, the driver monitoring camera is triggered to verify the driver's identity; Once the verification is successful, the vehicle's onboard unit and the smartphone will automatically establish a high-speed Wi-Fi Direct connection. The vehicle's infotainment system creates a virtual display interface and notifies the smartphone to start screen mirroring. Application projection and interface reconstruction are used to adapt the application interface on the smartphone screen to a form suitable for the three-dimensional spatial display of the head-up display. Users interact and respond to the application interface on the head-up display using the multi-function buttons on the steering wheel. The driver monitoring camera detects changes in the driver's status in real time, enabling the driver to maintain their status and exit safely.

Images