Vehicle augmented reality head-up display interaction method and storage medium

Abstract

The invention discloses a vehicle augmented reality head-up display interaction method and a storage medium, and relates to the technical field of intelligent vehicles. The method comprises the following steps: acquiring a driving scene type; in response to the driving scene type being the first scene type, controlling the vehicle to enter a full-function interaction mode; in the full-function interaction mode, gazing information of a driver and a gesture type of the driver are obtained; and in response to the fact that the duration of watching any virtual key by the driver is larger than a preset time threshold value and a specified gesture type of the driver is captured in a preset time period, controlling the vehicle to start an augmented reality head-up display function corresponding to the virtual key. According to the invention, the technical problem of frequent false triggering caused by inaccurate intention recognition during man-machine interaction in the prior art is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent vehicles, in particular to a vehicle augmented reality head-up display interaction method and a storage medium. BACKGROUND

[0002] With the rapid development of intelligent vehicles and autonomous driving technology, augmented reality head-up display (AR-HUD) is gradually becoming an important tool for improving driving experience and safety as an innovative vehicle information display method. AR-HUD can display augmented reality information on the front windshield of the vehicle, seamlessly integrating real-time navigation, vehicle status information, and the front road environment, allowing the driver to obtain key information without shifting their gaze. However, the existing line-of-sight tracking function, although it can provide a touchless interaction experience, has limitations in terms of poor recognition robustness and frequent misoperations when used alone in actual applications.

[0003] There is no effective solution to the above problems. SUMMARY

[0004] The embodiments of the present application provide a vehicle augmented reality head-up display interaction method and a storage medium to at least solve the technical problem of inaccurate intention recognition during human-computer interaction in related technologies, leading to frequent false triggering.

[0005] According to one embodiment of the present application, a vehicle augmented reality head-up display interaction method is provided, comprising: obtaining a driving scene type; in response to the driving scene type being a first scene type, controlling the vehicle to enter a full-function interaction mode, wherein the first scene type represents a scene with a driving risk lower than a first risk threshold, and the augmented reality head-up display function in the full-function interaction mode is in a normal state; in the full-function interaction mode, obtaining the gaze information of the driver and the gesture type of the driver; in response to capturing the duration of the driver's gaze at any virtual key being greater than a preset time threshold, and simultaneously capturing the specified gesture type of the driver within a preset time period, controlling the vehicle to start the augmented reality head-up display function corresponding to the virtual key, wherein the virtual key is a control key of the augmented reality head-up display function, and the specified gesture type is used to confirm the start of the augmented reality head-up display function.

[0006] Optionally, the vehicle augmented reality head-up display interaction method further comprises: in response to the driving scene type being a second scene type, controlling the vehicle to enter a simplified interaction mode, wherein the second scene type is used to represent a scene with a driving risk greater than a first risk threshold and less than a second risk threshold, the first risk threshold is less than the second risk threshold, and part of the augmented reality head-up display function in the simplified interaction mode is in a closed state; wherein in the simplified interaction mode, the gaze information of the driver is used to monitor the driving state of the driver, the gesture type of the driver is used to confirm the opening of part of the augmented reality head-up display function, and the display screen of the augmented reality head-up display function is used to display core driving information.

[0007] Optionally, the vehicle augmented reality head-up display interaction method further comprises: in response to the driving scene type being a third scene type, controlling the vehicle to enter a safety intervention mode, wherein the third scene type is used to represent a scene with a driving risk greater than the second risk threshold, and part of the augmented reality head-up display function in the safety intervention mode is in a closed state; wherein in the safety intervention mode, the gaze information of the driver is used to monitor the driving state of the driver, the gesture type-based identification function is locked, and the display screen of the augmented reality head-up display function is used to display emergency safety information.

[0008] Optionally, the vehicle augmented reality head-up display interaction method further comprises: in response to the driver gazing at any virtual key for a duration greater than a preset time threshold, controlling the augmented reality head-up display function to enter a preparation state; in the preparation state, controlling the virtual key to display in a preset state to prompt the driver to feed back a specified gesture type, wherein the preset state includes highlighting and superimposing a pulsating animation.

[0009] Optionally, the vehicle augmented reality head-up display interaction method further comprises: continuously monitoring the driving scene type of the vehicle; and in response to the driving scene type being a first scene type, controlling the vehicle to enter a full-function interaction mode.

[0010] Optionally, the control of the vehicle to turn on the augmented reality head-up display function corresponding to the virtual key comprises: converting an interaction instruction between the driver and the augmented reality head-up display function into a vehicle-mounted network message; and controlling the vehicle to turn on the augmented reality head-up display function corresponding to the virtual key according to the vehicle-mounted network message.

[0011] Optionally, obtaining the driving scenario type includes: obtaining the vehicle speed and the response information of the advanced driver assistance system; determining the driving scenario type as a first scenario type in response to the vehicle speed being lower than a first speed threshold and the advanced driver assistance system not issuing an alarm; determining the driving scenario type as a second scenario type in response to the vehicle speed being greater than a second speed threshold or the advanced driver assistance system issuing a first alarm, wherein the first alarm corresponds to a first processing priority and the second speed threshold is greater than the first speed threshold; and determining the driving scenario type as a third scenario type in response to the advanced driver assistance system issuing a second alarm, wherein the second alarm corresponds to a second processing priority and the second processing priority is higher than the first processing priority.

[0012] Optionally, the vehicle augmented reality head-up display interaction method also includes: in the ready state, controlling the vehicle's audio components to emit prompt sound effects, or superimposing haptic vibration prompts on the vehicle's steering wheel.

[0013] Optionally, the specified gesture types include: pinch, swipe, fist, palm rotation, and air click.

[0014] According to one embodiment of the present invention, a vehicle augmented reality head-up display (HUD) interactive device is also provided, comprising: a first acquisition module for acquiring a driving scene type; a first control module for controlling the vehicle to enter a full-function interactive mode in response to the driving scene type being a first scene type, wherein the first scene type represents a scene where the driving danger level is lower than a first danger threshold, and the augmented reality head-up display function in the full-function interactive mode is in a normal state; a second acquisition module for acquiring the driver's gaze information and the driver's gesture type in the full-function interactive mode; and a second control module for controlling the vehicle to activate the augmented reality head-up display function corresponding to the virtual button in response to capturing the duration of the driver's gaze at any virtual button being greater than a preset time threshold, and simultaneously capturing the driver's specified gesture type within a preset time period, wherein the virtual button is a control button for the augmented reality head-up display function, and the specified gesture type is used to confirm the activation of the augmented reality head-up display function.

[0015] According to one embodiment of the present invention, a vehicle is also provided, including: a memory storing an executable program; and a processor for running the program, wherein the program executes the vehicle augmented reality head-up display interaction method described above when it runs.

[0016] According to one embodiment of the present invention, a computer-readable storage medium is also provided, wherein the storage medium stores a computer program, wherein the computer program is configured to execute the vehicle augmented reality head-up display interaction method described above when run on a computer or processor.

[0017] According to one embodiment of the present invention, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the vehicle augmented reality head-up display interaction method described above.

[0018] According to one embodiment of the present invention, a computer program product is also provided, including a computer program that, when executed by a processor, implements the vehicle augmented reality head-up display interaction method described above.

[0019] In this embodiment of the invention, the driving scenario type is first obtained. If the driving scenario type is the first scenario type, the vehicle is controlled to enter the full-function interaction mode. Then, in the full-function interaction mode, the driver's gaze information and the driver's gesture type are obtained. If the duration of the driver's gaze at any virtual button is captured to be greater than a preset time threshold, and the driver's specified gesture type is captured simultaneously within a preset time period, the vehicle is controlled to activate the augmented reality head-up display function corresponding to the virtual button. This achieves the purpose of activating the augmented reality head-up display function through synchronous eye tracking and gesture recognition, thereby significantly reducing the error rate and ensuring the accurate transmission of the driver's intention. This solves the technical problem in related technologies where inaccurate intention recognition during human-computer interaction leads to frequent false triggers. Attached Figure Description

[0020] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings: Figure 1 This is a flowchart of a vehicle augmented reality head-up display interaction method according to one embodiment of the present invention; Figure 2 This is an interaction flowchart in the full-function interactive mode according to one embodiment of the present invention; Figure 3 This is a schematic diagram of a full-function interactive mode display interface according to one embodiment of the present invention; Figure 4 This is a schematic diagram of a simplified interactive mode display interface according to one embodiment of the present invention; Figure 5 This is a schematic diagram of a security intervention mode display interface according to one embodiment of the present invention; Figure 6 This is an interaction flowchart under a security intervention mode according to one embodiment of the present invention; Figure 7 This is a flowchart of the mode switching process for an augmented reality head-up display function according to one embodiment of the present invention; Figure 8This is an architecture diagram of a vehicle augmented reality head-up display interactive system according to one embodiment of the present invention; Figure 9 This is a structural block diagram of a vehicle augmented reality head-up display interactive device according to one embodiment of the present invention. Detailed Implementation

[0021] For ease of understanding, some concepts related to embodiments of the present invention are illustrated below for reference. Advanced Driver Assistance Systems (ADAS) refer to a series of systems that use technologies such as sensors and cameras to provide drivers with safety warnings and assisted control. ADAS helps drivers identify potential hazards while driving, such as forward collision warning, lane departure warning, and blind spot monitoring. Some systems can also take automatic measures in emergency situations, such as automatic emergency braking and adaptive cruise control, to improve driving safety and comfort.

[0022] A Driver Monitoring System (DMS) is a system used to monitor the driver's condition. Its main function is to detect the driver's attention level, fatigue level, and driver identity through cameras or other sensors to prevent accidents caused by distracted driving, fatigued driving, or unauthorized use. A DMS can monitor the driver's facial expressions, eye opening degree, head movements, etc., to assess the driver's alertness in real time and issue warnings or take measures when it detects that the driver may be in an unsafe state. For example, if the driver's eyes remain closed for more than a preset time, the system will determine that the driver may be fatigued, thus reminding the driver to rest or take other measures to ensure driving safety.

[0023] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0024] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. In the description of these embodiments, unless otherwise stated, "a plurality of" means two or more. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0025] According to one embodiment of the present invention, an embodiment of a vehicle augmented reality head-up display interaction method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0026] This method embodiment can be executed in an electronic device, similar control device, or system that includes a memory and a processor. Taking an electronic device as an example, the electronic device may include one or more processors and a memory for storing data. Optionally, the electronic device may also include a communication device for communication functions and a display device. Those skilled in the art will understand that the above structural description is merely illustrative and does not limit the structure of the electronic device. For example, the electronic device may include more or fewer components than described above, or have a different configuration than described above.

[0027] A processor may include one or more processing units. For example, a processor may include a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processing (DSP) chip, a microcontroller unit (MCU), a field-programmable gate array (FPGA), a neural network processing unit (NPU), a tensor processing unit (TPU), or an artificial intelligence (AI) processor. Different processing units may be independent components or integrated into one or more processors. In some instances, electronic devices may also include one or more processors.

[0028] The memory can be used to store computer programs, such as the computer program corresponding to the vehicle augmented reality head-up display interaction method in this embodiment of the invention. The processor implements the aforementioned vehicle augmented reality head-up display interaction method by running the computer program stored in the memory. The memory may include high-speed random access memory and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to electronic devices via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0029] Communication devices are used to receive or send data via a network. Specific examples of such networks may include wireless networks provided by the mobile terminal's communication provider. In one example, the communication device includes a network interface controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the communication device may be a radio frequency (RF) module used for wireless communication with the Internet.

[0030] Display devices can be, for example, touchscreen liquid crystal displays (LCDs) and touch displays (also referred to as "touchscreens" or "touch displays"). The LCD allows users to interact with the user interface of the mobile terminal. In some embodiments, the mobile terminal has a graphical user interface (GUI), which allows users to interact with the GUI through finger contact and / or gestures on a touch-sensitive surface. Optional human-computer interaction functions include: creating web pages, drawing, word processing, creating electronic documents, playing games, video conferencing, instant messaging, sending and receiving emails, call interfaces, playing digital video, playing digital music, and / or web browsing, etc. Executable instructions for performing the above human-computer interaction functions are configured / stored in one or more processor-executable computer program products or readable storage media.

[0031] This embodiment provides a vehicle augmented reality head-up display interaction method running on an electronic device. Figure 1 This is a flowchart of a vehicle augmented reality head-up display interaction method according to one embodiment of the present invention, such as... Figure 1 As shown, the process includes the following steps: Step S10: Obtain the driving scenario type; In this embodiment of the invention, driving scenario type refers to classifying the driving environment based on the vehicle's current driving status and environmental conditions to identify and define the specific situation in which the current driving operation takes place. This embodiment of the invention mainly divides driving scenario types into a first scenario type, a second scenario type, and a third scenario type. The first scenario type primarily represents low-risk scenarios, the second scenario type primarily represents high-risk scenarios, and the third scenario type primarily represents emergency safety situations; however, this is not limited to these specific scenarios.

[0032] As can be seen, by acquiring the driving scenario type, this invention can more comprehensively understand the driving environment and driver state, providing a foundation for subsequent adjustments to the interaction mode.

[0033] Step S11: In response to the driving scenario type being the first scenario type, control the vehicle to enter the full-function interactive mode, wherein the first scenario type is used to represent a scenario where the driving danger level is lower than the first danger threshold, and the augmented reality head-up display function in the full-function interactive mode is in normal state. In this embodiment of the invention, the first scenario type refers to a scenario where the driving environment and conditions are considered relatively safe or low-risk. For example, when the vehicle is stationary, traveling at low speed, or in autonomous driving mode, the driver faces fewer external threats and can focus more on non-driving tasks.

[0034] The full-function interaction mode is the interaction mode enabled in the first scenario type. In full-function interaction mode, the augmented reality head-up display (AR-HUD) function is in normal condition, meaning that the AR-HUD can display the complete interactive interface, including all available virtual buttons and information controls. The driver can freely access and control the AR-HUD functions through a combination of eye tracking and gesture recognition, such as adjusting the in-car temperature, switching music playback, and viewing navigation details, without any restrictions.

[0035] The first hazard threshold is used to measure the level of potential danger in the driving environment. The first hazard threshold is set based on various factors, including but not limited to vehicle speed, weather conditions, traffic flow, and warning signals from other advanced driver assistance systems (ADAS) and driver monitoring systems (DMS). When the level of danger in the driving environment is below the first hazard threshold, the system classifies it as the first scenario type, thus allowing the full-function interactive mode to be activated. Conversely, if the level of danger exceeds the first hazard threshold, the system will automatically switch to a safer interactive mode, limiting or reducing the driver's interaction with the AR-HUD to avoid distracting the driver and ensure driving safety.

[0036] Responding to the driving scenario type being the first scenario type, controlling the vehicle to enter the full-function interaction mode can be understood as follows: when the system recognizes that the current driving environment belongs to the low-risk first scenario type, the system will automatically switch to the full-function interaction mode. In this mode, the vehicle's augmented reality head-up display (AR-HUD) function and other related interaction technologies (such as gesture recognition, voice control, etc.) are fully enabled, and the driver can access and control various functions of the vehicle through natural interaction methods without additional restrictions.

[0037] As can be seen, when the driving environment is identified as the first scenario type, the system will automatically switch to the full-function interaction mode, opening up all the interactive functions of the AR-HUD to provide a richer and more intuitive user interface control experience.

[0038] Step S12: In full-function interactive mode, obtain the driver's gaze information and the driver's gesture type; In this embodiment of the invention, gaze information refers to the positional information of the driver's eye focus point captured by the system through gaze tracking technology. In this invention, gaze information typically includes the coordinates (X_gaze, Y_gaze) of the driver's gaze on the AR-HUD virtual interface, but this is not limited to these coordinates. The coordinates (X_gaze, Y_gaze) can indicate the specific virtual control element (such as an icon, button, etc.) that the driver is viewing or intends to operate.

[0039] Gesture type refers to the category of a driver's hand movements identified through gesture recognition technology. For example, by analyzing images or video streams captured by a camera, the shape and movement patterns of the hands are identified, thereby determining the type of gesture performed by the driver.

[0040] In full-function interactive mode, obtaining the driver's gaze information and gesture type can be understood as follows: when the vehicle's AR-HUD function is in full-function interactive mode, the in-vehicle camera and eye-tracking technology are used to monitor and obtain the driver's gaze information and gesture type in real time.

[0041] As can be seen, by acquiring the driver's gaze information in real time, the system can accurately locate the driver's gaze focus, thereby determining the driver's immediate operational intentions. Combined with gesture type recognition, the system can distinguish between conscious and unconscious actions by the driver, significantly reducing the error rate, making the interaction process more intuitive and accurate, and reducing driver distraction during operation.

[0042] Step S13: In response to the capture of the driver looking at any virtual button for a duration greater than a preset time threshold, and the capture of the driver's specified gesture type within a preset time period, the vehicle is controlled to activate the augmented reality head-up display function corresponding to the virtual button. The virtual button is the control button for the augmented reality head-up display function, and the specified gesture type is used to confirm the activation of the augmented reality head-up display function.

[0043] In this embodiment of the invention, virtual buttons refer to interactive controls presented in the AR-HUD display interface, used to control specific in-vehicle functions. Virtual buttons are not physical entities, but rather graphical interface elements virtually overlaid within the driver's field of vision using AR technology. For example, the AR-HUD interface may display icons for adjusting the air conditioning temperature, buttons for switching music playback, symbols for confirming navigation commands, etc., and this is not a limitation.

[0044] The preset time threshold is a parameter used to determine whether the driver intends to operate a specific virtual button.

[0045] The preset time period refers to the time during which the system waits for a specified gesture to appear after the driver looks at the virtual button. The existence of the preset time period ensures that the system does not immediately execute the operation after recognizing the operation intention, but waits for the driver to confirm with a specific gesture, thereby avoiding unnecessary operations and the risk of possible misoperation.

[0046] The designated gesture type is a series of predefined hand actions in this embodiment of the invention, used to ultimately confirm the driver's intention to operate the virtual buttons. The designated gesture type includes, but is not limited to, pinching, swiping, clenching a fist, rotating the palm, and clicking in the air.

[0047] In response to the capture of the driver looking at any virtual button for a duration greater than a preset time threshold, and the capture of the driver's specified gesture type within a preset time period, the system controls the vehicle to activate the augmented reality head-up display function corresponding to the virtual button. This can be understood as follows: if the driver continuously looks at a virtual button on the AR-HUD interface for more than a preset time threshold (e.g., 400 milliseconds), and makes a predefined specified gesture type within a preset time period (e.g., 1 to 2 seconds), the system will execute the function related to the virtual button being looked at.

[0048] As can be seen, by combining the driver's gaze and gesture input, the system can intelligently judge and execute the driver's intentions, and activate the relevant functions on the AR-HUD, thereby providing a more natural, efficient and safe cockpit interaction experience.

[0049] Figure 2 This is an interaction flowchart in the full-function interactive mode according to one embodiment of the present invention, such as... Figure 2 As shown, if the vehicle is currently in a low-risk scenario, the system continuously monitors the driver's gaze. Once it detects that the driver's gaze on a virtual button exceeds a preset time threshold T (e.g., 400 milliseconds), the system determines that the driver has a potential intention to operate that virtual button. At this time, the virtual button will respond with visual feedback, such as highlighting and overlaying a pulsating animation, prompting the driver that the system has entered a ready state and is waiting for further gesture confirmation within a preset time period δt (e.g., 1 to 2 seconds). The system will check whether the driver's gaze remains locked on the target virtual button during the gesture. If the gesture and gaze are consistent in time and space, the system will recognize the interaction intention as valid and then execute the instruction corresponding to the virtual button. If the driver does not make the expected gesture during the ready state, or if the gaze leaves the target during the gesture confirmation process, the system will time out or handle the anomaly, which may include canceling the ready state, requiring a renewed gaze, or providing a second confirmation opportunity. If the vehicle is currently in a low-risk scenario, the AR-HUD will be controlled to enter a simplified interaction mode. The above steps ensure the accuracy of the interaction. By confirming with both eye contact and gestures, unintentional interaction actions are filtered out, reducing the possibility of accidental triggering.

[0050] Figure 3 This is a schematic diagram of a full-function interactive mode display interface according to one embodiment of the present invention, such as... Figure 3As shown, in full-function interactive mode, the in-vehicle AR-HUD system's display interface will present richer and more comprehensive information to provide the driver with the most comprehensive interactive experience. The content that the full-function interactive mode display interface may display includes, but is not limited to: core driving information (such as vehicle speed), navigation and traffic information, entertainment system control (such as music playback), vehicle status and settings (such as air conditioning control), and communication functions (such as answering or hanging up calls).

[0051] Through the above steps, the driving scenario type is first obtained. If the driving scenario type is the first scenario type, the vehicle is controlled to enter the full-function interaction mode. Then, in the full-function interaction mode, the driver's gaze information and the driver's gesture type are obtained. If the duration of the driver's gaze at any virtual button is captured to be greater than a preset time threshold, and the driver's specified gesture type is captured simultaneously within the preset time period, the vehicle is controlled to activate the augmented reality head-up display function corresponding to the virtual button. This achieves the goal of activating the augmented reality head-up display function through synchronous gaze tracking and gesture recognition, thereby significantly reducing the error rate and ensuring the accurate transmission of the driver's intentions. This solves the technical problem in related technologies where inaccurate intention recognition during human-computer interaction leads to frequent false triggers.

[0052] Optionally, the vehicle augmented reality head-up display interaction method also includes the following execution steps: Step S14: In response to the driving scenario type being the second scenario type, the vehicle is controlled to enter a simplified interaction mode. The second scenario type is used to represent a scenario where the driving hazard level is greater than a first hazard threshold but less than a second hazard threshold. The first hazard threshold is less than the second hazard threshold. In the simplified interaction mode, some augmented reality head-up display functions are turned off. In the simplified interaction mode, the driver's gaze information is used to monitor the driver's driving status, the driver's gesture type is used to confirm the activation of some augmented reality head-up display functions, and the display screen of the augmented reality head-up display function is used to display core driving information.

[0053] In this embodiment of the invention, the second scenario type refers to a driving scenario where the driving environment or conditions reach a certain level of complexity and danger. In contrast to the low-risk first scenario type, the second scenario type encompasses states between safe and smooth driving and high-risk driving, such as driving in busy urban traffic, driving in the rain, and driving at night.

[0054] The simplified interaction mode is the operating mode of the AR-HUD system in high-risk driving scenarios. This mode reduces the number and functions of interactive virtual buttons to decrease the need for driver attention.

[0055] The second danger threshold is a standard used by the system to judge the degree of danger in a driving scenario. When the degree of danger in a driving scenario is higher than the first danger threshold but lower than the second danger threshold, the system will enter a simplified interaction mode. The setting of the second danger threshold is based on vehicle environmental parameters (such as vehicle speed, road conditions, weather conditions, etc.) and driver state parameters (such as fatigue level, attention level, etc.), and there are no restrictions here.

[0056] Driving status refers to the driver's physiological and psychological state, as well as their control over the vehicle and road environment. In simplified interaction mode, the driver's gaze information is used only to continuously monitor the driver's driving status, including the driver's attention level, fatigue level, and eye movement, to ensure that the driver can maintain focus on the driving task.

[0057] Core driving information refers to the basic vehicle status and road information that drivers need to continuously monitor under any driving conditions. Figure 4 This is a schematic diagram of a simplified interactive mode display interface according to one embodiment of the present invention, such as... Figure 4 As shown, in simplified interaction mode, the AR-HUD display mainly shows core driving information: current vehicle speed, remaining fuel, navigation route instructions, distance to vehicles ahead, and safe driving reminders, ensuring that the driver can grasp key driving data while avoiding excessive information interference.

[0058] In response to the driving scenario being classified as Scenario 2, controlling the vehicle to enter simplified interaction mode can be understood as follows: when the driving scenario is Scenario 2, the system enters simplified interaction mode by disabling some augmented reality head-up display functions. In simplified interaction mode, the driver's gaze is primarily used to monitor the driving status, while gestures are limited to confirming only a few core functions. The AR-HUD display shows core driving information to reduce distractions and ensure driving safety.

[0059] As can be seen, when the driving hazard level exceeds the first hazard threshold but has not yet reached the second hazard threshold, the system automatically switches to simplified interaction mode. In simplified interaction mode, the system only displays core driving information and disables AR-HUD functions that are unrelated to driving safety, thereby reducing visual interference to the driver and ensuring that the driver can concentrate more on the driving task, significantly improving driving safety.

[0060] Optionally, the vehicle augmented reality head-up display interaction method also includes the following execution steps: Step S15: In response to the driving scenario type being the third scenario type, the vehicle is controlled to enter the safety intervention mode. The third scenario type is used to represent a scenario where the driving danger level is greater than the second danger threshold. In the safety intervention mode, some augmented reality head-up display functions are turned off. In the safety intervention mode, the driver's gaze information is used to monitor the driver's driving status, the gesture type recognition function is locked, and the display screen of the augmented reality head-up display function is used to display emergency safety information.

[0061] In this embodiment of the invention, the third scenario type refers to an extreme or emergency driving scenario where the degree of driving danger exceeds the second danger threshold, including but not limited to emergency braking on highways, sudden forward collision warnings, or severe obstacles in the driving environment, such as heavy fog, icy roads, and other extreme weather conditions, which are not limited here.

[0062] The safety intervention mode is a highly restricted interaction mode employed by the AR-HUD system in the third scenario type. In this mode, most non-core augmented reality head-up display functions are systematically disabled or minimized to reduce any factors that might distract the driver. The system primarily uses the driver's gaze information to monitor their driving status, including fatigue and attention levels, while gesture-based recognition functions are actively locked to avoid further risks caused by misoperation in emergency situations.

[0063] Emergency safety information refers to the information that the AR-HUD display will highlight in safety intervention mode that is directly related to ensuring driving safety. Figure 5 This is a schematic diagram of a security intervention mode display interface according to one embodiment of the present invention, such as... Figure 5 As shown, the safety intervention mode display interface may show the following: current vehicle speed, emergency safety information, brake assist prompts, and safety warning signs. Emergency safety information includes, but is not limited to, forward collision warning, obstacle distance, vehicle loss of control warning, and emergency braking indication. By highlighting emergency safety information, the system can provide clear and direct visual prompts when the driver needs to obtain critical information as quickly as possible, assisting the driver in making rapid safety decisions.

[0064] In response to the driving scenario being the third scenario type, controlling the vehicle to enter the safety intervention mode can be understood as follows: when the driving scenario is the third scenario type, the system enters the safety intervention mode and shuts down all non-core AR-HUD functions, ensuring that the driver's vision and attention are fully focused on the driving task itself, effectively improving driving safety in dangerous situations.

[0065] Figure 6 This is an interaction flowchart under a security intervention mode according to one embodiment of the present invention, such as...Figure 6 As shown, the system continuously receives the highest priority signals from ADAS and DMS, including vehicle driving status (such as vehicle speed), surrounding environment perception (such as forward collision warning), and driver status (such as fatigue score). If no highest priority alarm is received, the system continues to monitor the ADAS signal source. If a highest priority alarm is received, the system controls the AR-HUD to activate the safety intervention mode. In safety intervention mode, the AR-HUD clears non-core display content, retaining only safety information, notifies the dynamic decision-maker to reset the interactive state machine, and notifies the audio unit to play an alarm sound. If the alarm is successfully cleared, the system exits the safety intervention mode and restores the full-function interactive mode; if the alarm is not cleared, the system continues to monitor the ADAS signal source. By monitoring ADAS and DMS signals in real time, the system intelligently adjusts the AR-HUD display interface and interactive functions, thereby maximizing driving safety without affecting the driving experience.

[0066] It can be seen that in emergency situations, displaying too much information can lead to information overload for the driver, making it difficult to react quickly and accurately. The safety intervention mode avoids information overload by displaying concise emergency safety information, enabling the driver to quickly identify key warnings and take swift safety action.

[0067] Figure 7 This is a flowchart illustrating the mode switching process of an augmented reality head-up display function according to one embodiment of the present invention, such as... Figure 7As shown, the system continuously monitors the driving scenario in which the vehicle is located. If the safety arbitrator is triggered, the AR-HUD is controlled to enter safety intervention mode. In safety intervention mode, the control system clears non-core display content, locks gesture recognition function, resets all interactive states, and displays emergency safety information. When the alarm in safety intervention mode is cleared, the driving scenario is continuously monitored. If the safety arbitrator is not triggered, the system determines the current scenario type of the vehicle. If the vehicle is currently in a high-risk scenario, the AR-HUD is controlled to enter simplified interaction mode. In simplified interaction mode, the control system displays core driving information, suppresses gesture recognition function, uses the driver's line of sight only for DMS monitoring, and prohibits unnecessary interactions. If the vehicle is currently in a low-risk scenario, the AR-HUD is controlled to enter full-function interaction mode. In full-function interaction mode, the control system displays the complete AR interface, activates gesture recognition function, and activates the preparation-confirmation process to support the complete interactive operation of the AR-HUD. In addition, the conditions for judging a high-risk scenario include, but are not limited to: vehicle speed > 80km / h or ADAS medium warning or severe weather. The conditions for judging a low-risk scenario include, but are not limited to: vehicle speed < 30km / h and no ADAS warning. In addition, the response time for triggering the security arbitrator is less than 50ms, the response time for confirming the alarm cancellation is less than 100ms, the response time for assessing high-risk scenarios is less than 200ms, and the response time for assessing low-risk scenarios is less than 200ms. Figure 7 The system should proactively reduce the interaction burden in situations where the driver needs to concentrate highly, while providing rich interactive functions when safety conditions permit, in order to enhance the driving experience.

[0068] Optionally, the vehicle augmented reality head-up display interaction method also includes the following execution steps: Step S16: In response to the driver looking at any virtual button for a duration greater than a preset time threshold, the augmented reality head-up display function is controlled to enter the preparation state. Step S17: In the preparatory state, control the virtual buttons to display in a preset state to prompt the driver to provide feedback on a specified gesture type. The preset state includes: highlighting and overlaying a pulsating animation.

[0069] In this embodiment of the invention, the ready state is a state in which the system waits for confirmation after the driver has been continuously looking at the virtual button for more than a preset time threshold.

[0070] Preset state refers to a special visual effect presented by virtual buttons in a ready state, with the purpose of providing clear visual feedback to the driver. The display method of preset state may include, but is not limited to, highlighting the virtual button to be viewed and overlaying a pulsating animation on it.

[0071] The response that the duration of the driver's gaze at any virtual button exceeds a preset time threshold, and the control of the augmented reality head-up display function to enter the ready state can be understood as follows: when the AR-HUD system detects that the driver's gaze time at a certain virtual button reaches or exceeds a preset time threshold (such as 400 milliseconds), the system will determine that the driver may have the intention to operate the virtual button, and then enter the ready state.

[0072] In the ready state, displaying the virtual buttons in a preset state to prompt the driver to provide feedback on a specified gesture type can be understood as follows: in the ready state, the system will not immediately execute the function related to the virtual button being viewed, but will wait for further gesture confirmation signal. By displaying the virtual buttons in a preset state, the system prompts the driver to confirm the gesture.

[0073] As can be seen, by setting a pre-set state, the system can effectively filter out unintentional eye movements by the driver, ensuring that every operation is based on the driver's clear intention, thereby reducing operational errors caused by accidental triggering. The visual feedback of the pre-set state, such as highlighting and pulsating animations, provides intuitive and user-friendly interface prompts, allowing the driver to clearly understand that the system is responding to their gaze and awaiting their gesture confirmation, thus improving the intuitiveness of operation and the user experience.

[0074] Optionally, the vehicle augmented reality head-up display interaction method also includes the following execution steps: Step S18: Continuously monitor the vehicle's driving scenario type; Step S19: In response to the driving scenario type being the first scenario type, control the vehicle to enter the full-function interactive mode.

[0075] In this embodiment of the invention, continuous monitoring of the vehicle's driving scenario type can be understood as the system being able to determine the current driving scenario type of the vehicle in real time by monitoring key parameters such as vehicle speed, distance between the vehicle and surrounding obstacles, road conditions, weather conditions, and driver status (fatigue, attention level, etc.).

[0076] Responding to the driving scenario type being the first scenario type, controlling the vehicle to enter the full-function interactive mode can be understood as follows: when the driving scenario type is judged to be the first scenario type, it indicates that the current driving environment is relatively safe and there is no high-risk situation that poses a direct threat to driving safety, and the vehicle's AR-HUD system is controlled to enter the full-function interactive mode.

[0077] For example, when a vehicle is slowly moving in a parking lot or safely parked in a private driveway, the driving scenario type may be identified as the first scenario type. In this state, the AR-HUD will be fully enabled, and the driver can control complex functions such as music playback, adjusting the air conditioning temperature, and viewing navigation through gaze and various gestures without worrying about affecting driving safety.

[0078] As can be seen, by continuously monitoring the types of driving scenarios, the system can automatically adjust the AR-HUD's interaction mode according to changes in the driving environment, ensuring that the AR-HUD can provide the most appropriate interactive interface and functions under any circumstances.

[0079] Optionally, in step S13, controlling the vehicle to activate the augmented reality head-up display function corresponding to the virtual button includes the following execution steps: Step S131: Convert the interaction commands between the driver and the augmented reality head-up display function into vehicle network messages; Step S132: Control the vehicle to activate the augmented reality head-up display function corresponding to the virtual button according to the vehicle network message.

[0080] In this embodiment of the invention, converting the interaction commands between the driver and the augmented reality head-up display (AR-HUD) into in-vehicle network messages can be understood as converting the driver's interactions with the AR-HUD through gaze and gestures into a standardized in-vehicle network message format. This conversion process includes, but is not limited to: the system first parses the driver's interaction commands received from the AR-HUD, identifying the specific operational intent, such as adjusting volume or switching navigation information. Then, these operation commands are encoded into a message format conforming to an in-vehicle network protocol (such as the CAN bus protocol). To ensure that the commands can be correctly transmitted and understood between multiple electronic control units in the vehicle, the system needs to convert the interaction commands into standardized network messages, typically including specific identifiers, data fields, and control commands.

[0081] Controlling the vehicle to activate the augmented reality head-up display function corresponding to the virtual button based on the in-vehicle network message can be understood as the converted network message being sent to the corresponding electronic control unit, such as the audio control unit, air conditioning system control unit, navigation system, etc., through the in-vehicle network to execute the corresponding augmented reality head-up display function.

[0082] As can be seen, by converting the driver's interactive commands into vehicle network messages, and then using the vehicle network messages to control the vehicle to perform corresponding functions, this embodiment of the invention ensures the integration and coordination between the AR-HUD system and the vehicle control system. This not only improves the reliability and execution of interactive commands, but also enables the AR-HUD to participate in the intelligent management and automated operation of the vehicle as part of the vehicle control network, thereby enhancing the level of intelligence in the cockpit and the driving experience.

[0083] Optionally, in step S10, obtaining the driving scenario type includes the following execution steps: Step S101: Obtain vehicle speed and response information of the advanced driver assistance system; Step S102: In response to the vehicle speed being lower than the first speed threshold and the advanced driver assistance system not issuing an alarm, the driving scenario type is determined to be the first scenario type; Step S103: In response to the vehicle speed being greater than the second speed threshold or the advanced driver assistance system issuing a first alarm, the driving scenario type is determined to be the second scenario type, wherein the first alarm corresponds to the first processing priority, and the second speed threshold is greater than the first speed threshold; Step S104: In response to the advanced driver assistance system issuing a second alarm, the driving scenario type is determined to be a third scenario type, wherein the second alarm corresponds to a second processing priority, and the second processing priority is higher than the first processing priority.

[0084] In this embodiment of the invention, the first warning typically represents a high-risk scenario in the driving environment, requiring the driver to be more vigilant, but not immediately interrupting the current driving operation. Examples include slight lane departure, relatively slow approach of a vehicle ahead, etc., which are not limited here.

[0085] The second type of warning corresponds to an emergency situation in the driving environment that requires the driver to take immediate action. Examples include forward collision warning and emergency braking warning, etc., without limitation.

[0086] Obtaining vehicle speed and advanced driver assistance system (ADAS) response information can be understood as the system collecting real-time vehicle speed information through the vehicle's built-in sensors and monitoring the status of the ADAS to determine if any warnings have been generated by the ADAS.

[0087] In response to the vehicle speed being lower than the first speed threshold and the advanced driver assistance system not issuing an alert, determining the driving scenario type as the first scenario type can be understood as follows: when the vehicle's current speed is lower than the first speed threshold (e.g., 30 km / h) and the ADAS system does not detect any potential risks, the current driving environment is determined to be a relatively safe first scenario type.

[0088] For example, when driving slowly in a parking lot, the vehicle speed is below a first speed threshold and the ADAS system does not detect any risk, the system determines the driving scenario type as the first scenario type.

[0089] In response to a vehicle speed exceeding a second speed threshold or an ADAS system issuing a first warning, the driving scenario type is determined to be the second scenario type. This can be understood as follows: when the vehicle speed exceeds the second speed threshold (e.g., 80 km / h), or the ADAS system issues a first warning, but the level of danger does not reach the level of immediately threatening safety, the driving scenario type is determined to be the second scenario type.

[0090] For example, when a vehicle is traveling on a highway and its speed exceeds a second speed threshold, or when the ADAS detects a vehicle rapidly approaching from behind, the system determines the driving scenario type as the second scenario type.

[0091] Responding to the second warning issued by the advanced driver assistance system and determining the driving scenario type as the third scenario type can be understood as the ADAS system issuing the second warning indicating that the vehicle is in the third scenario type.

[0092] For example, in extreme cases, such as when an obstacle suddenly appears ahead or the ADAS system triggers an emergency braking warning, the system determines the driving scenario type as the third scenario type.

[0093] As can be seen, the embodiments of the present invention ensure that the AR-HUD system can determine the scene type according to the complexity of the current driving environment in order to optimize the information display and interaction process, thereby providing the best driving experience while ensuring driving safety.

[0094] Optionally, the vehicle augmented reality head-up display interaction method also includes the following execution steps: In the ready state, the vehicle's audio components emit prompt sounds, or tactile vibrations are superimposed on the vehicle's steering wheel.

[0095] In this embodiment of the invention, in the preparatory state, controlling the vehicle's audio components to emit prompt sound effects, or superimposing tactile vibration prompts on the vehicle's steering wheel, can be understood as, in the preparatory state, as an alternative, controlling the vehicle's audio components to emit prompt sound effects, or superimposing tactile vibration prompts on the vehicle's steering wheel, so that the driver can make timely gesture feedback to activate the corresponding AR-HUD function.

[0096] As can be seen, the above steps can provide additional sensory feedback when the driver is preparing to interact with the system through sight and gestures, enhancing the driver's awareness of the system status and ensuring the smooth progress of the interaction process.

[0097] Figure 8 This is an architecture diagram of a vehicle augmented reality head-up display interactive system according to one embodiment of the present invention, such as... Figure 8 As shown, the vehicle augmented reality head-up display (HUD) interaction system comprises a perception layer, a decision-making and control layer, and an execution layer. The perception layer is responsible for multi-source data acquisition, including eye tracking, gesture recognition, and vehicle status data. The decision-making and control layer is the core processing unit, primarily used for data fusion, intelligent decision-making, and safety arbitration. The execution layer is responsible for command execution and user feedback, including AR-HUD display, vehicle control, and audio prompts.

[0098] The perception layer includes a gaze tracking module, a gesture recognition module, and a vehicle data interface. The gaze tracking module primarily uses the in-cabin DMS camera to calculate the driver's facial orientation and eye rotation angle, thereby determining the coordinates of the driver's gaze point on the AR-HUD virtual display plane. The gesture recognition module uses the in-cabin camera to capture images of the driver's hands, and uses a pre-trained convolutional neural network model to identify hand posture, gesture type, and the approximate spatial location of the gesture. The vehicle data interface continuously acquires information such as vehicle speed, turn signal status, and ADAS signals through the vehicle's controller area network bus to reflect the safety status of the current driving environment. The decision and control layer includes a multimodal data fusion unit, a dynamic decision-maker, and a safety arbitrator. The multimodal data fusion unit is responsible for integrating and fusing multi-source data (gaze data, gesture data, and vehicle status data) from the perception layer. The dynamic decision-maker, based on the fused data, determines which interaction mode is suitable for the current driving scenario. In other words, the dynamic decision-maker can understand and predict the driver's intentions, judge whether the combination of gaze and gesture constitutes a valid interaction signal, and thus decide whether to execute an interaction operation and, if so, which operation to execute. The safety arbitrator operates independently of the main interaction flow, continuously monitoring ADAS signals (such as forward collision warning and automatic emergency braking) and DMS fatigue scores to assess driving safety risks. The execution layer includes an AR-HUD rendering engine, an audio feedback unit, and vehicle control actuators. The AR-HUD rendering engine dynamically adjusts and updates the information displayed on the AR-HUD, including highlighting virtual controls, providing animated feedback, and adjusting display content based on driving scenarios and user actions. The audio feedback unit provides audible prompts or confirmations. When the user performs a valid operation or the system state changes, the audio feedback unit can notify the user via voice or audio signals, such as voice prompts when the system enters a "ready state" or "confirm execution" operation. The vehicle control actuators translate commands issued by the decision-making layer into control actions that the vehicle can perform, such as adjusting vehicle functions (air conditioning, navigation, volume control, etc.) via the vehicle's controller area network bus.

[0099] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of the present invention.

[0100] This embodiment also provides a vehicle augmented reality head-up display interaction device, which is used to implement the above embodiments and preferred embodiments, and will not be repeated as already described. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0101] Figure 9 This is a structural block diagram of a vehicle augmented reality head-up display interactive device according to one embodiment of the present invention, such as... Figure 9 As shown, taking a vehicle augmented reality head-up display (HUD) interaction device 900 as an example, the device includes: a first acquisition module 901, used to acquire the driving scene type; a first control module 902, used to control the vehicle to enter a full-function interaction mode in response to the driving scene type being a first scene type, wherein the first scene type represents a scene where the driving danger level is lower than a first danger threshold, and the augmented reality head-up display function in the full-function interaction mode is in normal condition; a second acquisition module 903, used to acquire the driver's gaze information and the driver's gesture type in the full-function interaction mode; and a second control module 904, used to control the vehicle to activate the augmented reality head-up display function corresponding to the virtual button in response to the capture of the driver's gaze at any virtual button for a duration greater than a preset time threshold, and the capture of the driver's specified gesture type within a preset time period, wherein the virtual button is the control button for the augmented reality head-up display function, and the specified gesture type is used to confirm the activation of the augmented reality head-up display function.

[0102] It should be noted that the above modules can be implemented by software or hardware. For the latter, they can be implemented in the following ways, but are not limited to: all the above modules are located in the same processor; or, the above modules are located in different processors in any combination.

[0103] Embodiments of the present invention also provide a vehicle, including: a memory storing an executable program; and a processor for running the program, wherein the program executes the vehicle augmented reality head-up display interaction method described above during runtime.

[0104] Embodiments of the present invention also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to perform the steps in any of the above method embodiments when run on a computer or processor.

[0105] Optionally, in this embodiment, the computer-readable storage medium may be configured to store a computer program for performing the following steps: Step S10: Obtain the driving scenario type; Step S11: In response to the driving scenario type being the first scenario type, control the vehicle to enter the full-function interactive mode, wherein the first scenario type is used to represent a scenario where the driving danger level is lower than the first danger threshold, and the augmented reality head-up display function in the full-function interactive mode is in normal state. Step S12: In full-function interactive mode, obtain the driver's gaze information and the driver's gesture type; Step S13: In response to the capture of the driver looking at any virtual button for a duration greater than a preset time threshold, and the capture of the driver's specified gesture type within a preset time period, the vehicle is controlled to activate the augmented reality head-up display function corresponding to the virtual button. The virtual button is the control button for the augmented reality head-up display function, and the specified gesture type is used to confirm the activation of the augmented reality head-up display function.

[0106] Optionally, in this embodiment, the computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0107] Embodiments of the present invention also provide an electronic device including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0108] Optionally, in this embodiment, the processor in the above-described electronic device may be configured to run a computer program to perform the following steps: Step S10: Obtain the driving scenario type; Step S11: In response to the driving scenario type being the first scenario type, control the vehicle to enter the full-function interactive mode, wherein the first scenario type is used to represent a scenario where the driving danger level is lower than the first danger threshold, and the augmented reality head-up display function in the full-function interactive mode is in normal state. Step S12: In full-function interactive mode, obtain the driver's gaze information and the driver's gesture type; Step S13: In response to the capture of the driver looking at any virtual button for a duration greater than a preset time threshold, and the capture of the driver's specified gesture type within a preset time period, the vehicle is controlled to activate the augmented reality head-up display function corresponding to the virtual button. The virtual button is the control button for the augmented reality head-up display function, and the specified gesture type is used to confirm the activation of the augmented reality head-up display function.

[0109] Embodiments of the present invention also provide a computer program product, including a computer program that, when executed by a processor, implements the steps in any of the above method embodiments.

[0110] Optionally, in this embodiment, the computer program in the above-described computer program product can be configured to perform the following steps when executed by a processor: Step S10: Obtain the driving scenario type; Step S11: In response to the driving scenario type being the first scenario type, control the vehicle to enter the full-function interactive mode, wherein the first scenario type is used to represent a scenario where the driving danger level is lower than the first danger threshold, and the augmented reality head-up display function in the full-function interactive mode is in normal state. Step S12: In full-function interactive mode, obtain the driver's gaze information and the driver's gesture type; Step S13: In response to the capture of the driver looking at any virtual button for a duration greater than a preset time threshold, and the capture of the driver's specified gesture type within a preset time period, the vehicle is controlled to activate the augmented reality head-up display function corresponding to the virtual button. The virtual button is the control button for the augmented reality head-up display function, and the specified gesture type is used to confirm the activation of the augmented reality head-up display function.

[0111] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0112] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0113] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0114] In the several embodiments provided by this invention, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection can be through some interfaces; the indirect coupling or communication connection of units or modules can be electrical or other forms.

[0115] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0116] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0117] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0118] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A vehicle augmented reality head-up display interaction method, characterized in that, include: Obtain the driving scenario type; In response to the driving scenario type being a first scenario type, the vehicle is controlled to enter a full-function interactive mode, wherein the first scenario type is used to represent a scenario where the driving danger level is lower than a first danger threshold, and the augmented reality head-up display function in the full-function interactive mode is in a normal state. In the full-function interactive mode, the driver's gaze information and the driver's gesture type are obtained; In response to the capture of the driver gazing at any virtual button for a duration greater than a preset time threshold, and the capture of a specified gesture type of the driver within a preset time period, the vehicle is controlled to activate the augmented reality head-up display function corresponding to the virtual button, wherein the virtual button is the control button for the augmented reality head-up display function, and the specified gesture type is used to confirm the activation of the augmented reality head-up display function.

2. The method according to claim 1, characterized in that, The method further includes: In response to the driving scenario type being the second scenario type, the vehicle is controlled to enter a simplified interaction mode, wherein the second scenario type is used to represent a scenario where the driving danger level is greater than the first danger threshold and less than the second danger threshold, the first danger threshold is less than the second danger threshold, and some of the augmented reality head-up display functions are turned off in the simplified interaction mode; In the simplified interaction mode, the driver's gaze information is used to monitor the driver's driving status, the driver's gesture type is used to confirm the activation of part of the augmented reality head-up display function, and the display screen of the augmented reality head-up display function is used to display core driving information.

3. The method according to claim 1, characterized in that, The method further includes: In response to the driving scenario type being the third scenario type, the vehicle is controlled to enter a safety intervention mode, wherein the third scenario type is used to represent a scenario where the driving danger level is greater than the second danger threshold, and some of the augmented reality head-up display functions are turned off in the safety intervention mode; In the safety intervention mode, the driver's gaze information is used to monitor the driver's driving status, the gesture recognition function is locked, and the display screen of the augmented reality head-up display function is used to display emergency safety information.

4. The method according to claim 1, characterized in that, The method further includes: In response to the driver's gaze at any of the virtual buttons for a duration greater than the preset time threshold, the augmented reality head-up display function is controlled to enter a ready state; In the prepared state, the virtual button is controlled to be displayed in a preset state to prompt the driver to respond to the specified gesture type, wherein the preset state includes: highlighting and superimposing a pulsating animation.

5. The method according to claim 2 or 3, characterized in that, The method further includes: Continuously monitor the driving scenario type of the vehicle; In response to the driving scenario being the first scenario type, the vehicle is controlled to enter the full-function interactive mode.

6. The method according to claim 1, characterized in that, The step of controlling the vehicle to activate the augmented reality head-up display function corresponding to the virtual button includes: The interaction commands between the driver and the augmented reality head-up display function are converted into vehicle network messages. The vehicle is controlled to activate the augmented reality head-up display function corresponding to the virtual button based on the in-vehicle network message.

7. The method according to claim 1, characterized in that, The types of driving scenarios obtained include: Obtain the vehicle speed and the response information of the advanced driver assistance system; In response to the vehicle speed being lower than a first speed threshold and the advanced driver assistance system not issuing a warning, the driving scenario type is determined to be the first scenario type; In response to the vehicle speed being greater than a second speed threshold or the advanced driver assistance system issuing a first alarm, the driving scenario type is determined to be a second scenario type, wherein the first alarm corresponds to a first processing priority, and the second speed threshold is greater than the first speed threshold; In response to the advanced driver assistance system issuing a second alarm, the driving scenario type is determined to be a third scenario type, wherein the second alarm corresponds to a second processing priority, and the second processing priority is higher than the first processing priority.

8. The method according to claim 4, characterized in that, The method further includes: In the prepared state, the audio components of the vehicle are controlled to emit a prompt sound effect, or a tactile vibration prompt is superimposed on the steering wheel of the vehicle.

9. The method according to claim 1, characterized in that, The specified gesture types include: pinch, slide, clench fist, palm rotation, and air click.

10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, wherein the computer program is configured to execute the vehicle augmented reality head-up display interaction method as described in any one of claims 1 to 9 when running on a computer or processor.

Images