Method, apparatus, medium and program product for head-display driving

Abstract

Embodiments of the present disclosure relate to a method, an apparatus, a medium, and a program product for head-mounted driving. The method includes generating a road condition image including an augmented reality image based on environmental information of a vehicle. The method also includes obtaining state information of the vehicle. In addition, the method further comprises the step of displaying the road condition image and the state information through the head-mounted display device. Therefore, according to the scheme of the embodiment of the invention, the information can be directly projected to the front of the eyes of the driver by utilizing the head display equipment, so that the driver can perceive the driving environment and the vehicle state in a more immersive manner, and the driving efficiency, safety and comfort are improved.

Description

Technical Field

[0001] The embodiments of this disclosure generally relate to the field of computers, and more specifically, to methods, apparatus, media, and program products for head-mounted display driving. Background Technology

[0002] In-vehicle displays and interactive systems play a crucial role in modern automobiles. These systems significantly enhance the comfort and convenience of the driving experience by providing real-time information, intelligent navigation, and entertainment features. Excellent in-vehicle displays and interactive systems not only provide drivers with clear and intuitive data displays but also interact intelligently with them, helping them drive more safely and efficiently.

[0003] Human-Machine Interface (HMI) is a crucial means of information exchange and interaction between in-vehicle systems and drivers. Through HMI, drivers can communicate with vehicle systems in an intuitive and effective way, obtain the information they need, and control vehicle functions. HMI not only includes physical interfaces such as the instrument panel, touchscreen, and voice recognition, but also encompasses user experience design and optimized interaction logic, providing drivers with a convenient and intuitive user experience. Summary of the Invention

[0004] Embodiments of this disclosure provide a method, apparatus, medium, and program product for head-mounted display driving.

[0005] In a first aspect of this disclosure, a method for head-mounted display driving is provided. The method includes generating a road condition image comprising augmented reality images based on environmental information of a vehicle. The method also includes acquiring state information of the vehicle. Furthermore, the method includes displaying the road condition image and the state information via a head-mounted display device.

[0006] In a second aspect of this disclosure, an electronic device for head-mounted display driving is provided, the electronic device including a processor and a memory coupled to the processor and storing instructions that, when executed by the processor, cause the device to perform the steps of the method according to the first aspect of this disclosure.

[0007] In a third aspect of this disclosure, a computer-readable storage medium is provided that stores computer-executable instructions that, when executed, cause a computer to perform the steps of the method according to a first aspect of this disclosure.

[0008] According to a fourth aspect of this disclosure, a computer program product is provided. The computer program product is tangibly stored on a non-volatile computer-readable medium and includes machine-executable instructions that, when executed, cause a machine to perform the steps of the method according to a first aspect of this disclosure.

[0009] Please note that this summary is provided to introduce some concepts in a simplified form, which will be further described in the detailed embodiments below. The summary is not intended to identify key or essential features of this disclosure, nor is it intended to limit the scope of this disclosure. Attached Figure Description

[0010] The above and other objects, features, and advantages of this disclosure will become more apparent from a more detailed description of exemplary embodiments thereof, taken in conjunction with the accompanying drawings, in which:

[0011] Figure 1 A schematic diagram of an example environment in which the methods and / or processes according to embodiments of the present disclosure may be implemented is shown;

[0012] Figure 2 A flowchart illustrating a method for head-mounted driving according to an embodiment of the present disclosure is shown;

[0013] Figure 3 The illustration shows a schematic diagram of the process of generating a road condition image according to an embodiment of the present disclosure;

[0014] Figure 4 The illustration shows a schematic diagram of the process of adjusting vehicle settings according to an embodiment of the present disclosure;

[0015] Figure 5 The illustration shows a schematic diagram of the process of implementing a fused head-mounted display device and a vehicle HMI according to an embodiment of the present disclosure;

[0016] Figure 6 A schematic block diagram of an apparatus that can be used to implement embodiments of the present disclosure is shown.

[0017] In the various figures, the same or corresponding reference numerals indicate the same or corresponding parts. Detailed Implementation

[0018] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0019] In the description of embodiments of this disclosure, the term "comprising" and its variations should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc., may refer to different or the same objects unless explicitly indicated otherwise.

[0020] As mentioned earlier, in-vehicle displays and interactions are crucial for drivers, and the HMI (Hardware Interface) serves as the means for information exchange and interaction between the in-vehicle system and the driver. HMIs generally include switches, buttons, the vehicle's instrument panel, a head-up display (HUD), voice control, and more. However, existing HMIs have some shortcomings. For example, drivers need to look down at the instrument panel while driving, which can easily cause visual interruptions and lead to operational errors, potentially resulting in driving accidents. Furthermore, although a HUD (also known as a head-up display system) can project some of the necessary information onto the windshield directly in front of the driver's eyes, there is still a certain spatial distance between the HUD and the driver, preventing a truly immersive driving experience.

[0021] To address this, embodiments of this disclosure propose a head-mounted display (HMI) driving solution. This solution first generates a real-time road condition image based on the vehicle's environmental information, including an enhanced display image. Then, it acquires the vehicle's status information and finally displays the road condition image and status information via a head-mounted display device. Therefore, compared to traditional HMIs, the solution of this disclosure can directly project the road condition image and status information in front of the driver's eyes using a head-mounted display device, thus avoiding visual interruption caused by the driver looking down at the instrument panel. This allows the driver to be more immersive in perceiving the driving environment and vehicle status, thereby improving driving efficiency, safety, and comfort. Furthermore, using a head-mounted display device can also alleviate the computational and cost burden of HMIs and avoid the development burden of backward compatibility.

[0022] The following is for reference. Figures 1 to 6 The present disclosure is provided to illustrate the basic principles and several example implementations. It should be understood that these exemplary embodiments are given only to enable those skilled in the art to better understand and implement the embodiments of the present disclosure, and are not intended to limit the scope of the disclosure in any way.

[0023] Figure 1 A schematic diagram of an example environment 100 in which methods and / or processes according to embodiments of the present disclosure may be implemented is shown. Figure 1As shown, example environment 100 may include a vehicle 110. Vehicle 110 may include a head-mounted display device 120. A head-mounted display device is a wearable device worn on the head, typically including components such as a display screen, lenses, sensors, and speakers, and can provide augmented reality, virtual reality, or mixed reality experiences. Head-mounted display device 120 can generate road condition images 140 based on environmental information 130. Environmental information 130 can be acquired through cameras, LiDAR, or millimeter-wave radar on vehicle 110, and may include, but is not limited to, static and dynamic information about roads, vehicles, pedestrians, and other objects.

[0024] The head-mounted display device 140 can utilize environmental information 130 to generate a road condition image 140 that includes augmented reality images 142. Augmented reality images refer to images that integrate virtual digital information with real-world scenes, presenting a combination of real and virtual elements in the user's field of vision. Virtual elements can include images, videos, text, 3D models, etc., thereby enhancing the user's perception and understanding of the real world. For example, although a driver can see lane lines on the road, lane lines rendered with augmented reality are more prominent and clearer, making them easier for the driver to identify. These lane lines with superimposed augmented reality images not only provide additional visual cues but can also be combined with actual road markings to provide the driver with more comprehensive and intuitive road condition information.

[0025] Furthermore, the head-mounted display 140 can acquire status information 150. Status information 150 may include vehicle speed, RPM, light status, and warning information. For example, the head-mounted display 140 can wirelessly communicate with the vehicle control unit to acquire status information 150 from the vehicle control unit. Then, the head-mounted display 140 can display a road condition image 140, including augmented reality image 142, and status information 150 on its display screen. For example, users can not only see the actual road conditions through the head-mounted display 140, but also see virtual road elements rendered using augmented reality technology, and also see the current vehicle status information, thereby obtaining more comprehensive driving information.

[0026] It should be understood that the above examples are not intended to be limiting, but are merely illustrative for the purpose of aiding understanding, and the embodiments of this disclosure are not limited to the above examples.

[0027] Figure 2 A flowchart of a method 200 for head-mounted driving according to an embodiment of the present disclosure is illustrated. At block 202, a road condition image including augmented reality images can be generated based on environmental information of the vehicle. For example, combined with... Figure 1 The head-mounted display device 120 can generate a road condition image 150, including an augmented reality image 140, based on the environmental information 130 of the vehicle 110.

[0028] At box 204, vehicle status information can be obtained. For example, combined with... Figure 1 The head-mounted display device 120 can acquire the status information 150 of the vehicle 110. At box 206, road condition images and status information can be displayed via the head-mounted display device. For example, in combination with... Figure 1 The head-mounted display device 120 can display road condition images 140 and status information 150. For example, although the driver can see lane markings on the road, lane markings rendered with augmented reality are more prominent and clearer, making them easier for the driver to identify. These lane markings overlaid with augmented reality images not only provide additional visual cues but can also be combined with actual road markings to provide the driver with more comprehensive and intuitive road condition information. Furthermore, by displaying status information through the head-mounted display device, the driver can avoid looking down at the dashboard, keeping their eyes focused on the road conditions and reducing safety hazards.

[0029] Therefore, compared with traditional HMIs, the solution of the embodiments of this disclosure can directly project road condition images and status information to the driver's eyes using a head-mounted display device, thereby avoiding visual interruption caused by the driver looking down at the instrument panel. This allows the driver to be more immersed in perceiving the driving environment and vehicle status, thereby improving driving efficiency, safety, and comfort. In addition, using a head-mounted display device can also reduce the computational and cost burden of HMIs and avoid the development burden of backward compatibility for HMIs.

[0030] Figure 3 The illustration shows a schematic diagram of a process 300 for generating road condition images according to an embodiment of the present disclosure. At block 302, static and dynamic information about the vehicle's environment is acquired based on the vehicle's environmental information. In some embodiments, the environmental information can be acquired using cameras, lidar, or millimeter-wave radar. For example, vehicles are typically equipped with multiple cameras to capture images of the surrounding environment, providing visual information about roads, vehicles, pedestrians, and other objects. In embodiments of the present disclosure, the cameras may include, but are not limited to, front-facing cameras, rear-view cameras, side-facing cameras, fisheye lenses, and infrared cameras. For example, a front-facing camera can be used to capture road conditions, traffic signs, and other vehicles ahead of the vehicle; a fisheye lens can provide a wide-angle field of view for panoramic monitoring of the surrounding environment; and an infrared camera can provide better visibility at night or in adverse weather conditions.

[0031] In some embodiments, environmental information can be acquired using lidar. For example, lidar acquires distance information by emitting a laser beam and measuring the time it takes for it to reflect back, providing detailed information about the distance, shape, and surface features of surrounding objects. In some embodiments, environmental information can be acquired using millimeter-wave radar, which uses millimeter-wave signals to detect the position and velocity of surrounding objects. In some embodiments, data from cameras, lidar, and millimeter-wave radar can be fused to generate environmental information. Each sensor has its own advantages; by fusing data from multiple sensors, the ability to perceive the surrounding environment can be enhanced, dependence on a single sensor can be reduced, and the robustness and reliability of the system can be improved.

[0032] In some embodiments, static information in the vehicle's environment may include, but is not limited to, road structures, buildings, landmarks, natural environments, and other static objects (e.g., traffic lights or curbs). For example, road structures may include, but are not limited to, road markings (e.g., lane lines), road surface markings (e.g., arrows), and traffic signs (e.g., speed limit signs, prohibitory signs). Furthermore, road structures may also identify road environment features such as intersections, curves, and pedestrian crossings. This static information is relatively stable and does not change significantly in a short period, making it crucial for the vehicle's environmental perception and navigation. By identifying and understanding this static information, the vehicle can better understand its surroundings and plan safe and efficient driving routes.

[0033] In some embodiments, dynamic information about the vehicle's environment may include, but is not limited to, the status of other vehicles, pedestrian status, traffic light status, and weather conditions. For example, the status of other vehicles may include their position, speed, and acceleration. Traffic light status refers to the status information of traffic lights used to indicate traffic flow and driving priorities. Weather conditions may include, for example, rain and snow, and changes in weather conditions may affect vehicle visibility and road traction. This dynamic information is relatively instantaneous and may change over time, making it crucial for vehicle environmental perception and safe driving. By acquiring, analyzing, and understanding this dynamic information in a timely manner, intelligent decisions can be made to cope with different traffic and road conditions.

[0034] At box 304, augmented reality images can be generated based on static and dynamic information. Augmented reality images refer to images that integrate virtual digital information with real-world scenes, presenting a combination of real and virtual elements in the user's field of vision. Virtual elements can include images, videos, text, 3D models, etc., thereby enhancing the user's perception and understanding of the real world. In some embodiments, static and dynamic information can be preprocessed (e.g., data calibration, denoising, and registration) to ensure data accuracy and consistency. In some embodiments, environmental modeling can be performed based on static and dynamic information. For example, static and dynamic information can be integrated into a unified environmental model. Static information can be used to construct the basic structure of the environment, while dynamic information can be used to update the position and state of moving objects in the model. Then, the environmental model can be used to generate augmented reality images. In some embodiments, augmented reality images may include, but are not limited to, virtual traffic signs, lane lines, driving directions, traffic flow, and other elements.

[0035] At box 306, a road condition image can be generated based on environmental information and augmented reality (AR) images. In some embodiments, an AR image can be overlaid onto the environmental information to generate a road condition image that includes both real environmental information and virtual AR images. For example, the position and pose of virtual elements can be updated in real time using a tracking algorithm, and the position and pose of virtual elements can be transformed from a virtual coordinate system to a camera coordinate system for overlay with environmental information. For example, perspective transformation and projection transformation can be used to align and blend virtual elements with real environmental information to ensure a consistent visual effect with the real scene. The generated road condition image can then be output to a head-mounted display device for the driver to view and reference. By integrating static and dynamic information to generate AR images and overlaying AR images with environmental information to generate road condition images, richer and more intuitive visual information can be provided, helping drivers better perceive and understand the surrounding road environment.

[0036] In some embodiments, the generated road condition image can be displayed on a first portion of the head-mounted display's screen. In some embodiments, vehicle status information can be displayed on a second portion of the head-mounted display's screen. For example, status information may include vehicle speed, RPM, light status, and warning information. By displaying this status information, the driver can understand the vehicle's current status and operating condition. Furthermore, in some embodiments, a portion of the status information can be displayed on the head-mounted display and another portion on the dashboard. In some embodiments, the status information to be displayed on the head-mounted display and the dashboard can be determined based on predefined rules. In some embodiments, the information to be displayed on the head-mounted display can be determined based on the user's usage habits. For example, if the head-mounted display detects that the user frequently looks down (e.g., over a certain period of time), it can proactively ask (e.g., via voice or display) what the user is viewing and whether to display the corresponding status information on the head-mounted display. By coordinating the head-mounted display and the dashboard, and rationally allocating and optimizing the display of status information, a more intelligent, personalized, and safe driving experience can be provided to the driver, improving driving comfort and efficiency.

[0037] In some embodiments, the angle of the user's head orientation can be acquired. For example, sensors built into the head-mounted display (such as gyroscopes, accelerometers, etc.) can be used to acquire the user's head orientation angle. These sensors can detect the rotation and tilt of the user's head, thereby determining the user's gaze direction. In some embodiments, a road condition image corresponding to the angle can be displayed on a first portion of the display screen. For example, if the user's head is facing left, a road condition image of the left lane is displayed, which includes augmented reality images, such as virtual traffic signs, lane lines, etc. In this way, the displayed road condition image can be adjusted in real time according to the user's head orientation angle, so that the user can always see road condition information corresponding to their current gaze direction.

[0038] Figure 4The illustration shows a schematic diagram of a process 400 for adjusting vehicle settings according to an embodiment of the present disclosure. At block 402, user interaction can be received. In some embodiments, user interaction can be received via voice recognition. For example, user voice input can be acquired and processed to receive user interaction. In some embodiments, user interaction can be received via gesture tracking. For example, the movement trajectory and changes of the user's hand can be tracked in consecutive image frames, and the tracked gestures can be converted into corresponding control commands or operations. In some embodiments, user interaction can be received via eye tracking. For example, an eye-tracking device (such as an eye tracker) can be used to detect the movement of the user's eyes, and the eye movement data can be analyzed to identify the user's gaze point, and then the user's interaction intention can be determined based on the user's gaze point and eye movement pattern. In some embodiments, user interaction can be received via head rotation. For example, a head-tracking device (such as a gyroscope, accelerometer, etc.) or a sensor such as a camera can be used to detect the user's head posture and rotation, and the user's interaction intention can be determined based on the user's head rotation. In some embodiments, user interaction can be received via physical buttons. For example, the user's interaction intention can be determined by button signals or events received from physical buttons on a head-mounted display device.

[0039] At box 404, vehicle settings can be adjusted based on interaction. For example, a user can use voice to adjust the in-car air conditioning temperature; the head-mounted display can receive and analyze the user's voice, then send an adjustment signal to the vehicle control unit. Alternatively, a user can use gesture tracking to turn on the turn signals; the camera on the head-mounted display can capture and analyze the user's gestures, then turn on the corresponding turn signals based on preset gesture rules. Finally, a user can adjust audio using physical buttons, such as adjusting the volume on the head-mounted display.

[0040] At box 406, the adjustment results of vehicle settings can be displayed on the head-mounted display's screen. For example, after adjusting the vehicle settings, the head-mounted display can obtain the corresponding adjustment results. For instance, after adjusting the vehicle's air conditioning temperature, the adjusted temperature can be displayed on the head-mounted display's screen, which the driver can see without looking down while driving. For instance, when the user turns on the turn signal, a dynamic image of the turn signal can be displayed on the head-mounted display's screen. In some embodiments, if the user performs continuous setting adjustments, the head-mounted display's screen will continuously update the displayed content to reflect the latest setting adjustment results. In addition, while displaying the adjustment results, user interaction options can also be provided, allowing the user to further adjust the settings.

[0041] In some embodiments, external devices can be connected wirelessly based on user interaction. For example, a mobile phone can be wirelessly connected to the head-mounted display device through user interaction (e.g., voice interaction). In some embodiments, the user interface of the external device can be displayed on the display screen of the head-mounted display device. For example, the mobile phone interface can be displayed on the display screen of the head-mounted display device, so that the user does not have to look down to look at the phone. In addition, the application interface of the mobile phone, such as a social media application, can be displayed on the display screen of the head-mounted display device, so that the user does not have to look down when sending or receiving messages, and can always keep their eyes on the road, reducing safety hazards.

[0042] In some embodiments, one or more interactive entertainment options can be displayed when the vehicle is parked. For example, after a user parks, music playback, video playback, games, applications, etc., can be displayed on the head-mounted display screen to provide entertainment options for the user while waiting. Furthermore, compared to the entertainment options provided by conventional vehicle HMIs, head-mounted displays can offer a more immersive entertainment experience. Through virtual reality or augmented reality, users can immerse themselves in a virtual environment and obtain a better interactive entertainment experience. For example, while conventional vehicle HMIs can provide music playback functionality, through head-mounted displays, users can enter a virtual music concert, immersively watch virtual music performances, interact with virtual artists, or experience virtual stage effects.

[0043] Figure 5 The illustration shows a schematic diagram of a process 500 for implementing a fusion of a head-mounted display (HMI) and a vehicle HMI according to an embodiment of the present disclosure. At block 502, predetermined rules for combining the HMI and the HMI can be determined. For example, it can be specified which interactive operations can be performed through the HMI and which can be performed through the vehicle HMI. For instance, the HMI can perform interactions such as driving assistance functions, traffic information display, and navigation guidance, while the vehicle HMI is responsible for performing interactive operations such as audio control and air conditioning adjustment.

[0044] At box 504, the interface between the head-mounted display (HMI) and the vehicle HMI can be adjusted. For example, adjusting the interface between the HMI and the vehicle HMI ensures effective communication and interaction between them. Integration between the HMI and the vehicle HMI can be achieved, for example, by defining data transmission protocols, adjusting hardware connection interfaces, and updating software drivers.

[0045] At box 506, user interaction and display can be achieved through a head-mounted display (HMD) and the vehicle's Human-Machine Interface (HMI). For example, users can interact with the vehicle system via control buttons, gesture recognition, and voice commands on the HMD. Simultaneously, users can also interact with the vehicle system through physical buttons and a touchscreen on the HMI. For instance, users can adjust the air conditioning temperature via the touchscreen interface, select driving modes via a knob, or turn on the vehicle's hazard lights via a button. In some embodiments, both the HMD and the HMI can display relevant information and feedback in real time. For example, on the HMD, users can see navigation guidance, vehicle status information, and driver assistance function prompts; on the HMI, users can see a music playback interface, an air conditioning adjustment interface, and driving information displays. Furthermore, regardless of whether the user interacts through the HMD or the HMI, the system can respond to the user's actions in real time and provide timely feedback. In some embodiments, the HMD can be mounted above the driver's seat, reducing the driver's wearing burden and making it more comfortable for extended periods.

[0046] Therefore, by integrating head-mounted displays and vehicle HMIs, the cost of vehicle HMIs can be reduced, and the burden of backward compatibility can be avoided. As long as the interface is properly connected and the power supply is compatible, it can be compatible with any vehicle.

[0047] Figure 6 A schematic block diagram of a device 600 that can be used to implement embodiments of the present disclosure is shown. Device 600 can be the device or apparatus described in the embodiments of the present disclosure. Figure 6 As shown, device 600 includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes according to computer program instructions stored in read-only memory (ROM) 602 or loaded from storage unit 608 into random access memory (RAM) 603. RAM 603 may also store various programs and data required for the operation of device 600. CPU 601, ROM 602, and RAM 603 are interconnected via bus 604. Input / output (I / O) interface 605 is also connected to bus 604.

[0048] Multiple components in device 600 are connected to I / O interface 605, including: input unit 606, such as keyboard, mouse, etc.; output unit 607, such as various types of monitors, speakers, etc.; storage unit 608, such as disk, optical disk, etc.; and communication unit 609, such as network card, modem, wireless transceiver, etc. Communication unit 609 allows device 600 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0049] The various methods or processes described above can be executed by processing unit 601. For example, in some embodiments, the methods can be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program can be loaded and / or installed on device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by CPU 601, one or more steps or actions in the methods or processes described above can be performed.

[0050] In some embodiments, the methods and processes described above can be implemented as a computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of this disclosure.

[0051] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example, but not limited to, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0052] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper cables, fiber optic cables, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to computer-readable storage media within the respective computing / processing device.

[0053] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​and conventional procedural programming languages. The computer-readable program instructions may execute entirely on a user's computer, partially on a user's computer, as a standalone software package, partially on a user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0054] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, these instructions create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0055] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more blocks of a flowchart and / or block diagram.

[0056] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of devices, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0057] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for head-mounted display driving: Based on the vehicle's environmental information, generate road condition images including augmented reality images; Obtain the vehicle's status information; and The road condition images and status information are displayed using a head-mounted display device.

2. The method of claim 1, wherein generating the road condition image including the augmented reality image comprises: Based on the environmental information of the vehicle, obtain static and dynamic information in the environment of the vehicle; The augmented reality image is generated based on the static information and the dynamic information; as well as The road condition image is generated based on the environmental information and the augmented reality image.

3. The method according to claim 2, wherein displaying the road condition image and the status information includes: The road condition image is displayed on the first part of the display screen of the head-mounted display device; as well as The status information is displayed in a second portion of the display screen of the head-mounted display device, wherein the status information includes at least one of the vehicle's speed, rotational speed, and lighting status.

4. The method of claim 3, wherein displaying the traffic image comprises: Get the angle of the user's head orientation; as well as Based on the angle, the road condition image corresponding to the angle is displayed in the first part of the display screen.

5. The method according to claim 4, further comprising: Receive user interaction; Based on the user interaction, adjust the vehicle settings of the vehicle; as well as The adjustment results of the vehicle settings are displayed on the screen.

6. The method according to claim 4, further comprising: Connect to external devices via wireless network based on user interaction; as well as The user interface of the external device is displayed on the screen.

7. The method according to claim 4, further comprising: In response to the vehicle being parked, one or more interactive entertainment options are displayed; as well as Based on user interaction, select one or more interactive entertainment options from the interactive entertainment options.

8. The method according to any one of claims 5-7, wherein receiving the user interaction comprises at least one of the following: The user interaction is received via voice recognition; The user interaction is received via gesture tracking; The user interaction is received via eye tracking; The user interaction is received by rotating the head; or The user interaction is received via physical buttons.

9. The method according to claim 1, further comprising: Determine predetermined rules for human-computer interaction combining the head-mounted display and the vehicle; as well as Based on the predetermined rules, user interaction is processed through the human-computer interaction of the head-mounted display device and the vehicle, respectively.

10. The method of claim 1, wherein the head-mounted display is mounted above the driver's seat.

11. An electronic device, comprising: processor, and A memory, coupled to the processor, stores instructions that, when executed by the processor, cause the device to perform the method according to any one of claims 1 to 10.

12. A computer-readable storage medium storing computer-executable instructions that, when executed, cause a computer to perform the method according to any one of claims 1 to 10.

13. A computer program product tangibly stored on a non-volatile computer-readable medium and comprising machine-executable instructions that, when executed, cause a machine to perform the method according to any one of claims 1 to 10.

Images