Head-mounted display control method and device and computer readable storage medium
By building a communication link between the head-mounted display device and the vehicle, using the vehicle's computing power to process image data, the problem of insufficient computing power of the device is solved, lightweight and efficient image processing is achieved, and user experience and driving safety are improved. .
Patent Information
- Application Number
- CN202310573871.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-18
- Publication Date
- 2024-11-22
AI Technical Summary
Due to the limitation of chip computing power in head-mounted display devices, the image resolution and display frame rate cannot achieve the best user experience, and it cannot be combined with the smart cockpit, resulting in poor user experience.
By building a communication link between the head-mounted display device and the vehicle, sending image data to the vehicle for processing and rendering, utilizing the vehicle's powerful computing power to reduce power consumption and computing pressure on the device side, and optimizing image display priority and stitching Process to improve user experience.
It effectively reduces the hardware implementation cost and power consumption of on-board VR/AR equipment, realizes the lightweight and efficient image processing of the equipment, and improves the user's wearing comfort and driving safety.
Smart Images

Figure CN119002844A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of vehicle-mounted augmented reality / virtual reality devices, and in particular to a head-mounted display control method, device, and computer-readable storage medium. Background Art
[0002] Virtual reality (VR) technology can provide an interactive and immersive virtual three-dimensional space, and augmented reality (AR) technology can superimpose display content with the real world through holographic projection lenses. Their common feature is that they use computers to generate a virtual world and project multimedia information such as images, videos, and sounds onto organs that humans can perceive. In existing technologies, VR / AR has different technical implementation methods. Depending on the location of the computing chip, head-mounted display devices such as VR or AR devices can be divided into an all-in-one mode and a split-machine mode. In the all-in-one mode, the computing chip is on the head-mounted display device. The VR / AR all-in-one machine receives game scene information from the network, completes the spatial positioning of the head display based on its own sensors, and realizes the creation and image rendering of the spatial scene by the all-in-one machine, and then displays the image information on the LCD display screen. In the split-machine solution, the computing chip is not on the head-mounted display device, but in an external PC or other host.
[0003] Currently, there is no application that combines VR / AR technology with smart cockpits. And due to the limitation of chip computing power in head-mounted display devices, image resolution and display frame rate cannot achieve the best user experience. The development of new energy vehicles and new technologies has enabled the central computing platform of vehicles to have more powerful computing power and image rendering capabilities than the chips in general head-mounted display devices.
[0004] Therefore, an improved VR / AR implementation method needs to be proposed. Summary of the invention
[0005] The main purpose of this application is to provide a head display control method, device and computer-readable storage medium, aiming to reduce the power consumption and computing pressure of the vehicle-mounted augmented reality / virtual reality device side.
[0006] To achieve the above-mentioned purpose, the present application provides a head display control method, which is applied to a head-mounted display device, and the method includes: When the distance between the head mounted display device and the vehicle is less than a first preset distance threshold and the vehicle is powered on, establishing a first communication link with the vehicle; After the first communication link is successfully established, the current data to be processed is sent to the vehicle through the first communication link, wherein the current data to be processed is the current image data to be processed; The first processing result data returned by the vehicle in response to the current data to be processed is received, and an image is displayed according to the first processing result data.
[0007] Optionally, the step of performing image display according to the first processing result data includes: Detecting whether there are at least two image frames to be displayed in the first processing result data; If yes, then analyzing each of the image frames to be displayed to obtain a display priority corresponding to each of the image frames to be displayed; The image frames to be displayed are displayed in sequence from high to low according to the display priority.
[0008] Optionally, after the step of parsing and obtaining the display priority corresponding to each of the image frames to be displayed, the method further includes: Detecting whether the display priority of each of the image frames to be displayed is lower than a preset priority; If not, executing: the step of displaying the image frames to be displayed in order from high to low according to the display priority; If so, scaling is performed on each of the image frames to be displayed according to a preset scaling rule, and each of the scaled image frames corresponding to the image frames to be displayed, wherein the preset scaling rule is that the higher the display priority of the image frame to be displayed, the larger the area of the scaled image frame corresponding to the image frame to be displayed; The scaled image frames are stitched together according to a preset stitching template to obtain a stitched image frame, and the stitched image frame is displayed in a window display area of the head-mounted display device, wherein the area size of the stitched image frame is adapted to the window display area.
[0009] Optionally, the step of performing stitching processing on the scaled image frames according to a preset stitching template to obtain a stitched image frame includes: According to the areas of the zoomed image frames from large to small, the zoomed image frames are sequentially spliced from the center of the area of the preset splicing template to the periphery to obtain a spliced image frame.
[0010] Optionally, before the step of establishing a first communication link with the vehicle, the method further includes: Detecting whether the head mounted display device establishes a second communication link with the user terminal device; If so, disconnect the second communication link and execute: the step of establishing a first communication link with the vehicle.
[0011] Optionally, after the step of displaying an image according to the first processing result data, the method further comprises: After determining that the first communication link cannot be successfully established with the vehicle, establishing a second communication link with the user terminal device; After the second communication link is successfully established, sending the current data to be processed to the user terminal device through the second communication link; Receive second processing result data returned by the user terminal device in response to the current data to be processed, and display an image according to the second processing result data.
[0012] Optionally, after the step of disconnecting the second communication link, the method further includes: Sending link establishment notification information to the vehicle, wherein the link establishment notification information is used to trigger the vehicle to establish a third communication link with the user terminal device; After the third communication link is successfully established, user data information from the user terminal device is acquired through the third communication link and the first communication link.
[0013] In addition, the present application also provides a head display control method, which is applied to a vehicle, and the method includes: Detecting whether the vehicle is in a powered-on state; If so, when the distance between the vehicle and the head mounted display device is less than a first preset distance threshold, establishing a first communication link with the head mounted display device; After the first communication link is successfully established, obtaining current data to be processed of the head mounted display device through the first communication link, wherein the current data to be processed is image data to be processed; Performing computational processing on the current data to be processed to obtain first processing result data; The first processing result data is sent to the head mounted display device, so that the head mounted display device displays an image in response to the first processing result data.
[0014] The present application also provides a head-mounted display control device, which is a physical device, and includes: a memory, a processor, and a program of the head-mounted display control method stored in the memory and executable on the processor. When the program of the head-mounted display control method is executed by the processor, the steps of the head-mounted display control method as described above can be implemented.
[0015] The present application also provides a computer-readable storage medium, on which a program for implementing the head display control method is stored, and the program for implementing the head display control method is executed by a processor to implement the steps of the head display control method as described above.
[0016] The present application also provides a computer program product, including a computer program, which implements the steps of the head display control method as described above when the computer program is executed by a processor.
[0017] The present application provides a head-mounted display control method, device and computer-readable storage medium. The technical solution of the present application is to establish a first communication link with the vehicle when the distance between the head-mounted display device and the vehicle is less than a first preset distance threshold and the vehicle is in a powered-on state; after the first communication link is successfully established, send current data to be processed to the vehicle through the first communication link, wherein the current data to be processed is current image data to be processed; receive first processing result data returned by the vehicle in response to the current data to be processed, and display an image according to the first processing result data.
[0018] For automobiles, with the development of automobile intelligence, electrification and networking, automobiles can replace mobile phones as the computing center of augmented reality / virtual reality devices by relying on their own intelligent infrastructure. The present application reduces the hardware implementation cost of the vehicle-mounted augmented reality / virtual reality devices by transferring the computing center of the vehicle-mounted augmented reality / virtual reality devices to the vehicle-mounted system and reusing the computing power of the vehicle-mounted system, thereby achieving lightweight vehicle-mounted augmented reality / virtual reality devices, using the vehicle-mounted system to provide hardware support for the augmented reality / virtual reality devices, and using the computing power of the vehicle-mounted system to perform image rendering or screen posture synchronization on the image data to be processed, and then sending it to the vehicle-mounted augmented reality / virtual reality device for display, thereby effectively reducing the power consumption and computing pressure on the vehicle-mounted augmented reality / virtual reality device side. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0020] In order to more clearly illustrate the technical solutions in this embodiment or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0021] Figure 1 This is a flowchart of the first embodiment of the head display control method of the present application; Figure 2 This is a flow chart of the second embodiment of the head display control method of the present application; Figure 3 This is a flowchart of the third embodiment of the head display control method of the present application; Figure 4 This is a schematic diagram of the module structure of the head display control device according to an embodiment of the present application; Figure 5 Schematic diagram of the device structure of the hardware operating environment involved in the head mounted display device in this embodiment.
[0022] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0023] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the scope of protection of the present invention.
[0024] In this embodiment, the head-mounted display device includes but is not limited to mixed reality (Mixed Reality)-MR devices (such as MR glasses or MR helmets), augmented reality (Augmented Reality)-AR devices (such as AR glasses or AR helmets), virtual reality-(Virtual Reality)-VR devices (such as VR glasses or VR helmets), extended reality (Extended Reality)-XR devices or some combination thereof, etc. head-mounted display devices.
[0025] Embodiment 1 Virtual reality (VR) technology can provide an interactive and immersive virtual three-dimensional space, and augmented reality (AR) technology can superimpose display content with the real world through holographic projection lenses. Their common feature is that they use computers to generate a virtual world and project multimedia information such as images, videos, and sounds onto organs that humans can perceive. In existing technologies, VR / AR has different technical implementation methods. Depending on the location of the computing chip, head-mounted display devices such as VR or AR devices can be divided into an all-in-one mode and a split-machine mode. In the all-in-one mode, the computing chip is on the head-mounted display device. The VR / AR all-in-one machine receives game scene information from the network, completes the spatial positioning of the head display based on its own sensors, and realizes the creation and image rendering of the spatial scene by the all-in-one machine, and then displays the image information on the LCD display screen. In the split-machine solution, the computing chip is not on the head-mounted display device, but in an external PC or other host.
[0026] Currently, there is no application that combines VR / AR technology with smart cockpits. And due to the limitation of chip computing power in head-mounted display devices, image resolution and display frame rate cannot achieve the best user experience. The development of new energy vehicles and new technologies has enabled the central computing platform of vehicles to have more powerful computing power and image rendering capabilities than the chips in general head-mounted display devices.
[0027] Therefore, an improved VR / AR implementation method needs to be proposed.
[0028] Based on this, please refer to Figure 1 This embodiment provides a head display control method, the method comprising: Step S10, when the distance between the head mounted display device and the vehicle is less than a first preset distance threshold and the vehicle is powered on, establishing a first communication link with the vehicle; In this embodiment, the first preset distance threshold can be set by those skilled in the art according to actual conditions, and this embodiment does not make any specific limitation.
[0029] It is easy to understand that the first communication link refers to the communication link for communication connection between the head mounted display device and the vehicle.
[0030] Step S20, after the first communication link is successfully established, sending the current data to be processed to the vehicle through the first communication link, wherein the current data to be processed is the current image data to be processed; Step S30, receiving first processing result data returned by the vehicle in response to the current data to be processed, and performing image display according to the first processing result data.
[0031] Head-mounted display devices such as VR or AR devices are used in the vehicle field as auxiliary tools for driving. Since the driver needs to wear them for a long time while driving, they should be as light as possible. In order to ensure lightweight, head-mounted display devices currently use a split solution, that is, they are paired with mobile phones, and the computing center is transferred to the mobile phone end. The head-mounted display device is responsible for data collection and display, thereby reducing the power consumption and computing pressure on the glasses side and reducing the weight of the glasses. However, the chip computing power of the mobile phone is also limited, and the image resolution and display frame rate still cannot achieve the best user experience.
[0032] As for vehicles, with the development of automobile intelligence, electrification and networking, cars can replace mobile phones and become the computing center of head-mounted display devices with their own intelligent infrastructure.
[0033] Specifically, the embodiments of the present application provide a head-mounted display control method, device and computer-readable storage medium. The technical solution of the embodiments of the present application is to establish a first communication link with the vehicle when the distance between the head-mounted display device and the vehicle is less than a first preset distance threshold and the vehicle is in a powered-on state; after the first communication link is successfully established, the current data to be processed is sent to the vehicle through the first communication link, wherein the current data to be processed is the current image data to be processed; and the first processing result data returned by the vehicle in response to the current data to be processed is received, and an image is displayed according to the first processing result data.
[0034] For automobiles, with the development of automobile intelligence, electrification and networking, automobiles can replace mobile phones as the computing center of augmented reality / virtual reality devices by relying on their own intelligent infrastructure. The embodiments of the present application transfer the computing center of the on-board augmented reality / virtual reality device to the vehicle's on-board system, reuse the computing power of the on-board system, thereby reducing the hardware implementation cost of the on-board augmented reality / virtual reality device, and realizing the lightweight of the on-board augmented reality / virtual reality device. The on-board system is used to provide hardware support for the augmented reality / virtual reality device, and the computing power of the on-board system is used to perform image rendering or screen posture synchronization on the image data to be processed, and then send it to the on-board augmented reality / virtual reality device for display, thereby effectively reducing the power consumption and computing pressure on the on-board augmented reality / virtual reality device side.
[0035] In addition, this embodiment greatly reduces the weight of the head-mounted display device and increases the wearing comfort of the user by transferring the computing power center of the head-mounted display device to the vehicle end.
[0036] For example, please refer to Figure 2 , the step of displaying an image according to the first processing result data comprises: Step S31, detecting whether there are at least two image frames to be displayed in the first processing result data; Step S32: if yes, then analyze each of the image frames to be displayed to obtain the display priority corresponding to each of the image frames to be displayed; In this embodiment, each image frame to be displayed can be identified to identify the image category corresponding to each image frame to be displayed. Specifically, it can be divided into a first category of images with a priority lower than a preset priority, such as media images and entertainment images, and also includes the second category of audio including security images and communication images. In this embodiment, each image frame to be displayed is analyzed to obtain the display priority corresponding to each image frame to be displayed, and then the display order of each image frame to be displayed is determined based on the display priority corresponding to each image frame to be displayed, so that users can have a better image perception experience in the car without missing important image information prompts.
[0037] Step S33, displaying the image frames to be displayed in order from high to low display priorities.
[0038] This embodiment analyzes each image frame to be displayed when it is detected that there are at least two image frames to be displayed in the first processing result data, and obtains the display priority corresponding to each image frame to be displayed. Then, each image frame to be displayed is displayed in order from high to low according to the display priority, thereby realizing display priority sorting of each image frame to be displayed, avoiding safety hazards caused by untimely display of safety information with higher display priority (such as collision warning information or sharp bend road condition information), preventing the driver from failing to promptly realize the dangerous working condition of the current vehicle, and thus effectively improving driving safety.
[0039] In an practicable manner, after the step of parsing and obtaining the display priority corresponding to each of the image frames to be displayed, the method further includes: Step A10, detecting whether the display priority of each of the image frames to be displayed is lower than the preset priority; In this embodiment, the display priority of each image frame to be displayed is lower than the preset priority, indicating that each image frame to be displayed is a scalable image frame. If there is at least one image frame to be displayed with a priority higher than the preset priority among the image frames to be displayed, it means that each image frame to be displayed cannot be scaled, thereby avoiding the safety hazards caused by the failure to attract the driver's attention after scaling of safety information with higher display priority (such as collision warning information or sharp curve road condition information), preventing the driver from failing to realize the dangerous working condition of the current vehicle in time, and effectively improving driving safety.
[0040] The preset priority level can be set by those skilled in the art according to actual conditions, and is not specifically limited in this embodiment.
[0041] Step A20, if not, then executing: the step of displaying the image frames to be displayed in order from high to low according to the display priority; Step A30: if yes, scaling is performed on each of the to-be-displayed image frames according to a preset scaling rule, and each of the to-be-displayed image frames corresponds to a scaled image frame, wherein the preset scaling rule is that the higher the display priority of the to-be-displayed image frame, the larger the area of the scaled image frame corresponding to the to-be-displayed image frame; Step A40, stitching each scaled image frame according to a preset stitching template to obtain a stitched image frame, and displaying the stitched image frame in the window display area of the head mounted display device, wherein the area size of the stitched image frame is adapted to the window display area.
[0042] In this embodiment, when the display priorities are all lower than the preset priority, that is, for example, all the image contents to be displayed are entertainment images, not safety images, and there is no need to display them separately to highlight the information, then the non-safety images can be scaled and spliced to be displayed all at once, thereby ensuring driving safety while allowing users to simultaneously obtain multiple information, thereby allowing users to have a better image perception experience in the car while allowing users to promptly notice prompts of important image information.
[0043] In a possible implementation manner, the step of performing stitching processing on the scaled image frames according to a preset stitching template to obtain a stitched image frame includes: Step B10, according to the areas of the zoomed image frames from large to small, the zoomed image frames are sequentially spliced from the center of the area of the preset splicing template to the periphery to obtain a spliced image frame.
[0044] This embodiment obtains a stitched image frame by stitching each zoomed image frame from the center of the area of a preset stitching template to the periphery according to the area of the zoomed image frame from large to small, thereby optimizing the UI (User Interface) layout, further ensuring that the user has a better image perception experience in the car while allowing the user to promptly notice prompts of important image information.
[0045] In order to help understand the technical concept of the present application, a specific embodiment is listed below: The smart cockpit is far superior to mobile phones in terms of computing power, battery life, and heat dissipation, so the smart cockpit is the preferred choice as the computing center of the split AR.
[0046] 1. The user wears a mobile phone and AR glasses and approaches the vehicle from a distance outside the vehicle. 1) When the vehicle's intelligent cockpit system is started, it detects the user's mobile phone's wireless network hotspot, 2) At this time, the smart cockpit system connects to the mobile phone hotspot and sends a computing center transfer message to the mobile phone 3) The mobile phone sends a computing center transfer message to the AR glasses, which connects to the car computer hotspot and uses the car computer as the new computing center. After the AR glasses complete the transfer of the computing center, the mobile phone will connect itself to the hotspot of the vehicle. The smart cockpit system, as the new computing center of the AR glasses, can transmit vehicle-related data to the AR glasses. At the same time, the mobile phone can use the smart cockpit as the data forwarding center to transmit the data information of the mobile phone to the AR glasses.
[0047] For example, users in the car can receive WeChat messages from their mobile phones while checking the remaining journey time and distance to the destination transmitted by the car computer.
[0048] 2. When the user wears a mobile phone and AR glasses and gets out of the car 1) The vehicle's smart cockpit system is turned off or the vehicle is far away, causing the AR glasses and mobile phone to be unable to connect to the smart cockpit hotspot 2) Turn on your phone's hotspot 3) AR glasses automatically search for connected hotspots from the glasses storage and connect to the mobile phone hotspot At this point, users can continue to use their phones as the computing center of the AR glasses and operate them normally.
[0049] It should be noted that the many details described in this specific embodiment are only helpful for understanding the technical concept of the present application and do not constitute a limitation of the present application. More simple transformations based on the technical concept of the present application should all be within the scope of protection of the present application.
[0050] Embodiment 2 Based on the first embodiment of the present application, in another embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. On this basis, before the step of establishing a first communication link with the vehicle, the method further includes: Step C10, detecting whether the head mounted display device establishes a second communication link with the user terminal device; Step C20: If yes, disconnect the second communication link and execute: the step of establishing a first communication link with the vehicle.
[0051] The user terminal device may be a mobile phone, a tablet computer, a laptop computer, or a smart wearable device, etc., and this embodiment does not make any specific limitation to this.
[0052] In this embodiment, the second communication link refers to a communication link for communication connection between the head mounted display device and the user terminal device.
[0053] In a possible implementation, after the step of displaying an image according to the first processing result data, the method further includes: Step D10, after determining that the first communication link cannot be successfully established with the vehicle, establishing a second communication link with the user terminal device; Step D20, after the second communication link is successfully established, sending the current data to be processed to the user terminal device through the second communication link; Step D30, receiving second processing result data returned by the user terminal device in response to the current data to be processed, and performing image display according to the second processing result data.
[0054] This embodiment detects whether the head mounted display device has established a second communication link with the user terminal device before the step of establishing a first communication link with the vehicle. If the head mounted display device has established the second communication link with the user terminal device, the second communication link is disconnected, and the step of establishing the first communication link with the vehicle is executed. After determining that the first communication link with the vehicle cannot be successfully established, the second communication link is established with the user terminal device. After the second communication link is successfully established, the current data to be processed is sent to the user terminal device through the second communication link, and the second processing result data returned by the user terminal device in response to the current data to be processed is received. Image display is performed according to the second processing result data, thereby realizing the interconnection strategy design among the vehicle smart cockpit, the mobile phone, and the AR glasses, so that the user head mounted display device can share the computing power of the smart cockpit and the user terminal device (such as a mobile phone) in a time-sharing manner, and the computing power center of the head mounted display device can be seamlessly switched inside and outside the vehicle, so that the user can have a smooth use experience.
[0055] In an practicable manner, after the step of disconnecting the second communication link, the method further includes: Step E10, sending link establishment notification information to the vehicle, wherein the link establishment notification information is used to trigger the vehicle to establish a third communication link with the user terminal device; After the third communication link is successfully established, user data information from the user terminal device is acquired through the third communication link and the first communication link.
[0056] In this embodiment, the third communication link refers to a communication link for communication connection between the vehicle and the user terminal device.
[0057] This embodiment sends a link establishment notification message to the vehicle after disconnecting the second communication link, wherein the link establishment notification message is used to trigger the vehicle to establish a third communication link with the user terminal device. After the third communication link is successfully established, the user data information from the user terminal device is obtained through the third communication link and the first communication link. Therefore, through the interconnection strategy design of the vehicle's smart cockpit, mobile phone, and AR glasses, the user can take into account both the smart cockpit and mobile phone information data streams in the car, and the user can simultaneously obtain information from different media devices, thereby enabling the user to have a better image perception experience in the car while allowing the user to promptly notice the prompts of important image information.
[0058] Embodiment 3 The present application also provides a head display control method, which is applied to a vehicle. Please refer to Figure 3 , the method further comprises: Step S40, detecting whether the vehicle is in a powered-on state; Step S50: If yes, when the distance between the vehicle and the head mounted display device is less than a first preset distance threshold, a first communication link is established with the head mounted display device; Step S60, after the first communication link is successfully established, obtaining current data to be processed of the head mounted display device through the first communication link, wherein the current data to be processed is image data to be processed; Step S70, performing operation processing on the current data to be processed to obtain first processing result data; Step S80: Send the first processing result data to the head mounted display device, so that the head mounted display device displays an image in response to the first processing result data.
[0059] Specifically, an embodiment of the present application provides a head-mounted display control method, which is applied to a vehicle. The technical solution of the embodiment of the present application is to establish a first communication link with the head-mounted display device when the distance between the vehicle and the head-mounted display device is less than a first preset distance threshold and the vehicle is in a powered-on state; after the first communication link is successfully established, current data to be processed of the head-mounted display device is obtained through the first communication link, wherein the current data to be processed is current image data to be processed, and then the current data to be processed is processed to obtain first processing result data, and then the first processing result data is sent to the head-mounted display device, so that the head-mounted display device displays an image in response to the first processing result data.
[0060] For automobiles, with the development of automobile intelligence, electrification and networking, automobiles can replace mobile phones as the computing center of augmented reality / virtual reality devices by relying on their own intelligent infrastructure. The embodiments of the present application transfer the computing center of the on-board augmented reality / virtual reality device to the vehicle's on-board system, reuse the computing power of the on-board system, thereby reducing the hardware implementation cost of the on-board augmented reality / virtual reality device, and realizing the lightweight of the on-board augmented reality / virtual reality device. The on-board system is used to provide hardware support for the augmented reality / virtual reality device, and the computing power of the on-board system is used to perform image rendering or screen posture synchronization on the image data to be processed, and then send it to the on-board augmented reality / virtual reality device for display, thereby effectively reducing the power consumption and computing pressure on the on-board augmented reality / virtual reality device side.
[0061] In addition, this embodiment greatly reduces the weight of the head-mounted display device and increases the wearing comfort of the user by transferring the computing power center of the head-mounted display device to the vehicle end.
[0062] Embodiment 4 The embodiment of the present invention also provides a head display control device, please refer to Figure 4, the head display control device is applied to a head mounted display device, and the device comprises: A building module 10 is used to build a first communication link with the vehicle when the distance between the head mounted display device and the vehicle is less than a first preset distance threshold and the vehicle is powered on; A communication module 20, configured to send the current data to be processed to the vehicle through the first communication link after the first communication link is successfully established, wherein the current data to be processed is the current image data to be processed; The display module 30 is used to receive first processing result data returned by the vehicle in response to the current data to be processed, and to perform image display according to the first processing result data.
[0063] Optionally, the display module 30 is further used for: Detecting whether there are at least two image frames to be displayed in the first processing result data; If yes, then analyzing each of the image frames to be displayed to obtain a display priority corresponding to each of the image frames to be displayed; The image frames to be displayed are displayed in sequence from high to low according to the display priority.
[0064] Optionally, the display module 30 is further used for: Detecting whether the display priority of each of the image frames to be displayed is lower than a preset priority; If not, executing: the step of displaying the image frames to be displayed in order from high to low according to the display priority; If so, scaling is performed on each of the image frames to be displayed according to a preset scaling rule, and each of the scaled image frames corresponding to the image frames to be displayed, wherein the preset scaling rule is that the higher the display priority of the image frame to be displayed, the larger the area of the scaled image frame corresponding to the image frame to be displayed; The scaled image frames are stitched together according to a preset stitching template to obtain a stitched image frame, and the stitched image frame is displayed in a window display area of the head-mounted display device, wherein the area size of the stitched image frame is adapted to the window display area.
[0065] Optionally, the display module 30 is further used for: According to the areas of the zoomed image frames from large to small, the zoomed image frames are sequentially spliced from the center of the area of the preset splicing template to the periphery to obtain a spliced image frame.
[0066] Optionally, the building block 10 is further used for: Output the preset unlocking method selection window; Detecting whether the head mounted display device establishes a second communication link with the user terminal device; If so, disconnect the second communication link and execute: the step of establishing a first communication link with the vehicle.
[0067] Optionally, the building block 10 is further used for: After determining that the first communication link cannot be successfully established with the vehicle, establishing a second communication link with the user terminal device; After the second communication link is successfully established, sending the current data to be processed to the user terminal device through the second communication link; Receive second processing result data returned by the user terminal device in response to the current data to be processed, and perform image display according to the second processing result data.
[0068] Optionally, the building block 10 is further used for: Sending link establishment notification information to the vehicle, wherein the link establishment notification information is used to trigger the vehicle to establish a third communication link with the user terminal device; After the third communication link is successfully established, user data information from the user terminal device is acquired through the third communication link and the first communication link.
[0069] The head display control device provided in the embodiment of the present invention adopts the head display control method in the above embodiment 1 or embodiment 2, which can reduce the power consumption and computing pressure of the vehicle-mounted augmented reality / virtual reality device side. Compared with the prior art, the beneficial effects of the head display control device provided in the embodiment of the present invention are the same as the beneficial effects of the head display control method provided in the above embodiment, and other technical features in the head display control device are the same as the features disclosed in the above embodiment method, which will not be repeated here.
[0070] Embodiment 5 An embodiment of the present invention provides a head-mounted display device, the head-mounted display device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the head-mounted display control method in the above-mentioned embodiment 1.
[0071] Reference below Figure 5, which shows a schematic diagram of the structure of a head-mounted display device suitable for implementing the embodiment of the present disclosure. The head-mounted display device in the embodiment of the present disclosure includes, but is not limited to, mixed reality (Mixed Reality)-MR devices (such as MR glasses or MR helmets), augmented reality (Augmented Reality)-AR devices (such as AR glasses or AR helmets), virtual reality-(Virtual Reality)-VR devices (such as VR glasses or VR helmets), extended reality (Extended Reality)-XR devices or some combination thereof, etc. head-mounted display devices. Figure 5 The head mounted display device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.
[0072] like Figure 5 As shown, the head-mounted display device may include a processing device 1001 (such as a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM1002) or a program loaded from a storage device into a random access memory (RAM1004). In RAM1004, various programs and data required for the operation of the head-mounted display device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other via a bus 1005. An input / output (I / O) interface is also connected to the bus 1005.
[0073] Typically, the following systems may be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 1003 including, for example, a tape, a hard disk, etc.; and communication devices 1009. The communication device 1009 may allow the head-mounted display device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a head-mounted display device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be implemented or have instead.
[0074] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.
[0075] The head mounted display device provided by the present invention adopts the head mounted display control method in the above embodiment, which can reduce the power consumption and computing pressure of the vehicle-mounted augmented reality / virtual reality device side. Compared with the prior art, the beneficial effects of the head mounted display device provided by the embodiment of the present invention are the same as the beneficial effects of the head mounted display control method provided by the above embodiment, and the other technical features in the head mounted display device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.
[0076] It should be understood that the various parts of the present disclosure can be implemented with hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.
[0077] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.
[0078] Embodiment 6 An embodiment of the present invention provides a computer-readable storage medium having computer-readable program instructions stored thereon, and the computer-readable program instructions are used to execute the head display control method in the above embodiment.
[0079] The computer-readable storage medium provided in the embodiment of the present invention may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.
[0080] The computer-readable storage medium may be included in the head-mounted display device; or may exist independently without being assembled into the head-mounted display device.
[0081] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the head-mounted display device, the head-mounted display device: establishes a first communication link with the vehicle when the distance between the head-mounted display device and the vehicle is less than a first preset distance threshold and the vehicle is in a powered-on state; after the first communication link is successfully established, sends current data to be processed to the vehicle through the first communication link, wherein the current data to be processed is image data to be processed; receives first processing result data returned by the vehicle in response to the current data to be processed, and displays an image according to the first processing result data.
[0082] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0083] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0084] The modules involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.
[0085] The computer-readable storage medium provided by the present invention stores computer-readable program instructions for executing the above-mentioned head display control method, which can reduce the power consumption and computing pressure of the vehicle-mounted augmented reality / virtual reality device side. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the embodiment of the present invention are the same as the beneficial effects of the head display control method provided by the above-mentioned embodiment 1 or embodiment 2, and will not be repeated here.
[0086] Embodiment 7 An embodiment of the present invention further provides a computer program product, including a computer program, which implements the steps of the head display control method as described above when the computer program is executed by a processor.
[0087] The computer program product provided by the present application can reduce the power consumption and computing pressure of the vehicle-mounted augmented reality / virtual reality device side. Compared with the prior art, the beneficial effects of the computer program product provided by the embodiment of the present invention are the same as the beneficial effects of the head display control method provided by the above-mentioned embodiment 1 or embodiment 2, and will not be repeated here.
[0088] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent processing scope of the present application.
Claims
1. A head mounted display control method, characterized in that: The head display control method is applied to a head mounted display device, and the method comprises: When the distance between the head mounted display device and the vehicle is less than a first preset distance threshold and the vehicle is powered on, establishing a first communication link with the vehicle; After the first communication link is successfully established, the current data to be processed is sent to the vehicle through the first communication link, wherein the current data to be processed is the current image data to be processed; The first processing result data returned by the vehicle in response to the current data to be processed is received, and an image is displayed according to the first processing result data.
2. The head display control method according to claim 1, characterized in that: The step of displaying an image according to the first processing result data comprises: Detecting whether there are at least two image frames to be displayed in the first processing result data; If yes, then analyzing each of the image frames to be displayed to obtain a display priority corresponding to each of the image frames to be displayed; The image frames to be displayed are displayed in sequence from high to low according to the display priority.
3. The head display control method according to claim 2, characterized in that: After the step of parsing and obtaining the display priority corresponding to each of the image frames to be displayed, the method further includes: Detecting whether the display priority of each of the image frames to be displayed is lower than a preset priority; If not, executing: the step of displaying the image frames to be displayed in order from high to low according to the display priority; If so, scaling is performed on each of the image frames to be displayed according to a preset scaling rule, and each of the scaled image frames corresponding to the image frames to be displayed, wherein the preset scaling rule is that the higher the display priority of the image frame to be displayed, the larger the area of the scaled image frame corresponding to the image frame to be displayed; The scaled image frames are stitched together according to a preset stitching template to obtain a stitched image frame, and the stitched image frame is displayed in a window display area of the head-mounted display device, wherein the area size of the stitched image frame is adapted to the window display area.
4. The head display control method according to claim 3, characterized in that: The step of performing stitching processing on each zoomed image frame according to a preset stitching template to obtain a stitched image frame comprises: According to the areas of the zoomed image frames from large to small, the zoomed image frames are sequentially spliced from the center of the area of the preset splicing template to the periphery to obtain a spliced image frame.
5. The head display control method according to claim 1, characterized in that: Before the step of establishing a first communication link with the vehicle, the method further includes: Detecting whether the head mounted display device establishes a second communication link with the user terminal device; If so, disconnect the second communication link and execute: the step of establishing a first communication link with the vehicle.
6. The head display control method according to claim 5, characterized in that: After the step of displaying an image according to the first processing result data, the method further includes: After determining that the first communication link cannot be successfully established with the vehicle, establishing a second communication link with the user terminal device; After the second communication link is successfully established, sending the current data to be processed to the user terminal device through the second communication link; Receive second processing result data returned by the user terminal device in response to the current data to be processed, and display an image according to the second processing result data.
7. The head display control method according to claim 5, characterized in that: After the step of disconnecting the second communication link, the method further includes: Sending link establishment notification information to the vehicle, wherein the link establishment notification information is used to trigger the vehicle to establish a third communication link with the user terminal device; After the third communication link is successfully established, user data information from the user terminal device is acquired through the third communication link and the first communication link.
8. A head mounted display control method, characterized in that: The head display control method is applied to a vehicle, and the method comprises: Detecting whether the vehicle is in a powered-on state; If so, when the distance between the vehicle and the head mounted display device is less than a first preset distance threshold, establishing a first communication link with the head mounted display device; After the first communication link is successfully established, obtaining current data to be processed of the head mounted display device through the first communication link, wherein the current data to be processed is image data to be processed; Performing computational processing on the current data to be processed to obtain first processing result data; The first processing result data is sent to the head mounted display device, so that the head mounted display device displays an image in response to the first processing result data.
9. A head mounted display control device, characterized in that: The head display control device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the steps of the head display control method according to any one of claims 1 to 8.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program for implementing a head display control method, and the program for implementing a head display control method is executed by a processor to implement the steps of the head display control method according to any one of claims 1 to 8.
