Control method and device, head-mounted display equipment and medium
By detecting the wearer's gaze area, blurring virtual objects that do not need to be rendered and stopping image data transmission, it solves the problem of excessive power consumption in on-board applications, and realizes the optimization of rendering content in different scenarios and reduces delay and power consumption to improve user experience.
Patent Information
- Application Number
- CN202310548125.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-12
- Publication Date
- 2024-11-12
AI Technical Summary
AR glasses consume too much power when rendering scenes with multiple applications, resulting in heat generation and delay, affecting the user experience.
By detecting the wearer's current gaze area, virtual objects in applications that interact with external objects of the vehicle and stop image data transmission, the display screen brightness is reduced when the gaze area is located in the vehicle; conversely, when the gaze area is located outside the vehicle, the display screen brightness is reduced; otherwise, when the gaze area is located outside the vehicle , blur the virtual objects in applications that interact with objects in the vehicle and stop image data transmission to improve brightness.
In different usage scenarios, focus on rendering content, reduce rendering content, reduce delay and power consumption, and improve the performance and user experience of AR glasses.
Smart Images

Figure CN118939164A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present disclosure relate to the technical field of wearable devices, and more specifically, to a control method, a control device, a head-mounted display device, and a computer-readable storage medium. Background Art
[0002] With the development of augmented reality technology, AR products such as AR glasses have appeared in more and more application scenarios, such as car-mounted AR glasses.
[0003] AR glasses consume very high power when using multiple applications to render scenes, which leads to problems such as heating and high latency that affect user experience. Summary of the invention
[0004] The purpose of the embodiments of the present disclosure is to provide a control method, device, head-mounted display device and medium.
[0005] According to a first aspect of an embodiment of the present disclosure, a control method is provided, the method comprising:
[0006] Detecting a current gaze area of a wearer of the head mounted display device;
[0007] When the current gaze area is located within the vehicle, blurring a first virtual object in a first application rendered and displayed by a first canvas and / or stopping transmission of image data to be transmitted of the first application; wherein the first application is an application that interacts with an object outside the vehicle, and the first virtual object corresponds to the object outside the vehicle;
[0008] When the current gaze area is outside the vehicle, blur the second virtual object in the second application rendered and displayed by the second canvas and / or stop the transmission of the image data to be transmitted of the second application; wherein, the second application is an application that interacts with objects within the vehicle, and the second virtual object corresponds to the object within the vehicle.
[0009] Optionally, detecting a current gaze area of a wearer of the head mounted display device includes:
[0010] Acquiring target parameter information; wherein the target parameter information includes current posture information of the vehicle, a three-dimensional model of the vehicle and current sitting position information of the wearer;
[0011] constructing a collision box of the three-dimensional model of the vehicle according to the target parameter information;
[0012] Generate a virtual identifier based on the wearer's current gaze information;
[0013] In a case where the virtual marker collides with a first area of a collision box of the three-dimensional model of the vehicle, determining that a current gaze area of the wearer of the head mounted display device is located within the vehicle;
[0014] When the virtual marker collides with a second area of the collision box of the three-dimensional model of the vehicle, it is determined that a current gaze area of the wearer of the head mounted display device is outside the vehicle.
[0015] Optionally, the head mounted display device is a pair of smart glasses, the smart glasses include lenses made of electrochromic glass, and the method further includes:
[0016] When the current gaze area is located in the vehicle, setting the color of the lens to a first color to reduce the brightness of the display screen of the head mounted display device;
[0017] When the current gaze area is outside the vehicle, the color of the lens is set to a second color to increase the brightness of the display screen of the head-mounted display device.
[0018] Optionally, the acquiring target parameter information includes:
[0019] establishing a short-range communication connection with the vehicle;
[0020] Based on the short-range communication connection, receiving the current position data of the vehicle and the three-dimensional model of the vehicle transmitted by the vehicle; and,
[0021] Acquire a first image captured by a camera device of the head mounted display device;
[0022] According to the first image, current sitting position information of the wearer is determined.
[0023] Optionally, after determining the current sitting position information of the wearer according to the first image, the method further includes:
[0024] Providing a modification entry for modifying the current sitting position information of the wearer;
[0025] acquiring the current sitting position information of the wearer re-entered through the modification entry;
[0026] The input current sitting position information of the wearer is used as the current sitting position information of the wearer.
[0027] According to a second aspect of an embodiment of the present disclosure, a control device is provided, the device comprising:
[0028] A detection module, used to detect a current gaze area of a wearer of the head mounted display device;
[0029] a control module, configured to, when the current gaze area is located within the vehicle, blur a first virtual object in a first application rendered and displayed by a first canvas and / or stop transmission of image data to be transmitted of the first application; wherein the first application is an application that interacts with an object outside the vehicle, and the first virtual object corresponds to the object outside the vehicle;
[0030] The control module is used to blur the second virtual object in the second application rendered and displayed by the second canvas and / or stop the transmission of the image data to be transmitted of the second application when the current gaze area is outside the vehicle; wherein the second application is an application that interacts with the object inside the vehicle, and the second virtual object corresponds to the object inside the vehicle.
[0031] Optionally, the detection module is specifically used to:
[0032] Acquiring target parameter information; wherein the target parameter information includes current posture information of the vehicle, a three-dimensional model of the vehicle and current sitting position information of the wearer;
[0033] constructing a collision box of the three-dimensional model of the vehicle according to the target parameter information;
[0034] Generate a virtual identifier based on the wearer's current gaze information;
[0035] In a case where the virtual marker collides with a first area of a collision box of the three-dimensional model of the vehicle, determining that a current gaze area of the wearer of the head mounted display device is located within the vehicle;
[0036] When the virtual marker collides with a second area of the collision box of the three-dimensional model of the vehicle, it is determined that a current gaze area of the wearer of the head mounted display device is located outside the vehicle.
[0037] Optionally, the control module is further used to:
[0038] When the current gaze area is located inside the vehicle, setting the color of the lens to a first color to reduce the brightness of the display screen of the head mounted display device;
[0039] When the current gaze area is outside the vehicle, the color of the lens is set to a second color to increase the brightness of the display screen of the head-mounted display device.
[0040] According to a third aspect of an embodiment of the present disclosure, a head-mounted display device is provided, the head-mounted display device comprising:
[0041] Memory for storing executable computer instructions;
[0042] A processor is used to execute the control method according to the first aspect above under the control of the executable computer instructions.
[0043] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which computer instructions are stored. When the computer instructions are executed by a processor, the control method described in the first aspect above is executed.
[0044] One beneficial effect of the embodiments of the present disclosure is that the head-mounted display device detects the current gaze area of the wearer of the head-mounted display device, and when the current gaze area is located inside the vehicle, the virtual objects in the application rendered and displayed by the first canvas that interact with objects outside the vehicle can be blurred, and / or the transmission of the image data to be transmitted of the application can be stopped, and the virtual objects in the application correspond to the objects outside the vehicle. When the current gaze area is located outside the vehicle, the virtual objects in the application rendered and displayed by the second canvas that interact with objects inside the vehicle can be blurred, and / or the transmission of the image data to be transmitted of the application can be stopped, and the virtual objects in the application correspond to the objects inside the vehicle. In other words, through the embodiments of the present application, different usage scenarios can be distinguished, and different content can be rendered in different usage scenarios, thereby reducing the rendering content, delay and power consumption.
[0045] Other features and advantages of the present specification will become apparent from the following detailed description of exemplary embodiments of the present specification with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the specification and, together with the description, serve to explain the principles of the specification.
[0047] Figure 1 is a schematic diagram of the hardware configuration of a control system according to an embodiment of the present disclosure;
[0048] Figure 2 is a flow chart of a control method according to an embodiment of the present disclosure;
[0049] Figure 3 is a principle block diagram of a control device according to an embodiment of the present disclosure;
[0050] Figure 4 It is a principle block diagram of a head mounted display device according to an embodiment of the present disclosure. DETAILED DESCRIPTION
[0051] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that unless otherwise specifically stated, the relative arrangement of components and steps, numerical expressions and numerical values set forth in these embodiments do not limit the scope of the embodiments of the present disclosure.
[0052] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.
[0053] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.
[0054] In all examples shown and discussed herein, any specific values should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.
[0055] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.
[0056] <Hardware Configuration>
[0057] Figure 1 The figure is a schematic diagram of the hardware configuration of a control system that can be used to implement a control method of an embodiment. Figure 1 A head mounted display device 100, a vehicle 200 and a network 300 are shown. The head mounted display device 100 can be connected to the network 300, and can also be connected to the vehicle 200 through a communication method such as Bluetooth. In one embodiment, the head mounted display device 100 is connected to the vehicle 200 only through a communication method such as Bluetooth. Among them, a plurality of servers 301, 302 can be set in the network 300. The network 300 can be a wireless communication network or a wired communication network. The network 300 can be a local area network or a wide area network. The network 300 can be a short distance communication or a long distance communication.
[0058] In one embodiment, the head mounted display device 100 may be vehicle mounted AR (Augmented Reality) glasses and vehicle mounted MR (Mixed Reality) glasses, etc., which is not limited in the embodiments of the present disclosure.
[0059] like Figure 1As shown, the head mounted display device 100 may include a processor 101 and a memory 102. The head mounted display device 100 also includes a communication device 103, a display device 104, a user interface 105, a camera device 106, an audio / video interface 107, a sensor 108, etc. In addition, the head mounted display device 100 may also include a power management chip 109 and a battery 110, etc.
[0060] Among them, the processor 101 can be various processors. The memory 102 can store the underlying software, system software, application software, data, etc. required for the operation of the head-mounted display device 100. The memory 102 can include various forms of memory, such as ROM, RAM, Flash, etc. The communication device 103 can include, for example, a WiFi communication device, a Bluetooth communication device, a 3G, 4G and 5G communication device, etc. Through the communication device 103, the head-mounted display device 100 can be arranged in the network. The display device 104 can be a liquid crystal display screen, an OLED display screen, etc. In one example, the display device 104 can be a touch screen. The user can perform input operations through the display device 104. In addition, the user can also perform fingerprint recognition through the touch screen. The user interface 105 may include a USB interface, a lightning interface, a keyboard, etc. The camera device 106 can be a single camera or multiple cameras. The audio / video interface 107 can include, for example, a speaker interface, a microphone interface, a video transmission interface such as HDMI, etc. The sensor 108 can include, for example, a gyroscope, an accelerometer, a temperature sensor, a humidity sensor, a pressure sensor, etc. For example, the posture information of the head-mounted display device can be determined by the sensor. The power management chip 109 can be used to manage the power input to the head mounted display device 100, and can also manage the battery 110 to ensure greater utilization efficiency. The battery 110 is, for example, a lithium ion battery.
[0061] Figure 1 The various components shown are only schematic. The head mounted display device 100 may include Figure 1 One or more of the components shown, but not necessarily including Figure 1 All parts in. Figure 1 The illustrated head mounted display device 100 is illustrative only and is in no way intended to limit the embodiments herein, their applications, or uses.
[0062] In this embodiment, the memory 102 of the head mounted display device 100 is used to store program instructions, which are used to control the processor 101 to operate to execute the control method. The technician can design instructions according to the scheme disclosed in the present invention. How the instructions control the processor to operate is well known in the art, so it will not be described in detail here.
[0063] In one embodiment, Figure 1As shown, the vehicle 200 may include a processor 201 and a memory 202. The vehicle 200 also includes a communication device 203, a display device 204, a user interface 205, a camera device 206, an audio / video interface 207, a sensor 208, etc. In addition, the second device 200 may also include a power management chip 209 and a battery 210, etc.
[0064] The vehicle 200 may be a mobile phone, a portable computer, a tablet computer, a PDA, a wearable device, etc., which is not limited in the embodiments of the present disclosure. Figure 1 The various components shown are for illustration only. Vehicle 200 may include Figure 1 One or more of the components shown, but not necessarily including Figure 1 All parts in. Figure 1 The illustrated vehicle 200 is for illustrative purposes only and is in no way intended to limit the embodiments herein, their application, or uses.
[0065] It should be understood that although Figure 1 Only one head mounted display device 100 and one vehicle 200 are shown, but this does not mean that the respective numbers are limited, and the control system may include multiple head mounted display devices 100 and multiple vehicles 200.
[0066] In the above description, the technician can design instructions according to the solution provided by the present disclosure. How instructions control the processor to operate is well known in the art, so it will not be described in detail here.
[0067] <Method Example>
[0068] Figure 2 A control method according to an embodiment of the present disclosure is shown. The control method can be performed by Figure 1 The head mounted display device shown in the figure can be a vehicle mounted AR glasses and a vehicle mounted MR glasses, etc., which is not limited in this embodiment. Figure 2 As shown, the control method of this embodiment may include the following steps S2100 to S2300:
[0069] Step S2100: Detect the current gaze area of the wearer of the head mounted display device.
[0070] In this embodiment, the desktop launcher of the head mounted display device is called AR Launcher. When the head mounted display device is turned on, the AR Launcher is automatically started, and the desktop environment of the head mounted display device starts to run. After the user wears the head mounted display device, the head mounted display device displays the desktop environment within the user's field of vision. It can be understood that the desktop environment is usually a 3D desktop environment.
[0071] In this embodiment, during the operation of the 3D desktop environment, the 3D desktop environment creates multiple canvases and establishes a correspondence between the canvases and the rendered applications. For example, canvas 1 renders and displays image data of a map application, and canvas 2 renders and displays image data of a game application.
[0072] The wearer's current gaze area may be inside the vehicle or outside the vehicle.
[0073] In this embodiment, the step S2100 of detecting the current gaze area of the wearer of the head mounted display device may further include the following steps S2110 to S2150:
[0074] Step S2110, obtaining target parameter information.
[0075] Among them, the target parameter information includes the current posture information of the vehicle, the three-dimensional model of the vehicle and the current sitting position information of the wearer.
[0076] Specifically, obtaining the target parameter information in step S2110 may further include establishing a short-distance communication connection with the vehicle; based on the short-distance communication connection, receiving the current posture data of the vehicle and the three-dimensional model of the vehicle transmitted by the vehicle; and obtaining a first image captured by the camera device of the head-mounted display device; and determining the current sitting position information of the wearer based on the first image.
[0077] The short-range communication connection may be a Bluetooth communication connection, a Near Field Communication (NFC) connection, etc., which is not limited in this embodiment.
[0078] In this step S2110, on the one hand, after the user turns on the head-mounted display device, the head-mounted display device will establish a short-distance communication connection with the vehicle, so that the vehicle transmits the current posture data of the vehicle and the three-dimensional model of the vehicle to the head-mounted display device based on the short-distance communication connection, that is, the head-mounted display device receives the current posture data of the vehicle and the three-dimensional model of the vehicle transmitted by the vehicle based on the short-distance communication connection. On the other hand, after the user turns on the head-mounted display device, the 6Dof camera in the head-mounted display device captures a first image of the wearer's environment, and the processor in the head-mounted display device recognizes the first image to obtain the wearer's current sitting position information, wherein the current sitting position information can be one of the main driving position, the co-pilot position, the left rear position, the middle rear position, and the right rear position.
[0079] It can be understood that how to construct a three-dimensional model of a vehicle belongs to the prior art and will not be described in detail in this embodiment.
[0080] Optionally, after the head-mounted display device determines the current sitting position information of the wearer based on the first image, the head-mounted display device also provides a modification entry for modifying the current sitting position information of the wearer; obtains the current sitting position information of the wearer re-entered through the modification entry; and uses the input current sitting position information of the wearer as the current sitting position information of the wearer.
[0081] The modification entry may be a modification control. If the wearer believes that the wearer's current sitting position information calculated by the head mounted display device is incorrect, the wearer may click the modification control to input the wearer's current sitting position information. Correspondingly, the head mounted display device also provides a confirmation entry, which may be a confirmation control. If the wearer believes that the wearer's current sitting position information calculated by the head mounted display device is correct, the wearer may click the confirmation control to confirm the wearer's current sitting position information.
[0082] Specifically, after the head-mounted display device determines the wearer's current sitting position information based on the first image captured by the 6Dof camera, it will display the wearer's current sitting position information on the display screen of the head-mounted display device and provide a modification control and a confirmation control. If the wearer clicks the confirmation control, the head-mounted display device can use the calculated wearer's current sitting position information as the wearer's current sitting position information. If the wearer clicks the modification control, he can input his own current sitting position information, and the head-mounted display device will use the user's own current sitting position information input by the user as the wearer's current sitting position information.
[0083] Step S2120: constructing a collision box of the three-dimensional model of the vehicle according to the target parameter information.
[0084] In this step S2120, after the head-mounted display device obtains the current posture information of the vehicle, the three-dimensional model of the vehicle and the current sitting position information of the wearer, it can construct a collision box of the three-dimensional model of the vehicle based on the current sitting position information of the wearer according to the current posture information of the vehicle and the three-dimensional model of the vehicle, and then perform collision detection based on the collision box of the three-dimensional model.
[0085] Step S2130: Generate a virtual identifier based on the wearer's current gaze information.
[0086] The current gaze information of the wearer may be the gaze point of the wearer.
[0087] The virtual mark may be a virtual ray, the virtual ray may be a straight line, or the virtual ray may be a curve.
[0088] In this step S2130, the head mounted display device turns on the eye tracking function, determines the gaze point of the wearer based on the eye tracking function, and extends a virtual ray based on the direction of the gaze point.
[0089] Step S2140: When the virtual marker collides with a first area of a collision box of the three-dimensional model of the vehicle, determining that a current gaze area of the wearer of the head mounted display device is located within the vehicle.
[0090] The first area of the collision box of the three-dimensional model of the vehicle may be, for example, a non-window area of the three-dimensional model of the vehicle, and the second area of the collision box of the three-dimensional model of the vehicle may be, for example, a window area of the three-dimensional model of the vehicle.
[0091] Exemplarily, if the virtual marker collides with a non-window area of a collision box of the three-dimensional model of the vehicle, it is determined that the current gaze area of the wearer of the head mounted display device is located inside the vehicle.
[0092] Step S2150: When the virtual marker collides with a second area of the collision box of the three-dimensional model of the vehicle, determining that the current gaze area of the wearer of the head mounted display device is outside the vehicle.
[0093] Exemplarily, if the virtual marker collides with a window area of a collision box of the three-dimensional model of the vehicle, it is determined that the current gaze area of the wearer of the head mounted display device is outside the vehicle.
[0094] Step S2200: when the current gaze area is located within the vehicle, blurring the first virtual object in the first application rendered and displayed by the first canvas and / or stopping the transmission of the image data to be transmitted of the first application.
[0095] It can be understood that when the current gaze area is within the vehicle, it indicates that the newly opened application is an application that does not need to interact with the external environment. At this time, the coordinate system of the head-mounted display device and the vehicle coordinate system can be controlled to remain consistent, so that the virtual objects in the application are anchored in the vehicle.
[0096] The first canvas may be one of the canvases created by the head mounted display device.
[0097] The first application is an application that interacts with an object outside the vehicle, and the first virtual object corresponds to the object outside the vehicle, that is, the first virtual object corresponds to a physical object outside the vehicle. For example, if the first application is a map application, the first virtual object may be a virtual form of a road sign outside the vehicle, or a virtual form of a pedestrian outside the vehicle.
[0098] For example, the first canvas may be the canvas 1 mentioned above, and the first application may be the map application mentioned above, and the canvas 1 renders and displays the image data of the map application. If the wearer's current gaze area is located within the vehicle, the head mounted display device may blur the virtual forms of road signs and pedestrians in the map application rendered and displayed by canvas 1 to reduce its rendering quality, and may also stop the transmission of the image data to be transmitted of the map application to achieve the effect of reducing rendering power consumption and reducing rendering delay.
[0099] Optionally, when the head-mounted display device is smart glasses, the smart glasses include lenses made of electrochromic glass, and the method further includes: when the current gaze area is located within a vehicle, setting the color of the lenses to a first color to reduce the brightness of a display screen of the head-mounted display device.
[0100] Among them, the lenses of the smart glasses are made of electrochromic glass, the optical properties of which can undergo stable and reversible color changes under the action of an external electric field, which appears as a reversible color change.
[0101] The first color and the second color mentioned later are relative. For example, the first color may be a light color, and correspondingly, the second color may be a dark color.
[0102] That is to say, when the current gaze area is located inside the vehicle, the head-mounted display device can also change the color of the lens to a light color to reduce the screen brightness of the display screen of the head-mounted display device.
[0103] Step S2300, when the current gaze area is outside the vehicle, control the coordinate system of the head-mounted display device to remain unchanged relative to the world coordinate system, blur the second virtual object in the second application rendered and displayed by the second canvas and / or stop the transmission of the image data to be transmitted of the second application.
[0104] It can be understood that when the current gaze area is outside the vehicle, it indicates that the newly opened application is an application that needs to interact with the external environment. At this time, the coordinate system of the head-mounted display device and the world coordinate system can be controlled to be consistent, so that the virtual objects in the application are anchored in the space of the real world.
[0105] The second canvas may be another canvas other than the first canvas in the head mounted display device.
[0106] The second application is an application that interacts with objects in the vehicle, and the second virtual object corresponds to the object in the vehicle, that is, the second virtual object corresponds to the physical object in the vehicle. For example, if the first application is a game application, the second virtual object can be a virtual form of a passenger in the vehicle.
[0107] For example, the second canvas may be the canvas 2 mentioned above, and the second application may be the game application mentioned above, and the canvas 2 renders and displays the image data of the game application. Then, when the wearer's current gaze area is outside the vehicle, the head mounted display device may blur the virtual form of the passenger in the game application rendered and displayed by canvas 2 to reduce its rendering quality, and may also stop the transmission of the image data to be transmitted of the game application, so as to achieve the effect of stopping the reduction of rendering power consumption and reducing rendering delay.
[0108] Optionally, in the case where the head-mounted display device is smart glasses, the smart glasses include lenses made of electrochromic glass, and the method further includes: when the current gaze area is located inside the vehicle, and when the current gaze area is located outside the vehicle, setting the color of the lenses to a second color to increase the brightness of the display screen of the head-mounted display device.
[0109] As described above, when the first color is light, the second color is dark, that is, when the current viewing area is outside the vehicle, the head-mounted display device can also change the color of the lens to a dark color to increase the screen brightness of the display screen of the head-mounted display device.
[0110] It can be understood that for the first application, that is, for the application that needs to interact with objects outside the vehicle, the head mounted display device coordinate system is consistent with the world coordinate system, so that the first virtual object is anchored relative to the external environment. For the second application, that is, for the application that needs to interact with objects inside the vehicle, the head mounted display device coordinate system is consistent with the vehicle coordinate system, so that the second virtual object is anchored relative to the vehicle.
[0111] According to an embodiment of the present disclosure, the head-mounted display device detects the current gaze area of the wearer of the head-mounted display device, and when the current gaze area is located inside the vehicle, the virtual objects in the application rendered and displayed by the first canvas that interact with objects outside the vehicle can be blurred, and / or the transmission of the image data to be transmitted of the application can be stopped, and the virtual objects in the application correspond to the objects outside the vehicle. When the current gaze area is located outside the vehicle, the virtual objects in the application rendered and displayed by the second canvas that interact with objects inside the vehicle can be blurred, and / or the transmission of the image data to be transmitted of the application can be stopped, and the virtual objects in the application correspond to the objects inside the vehicle. In other words, through the embodiments of the present application, different usage scenarios can be distinguished, and different content can be rendered in different usage scenarios, thereby reducing the rendering content, delay and power consumption.
[0112] <Device Example>
[0113] Figure 3 is a schematic diagram of a control device according to an embodiment, referring to Figure 3 As shown, the control device 300 includes a detection module 310 and a control module 320 .
[0114] A detection module 310 is used to detect a current gaze area of a wearer of a head mounted display device;
[0115] The control module 320 is configured to blur a first virtual object in a first application rendered and displayed by a first canvas and / or stop transmission of image data to be transmitted of the first application when the current gaze area is located within the vehicle; wherein the first application is an application that interacts with an object outside the vehicle, and the first virtual object corresponds to the object outside the vehicle;
[0116] The control module 320 is used to blur the second virtual object in the second application rendered and displayed by the second canvas and / or stop the transmission of the image data to be transmitted of the second application when the current gaze area is outside the vehicle; wherein the second application is an application that interacts with the object inside the vehicle, and the second virtual object corresponds to the object inside the vehicle.
[0117] In one embodiment, the detection module 310 is specifically used to: obtain target parameter information; wherein the target parameter information includes the current posture information of the vehicle, the three-dimensional model of the vehicle and the current sitting position information of the wearer; construct a collision box of the three-dimensional model of the vehicle according to the target parameter information; generate a virtual identifier according to the wearer's current gaze information; when the virtual identifier collides with a first area of the collision box of the three-dimensional model of the vehicle, determine that the current gaze area of the wearer of the head-mounted display device is located inside the vehicle; when the virtual identifier collides with a second area of the collision box of the three-dimensional model of the vehicle, determine that the current gaze area of the wearer of the head-mounted display device is located outside the vehicle.
[0118] In one embodiment, the control module 310 is further used to: when the current gaze area is located inside the vehicle, set the color of the lens to a first color to reduce the brightness of the display screen of the head-mounted display device; when the current gaze area is located outside the vehicle, set the color of the lens to a second color to increase the brightness of the display screen of the head-mounted display device.
[0119] In one embodiment, the detection module 310 is specifically used to: establish a short-range communication connection with the vehicle; based on the short-range communication connection, receive the current posture data of the vehicle and the three-dimensional model of the vehicle transmitted by the vehicle; and obtain a first image captured by the camera device of the head-mounted display device; and determine the current sitting position information of the wearer based on the first image.
[0120] In one embodiment, the detection module 310 is further used to: provide a modification entry for modifying the current sitting position information of the wearer; obtain the current sitting position information of the wearer re-entered through the modification entry; and use the input current sitting position information of the wearer as the current sitting position information of the wearer.
[0121] According to an embodiment of the present disclosure, the head-mounted display device detects the current gaze area of the wearer of the head-mounted display device, and when the current gaze area is located inside the vehicle, the virtual objects in the application rendered and displayed by the first canvas that interact with objects outside the vehicle can be blurred, and / or the transmission of the image data to be transmitted of the application can be stopped, and the virtual objects in the application correspond to the objects outside the vehicle. When the current gaze area is located outside the vehicle, the virtual objects in the application rendered and displayed by the second canvas that interact with objects inside the vehicle can be blurred, and / or the transmission of the image data to be transmitted of the application can be stopped, and the virtual objects in the application correspond to the objects inside the vehicle. In other words, through the embodiments of the present application, different usage scenarios can be distinguished, and different content can be rendered in different usage scenarios, thereby reducing the rendering content, delay and power consumption.
[0122] <Equipment Embodiment>
[0123] Figure 4 FIG. 1 is a schematic diagram of the hardware structure of a head mounted display device according to an embodiment. Figure 4 As shown, the head mounted display device 400 includes a processor 410 and a memory 420 .
[0124] The memory 420 may be used to store executable computer instructions.
[0125] The processor 410 can be used to execute the control method described in the embodiment of the method of the present disclosure according to the control of the executable computer instructions.
[0126] The head mounted display device 400 may be as follows Figure 1 The head mounted display device 100 shown may also be a device with other hardware structures, which is not limited here.
[0127] In another embodiment, the head mounted display device 400 may include the above control device 300 .
[0128] In one embodiment, each module of the above control device 300 can be implemented by the processor 410 running computer instructions stored in the memory 420 .
[0129] <Computer Readable Storage Medium>
[0130] The embodiment of the present disclosure further provides a computer-readable storage medium on which computer instructions are stored. When the computer instructions are executed by a processor, the control method provided by the embodiment of the present disclosure is executed.
[0131] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.
[0132] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.
[0133] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.
[0134] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.
[0135] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.
[0136] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.
[0137] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.
[0138] The flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of an instruction, and the module, a program segment or a part of an instruction contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of the boxes in the block diagram and / or the flowchart can be implemented by a dedicated hardware-based system that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions. It is well known to those skilled in the art that it is equivalent to implement it by hardware, implement it by software, and implement it by combining software and hardware.
[0139] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The choice of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein. The scope of the present disclosure is defined by the appended claims.
Claims
1. A control method, characterized in that: The method comprises: Detecting a current gaze area of a wearer of the head mounted display device; When the current gaze area is located within the vehicle, blurring a first virtual object in a first application rendered and displayed by a first canvas and / or stopping transmission of image data to be transmitted of the first application; wherein the first application is an application that interacts with an object outside the vehicle, and the first virtual object corresponds to the object outside the vehicle; When the current gaze area is outside the vehicle, blur the second virtual object in the second application rendered and displayed by the second canvas and / or stop the transmission of the image data to be transmitted of the second application; wherein, the second application is an application that interacts with objects within the vehicle, and the second virtual object corresponds to the object within the vehicle.
2. The method according to claim 1, characterized in that The detecting the current gaze area of the wearer of the head mounted display device comprises: Acquiring target parameter information; wherein the target parameter information includes current posture information of the vehicle, a three-dimensional model of the vehicle and current sitting position information of the wearer; constructing a collision box of the three-dimensional model of the vehicle according to the target parameter information; Generate a virtual identifier based on the wearer's current gaze information; In a case where the virtual marker collides with a first area of a collision box of the three-dimensional model of the vehicle, determining that a current gaze area of the wearer of the head mounted display device is located within the vehicle; When the virtual marker collides with a second area of the collision box of the three-dimensional model of the vehicle, it is determined that a current gaze area of the wearer of the head mounted display device is outside the vehicle.
3. The method according to claim 1, characterized in that The head mounted display device is a smart glasses, the smart glasses include lenses made of electrochromic glass, and the method further includes: When the current gaze area is located inside the vehicle, setting the color of the lens to a first color to reduce the brightness of the display screen of the head mounted display device; When the current gaze area is outside the vehicle, the color of the lens is set to a second color to increase the brightness of the display screen of the head-mounted display device.
4. The method according to claim 2, characterized in that: The obtaining of target parameter information includes: establishing a short-range communication connection with the vehicle; Based on the short-range communication connection, receiving the current position data of the vehicle and the three-dimensional model of the vehicle transmitted by the vehicle; and, Acquire a first image captured by a camera device of the head mounted display device; According to the first image, current sitting position information of the wearer is determined.
5. The method according to claim 4, characterized in that After determining the current sitting position information of the wearer according to the first image, the method further includes: Providing a modification entry for modifying the current sitting position information of the wearer; acquiring the current sitting position information of the wearer re-entered through the modification entry; The input current sitting position information of the wearer is used as the current sitting position information of the wearer.
6. A control device, characterized in that: The device comprises: A detection module, used to detect a current gaze area of a wearer of the head mounted display device; a control module, configured to, when the current gaze area is located within the vehicle, blur a first virtual object in a first application rendered and displayed by a first canvas and / or stop transmission of image data to be transmitted of the first application; wherein the first application is an application that interacts with an object outside the vehicle, and the first virtual object corresponds to the object outside the vehicle; The control module is used to blur the second virtual object in the second application rendered and displayed by the second canvas and / or stop the transmission of the image data to be transmitted of the second application when the current gaze area is outside the vehicle; wherein the second application is an application that interacts with the object inside the vehicle, and the second virtual object corresponds to the object inside the vehicle.
7. The device according to claim 6, characterized in that The detection module is specifically used for: Acquiring target parameter information; wherein the target parameter information includes current posture information of the vehicle, a three-dimensional model of the vehicle and current sitting position information of the wearer; constructing a collision box of the three-dimensional model of the vehicle according to the target parameter information; Generate a virtual identifier based on the wearer's current gaze information; In a case where the virtual marker collides with a first area of a collision box of the three-dimensional model of the vehicle, determining that a current gaze area of the wearer of the head mounted display device is located within the vehicle; When the virtual marker collides with a second area of the collision box of the three-dimensional model of the vehicle, it is determined that a current gaze area of the wearer of the head mounted display device is outside the vehicle.
8. The apparatus according to claim 6, wherein the head mounted display device is a pair of smart glasses, wherein the smart glasses include lenses made of electrochromic glass, and wherein the control module is further configured to: When the current gaze area is located in the vehicle, setting the color of the lens to a first color to reduce the brightness of the display screen of the head mounted display device; When the current gaze area is outside the vehicle, the color of the lens is set to a second color to increase the brightness of the display screen of the head-mounted display device.
9. A head mounted display device, characterized in that: The head mounted display device comprises: Memory for storing executable computer instructions; A processor, configured to execute the control method according to any one of claims 1 to 5 under the control of the executable computer instructions.
10. A computer-readable storage medium having computer instructions stored thereon, wherein the computer instructions are executed by a processor to execute the control method according to any one of claims 1 to 5.
