Method and device for reducing power consumption of head-mounted display equipment, equipment and medium

Abstract

The invention discloses a method and device for reducing power consumption of head-mounted display equipment, equipment and a medium, and relates to the technical field of image processing. The method is applied to the head-mounted display device, the head-mounted display device comprises an eyeball tracking module, a camera and a display screen, and the method comprises the following steps: determining a gazing area and a non-gazing area of a wearer of the head-mounted display device on the display screen according to eye movement information collected by the eyeball tracking module; the collection frame rate of a first pixel collection point is set as a first collection frame rate, the collection frame rate of a second pixel collection point is set as a second collection frame rate, and the first pixel collection point is a pixel collection point of an environment image displayed by a pixel point of a collection fixation area of the camera; the second pixel collection point is a pixel collection point of an environment image displayed by a pixel point of a collection non-fixation area of the camera, and the first collection frame rate is greater than the second collection frame rate; and controlling the first pixel acquisition point to acquire the environment image according to the first acquisition frame rate and the second pixel acquisition point to acquire the environment image according to the second acquisition frame rate.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and more specifically, to a method, apparatus, device, and medium for reducing the power consumption of head-mounted display devices. Background Technology

[0002] Extended Reality (XR) devices, as a typical representative of cutting-edge technology, are intelligent terminal devices that use computer technology to deeply integrate the real world and the virtual world, thereby constructing a virtual environment that supports human-computer interaction.

[0003] Currently, one of the biggest pain points of head-mounted display devices is their high power consumption, which greatly reduces the usage time of XR devices and consequently diminishes the user experience. Summary of the Invention

[0004] One objective of this application is to provide a new technical solution for reducing the power consumption of head-mounted display devices.

[0005] According to a first aspect of this application, a method for reducing power consumption of a head-mounted display device is provided, applied to the head-mounted display device, the head-mounted display device including an eye-tracking module, a camera, and a display screen, the method comprising:

[0006] Based on the eye movement information collected by the eye-tracking module, the gaze area and non-gaze area of ​​the wearer of the head-mounted display device on the display screen are determined;

[0007] The first pixel acquisition point is set to a first acquisition frame rate and the second pixel acquisition point is set to a second acquisition frame rate. The first pixel acquisition point is the pixel acquisition point of the camera that acquires the pixels of the gaze area and displays the environmental image. The second pixel acquisition point is the pixel acquisition point of the camera that acquires the pixels of the non-gaze area and displays the environmental image. The first acquisition frame rate is greater than the second acquisition frame rate.

[0008] The first pixel acquisition point is controlled to acquire environmental images at the first acquisition frame rate, and the second pixel acquisition point is controlled to acquire environmental images at the second acquisition frame rate.

[0009] Optionally, setting the acquisition frame rate of the second pixel acquisition point to a second acquisition frame rate includes:

[0010] Based on the distance between the pixels of the environmental image captured by the second pixel acquisition point and the gaze area, the acquisition frame rate of the second pixel acquisition point is set to a second acquisition frame rate that matches the distance.

[0011] Wherein, the second acquisition frame rate of the second pixel acquisition point corresponding to the closer distance is greater than or equal to the second acquisition frame rate of the second pixel acquisition point corresponding to the farther distance.

[0012] Optionally, after determining the gaze area and non-gaze area of ​​the wearer of the head-mounted display device on the display screen based on the eye movement information collected by the eye-tracking module, the method further includes:

[0013] Enable the first pixel acquisition point;

[0014] Enable the second pixel acquisition point of the target resolution corresponding to the pixel in the first non-focused area. The first non-focused area is composed of pixels whose distance from the focus area is less than or equal to a preset distance. The target resolution is less than the full resolution of the second pixel acquisition point corresponding to the pixel in the first non-focused area.

[0015] The second non-focused region is closed to the second pixel acquisition point, and the second non-focused region is composed of pixels whose distance from the focusing region is greater than the preset distance.

[0016] Optionally, activating the second pixel acquisition point of the target resolution corresponding to the pixel point in the first non-focused region includes:

[0017] Based on the distance between the pixels of the environmental image captured by the second pixel acquisition point and the gaze area, activate the second pixel acquisition point with a target resolution that matches the distance to the pixels in the first non-gaze area;

[0018] Wherein, the target resolution of the second pixel acquisition point corresponding to the closer distance is greater than or equal to the target resolution of the second pixel acquisition point corresponding to the farther distance.

[0019] Optionally, in the method of controlling the first pixel acquisition point to acquire environmental images at the first acquisition frame rate and the second pixel acquisition point to acquire environmental images at the second acquisition frame rate, the method further includes:

[0020] Acquire the current real-world image captured by the camera and the corresponding current virtual-world image;

[0021] When the size of the current real environment image is smaller than the size of the current virtual image, the current real environment image is fused with the first virtual image to obtain a first fused image. The first virtual image is the virtual image in the current virtual image that is to be displayed in the gaze area.

[0022] The first fused image and the second fused image are stitched together to form the current complete fused image. The second fused image is the fused image in the previous complete fused image excluding the corresponding area of ​​the first fused image, or the second fused image is the virtual image in the current virtual image to be displayed in the non-focused area.

[0023] Optionally, after controlling the first pixel acquisition point to acquire environmental images at the first acquisition frame rate and the second pixel acquisition point to acquire environmental images at the second acquisition frame rate, the method further includes:

[0024] The frame rate of the corresponding gaze area in the display screen of the head-mounted display device is set to a first display frame rate and the frame rate of the corresponding non-gaze area is set to a second display frame rate, wherein the first display frame rate is greater than the second display frame rate;

[0025] The display screen is controlled to display images at a first display frame rate in the gaze area and at a second display frame rate in the non-gaze area.

[0026] Optionally, after controlling the first pixel acquisition point to acquire environmental images at the first acquisition frame rate and the second pixel acquisition point to acquire environmental images at the second acquisition frame rate, the method further includes:

[0027] The brightness of the corresponding gaze area in the display screen of the head-mounted display device is set as a first brightness and the brightness of the corresponding non-gaze area is set as a second brightness, wherein the first brightness is greater than the second brightness;

[0028] The display screen is controlled to display an image at a first brightness in the viewing area and at a second brightness in the non-viewing area.

[0029] According to a second aspect of this application, an apparatus for reducing power consumption of a head-mounted display device is provided, applied to the head-mounted display device, the head-mounted display device including an eye-tracking module, a camera, and a display screen, the apparatus comprising:

[0030] The determination module is used to determine the gaze area and non-gaze area of ​​the wearer of the head-mounted display device on the display screen based on the eye movement information collected by the eye-tracking module.

[0031] The setting module is used to set the acquisition frame rate of the first pixel acquisition point as the first acquisition frame rate and the acquisition frame rate of the second pixel acquisition point as the second acquisition frame rate. The first pixel acquisition point is the pixel acquisition point of the camera that acquires the pixels of the gaze area and displays the environmental image. The second pixel acquisition point is the pixel acquisition point of the camera that acquires the pixels of the non-gaze area and displays the environmental image. The first acquisition frame rate is greater than the second acquisition frame rate.

[0032] The control module is used to control the first pixel acquisition point to acquire environmental images according to the first acquisition frame rate and the second pixel acquisition point to acquire environmental images according to the second acquisition frame rate, so as to obtain a real environmental image.

[0033] According to a third aspect of this application, a head-mounted display device is provided, the head-mounted display device including the apparatus as described in the second aspect, an eye-tracking module, a camera, and a display screen;

[0034] Alternatively, the head-mounted display device includes the eye-tracking module, the camera, the display screen, a memory, and a processor. The eye-tracking module, the camera, the display screen, and the memory are all connected to the processor. The memory is used to store computer instructions, and the processor is used to retrieve the computer instructions from the memory to perform the method as described in any one of the first aspects.

[0035] According to a fourth aspect of this application, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method according to any one of the first aspects.

[0036] This application provides a method for reducing the power consumption of a head-mounted display device. The head-mounted display device includes an eye-tracking module, a camera, and a display screen. The method includes: determining the gaze area and non-gaze area of ​​the wearer on the display screen based on eye movement information collected by the eye-tracking module; setting a first frame rate for a first pixel acquisition point and a second frame rate for a second pixel acquisition point, where the first pixel acquisition point is the pixel acquisition point of the environmental image displayed by the camera's acquisition of the gaze area, and the second pixel acquisition point is the pixel acquisition point of the environmental image displayed by the camera's acquisition of the non-gaze area, with the first frame rate being greater than the second frame rate; and controlling the first pixel acquisition point to acquire the environmental image at the first frame rate and the second pixel acquisition point to acquire the environmental image at the second frame rate. This method reduces the power consumption of the camera while maintaining a good user visual experience. With the reduced power consumption of the camera, the power consumption of the head-mounted display device also decreases.

[0037] Other features and advantages of this application will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0038] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments of the present application and, together with their description, serve to explain the principles of the present application.

[0039] Figure 1 This is a flowchart illustrating a method for reducing power consumption in a head-mounted display device provided in this application;

[0040] Figure 2 This is a schematic diagram of a device for reducing power consumption of a head-mounted display device provided in this application;

[0041] Figure 3 This is a schematic diagram of the structure of a head-mounted display device provided in an embodiment of this application. Detailed Implementation

[0042] Various exemplary embodiments of the present application will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the present application.

[0043] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the scope of this application and its application or use.

[0044] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0045] In all the examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0046] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0047] This application provides a method for reducing the power consumption of a head-mounted display device. The method is applied to the head-mounted display device, which is an XR device, such as a MR device or an AR device. The head-mounted display device includes an eye-tracking module, a camera, and a display screen.

[0048] The eye-tracking module collects eye movement information from the wearer of the head-mounted display. The camera captures environmental images from the wearer's perspective, enabling the head-mounted display to provide perspective functionality. The camera consists of multiple pixel acquisition points; each pixel is an image sensing unit capable of independently capturing the smallest unit of environmental image. The environmental image captured by one pixel is displayed as a single pixel on the screen. The display screen is used for image display.

[0049] like Figure 1As shown, the method for reducing the power consumption of a head-mounted display device provided in this application includes the following steps S110 to S130.

[0050] Step S110: Based on the eye movement information collected by the eye-tracking module, determine the gaze area and non-gaze area of ​​the wearer of the head-mounted display device on the display screen.

[0051] In one embodiment of this application, the eye-tracking module includes an infrared illumination system and an infrared camera. The infrared illumination system emits infrared light towards the human eye. When the human eye receives infrared light, the pupil absorbs the infrared light, creating a dark area, while the cornea reflects the infrared light, creating a bright spot. The infrared camera captures images of the human eye, obtaining an infrared image of the eye as eye movement information. Further, by analyzing the eye movement information using an eye-tracking algorithm, the wearer's gaze point on the display screen can be determined. Furthermore, a circle on the display screen with the gaze point as its center and a predetermined number of pixels as its radius can be designated as the gaze area, and the area on the display screen excluding the gaze area can be designated as the non-gaze area. The predetermined number is half the number of pixels the wearer can see directly when looking at the display screen, and can be set based on experience or experimentation.

[0052] Step S120: Set the acquisition frame rate of the first pixel acquisition point to the first acquisition frame rate and the acquisition frame rate of the second pixel acquisition point to the second acquisition frame rate.

[0053] The first pixel acquisition point is the pixel acquisition point of the environmental image displayed by the pixel points of the camera's gaze area, and the second pixel point is the pixel acquisition point of the environmental image displayed by the pixel points of the camera's non-gaze area. The first acquisition frame rate is greater than the second acquisition frame rate.

[0054] In this embodiment, since the wearer views the environmental image displayed in the gaze area with direct eye contact and the environmental image displayed in the non-gaze area with peripheral vision, and the first pixel acquisition point is the pixel acquisition point of the camera that captures the environmental image displayed in the gaze area, and the second pixel acquisition point is the pixel acquisition point of the camera that captures the environmental image displayed in the non-gaze area, the acquisition frame rate of the first pixel acquisition point is set to a relatively high first acquisition frame rate, and the acquisition frame rate of the second pixel acquisition point is set to a relatively low second acquisition frame rate. This ensures that when the environmental image captured by the first pixel acquisition point is displayed in the gaze area, the wearer's visual experience is not degraded. The first acquisition frame rate is typically the frame rate of pixel sampling points in a camera in conventional technology. Furthermore, since the wearer's first acquisition frame rate is typically the frame rate of pixel sampling points in a camera in conventional technology, the environmental image captured by the second pixel acquisition point, when displayed in the non-gaze area, will not affect the wearer's visual experience. Simultaneously, the second pixel acquisition point acquires images at a lower second acquisition frame rate, reducing the camera's power consumption.

[0055] As can be seen from the above, step S120 reduces camera power consumption while ensuring a good user visual experience. With reduced camera power consumption, the power consumption of the head-mounted display device also decreases.

[0056] Step S130: Control the first pixel acquisition point to acquire environmental images at the first acquisition frame rate and the second pixel to acquire environmental images at the second acquisition frame rate.

[0057] By following step S130 above, a real environment image to be fused with the virtual image can be obtained.

[0058] This application provides a method for reducing the power consumption of a head-mounted display device. The head-mounted display device includes an eye-tracking module, a camera, and a display screen. The method includes: determining the gaze area and non-gaze area of ​​the wearer on the display screen based on eye movement information collected by the eye-tracking module; setting a first frame rate for a first pixel acquisition point and a second frame rate for a second pixel acquisition point, where the first pixel acquisition point is the pixel acquisition point of the environmental image displayed by the camera's acquisition of the gaze area, and the second pixel acquisition point is the pixel acquisition point of the environmental image displayed by the camera's acquisition of the non-gaze area, with the first frame rate being greater than the second frame rate; and controlling the first pixel acquisition point to acquire the environmental image at the first frame rate and the second pixel acquisition point to acquire the environmental image at the second frame rate. This method reduces the power consumption of the camera while maintaining a good user visual experience. With the reduced power consumption of the camera, the power consumption of the head-mounted display device also decreases.

[0059] Understandably, in traditional technologies, each pixel in a camera uses the same frame rate to capture images. Therefore, the size of the real environment image captured by the camera is consistent and the same as the size of the virtual image to be merged with the real environment image. However, in this embodiment, since the first pixel captures the environment image at a first frame rate and the second pixel captures the environment image at a second frame rate, there are cases where the size of the real environment image captured by the camera is inconsistent, resulting in it being smaller than the virtual image to be merged with the real environment image. For example, taking a first frame rate of 60Hz and a second frame rate of 30Hz as an example, at the start of the capture, the first pixel captures the environment image corresponding to the gaze area, and the second pixel captures the environment image corresponding to the non-gaze area. At this time, the camera captures a complete real environment image. However, at the next capture moment, the first pixel continues to capture the environment image corresponding to the gaze area, while the second pixel no longer captures the environment image corresponding to the non-gaze area. At this time, the camera only captures the real environment image corresponding to the gaze area, which results in the size of the real environment image captured by the camera being smaller than the size of the virtual image to be merged with it. In order to ensure the fusion of the real environment image and the virtual image to be fused when the size of the real environment image is smaller than the size of the virtual image to be fused, in one embodiment of this application, the method for reducing the power consumption of the head-mounted display device provided in this application further includes the following steps S140 to S160 after the above step S130.

[0060] Step S140: Acquire the current real environment image captured by the camera and the corresponding current virtual environment image.

[0061] In this embodiment, the real environment image captured by the camera at the current moment is determined as the current real environment image. The virtual environment image to be fused with the current real environment image is denoted as the current virtual environment image.

[0062] It can be understood that the current real-world image can be either the environment image captured by the first pixel sensor or the environment image captured by the second pixel sensor, or the current real-world image can be the environment image captured by the first pixel sensor. If the current real-world image is the environment image captured by both the first and second pixel sensors, then the size of the current real-world image is equal to the size of the current virtual image. If the current real-world image is the environment image captured by the first pixel sensor, then the size of the current real-world image is smaller than the size of the current virtual image.

[0063] Step S150: When the size of the current real environment image is smaller than the size of the current virtual image, the current real environment image is fused with the first virtual image to obtain the first fused image.

[0064] The first virtual image is the virtual image to be displayed in the gaze area of ​​the current virtual image.

[0065] When the size of the current real-world image is smaller than the size of the current virtual image, only the first virtual image is merged with the current real-world image to obtain the first fused image. It can be understood that the first fused image is the combined virtual and real image subsequently displayed in the gaze area.

[0066] To display the image of the non-focused region, proceed to step S160 below.

[0067] Step S160: The first fused image and the second fused image are stitched together to form the current complete fused image.

[0068] The second fused image is either the fused image in the previous complete fused image excluding the corresponding area of ​​the first fused image, or the second fused image is a virtual image in the current virtual image to be displayed in the non-focused area.

[0069] In this embodiment, the current complete fused frame is used for direct display on the screen. Furthermore, the fused image from the previous complete fused image, excluding the corresponding area of ​​the first fused image, is used as the second fused image and stitched together with the first fused image to obtain the current complete fused image. That is, the previous complete fused image is used to complete the first fused image into the current complete fused image.

[0070] Alternatively, the virtual image to be displayed in the non-focused area of ​​the current virtual image can be used directly as a partial fused image to complete the first fused image into the current complete fused image.

[0071] In one embodiment of this application, the setting of the acquisition frame rate of the second pixel acquisition point in step S120 is the second acquisition frame rate, which is specifically implemented through the following step S121.

[0072] Step S121: Based on the distance between the pixels of the environmental image captured by the second pixel acquisition point and the gaze area, set the acquisition frame rate of the second pixel acquisition point to a second acquisition frame rate that matches the distance.

[0073] Among them, the second acquisition frame rate of the second pixel acquisition point corresponding to the closer distance is greater than or equal to the second acquisition frame rate of the second pixel acquisition point corresponding to the farther distance.

[0074] In this embodiment, the distance between the pixels of the environmental image captured by the second pixel acquisition point and the gaze area can be represented by the number of pixels between the pixels of the environmental image captured by the second pixel acquisition point and the center pixel of the gaze area.

[0075] Different distance ranges between pixels in the environmental image captured by the second pixel acquisition point and the gaze area correspond to different second acquisition frame rates. Specifically, the larger the distance range, the smaller the second acquisition frame rate. Thus, through the above step S121, the second acquisition frame rate of the second pixel acquisition point can be lower for pixels farther away from the gaze area. This results in lower smoothness of the environmental image further away from the gaze area, which is consistent with the wearer's visual characteristics.

[0076] Through the above step S121, the second acquisition frame rate of the second pixel acquisition point can be reduced in a gradually decreasing manner, which can further reduce the power consumption of the camera, and thus further reduce the power consumption of the head-mounted display device.

[0077] In one embodiment of this application, the method for reducing the power consumption of a head-mounted display device provided in this application further includes the following steps S111 to S113 after the above step S110.

[0078] Step S111: Start the first pixel acquisition point.

[0079] Through step S111 above, all first pixel acquisition points are activated, enabling all first pixel acquisition points to acquire environmental images. This allows the first pixel acquisition points to acquire the entire environmental image displayed in the gaze area at full resolution.

[0080] Step S112: Enable the acquisition of the second pixel point of the target resolution corresponding to the pixel point in the first non-focus area.

[0081] The first non-focused region is composed of pixels that are less than a preset distance from the focused region, and the target resolution is less than the full resolution of the second pixel acquisition point corresponding to the pixels in the first non-focused region.

[0082] In this embodiment, the distance between a pixel and the gaze area can be specifically represented by the number of pixels between the pixel and the center pixel of the gaze area. The preset distance is the minimum distance between a pixel that the wearer cannot see and the gaze area, and can be determined empirically.

[0083] In step S112 above, since the target resolution is less than the full resolution of the second pixel acquisition point corresponding to the first non-focused area, only a portion of the second pixel acquisition points corresponding to the pixels in the first non-focused area are enabled. This allows the acquisition of the environmental image of the first non-focused area using a portion of the second pixel acquisition points. Since the first non-focused area is a non-focused area, although using a portion of the second pixel acquisition points in the first non-focused area to acquire the environmental image of the first non-focused area will reduce the clarity of the environmental image, this will not reduce the wearer's visual experience. This further reduces the power consumption of the camera, thereby further reducing the power consumption of the head-mounted display device.

[0084] Step S113: Turn off the second pixel acquisition point corresponding to the pixel point in the second non-focused area.

[0085] The second non-focused region consists of pixels that are at a distance greater than a preset distance from the focused region.

[0086] Corresponding to step S112 above, since the second non-focused area is composed of pixels whose distance from the focused area is greater than a preset distance, that is, the second non-focused area is an area that the wearer cannot look at. Based on this, the second pixel acquisition point corresponding to the pixels in the second non-focused area is turned off to stop the acquisition of environmental images of the second non-focused area. This further reduces the power consumption of the camera, thereby further reducing the power consumption of the head-mounted display device.

[0087] In one embodiment of this application, the above step S112 is specifically implemented by the following step S1121.

[0088] Step S1121: Based on the distance between the pixels of the environmental image acquired by the second pixel acquisition point and the gaze area, activate the second pixel acquisition point corresponding to the pixel in the first non-gaze area with a target resolution that matches the distance.

[0089] Among them, the target resolution of the second pixel acquisition point corresponding to the closer distance is greater than or equal to the target resolution of the second pixel acquisition point corresponding to the farther distance.

[0090] Similar to step S121 above, in this embodiment, the distance range between the pixels of the environmental image captured by different display second pixel acquisition points and the gaze area corresponds to different target resolutions. Specifically, the larger the distance range, the smaller the target resolution. Thus, through step S1121 above, the environmental image resolution of areas farther from the gaze area can be reduced, resulting in lower clarity of environmental images far from the gaze area, which aligns with the wearer's visual characteristics.

[0091] Through the above step S1121, the sharpness of the non-focused area can be reduced in a gradually decreasing manner, which can further reduce the power consumption of the camera, and thus further reduce the power consumption of the head-mounted display device.

[0092] Based on any of the above embodiments, the method for reducing the power consumption of head-mounted devices provided in this application further includes the following steps S170 and S180 after step S130.

[0093] Step S170: Set the frame rate of the corresponding gaze area in the display screen of the head-mounted display device to a first display frame rate and the frame rate of the corresponding non-gaze area to a second display frame rate, wherein the first display frame rate is greater than the second display frame rate.

[0094] Step S180: Control the display screen to display an image at a first display frame rate in the gaze area and at a second display frame rate in the non-gaze area.

[0095] In this embodiment, by using the above step S170, when the display screen displays an image, the first display frame rate of the wearer's gaze area is higher than the second display frame rate of the non-gaze area. This reduces the power consumption of the display screen, thereby reducing the power consumption of the head-mounted display device while ensuring the wearer's visual experience.

[0096] It is understandable that, similar to step S121 above, a second display frame rate corresponding to the distance range can be set according to the distance from the viewing area. The larger the distance range, the smaller the second display frame rate. This can further reduce the power consumption of the display screen, thereby reducing the power consumption of the head-mounted display device.

[0097] Based on any of the above embodiments, the method for reducing the power consumption of head-mounted devices provided in this application further includes the following steps S190 and S1100 after step S130.

[0098] Step S190: Set the brightness of the corresponding gaze area in the display screen of the head-mounted display device to a first brightness and the brightness of the corresponding non-gaze area to a second brightness, wherein the first brightness is greater than the second brightness.

[0099] Step S1100: Control the display screen to display an image at a first brightness in the viewing area and at a second brightness in the non-viewing area.

[0100] In this embodiment, by using the above step S190, the brightness of the wearer's gaze area can be higher than the brightness of the non-gaze area when the display screen shows an image. This reduces the power consumption of the display screen, thereby reducing the power consumption of the head-mounted display device while ensuring the wearer's visual experience.

[0101] It is understandable that, similar to step S121 above, a second brightness corresponding to the distance range can be set according to the distance from the viewing area. The larger the distance range, the smaller the second brightness. This can further reduce the power consumption of the display screen, thereby reducing the power consumption of the head-mounted display device.

[0102] In one embodiment of this application, step S110 is specifically implemented through steps S114 and S115.

[0103] Step S114: Check whether the set conditions have been triggered.

[0104] Step S115: When triggered, determine the gaze area and non-gaze area of ​​the wearer of the head-mounted display device on the display screen based on the eye movement information collected by the eye-tracking module.

[0105] The setting conditions include any one of the following: the setting button being pressed, the setting voice being received, and the head-mounted display device producing a setting posture change.

[0106] In one embodiment of this application, a setting button is provided on the head-mounted display device, which is used to trigger the above-described step S110. Specifically, when the wearer wants to control the head-mounted display device to enter a low-power mode, they press the setting button. At this time, the setting button is detected to be pressed, that is, the setting condition is triggered.

[0107] In another embodiment of this application, when the wearer wants to control the head-mounted display device to enter a low-power mode, they input a setting voice such as "Please turn on low-power mode". At this time, the head-mounted display device receives the setting voice, that is, it detects that the setting condition has been triggered.

[0108] In another embodiment of this application, when the wearer wants to control the head-mounted display device to enter a low-power mode, he performs a specific head movement. At this time, it is determined that the head-mounted display device has generated a set posture change, that is, the set condition is detected to be triggered.

[0109] When the set condition is detected to be triggered, step S110 is triggered to execute the method of reducing the power consumption of the head-mounted display device as shown in steps S110 to S130, or steps S110 to S160, or steps S110 to S1100, so that the head-mounted display device enters a low-power mode.

[0110] Of course, if the wearer presses the setting button again when the head-mounted display device enters low power mode, or if the wearer enters the voice command "Please turn off low power mode", or if the wearer performs a specific head movement again, the above step S110 can be stopped from being triggered, so that the head-mounted display device exits low power mode.

[0111] like Figure 2 As shown, this application also provides a device 200 for reducing the power consumption of a head-mounted display device, applied to a head-mounted display device including an eye-tracking module, a camera, and a display screen. The device 200 includes:

[0112] The determination module 210 is used to determine the gaze area and non-gaze area of ​​the wearer of the head-mounted display device on the display screen based on the eye movement information collected by the eye-tracking module.

[0113] Setting module 220 is used to set the acquisition frame rate of the first pixel acquisition point to a first acquisition frame rate and the acquisition frame rate of the second pixel acquisition point to a second acquisition frame rate. The first pixel acquisition point is the pixel acquisition point of the camera that acquires the pixels of the gaze area to display the environmental image. The second pixel acquisition point is the pixel acquisition point of the camera that acquires the pixels of the non-gaze area to display the environmental image. The first acquisition frame rate is greater than the second acquisition frame rate.

[0114] The control module 230 is used to control the first pixel acquisition point to acquire environmental images according to the first acquisition frame rate and the second pixel acquisition point to acquire environmental images according to the second acquisition frame rate.

[0115] In one embodiment of this application, the setting module 220 is specifically used to set the acquisition frame rate of the second pixel acquisition point to a second acquisition frame rate that matches the distance, according to the distance between the pixel of the environmental image acquired by the second pixel acquisition point and the gaze area.

[0116] Wherein, the second acquisition frame rate of the second pixel acquisition point corresponding to the closer distance is greater than or equal to the second acquisition frame rate of the second pixel acquisition point corresponding to the farther distance.

[0117] In one embodiment of this application, the control module 230 is further configured to activate the first pixel acquisition point;

[0118] Enable the second pixel acquisition point of the target resolution corresponding to the pixel in the first non-focused area. The first non-focused area is composed of pixels whose distance from the focus area is less than or equal to a preset distance. The target resolution is less than the full resolution of the second pixel acquisition point corresponding to the pixel in the first non-focused area.

[0119] The second non-focused region is closed to the second pixel acquisition point, and the second non-focused region is composed of pixels whose distance from the focusing region is greater than the preset distance.

[0120] In one embodiment of this application, the control module 230 is specifically used to activate the second pixel acquisition point with a target resolution that matches the distance between the pixel of the environmental image acquired by the second pixel acquisition point and the gaze area.

[0121] Wherein, the target resolution of the second pixel acquisition point corresponding to the closer distance is greater than or equal to the target resolution of the second pixel acquisition point corresponding to the farther distance.

[0122] In one embodiment of this application, the apparatus 200 for reducing the power consumption of a head-mounted display device provided in this application further includes:

[0123] The fusion module is used to acquire the current real environment image captured by the camera and the corresponding current virtual environment image;

[0124] When the size of the current real environment image is smaller than the size of the current virtual image, the current real environment image is fused with the first virtual image to obtain a first fused image. The first virtual image is the virtual image in the current virtual image that is to be displayed in the gaze area.

[0125] The first fused image and the second fused image are stitched together to form the current complete fused image. The second fused image is the fused image in the previous complete fused image excluding the corresponding area of ​​the first fused image, or the second fused image is the virtual image in the current virtual image to be displayed in the non-focused area.

[0126] In one embodiment of this application, the setting module 220 is further configured to set the frame rate of the corresponding gaze area in the display screen of the head-mounted display device to be a first display frame rate and the frame rate of the corresponding non-gaze area to be a second display frame rate, wherein the first display frame rate is greater than the second display frame rate;

[0127] The control module 230 is also configured to control the display screen to display images in the gaze area at a first display frame rate and in the non-gaze area at a second display frame rate.

[0128] In one embodiment of this application, the setting module 220 is further configured to set the brightness of the corresponding gaze area in the display screen of the head-mounted display device to a first brightness and the brightness of the corresponding non-gaze area to a second brightness, wherein the first brightness is greater than the second brightness;

[0129] The control module 230 is also used to control the display screen to display an image at a first brightness in the viewing area and at a second brightness in the non-viewing area.

[0130] This application also provides a head-mounted display device, which includes any of the devices for reducing the power consumption of the head-mounted display device as provided in the above device embodiments, an eye-tracking module, a camera, and a display screen;

[0131] Or, such as Figure 3 As shown, the head-mounted display device 300 includes an eye-tracking module 310, a camera 320, a display screen 330, a memory 340, and a processor 350. The eye-tracking module 310, the camera 320, the display screen 330, and the memory 340 are all connected to the processor 350. The memory 340 is used to store computer instructions, and the processor 350 is used to retrieve the computer instructions from the memory 340 to execute the method as described in any of the above method embodiments.

[0132] In one embodiment of this application, the processor 350 is a collection of the main control chip, camera module, and display module hardware with processing functions in the head-mounted display device.

[0133] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method according to any one of the above-described method embodiments.

[0134] This application may be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this application.

[0135] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0136] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0137] The computer program instructions used to perform the operations of this application may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuits, such as programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), are personalized by utilizing state information from the computer-readable program instructions. These electronic circuits can execute the computer-readable program instructions to implement various aspects of this application.

[0138] Various aspects of this application are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0139] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0140] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0141] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions. It will be well known to those skilled in the art that implementation in hardware, implementation in software, and implementation using a combination of software and hardware are equivalent.

[0142] The various embodiments of this application have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical applications, or technical improvements to the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein. The scope of this application is defined by the appended claims.

Claims

1. A method for reducing the power consumption of a head-mounted display device, characterized in that, The method, applied to a head-mounted display device including an eye-tracking module, a camera, and a display screen, comprises: Based on the eye movement information collected by the eye-tracking module, the gaze area and non-gaze area of ​​the wearer of the head-mounted display device on the display screen are determined; The first pixel acquisition point is set to a first acquisition frame rate and the second pixel acquisition point is set to a second acquisition frame rate. The first pixel acquisition point is the pixel acquisition point of the camera that acquires the pixels of the gaze area and displays the environmental image. The second pixel acquisition point is the pixel acquisition point of the camera that acquires the pixels of the non-gaze area and displays the environmental image. The first acquisition frame rate is greater than the second acquisition frame rate. The first pixel acquisition point is controlled to acquire environmental images at the first acquisition frame rate, and the second pixel acquisition point is controlled to acquire environmental images at the second acquisition frame rate.

2. The method according to claim 1, characterized in that, Setting the acquisition frame rate of the second pixel acquisition point to the second acquisition frame rate includes: Based on the distance between the pixels of the environmental image captured by the second pixel acquisition point and the gaze area, the acquisition frame rate of the second pixel acquisition point is set to a second acquisition frame rate that matches the distance. Wherein, the second acquisition frame rate of the second pixel acquisition point corresponding to the closer distance is greater than or equal to the second acquisition frame rate of the second pixel acquisition point corresponding to the farther distance.

3. The method according to claim 1, characterized in that, After determining the gaze area and non-gaze area of ​​the wearer of the head-mounted display device on the display screen based on the eye movement information collected by the eye-tracking module, the method further includes: Enable the first pixel acquisition point; Enable the second pixel acquisition point of the target resolution corresponding to the pixel in the first non-focused area. The first non-focused area is composed of pixels whose distance from the focus area is less than or equal to a preset distance. The target resolution is less than the full resolution of the second pixel acquisition point corresponding to the pixel in the first non-focused area. The second non-focused region is closed to the second pixel acquisition point, and the second non-focused region is composed of pixels whose distance from the focusing region is greater than the preset distance.

4. The method according to claim 3, characterized in that, The step of activating the second pixel acquisition point of the target resolution corresponding to the pixel point in the first non-focused region includes: Based on the distance between the pixels of the environmental image captured by the second pixel acquisition point and the gaze area, activate the second pixel acquisition point with a target resolution that matches the distance to the pixels in the first non-gaze area; Wherein, the target resolution of the second pixel acquisition point corresponding to the closer distance is greater than or equal to the target resolution of the second pixel acquisition point corresponding to the farther distance.

5. The method according to claim 1, characterized in that, The method further includes controlling the first pixel acquisition point to acquire environmental images at the first acquisition frame rate and the second pixel acquisition point to acquire environmental images at the second acquisition frame rate: Acquire the current real-world image captured by the camera and the corresponding current virtual-world image; When the size of the current real environment image is smaller than the size of the current virtual image, the current real environment image is fused with the first virtual image to obtain a first fused image. The first virtual image is the virtual image in the current virtual image that is to be displayed in the gaze area. The first fused image and the second fused image are stitched together to form the current complete fused image. The second fused image is the fused image in the previous complete fused image excluding the corresponding area of ​​the first fused image, or the second fused image is the virtual image in the current virtual image to be displayed in the non-focused area.

6. The method according to any one of claims 1-5, characterized in that, After controlling the first pixel acquisition point to acquire environmental images at the first acquisition frame rate and the second pixel acquisition point to acquire environmental images at the second acquisition frame rate, the method further includes: The frame rate of the corresponding gaze area in the display screen of the head-mounted display device is set to a first display frame rate and the frame rate of the corresponding non-gaze area is set to a second display frame rate, wherein the first display frame rate is greater than the second display frame rate; The display screen is controlled to display images at a first display frame rate in the gaze area and at a second display frame rate in the non-gaze area.

7. The method according to any one of claims 1-5, characterized in that, After controlling the first pixel acquisition point to acquire environmental images at the first acquisition frame rate and the second pixel acquisition point to acquire environmental images at the second acquisition frame rate, the method further includes: The brightness of the corresponding gaze area in the display screen of the head-mounted display device is set as a first brightness and the brightness of the corresponding non-gaze area is set as a second brightness, wherein the first brightness is greater than the second brightness; The display screen is controlled to display an image at a first brightness in the viewing area and at a second brightness in the non-viewing area.

8. A device for reducing power consumption of a head-mounted display device, characterized in that, Applied to head-mounted display devices, the head-mounted display devices including eye-tracking modules, cameras, and displays, the device includes: The determination module is used to determine the gaze area and non-gaze area of ​​the wearer of the head-mounted display device on the display screen based on the eye movement information collected by the eye-tracking module. The setting module is used to set the acquisition frame rate of the first pixel acquisition point as the first acquisition frame rate and the acquisition frame rate of the second pixel acquisition point as the second acquisition frame rate. The first pixel acquisition point is the pixel acquisition point of the camera that acquires the pixels of the gaze area and displays the environmental image. The second pixel acquisition point is the pixel acquisition point of the camera that acquires the pixels of the non-gaze area and displays the environmental image. The first acquisition frame rate is greater than the second acquisition frame rate. The control module is used to control the first pixel acquisition point to acquire environmental images according to the first acquisition frame rate and the second pixel acquisition point to acquire environmental images according to the second acquisition frame rate, so as to obtain a real environmental image.

9. A head-mounted display device, characterized in that, The head-mounted display device includes the apparatus as described in claim 8, an eye-tracking module, a camera, and a display screen; Alternatively, the head-mounted display device includes the eye-tracking module, the camera, the display screen, a memory, and a processor. The eye-tracking module, the camera, the display screen, and the memory are all connected to the processor. The memory is used to store computer instructions, and the processor is used to retrieve the computer instructions from the memory to perform the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the method according to any one of claims 1-7.

Images