Screen control method and control apparatus for head-mounted display device

Abstract

The present disclosure provides a screen control method and control device of a head-mounted display device, the method comprising: acquiring environmental brightness information at a first time and pupil information of a wearer of the head-mounted display device, the first time being a time when the head-mounted display device is worn; in the case where the environmental brightness information changes, monitoring pupil changes of the wearer of the head-mounted display device; and adjusting screen display parameters according to the pupil changes of the wearer of the head-mounted display device.

Description

Technical Field

[0001] This disclosure relates to the field of head-mounted display technology, and more specifically, to a screen control method, control device, apparatus, and readable storage medium for head-mounted display devices. Background Technology

[0002] Currently, head-mounted display devices offer two main methods for adjusting screen brightness: first, requiring the wearer to manually adjust the brightness based on their perception, which results in low precision and inaccuracy; and second, adjusting screen brightness based on ambient light, which cannot flexibly adapt to wearers with varying visual sensitivities. Summary of the Invention

[0003] One object of this disclosure is to provide a screen control method for a head-mounted display device.

[0004] According to a first aspect of the present disclosure, a screen control method for a head-mounted display device is provided, the method comprising:

[0005] Acquire ambient brightness information and pupil information of the wearer of the head-mounted display device at a first moment, where the first moment is the moment when the head-mounted display device is worn;

[0006] When the ambient brightness information changes, monitor the pupil changes of the wearer of the head-mounted display device;

[0007] The screen display parameters are adjusted based on the pupil changes of the wearer of the head-mounted display device.

[0008] Optionally, adjusting the screen display parameters based on changes in the pupils of the wearer of the head-mounted display device includes:

[0009] Capture the gaze area of ​​the wearer's two eyes at the current moment;

[0010] Without changing the fixation point area, the display parameters of the fixation point area are first adjusted according to the pupil changes of the wearer of the head-mounted display device, and then the display parameters of the surrounding area of ​​the fixation point area are adjusted step by step.

[0011] When the gaze point area changes, the display parameters of the current gaze point area are first adjusted according to the pupil changes of the wearer of the head-mounted display device, and then the display parameters of the surrounding area are adjusted step by step.

[0012] Optionally, adjusting the screen display parameters based on changes in the pupils of the wearer of the head-mounted display device includes:

[0013] Set a first threshold and a second threshold for pupil size. The first threshold is the pupil information when the pupil dilates, and the second threshold is the pupil information when the pupil shrinks.

[0014] If the pupil information of the wearer of the head-mounted display device is greater than a first threshold, the screen brightness is increased.

[0015] If the pupil information of the wearer of the head-mounted display device is greater than a second threshold, the screen brightness is adjusted to decrease.

[0016] Optionally, after adjusting the screen display parameters according to the pupil changes of the wearer of the head-mounted display device, the method further includes:

[0017] Record each set of ambient brightness information, the pupil information of the wearer of the head-mounted display device, and the screen brightness to generate a mapping relationship;

[0018] When the ambient brightness information changes, the screen display parameters are adjusted according to the mapping relationship.

[0019] Optionally, before adjusting the screen display parameters based on changes in the pupils of the wearer of the head-mounted display device, the method further includes:

[0020] Based on the mapping relationship, set the screen brightness adjustment time and screen brightness adjustment range.

[0021] Optionally, adjusting the screen display parameters based on changes in the pupils of the wearer of the head-mounted display device includes:

[0022] Obtain the ambient brightness information at the second moment;

[0023] If the ambient brightness information at the second moment is the same as that at the first moment, the screen display parameters are adjusted back to the historical screen display parameters.

[0024] According to a second aspect of the present disclosure, a screen control device for a head-mounted display device is provided, the device comprising:

[0025] The acquisition module is used to acquire ambient brightness information and pupil information of the wearer of the head-mounted display device at a first moment, wherein the first moment is the moment when the head-mounted display device is worn;

[0026] The monitoring module is used to monitor changes in the pupil size of the wearer of the head-mounted display device in the event of changes in ambient brightness information.

[0027] The adjustment module is used to adjust the screen display parameters according to the pupil changes of the wearer of the head-mounted display device.

[0028] Optionally, the adjustment module further includes:

[0029] The capture module is used to capture the gaze point area of ​​the wearer's eyes at the current moment.

[0030] The first adjustment module is used to adjust the display parameters of the gaze point area first, and then adjust the display parameters of the surrounding area of ​​the gaze point area step by step, based on the pupil changes of the wearer of the head-mounted display device, without changing the gaze point area.

[0031] The second adjustment module is used to adjust the display parameters of the current gaze point area first, and then adjust the display parameters of the surrounding area of ​​the current gaze point area step by step, based on the pupil changes of the wearer of the head-mounted display device when the gaze point area changes.

[0032] According to a third aspect of the present disclosure, a head-mounted display device is provided, comprising:

[0033] Memory is used to store executable computer instructions;

[0034] A processor, configured to execute the screen control method according to any one of the first aspects above, under the control of the executable computer instructions.

[0035] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided, on which computer instructions are stored, which, when executed by a processor, perform the screen control method described in any one of the first aspects above.

[0036] One beneficial effect of this disclosure is that it detects changes in the pupils of the wearer of the head-mounted display device when the ambient brightness information changes, learns the sensitivity of different wearers' pupils to ambient brightness, and adjusts the screen brightness according to the wearer's pupil changes. This can flexibly adapt to wearers with different pupil sensitivities and does not rely on the wearer to manually adjust the screen brightness, resulting in higher adjustment accuracy.

[0037] Other features and advantages of this specification will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0038] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments of this specification and, together with their description, serve to explain the principles of this specification.

[0039] Figure 1 This is a schematic flowchart of a screen control method for a head-mounted display device according to an embodiment of the present disclosure;

[0040] Figure 2 This is a schematic diagram illustrating screen brightness prediction when ambient brightness increases according to an embodiment of the present disclosure;

[0041] Figure 3 This is a schematic diagram illustrating screen brightness prediction when ambient brightness decreases according to an embodiment of the present disclosure;

[0042] Figure 4 This is a schematic diagram of the brightness adjustment area according to an embodiment of the present disclosure, with the wearer's line of sight remaining unchanged;

[0043] Figure 5 This is a schematic diagram of the brightness adjustment area according to an embodiment of the present disclosure when the wearer's line of sight changes;

[0044] Figure 6 This is a schematic diagram of the structure of the screen control device of a head-mounted display device according to an embodiment of the present disclosure;

[0045] Figure 7 This is a schematic diagram of the hardware structure of a head-mounted display device according to an embodiment of the present disclosure. Detailed Implementation

[0046] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the embodiments of the present disclosure.

[0047] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit this disclosure or its application or use.

[0048] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0049] In all the examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0050] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0051] <Method Implementation>

[0052] Figure 1 An embodiment of the present disclosure illustrates a screen control method for a head-mounted display device, such as... Figure 1 As shown, the screen control method of the head-mounted display device in this embodiment may include the following steps S101 to S103:

[0053] S101, acquire ambient brightness information and pupil information of the wearer of the head-mounted display device at a first moment, where the first moment is the moment when the head-mounted display device is worn.

[0054] In this embodiment, at the first moment, an eye-tracking model of the wearer is established using eye-tracking technology, and the wearer's pupil data, iris data, etc. are extracted to improve the utilization rate of data, facilitate the recording and monitoring of the wearer's pupil size, and export the wearer's pupil information from the eye model.

[0055] Since wearers of head-mounted display devices will be navigating between environments with significant brightness contrasts, such as driving from the ground level into a dimly lit underground parking garage, the screen control method of this embodiment allows the head-mounted display device to automatically adjust the field of vision brightness according to changes in external brightness, adapting to the current dim reality, reducing eye strain and minimizing danger.

[0056] In this embodiment, an RGB camera is used to acquire a scene image of the current scene where the head-mounted display device is located at the first moment, and an algorithm is used to adjust the brightness of the image to the actual ambient brightness of the current scene before displaying it on the screen of the head-mounted display device.

[0057] S102, when the ambient brightness information changes, monitor the pupil changes of the wearer of the head-mounted display device.

[0058] In this embodiment, after the first moment, the head-mounted display device monitors the ambient brightness information in real time, and when the ambient brightness information changes, the established eye model of the wearer implements eye tracking technology to monitor the changes in the wearer's pupils in real time.

[0059] The changes in the pupil include, but are not limited to, changes in pupil area or changes in the major and minor axes of the ellipse fitted by the pupil.

[0060] S103, adjust the screen display parameters according to the pupil changes of the wearer of the head-mounted display device.

[0061] In this embodiment, since each wearer has a different ability to adapt to brightness, the screen display parameters are adjusted according to changes in the wearer's pupils to flexibly adapt to the wearer's pupil sensitivity. These screen display parameters include screen brightness.

[0062] This embodiment can adjust the screen brightness to display images based on the wearer's eye sensitivity to brightness, not only according to the brightness changes in the real scene, but also by monitoring the wearer's sensitivity to the brightness of the current scene. This enhances the wearer's experience and reduces the adaptation time caused by brightness changes.

[0063] In one embodiment, when the wearer moves from a brighter area to a darker area, the head-mounted display device can proactively and adaptively increase the screen brightness in advance. Specifically, it first increases the brightness of the darker area, and then, after time T, adjusts it to the current actual brightness of the darker area, reducing the wearer's adaptation time to the sudden change to a darker area. Conversely, when the wearer moves from a darker area to a brighter area, the head-mounted display device can proactively and adaptively decrease the screen brightness in advance. Specifically, it first decreases the brightness of the brighter area, and then, after time T, adjusts it to the current actual brightness of the brighter area, again reducing the wearer's adaptation time to the sudden change to a brighter area.

[0064] In one embodiment, such as Figure 4 and Figure 5 As shown, adjusting screen display parameters based on changes in the pupils of the wearer of the head-mounted display device includes:

[0065] S1031, Capture the gaze point area of ​​the wearer's eyes at the current moment;

[0066] S1032, without changing the gaze point area, adjust the display parameters of the gaze point area first according to the pupil changes of the wearer of the head-mounted display device, and then adjust the display parameters of the surrounding area of ​​the gaze point area step by step.

[0067] S1033, when the gaze point area changes, according to the pupil changes of the wearer of the head-mounted display device, first adjust the display parameters of the current gaze point area, and then adjust the display parameters of the surrounding area of ​​the current gaze point area step by step.

[0068] In this embodiment, the gaze point is the region where the wearer's binocular gaze intersects, calculated using eye-tracking technology. Figure 4 As shown, the dashed box represents the wearer's gaze area, and the shaded area represents the brightness adjustment area. Figure 4 This describes the process of adjusting brightness in stages, centered on the focal point area. With the wearer's line of sight remaining constant (i.e., the focal point area unchanged), the display area of ​​the head-mounted display is centered on the focal point area. Other areas surrounding the focal point are then divided into levels based on their distance from it, from smallest to largest. When adjusting screen brightness, the brightness of the focal point area is adjusted first, followed by the gradual adjustment of the brightness of other areas according to their respective levels.

[0069] like Figure 5 As shown, the dashed box represents the wearer's current gaze area, and the shaded area represents the brightness adjustment area. Figure 5This diagram illustrates the brightness adjustment process of the first, second, ..., Nth fixation point regions as the fixation point moves, showing the process of adjusting brightness level by level with the current Nth fixation point region as the center. When the wearer's gaze moves, i.e., the fixation point region changes, it is divided into the first, second, ..., Nth fixation point regions. These regions are the areas that the wearer's line of sight passes through sequentially. The display area of ​​the head-mounted display device is centered on the first, second, ..., Nth fixation point regions. Other areas surrounding the Nth fixation point region are divided into levels according to their distance from the fixation point region, from smallest to largest. When adjusting the screen brightness, the brightness of the first, second, ..., Nth fixation point regions is adjusted first, and then the brightness of other areas is adjusted level by level according to these levels.

[0070] The screen control method provided in this embodiment can render brightness according to the position of the gaze point, ensuring the fastest and most timely adjustment of the brightness of the wearer's gaze point position, adapting to changes in the wearer's pupils, and then adjusting the brightness information of the area position so that the brightness of the entire display area of ​​the head-mounted display device is consistent, and flexibly adapting to the pupil sensitivity of different wearers.

[0071] In one embodiment, adjusting screen display parameters based on pupil changes of the wearer of the head-mounted display device includes:

[0072] S1034, set a first threshold and a second threshold for the pupil, where the first threshold is the pupil information when the pupil dilates and the second threshold is the pupil information when the pupil shrinks.

[0073] S1035, if the pupil information of the wearer of the head-mounted display device is greater than the first threshold, adjust the screen brightness to increase;

[0074] S1036, if the pupil information of the wearer of the head-mounted display device is greater than the second threshold, adjust the screen brightness to decrease.

[0075] In this embodiment, if the ambient brightness in front of the wearer decreases, based on the ambient brightness information and the wearer's pupil information obtained at the first moment, it is monitored that the pupil has become larger than the first threshold. At this time, the first threshold is set to 5% of the pupil area, and the brightness of the image displayed on the current head-mounted display device needs to be increased.

[0076] If the ambient brightness in front of the wearer increases, based on the ambient brightness information and the wearer's pupil information obtained at the first moment, if the pupil size is detected to exceed the second threshold, the second threshold is set to 5% of the pupil area, and the brightness of the image displayed on the current head-mounted display device needs to be reduced.

[0077] In one embodiment, when the brightness of the image displayed by the head-mounted display is reduced, the brightness of the image displayed by the head-mounted display in the next frame is gradually increased to reduce eye strain on the user. Similarly, when the brightness of the image displayed by the head-mounted display is increased, the brightness of the image displayed by the head-mounted display in the next frame is gradually reduced to shorten the adaptation time for the wearer's eyes.

[0078] In one embodiment, when the ambient brightness in front of the wearer decreases or increases, the changes in the wearer's pupils are monitored. If the wearer's pupils dilate but do not exceed a first threshold or constrict but do not exceed a second threshold, it indicates that the wearer's pupils are adapting to the current brightness change and will not cause significant changes in the pupils. The head-mounted display device can then directly display the current ambient brightness.

[0079] In one embodiment, after adjusting the screen display parameters based on changes in the pupils of the wearer of the head-mounted display device, the method further includes:

[0080] S104, record each set of ambient brightness information, the pupil information of the wearer of the head-mounted display device, and the screen brightness, and generate a mapping relationship;

[0081] S105, when the ambient brightness information changes, adjust the screen display parameters according to the mapping relationship.

[0082] In this embodiment, after the wearer puts on the head-mounted display device, the device records the wearer's pupil data and required screen brightness under each set of ambient brightness information, and generates a mapping relationship between the three. When the ambient brightness information changes, the screen brightness is adjusted according to the mapping relationship, decreasing or increasing the brightness while continuing to monitor the wearer's pupil size in real time.

[0083] In one embodiment, before adjusting the screen display parameters based on changes in the pupils of the wearer of the head-mounted display device, the method further includes:

[0084] S106, Based on the mapping relationship, set the screen brightness adjustment time and screen brightness adjustment range.

[0085] In this embodiment, different wearers have different sensitivities to brightness. By analyzing the mapping relationship of wearers stored in the head-mounted display device, the screen brightness adjustment time and screen brightness adjustment range are set.

[0086] In one embodiment, the wearer sets the screen brightness adjustment time to 3 seconds based on their sensitivity to brightness, and the screen brightness adjustment increment to Mcd / m² per second. 2(candela / square meter) After setting, determine the brightness that needs to be reduced or reduced according to the mapping relationship, and adjust the brightness evenly according to the set screen brightness adjustment time and screen brightness adjustment range to prevent excessive changes from causing discomfort to the wearer's pupils.

[0087] In one embodiment, adjusting the screen display parameters based on changes in the pupils of the wearer of the head-mounted display device includes:

[0088] S107, Obtain the ambient brightness information at the second moment;

[0089] S108, if the ambient brightness information at the second moment is the same as the ambient brightness information at the first moment, the screen display parameters are adjusted back to the historical screen display parameters.

[0090] In this embodiment, as Figure 2 As shown, the system acquires ambient brightness information and the wearer's pupil information when wearing the head-mounted display device. When the ambient brightness increases, it searches for historical data corresponding to the current ambient brightness in a mapping relationship based on the magnitude of the increase, thus predicting the wearer's pupil change magnitude. Based on the wearer's pupil change, the screen display parameters are adjusted. At the second moment, when the ambient brightness information changes, and it matches the historical ambient brightness information from the first moment, the screen display parameters are slowly restored to the historical parameters. The restored screen display parameters are adapted to the wearer's pupils, and the wearer's pupil data is restored to the state at the first moment.

[0091] In this embodiment, as Figure 3 As shown, the system acquires ambient brightness information and the wearer's pupil information when wearing the head-mounted display device. When the ambient brightness decreases, it searches for historical data corresponding to the current ambient brightness in a mapping relationship based on the magnitude of the decrease, thereby predicting the wearer's pupil change magnitude. Based on the wearer's pupil change, the screen display parameters are adjusted. At the second moment, the ambient brightness information changes. When the ambient brightness information at the second moment matches the historical ambient brightness information at the first moment, the screen display parameters are slowly restored to the historical screen display parameters. The restored screen display parameters are adapted to the wearer's pupils, and the wearer's pupil data is restored to the state at the first moment.

[0092] The historical screen brightness is the screen brightness corresponding to the historical ambient brightness information that is consistent with the current ambient brightness information in the mapping relationship.

[0093] According to embodiments of this disclosure, changes in the pupils of the wearer of the head-mounted display device are detected when ambient brightness information changes, the sensitivity of different wearers' pupils to ambient brightness is determined, and the screen brightness is adjusted according to the wearer's pupil changes. This can flexibly adapt to wearers with different pupil sensitivities and does not rely on the wearer manually adjusting the screen brightness, resulting in higher adjustment accuracy.

[0094] <Device Embodiment>

[0095] Figure 6 This is a schematic diagram of the screen control device of a head-mounted display device according to one embodiment. Figure 6 As shown, the screen control device 600 includes a first acquisition module 610, a first monitoring module 620, and a first adjustment module 630.

[0096] The first acquisition module 610 is used to acquire ambient brightness information and pupil information of the wearer of the head-mounted display device at a first moment, wherein the first moment is the moment when the head-mounted display device is worn.

[0097] The first monitoring module 620 is used to monitor changes in the pupils of the wearer of the head-mounted display device when the ambient brightness information changes.

[0098] The first adjustment module 630 is used to adjust the screen display parameters according to the pupil changes of the wearer of the head-mounted display device.

[0099] In one embodiment, the first adjustment module 630 further includes (not shown in the figure):

[0100] The capture module 6301 is used to capture the gaze point area of ​​the wearer's eyes at the current moment;

[0101] The second adjustment module 6302 is used to adjust the display parameters of the gaze point area first, and then adjust the display parameters of the surrounding area of ​​the gaze point area step by step, based on the pupil changes of the wearer of the head-mounted display device, without changing the gaze point area.

[0102] The third adjustment module 6303 is used to adjust the display parameters of the current gaze point area first, and then adjust the display parameters of the surrounding area of ​​the current gaze point area step by step, according to the pupil changes of the wearer of the head-mounted display device when the gaze point area changes.

[0103] In one embodiment, the first adjustment module 630 further includes (not shown in the figure):

[0104] The first setting module 6304 is used to set a first threshold and a second threshold for the pupil, wherein the first threshold is the pupil information when the pupil dilates and the second threshold is the pupil information when the pupil shrinks.

[0105] The fourth adjustment module 6305 is used to increase the screen brightness when the pupil information of the wearer of the head-mounted display device is greater than a first threshold.

[0106] The fifth adjustment module 6306 is used to adjust the screen brightness to decrease when the pupil information of the wearer of the head-mounted display device is greater than the second threshold.

[0107] In one embodiment, the screen control device 600 further includes (not shown):

[0108] Storage module 640 is used to record each set of ambient brightness information, the pupil information of the wearer of the head-mounted display device, and screen brightness, and generate a mapping relationship;

[0109] The sixth adjustment module 650 is used to adjust the screen display parameters according to the mapping relationship when the ambient brightness information changes.

[0110] In one embodiment, the screen control device 600 further includes (not shown):

[0111] The second setting module 660 is used to set the screen brightness adjustment time and screen brightness adjustment range according to the mapping relationship.

[0112] In one embodiment, the screen control device 600 further includes (not shown):

[0113] The second acquisition module 670 is used to acquire ambient brightness information at the second moment.

[0114] The seventh adjustment module 680 adjusts the screen display parameters back to historical screen display parameters when the ambient brightness information at the second moment is the same as that at the first moment.

[0115] According to embodiments of this disclosure, changes in the pupils of the wearer of the head-mounted display device are detected when ambient brightness information changes, the sensitivity of different wearers' pupils to ambient brightness is determined, and the screen brightness is adjusted according to the wearer's pupil changes. This can flexibly adapt to wearers with different pupil sensitivities and does not rely on the wearer manually adjusting the screen brightness, resulting in higher adjustment accuracy.

[0116] <Equipment Example>

[0117] Figure 7 This is a schematic diagram of the hardware structure of a head-mounted display device according to one embodiment. Figure 7 As shown, the head-mounted display device 700 includes a processor 710 and a memory 720.

[0118] The memory 720 can be used to store executable computer instructions.

[0119] The processor 710 can be used to execute the screen control method according to the method embodiments of this disclosure, under the control of the executable computer instructions.

[0120] The head-mounted display device 700 can be a VR device or a device with other hardware structures; no limitation is made here.

[0121] In another embodiment, the head-mounted display device 700 may include the screen control device 600 described above.

[0122] In one embodiment, each module of the screen control device 600 can be implemented by the processor 710 running computer instructions stored in the memory 720.

[0123] Computer-readable storage media

[0124] This disclosure also provides a computer-readable storage medium storing computer instructions thereon, which, when executed by a processor, perform the screen control method provided in this disclosure.

[0125] This disclosure can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this disclosure.

[0126] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0127] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0128] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0129] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0130] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0131] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0132] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions. It will be known to those skilled in the art that implementation in hardware, implementation in software, and implementation in a combination of software and hardware are equivalent.

[0133] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, and are not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein. The scope of this disclosure is defined by the appended claims.

Claims

1. A screen control method for a head-mounted display device, characterized in that, The method includes: Acquire ambient brightness information and pupil information of the wearer of the head-mounted display device at a first moment, where the first moment is the moment when the head-mounted display device is worn; When the ambient brightness information changes, monitor the pupil changes of the wearer of the head-mounted display device; Adjust screen display parameters based on changes in the pupil size of the wearer of the head-mounted display device; The step of adjusting screen display parameters based on changes in the pupils of the wearer of the head-mounted display device includes: Capture the gaze area of ​​the wearer's two eyes at the current moment; Without changing the fixation point area, the display parameters of the fixation point area are first adjusted according to the pupil changes of the wearer of the head-mounted display device, and then the display parameters of the surrounding area of ​​the fixation point area are adjusted step by step. When the gaze point area changes, the display parameters of the current gaze point area are first adjusted according to the pupil changes of the wearer of the head-mounted display device, and then the display parameters of the surrounding area are adjusted step by step.

2. The method according to claim 1, characterized in that, The step of adjusting screen display parameters based on changes in the pupils of the wearer of the head-mounted display device includes: Set a first threshold and a second threshold for pupil size. The first threshold is the pupil information when the pupil dilates, and the second threshold is the pupil information when the pupil shrinks. If the pupil information of the wearer of the head-mounted display device is greater than a first threshold, the screen brightness is increased. If the pupil information of the wearer of the head-mounted display device is greater than a second threshold, the screen brightness is adjusted to decrease.

3. The method according to claim 1 or 2, characterized in that, After adjusting the screen display parameters based on the pupil changes of the wearer of the head-mounted display device, the method further includes: Record each set of ambient brightness information, the pupil information of the wearer of the head-mounted display device, and the screen brightness to generate a mapping relationship; When the ambient brightness information changes, the screen display parameters are adjusted according to the mapping relationship.

4. The method according to claim 3, characterized in that, Before adjusting the screen display parameters based on changes in the pupils of the wearer of the head-mounted display device, the method further includes: Based on the mapping relationship, set the screen brightness adjustment time and screen brightness adjustment range.

5. The method according to claim 1, characterized in that, The step of adjusting the screen display parameters based on the pupil changes of the wearer of the head-mounted display device includes: Obtain the ambient brightness information at the second moment; If the ambient brightness information at the second moment is the same as that at the first moment, the screen display parameters are adjusted back to the historical screen display parameters.

6. A screen control device for a head-mounted display device, characterized in that, The device includes: The acquisition module is used to acquire ambient brightness information and pupil information of the wearer of the head-mounted display device at a first moment, wherein the first moment is the moment when the head-mounted display device is worn; The monitoring module is used to monitor changes in the pupil size of the wearer of the head-mounted display device in the event of changes in ambient brightness information. An adjustment module is used to adjust screen display parameters based on changes in the pupil size of the wearer of the head-mounted display device; The adjustment module further includes: The capture module is used to capture the gaze point area of ​​the wearer's eyes at the current moment; The first adjustment module is used to adjust the display parameters of the gaze point area first, and then adjust the display parameters of the surrounding area of ​​the gaze point area step by step, based on the pupil changes of the wearer of the head-mounted display device, without changing the gaze point area. The second adjustment module is used to adjust the display parameters of the current gaze point area first, and then adjust the display parameters of the surrounding area of ​​the current gaze point area step by step, based on the pupil changes of the wearer of the head-mounted display device when the gaze point area changes.

7. A head-mounted display device, characterized in that, include: Memory is used to store executable computer instructions; A processor configured to execute the screen control method according to any one of claims 1-5, under the control of the executable computer instructions.

8. A computer-readable storage medium having computer instructions stored thereon, the computer instructions being executed by a processor to perform the screen control method according to any one of claims 1-5.

Images