Brightness adjustment method and device, head-mounted display equipment and readable storage medium

Abstract

The invention discloses a brightness adjustment method and device, head-mounted display equipment and a readable storage medium. The brightness adjustment method comprises the steps of obtaining a first image brightness value corresponding to a current to-be-played image frame; determining a lens cone current value mapped by the first image brightness value based on a preset brightness mapping relation; and taking the mapped lens cone current value as a target lens cone current value, and applying the target lens cone current value when the to-be-played image frame is played so as to control the current brightness of the lens cone to be the brightness corresponding to the target lens cone current value. According to the invention, even when the lens cone of the head-mounted display device plays the image frames with different image brightness values, the illumination intensity sensed by the eyes of a user is always within the illumination intensity range comfortable to be sensed by the human eyes, and the discomfort of the human eyes caused by strong light or weak light is avoided.

Description

Technical Field

[0001] The present application relates to the technical field of head-mounted display devices, and in particular to a brightness adjustment method, device, head-mounted display device, and readable storage medium. Background Art

[0002] Head-mounted display devices are devices that use head-mounted displays to block a person's vision of the outside world, and guide the user to feel as if they are in a virtual environment. For example, VR (Virtual Reality) devices, in order to avoid excessive eye damage when users wear VR devices (a type of head-mounted display devices), it is necessary to adjust the brightness of the VR device's lens barrel to prevent users from being in an environment with too bright or too dark screen brightness for a long time, thereby avoiding damage to the user's vision or causing discomfort to the user's eyes.

[0003] At present, the screen brightness of common electronic devices on the market is adjusted according to the ambient light around the user, that is, the ambient light intensity around the user is obtained through a photosensitive sensor, and the screen backlight is increased or decreased according to the ambient light intensity.

[0004] However, this adjustment method is only applicable to the case where the device screen and the outside world are in the same space. For head-mounted display devices (such as VR devices, etc.), the screen and the external ambient light are in two relatively independent spaces, and the VR device cannot directly adjust the screen brightness according to the external ambient light. Therefore, in the related technical solutions, the method of adjusting the screen backlight by ambient light in electronic devices is difficult to apply to head-mounted display devices.

[0005] The above contents are only used to assist in understanding the technical solution of the present invention and do not constitute an admission that the above contents are prior art. Summary of the invention

[0006] The main purpose of the present application is to provide a brightness adjustment method, device, head-mounted display device and readable storage medium, aiming to solve the technical problem that when users use head-mounted display devices, they are easily in an environment of too bright or too dark screen brightness, causing excessive eye damage.

[0007] To achieve the above object, the present application provides a brightness adjustment method, which is applied to a head-mounted display device, wherein the head-mounted display device includes a lens barrel for displaying an image, and the method includes: Obtaining a first image brightness value corresponding to the current image frame to be played; Based on a preset brightness mapping relationship, determining a barrel current value mapped to the brightness value of the first image, wherein the brightness mapping relationship is calibrated by a matching relationship between the barrel current value and the image brightness value corresponding to a comfortable photosensitivity range of human eyes; The mapped lens barrel current value is used as a target lens barrel current value, and when playing the image frame to be played, the target lens barrel current value is applied to control the current brightness of the lens barrel to be the brightness corresponding to the target lens barrel current value.

[0008] Optionally, before the step of obtaining the first image brightness value corresponding to the current image frame to be played, the method further includes: Get the RGB value of each pixel in the current image frame to be played; According to the RGB values ​​of the pixels, a first image brightness value corresponding to the current image frame to be played is calculated.

[0009] Optionally, the step of calculating the first image brightness value corresponding to the current image frame to be played according to the RGB value of each pixel includes: Accumulate the RGB values ​​of the pixels to obtain the total RGB value corresponding to the current image frame to be played; Based on a preset RGB value mapping relationship, determine the image brightness value of the RGB total value mapping; The mapped image brightness value is used as the first image brightness value corresponding to the current image frame to be played.

[0010] Optionally, the method further comprises: Play a test video of a preset duration, wherein the test video includes a plurality of test image frames with different image brightness values; Obtaining the second image brightness value corresponding to the currently played test image frame and the current actual barrel current value; The eye image information corresponding to the test image frame currently played by the user is collected, and the mapping relationship between the barrel current value and the image brightness value is calibrated according to the second image brightness value, the actual barrel current value and the eye image information to obtain the brightness mapping relationship.

[0011] Optionally, the step of calibrating the mapping relationship between the lens barrel current value and the image brightness value according to the second image brightness value, the actual lens barrel current value and the eye image information includes: Determining the pupil area and eye opening width of the user according to the eye image information; Collecting current light intensity information on the human eye side, wherein the light intensity information is affected by the brightness value of the second image and the actual lens barrel current value; Determining whether the light intensity information conforms to a comfortable light sensitivity range of human eyes according to the pupil area and the eye opening amplitude; The mapping relationship between the barrel current value and the image brightness value is calibrated according to the second image brightness value corresponding to the comfortable photosensitivity interval and the actual barrel current value.

[0012] Optionally, the step of calibrating the mapping relationship between the barrel brightness value and the image brightness value according to the second image brightness value corresponding to the comfortable photosensitivity interval and the actual barrel current value includes: The second image brightness value corresponding to the comfortable photosensitivity interval is associated with the actual barrel current value to obtain a mapping relationship between the barrel current value and the image brightness value.

[0013] Optionally, the step of determining whether the light intensity information conforms to a comfortable light sensitivity range of human eyes according to the pupil area and the eye opening amplitude includes: If the pupil area is within the preset standard pupil area range, and the eye opening range is within the preset standard opening range, it is determined that the light intensity information meets the comfortable light sensitivity range of the human eye; If the pupil area does not fall within the preset standard pupil area range, and / or the eye opening amplitude does not fall within the preset standard opening amplitude range, it is determined that the light intensity information does not conform to the comfortable photosensitivity range of the human eye.

[0014] In addition, the present application also provides a brightness adjustment device, which is applied to a head-mounted display device, wherein the head-mounted display device includes a lens barrel for displaying an image, and the brightness adjustment device includes: An acquisition module, used to acquire a first image brightness value corresponding to a current image frame to be played; A determination module, configured to determine the barrel current value mapped to the brightness value of the first image based on a preset brightness mapping relationship, wherein the brightness mapping relationship is calibrated by a matching relationship between the barrel current value and the image brightness value corresponding to a comfortable photosensitivity range of human eyes; The control module is used to use the mapped lens barrel current value as a target lens barrel current value, and apply the target lens barrel current value when playing the image frame to be played, so as to control the current brightness of the lens barrel to be the brightness corresponding to the target lens barrel current value.

[0015] The present application also provides a head-mounted display device, which is a physical device. The head-mounted display device includes: a memory, a processor, and a program of the brightness adjustment method stored in the memory and executable on the processor. When the program of the brightness adjustment method is executed by the processor, the steps of the brightness adjustment method described above can be implemented.

[0016] The present application also provides a computer-readable storage medium, on which is stored a program for implementing the brightness adjustment method, and the program for implementing the brightness adjustment method is executed by a processor to implement the steps of the brightness adjustment method as described above.

[0017] The present application also provides a computer program product, including a computer program, which implements the steps of the brightness adjustment method as described above when executed by a processor.

[0018] The present application obtains the first image brightness value corresponding to the current image frame to be played, and determines the lens barrel current value mapped to the first image brightness value based on a preset brightness mapping relationship, wherein the brightness mapping relationship is calibrated by the matching relationship between the lens barrel current value and the image brightness value corresponding to the comfortable photosensitivity range of the human eye, and then uses the mapped lens barrel current value as the target lens barrel current value, and applies the target lens barrel current value when playing the image frame to be played, so as to control the current brightness of the lens barrel to be the brightness corresponding to the target lens barrel current value, so that when the lens barrel of the VR device plays image frames with different image brightness values, the brightness L (i.e., the image brightness value) of the image frame is too bright. If the brightness L of the image frame is too dark, the brightness of the lens barrel is automatically adjusted down by reducing the lens barrel current value. If the brightness L of the image frame is too dark, the brightness of the lens barrel is automatically adjusted up by increasing the lens barrel current value, so that the light perception Y sensed by the user's eyes is always within a moderate light intensity range, and there will be no eye discomfort caused by strong light or weak light. The light intensity of the light environment where the user's eyes are located is always within the light intensity range that is comfortable for the human eye, thereby avoiding damage to the user's eyesight or discomfort to the user's eyes, thereby solving the technical problem that when the user uses a head-mounted display device, the screen brightness environment is too bright or too dark, causing excessive eye damage.

[0019] Through a preset intelligent brightness adjustment algorithm, the present application can allow the lens barrel to automatically adjust to the most appropriate light intensity when the user uses a head-mounted display device, effectively preventing the user's eye fatigue and damage to the user's eyes, meeting the user's expectations for the adjusted brightness, and greatly improving the convenience of adjusting the screen brightness and the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0021] In order to more clearly illustrate the technical solutions in this embodiment or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0022] Figure 1 This is a flow chart of the first embodiment of the brightness adjustment method of the present application; Figure 2 This is a flow chart of the second embodiment of the brightness adjustment method of the present application; Figure 3 This is a structural diagram of VR glasses according to a specific embodiment of the present application; Figure 4 This is a schematic diagram of the module structure of the brightness adjustment device according to an embodiment of the present application; Figure 5 Schematic diagram of the device structure of the hardware operating environment involved in the head mounted display device in this embodiment.

[0023] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0024] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the scope of protection of the present invention.

[0025] Embodiment 1 Head-mounted display devices are devices that use head-mounted displays to block a person's vision of the outside world, and guide the user to feel as if they are in a virtual environment. For example, VR (Virtual Reality) devices, in order to avoid excessive eye damage when users wear VR devices (a type of head-mounted display devices), it is necessary to adjust the brightness of the VR device's lens barrel to prevent users from being in an environment with too bright or too dark screen brightness for a long time, thereby avoiding damage to the user's vision or causing discomfort to the user's eyes.

[0026] At present, the screen brightness of common electronic devices on the market is adjusted according to the ambient light around the user, that is, the ambient light intensity around the user is obtained through a photosensitive sensor, and the screen backlight is increased or decreased according to the ambient light intensity.

[0027] However, this adjustment method is only applicable to the case where the device screen and the outside world are in the same space. For head-mounted display devices (such as VR devices, etc.), the screen and the external ambient light are in two relatively independent spaces, and the VR device cannot directly adjust the screen brightness according to the external ambient light. Therefore, in the related technical solutions, the method of adjusting the screen backlight by ambient light in electronic devices is difficult to apply to head-mounted display devices.

[0028] Based on this, please refer to Figure 1 This embodiment provides a brightness adjustment method, which is applied to a head-mounted display device, wherein the head-mounted display device includes a lens barrel for displaying an image, and the method includes: Step S10, obtaining a first image brightness value corresponding to the current image frame to be played; In this embodiment, the first image brightness value refers to the image brightness value of the current image frame to be played. It is understandable that, based on the visual characteristics of the human eye, image frames of different colors often correspond to different image brightness values. For example, compared with the image brightness value of the image frame to be played that is biased towards black, the image brightness value of the image frame to be played that is biased towards white is higher.

[0029] It is known that a head mounted display device (such as a VR device) generally includes two lens barrels for displaying images, and the two lens barrels correspond to the two eyes of the user respectively. It is known to those skilled in the art that the lens barrels are similar to a cylindrical display screen.

[0030] Step S20, determining the barrel current value mapped to the brightness value of the first image based on a preset brightness mapping relationship, wherein the brightness mapping relationship is calibrated by the matching relationship between the barrel current value and the image brightness value corresponding to the comfortable photosensitivity range of the human eye; In this embodiment, the brightness mapping relationship stores a one-to-one mapping relationship between the brightness value of the first image and the barrel current value.

[0031] It should be noted that the lens barrel current value is used to control the lens barrel brightness. The larger the lens barrel current value, the greater the lens barrel brightness, that is, the lens barrel current value corresponds to the lens barrel brightness. The lens barrel of the head-mounted display device is similar to a cylindrical display screen. The lens barrel brightness is similar to the backlight brightness of a cylindrical display screen.

[0032] It is easy to understand that after the user wears the VR device, the light intensity received by the user's eyes is determined by the combined factors of the lens barrel brightness and the image brightness value. That is, when the lens barrel brightness is equal, the greater the image brightness value, the greater the light intensity received by the user's eyes. Correspondingly, when the image brightness values ​​are equal, the greater the lens barrel brightness, the greater the light intensity received by the user's eyes.

[0033] In this embodiment, the comfortable light sensitivity range of the human eye refers to the range of light intensity that the human eye feels comfortable. That is, the comfortable light sensitivity range of the human eye is a relatively moderate light intensity range, and there will be no discomfort to the eyes due to strong light or weak light. Among them, the comfortable light sensitivity range of the human eye can be calibrated by those skilled in the art by testing the human eye in a large number of different lighting environments. This embodiment does not make a specific limitation on this, so as to better determine whether the light intensity in the current lighting environment belongs to the comfortable perception range of the human eye.

[0034] It is easy to understand that there is a one-to-one matching relationship (or called a combination relationship) between each lens barrel current value and each image brightness value, some of which are within the range of light intensity that is comfortable for the human eye to perceive, and those matching relationships within the range of light intensity that is comfortable for the human eye to perceive are the matching relationships between the lens barrel current value and the image brightness value corresponding to the comfortable photosensitivity range of the human eye.

[0035] After step S20, step S30 is performed to use the mapped barrel current value as the target barrel current value, and to apply the target barrel current value when playing the image frame to be played, so as to control the current brightness of the barrel to be the brightness corresponding to the target barrel current value.

[0036] In this embodiment, by obtaining the first image brightness value corresponding to the current image frame to be played, based on a preset brightness mapping relationship, the lens barrel current value mapped by the first image brightness value is determined, wherein the brightness mapping relationship is calibrated by the matching relationship between the lens barrel current value and the image brightness value corresponding to the comfortable photosensitivity range of the human eye, and then the mapped lens barrel current value is used as the target lens barrel current value, and when playing the image frame to be played, the target lens barrel current value is applied to control the current brightness of the lens barrel to be the brightness corresponding to the target lens barrel current value, so that when the lens barrel of the VR device plays image frames with different image brightness values, the brightness L (i.e., the image brightness value) of the image frame is too bright. If the brightness L of the image frame is too dark, the brightness of the lens barrel is automatically adjusted down by reducing the lens barrel current value. If the brightness L of the image frame is too dark, the brightness of the lens barrel is automatically adjusted up by increasing the lens barrel current value, so that the light perception Y sensed by the user's eyes is always within a moderate light intensity range, and there will be no eye discomfort caused by strong light or weak light. The light intensity of the light environment where the user's eyes are located is always within the light intensity range that is comfortable for the human eye, thereby avoiding damage to the user's eyesight or discomfort to the user's eyes, thereby solving the technical problem that when the user uses a head-mounted display device, the screen brightness environment is too bright or too dark, causing excessive eye damage.

[0037] This embodiment uses a preset intelligent brightness adjustment algorithm to allow the lens barrel to automatically adjust to the most appropriate light intensity when the user uses the VR device, effectively preventing the user's eye fatigue and damage to the user's eyes, meeting the user's expectations for the adjusted brightness, and greatly improving the convenience of adjusting the screen brightness and the user experience.

[0038] In an practicable manner, before the step of obtaining the first image brightness value corresponding to the current image frame to be played, the method further includes: Step A10, obtaining the RGB (red, green, blue, three primary colors) value of each pixel in the current image frame to be played; Step A20: Calculate the first image brightness value corresponding to the current image frame to be played according to the RGB value of each pixel.

[0039] In this embodiment, it is known to those skilled in the art that, based on the visual characteristics of the human eye, different RGB values ​​of pixels in an image frame often correspond to different image brightness values. For example, compared with the image brightness value of a to-be-played image frame that is biased toward black, the image brightness value of a to-be-played image frame that is biased toward white is higher. Among them, the RGB value (0, 0, 0) is black, and the RGB value (255, 255, 255) is white.

[0040] Furthermore, the step of calculating the first image brightness value corresponding to the current image frame to be played according to the RGB value of each pixel includes: Step B10, accumulating the RGB values ​​of the pixels to obtain a total RGB value corresponding to the current image frame to be played; Step B20, determining the image brightness value of the RGB total value mapping based on a preset RGB value mapping relationship; In this embodiment, the preset RGB value mapping relationship stores a one-to-one mapping relationship between RGB total values ​​and image brightness values.

[0041] Step B30: Using the mapped image brightness value as the first image brightness value corresponding to the current image frame to be played.

[0042] After repeated experimental studies, the applicant found that under the condition of the same barrel brightness, the human eye's perception of the brightness of the image frame is closely related to the RGB value of each pixel of the observed image frame. Specifically, the system can accumulate the RGB value of each pixel of the image frame to be played, and the total RGB value corresponding to the image frame to be played can be accumulated. The total RGB value is related to the human eye's perception of the brightness value of the image frame. The larger the RGB total value, the greater the human eye's perception of the brightness value of the image frame, that is, the greater the image brightness value of the image frame. The smaller the RGB total value, the smaller the human eye's perception of the brightness value of the image frame, that is, the smaller the image brightness value of the image frame.

[0043] This embodiment obtains the RGB value of each pixel in the current image frame to be played, and accurately calculates the image brightness value corresponding to the image frame to be played based on the RGB value of each pixel, thereby accurately obtaining the human eye's perception of the brightness value of the image frame to be played.

[0044] Embodiment 2 Based on the first embodiment of the present application, in another embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 2 , the method further comprises: Step S40, playing a test video of a preset duration, wherein the test video includes a plurality of test image frames with different image brightness values; In this embodiment, the test video may be pre-stored in the system of the head mounted display device.

[0045] Step S50, obtaining a second image brightness value corresponding to the currently played test image frame and a current actual barrel current value; The second image brightness value refers to the image brightness value corresponding to the currently played test image frame.

[0046] Step S60, collecting eye image information corresponding to the test image frame currently played when the user observes, and calibrating the mapping relationship between the barrel current value and the image brightness value according to the second image brightness value, the actual barrel current value and the eye image information, to obtain the brightness mapping relationship.

[0047] In this embodiment, the eye image information corresponding to the test image frame currently being played when the user is observing can be collected through the camera of the head mounted display device.

[0048] Further, in step S60, the step of calibrating the mapping relationship between the lens barrel current value and the image brightness value according to the second image brightness value, the actual lens barrel current value and the eye image information includes: Step C10, determining the pupil area and eye opening width of the user according to the eye image information; In this embodiment, the pupil is a small circular hole in the center of the iris in the human eye, which is a passage for light to enter the eye. The contraction of the pupillary sphincter on the iris can shrink the pupil, and the contraction of the pupil dilator muscle can dilate the pupil. The dilation and contraction of the pupil control the amount of light entering the pupil. The size of the pupil can control the amount of light entering the eye. That is, when the human eye perceives strong light, the pupil area will become smaller in order to reduce the incidence of light. When the human eye perceives weak light, the pupil area will become larger in order to increase the incidence of light.

[0049] Pupil features can be extracted from the collected eye image information. Specifically, a feature extraction technology consistent with the prior art can be adopted. For example, pupil contour extraction, pupil grayscale value processing, eyelash removal and other edge processing can be performed on the eye image information to identify the user's pupil diameter, and then the user's pupil area can be calculated based on the pupil diameter.

[0050] In this embodiment, the eye opening amplitude is used to characterize the eye opening amplitude, or the degree of eyelid opening. It is easy to understand that when the human eye perceives strong light, in order to reduce the light entering, the opening amplitude of the glasses will decrease to avoid the strong light. When the human eye perceives weak light, in order to increase the light entering, the opening amplitude of the glasses will increase to increase the perceived clarity of the image information.

[0051] After step C10, step C20 is performed to collect current light intensity information on the human eye side, wherein the light intensity information is affected by the second image brightness value and the actual lens barrel current value; Among them, the current light intensity information on the human eye side can be collected through the photosensitive sensor of the head-mounted display device.

[0052] In this embodiment, the current light intensity information on the human eye side refers to the light intensity information that can be sensed by the human eye (normal human eye), or the light intensity information of the lighting environment in which the human eye is located. It is easy to understand that when the user observes the currently played test image frame, the current light intensity information on the human eye side is affected by two factors: the brightness value of the second image and the actual lens barrel current value. That is, when the actual lens barrel brightness is equal, the greater the second image brightness value, the greater the value corresponding to the light intensity information. Correspondingly, when the second image brightness value is equal, the greater the actual lens barrel brightness, the greater the value corresponding to the light intensity information.

[0053] Step C30, determining whether the light intensity information conforms to a comfortable light sensitivity range of human eyes according to the pupil area and the eye opening amplitude; Exemplarily, the step of determining whether the light intensity information conforms to a comfortable light sensitivity range of human eyes according to the pupil area and the eye opening amplitude includes: Step D10, if the pupil area is within a preset standard pupil area range, and the eye opening amplitude is within a preset standard opening amplitude range, then determining that the light intensity information meets the comfortable light sensitivity range of the human eye; Step D20: If the pupil area does not fall within the preset standard pupil area range, and / or the eye opening amplitude does not fall within the preset standard opening amplitude range, it is determined that the light intensity information does not conform to the comfortable photosensitivity range of the human eye.

[0054] In this embodiment, the standard pupil area range can be set by those skilled in the art according to actual conditions, and this embodiment does not make any specific restrictions, so as to better determine whether the user's pupil shrinks when sensing strong light, or expands when sensing weak light. Correspondingly, the standard opening amplitude range can be set by those skilled in the art according to actual conditions, and this embodiment does not make any specific restrictions, so as to better determine whether the user's eyes reduce the degree of eyelid opening when sensing strong light, or expand the degree of eyelid opening when sensing weak light.

[0055] It should be noted that when the user just puts on the VR device (a type of head-mounted display device) but has not yet turned on the VR screen, the VR device uses the pupil size of the pupil image obtained in this scene as the wearer's reference pupil size. The reference pupil size is used by the device to determine whether the user's eyes are in a relatively comfortable lighting environment. At this time, the lighting device in the VR device emits light of preset brightness, collects the pupil image of the wearer at the preset brightness, and the preset brightness is debugged to simulate the light brightness of the wearer's pupil in a comfortable scene, determines the corresponding pupil size in the pupil image, and uses it as a reference pupil size. In the subsequent brightness adjustment, the standard pupil area range is constructed based on this reference pupil size, or the VR device generates a standard pupil area range for the user based on the reference pupil size, and registers based on the user's face or iris and other features with unique texture information. When the user subsequently uses the VR device, the user's account is automatically logged in after the face or iris of the user is recognized, and the standard pupil area range for the user is obtained from the storage information of the account, so as to more accurately identify whether the user's eyes reduce the degree of eyelid opening when sensing strong light, or expand the degree of eyelid opening when sensing weak light, and then adjust the corresponding brightness to keep the user in a comfortable light intensity environment at all times to avoid user eye fatigue.

[0056] Correspondingly, when the user just puts on the VR device but has not yet turned on the VR screen, the VR device can use the eye opening amplitude (or eyelid opening amplitude) of the eye image obtained in this scene as the wearer's reference opening amplitude. The reference opening amplitude is used by the device to determine whether the user's eyes are in a relatively comfortable lighting environment. At this time, the lighting device in the VR device emits light of preset brightness and collects the pupil image of the wearer at the preset brightness. The preset brightness is debugged to simulate the light brightness of the wearer in a comfortable scene, and the corresponding eyelid opening amplitude in the eye image is determined as a reference opening amplitude. In the subsequent brightness adjustment, the standard opening amplitude range is constructed based on this reference opening amplitude, or the VR device generates a standard opening amplitude range for the user based on the reference opening amplitude, and registers based on the user's face or iris and other features with unique texture information. When the user subsequently uses the VR device, the user's account is automatically logged in after the face or iris of the user is recognized, and the standard opening amplitude range for the user is obtained from the storage information of the account, so as to more accurately identify whether the user's eyes reduce the eyelid opening degree when sensing strong light, or expand the eyelid opening degree when sensing weak light, and then adjust the corresponding brightness to keep the user in a comfortable light intensity environment at all times to avoid user eye fatigue.

[0057] After step C30, step C40 is performed to calibrate the mapping relationship between the barrel current value and the image brightness value according to the second image brightness value corresponding to the comfortable photosensitivity interval and the actual barrel current value.

[0058] Exemplarily, the step of calibrating the mapping relationship between the barrel brightness value and the image brightness value according to the second image brightness value corresponding to the comfortable photosensitivity interval and the actual barrel current value includes: Step E10 , associating the second image brightness value corresponding to the comfortable photosensitivity interval with the actual barrel current value, and obtaining a mapping relationship between the barrel current value and the image brightness value.

[0059] It is easy to understand that the one-to-one matching combination relationship (or called matching relationship) between each actual barrel current value and each second image brightness value, some of which are within the range of light intensity that is comfortable for the human eye to perceive, and these combination relationships within the range of light intensity that is comfortable for the human eye to perceive, that is, the correlation relationship between the second image brightness value corresponding to the comfortable photosensitivity interval and the actual barrel current value, by associating the second image brightness value corresponding to the comfortable photosensitivity interval with the actual barrel current value, the mapping relationship between the barrel current value and the image brightness value is obtained by association, thereby completing the accurate calibration of the mapping relationship between the barrel brightness value and the image brightness value.

[0060] This embodiment plays a test video of a preset duration, wherein the test video includes multiple test image frames with different image brightness values, obtains a second image brightness value corresponding to the currently played test image frame, and a current actual lens barrel current value, then collects eye image information corresponding to the currently played test image frame when the user is observing, and then calibrates the mapping relationship between the lens barrel current value and the image brightness value according to the second image brightness value, the actual lens barrel current value and the eye image information, so as to accurately calibrate the brightness mapping relationship, so as to facilitate the subsequent determination of the lens barrel current value mapped by the first image brightness value based on the brightness mapping relationship, and use the mapped lens barrel current value as the target lens barrel current value to control the current brightness of the lens barrel to be the brightness corresponding to the target lens barrel current value, so as to achieve that even if the lens barrel of the VR device plays image frames with different image brightness values, the light intensity felt by the user's eyes still remains within the light intensity range that is comfortable for the human eye to perceive, thereby avoiding discomfort to the human eye due to strong light or weak light.

[0061] In order to help understand the technical concept of this application, a specific embodiment is listed, please refer to Figure 3 , Figure 3 This is a structural diagram of VR glasses (a type of head-mounted display device) in a specific embodiment of the present application, including the following key components: The photosensor is located around the lens barrel and is used to collect the current light intensity when the user's glasses are in a closed environment (the user wears VR glasses).

[0062] The camera is used to collect the user's eye opening and closing status, opening and closing amplitude, pupil zoom status, etc.

[0063] The lens barrel is the display screen of VR glasses, and the brightness of the lens barrel can be controlled by the value of the lens barrel current.

[0064] The VR glasses can be combined with a preset brightness adjustment program to ensure that the light intensity felt by the user when using the VR device is always within the light intensity range that is comfortable for the human eye. The specific steps include: 1. Collect the intensity Q of light suitable for the user After the user puts on the VR glasses, the user will be prompted to perform a light perception test Y, and a video will be played. The user only needs to look at the display screen, and the camera will collect the changes in the user's eye opening and closing amplitude and pupil zoom changes (when the human eye perceives strong light, it will unconsciously close its eyes or reduce the opening and closing amplitude of the glasses to avoid strong light, and at the same time, in order to reduce the input of light, the pupil will become smaller; when the human eye perceives weak light, it will also unconsciously open the opening and closing amplitude of the eyes to see the objects in front clearly, and at the same time, in order to increase the input of light, the pupil will become larger). The system tests the user's most comfortable state through light tests of different intensities, that is, the state where the glasses open and close with the largest amplitude and the pupil size is moderate. And use the photosensitive sensor to record the intensity Q of the light currently suitable for the user.

[0065] 2. Use the light sensor to self-test the brightness of different scenes While playing the video in the previous step, the system also collects the brightness L of different image frames of the video, as well as the light intensity fed back by the photosensitive sensor when playing this image frame and the current D of the lens barrel at this time (the brightness of the lens barrel is controlled by the current passing through). Through the intensity Q of the light suitable for the user, it can be inferred that when playing the image frame brightness L, how much current D should be given to the lens barrel (i.e., controlling the brightness of the lens barrel) to adapt to the user's light perception Y, that is, if the brightness L of the image frame is too bright, the brightness of the lens barrel will be automatically lowered, and if the brightness L of the image frame is too dark, the brightness of the lens barrel will be automatically increased.

[0066] 3. Calculation of light perception Y (1) The light intensity Q suitable for the user obtained by the photosensor (2) The brightness L of the current playing image frame obtained by the system (3) Lens tube current D Reverse deduction process: According to the acquired light intensity Q suitable for the user, when playing the video, every time the light sensor acquires the current light intensity as Q, it will record the current frame brightness and the current lens barrel current value. When the acquired data is large enough, it can calculate the lens barrel current value D corresponding to the image frame brightness L. For example, when the frame brightness is L+1 (the frame brightness increases), the lens barrel current value is changed to D-1 (the current value decreases and the lens barrel becomes darker). At this time, the current light intensity is still the intensity Q suitable for the user.

[0067] 4. Usage scenarios: When the user uses it for the first time, the system will prompt the user to configure the user's light perception Y to protect the user's eyes, and let the user watch a video according to the user's usual eye habits. After the user finishes watching the video, the system extracts and configures the user's light perception Y. When the user plays games or watches movies, the system will adjust the current value of the lens barrel to control the brightness of the lens barrel, so that the user's visual experience is always at light perception Y, and the user will not feel uncomfortable due to sudden changes in light or dimming of the eyes.

[0068] This specific embodiment adds a photosensitive sensor and a camera, and adjusts the lens barrel current. Through an algorithm, the lens barrel can automatically adjust to the most suitable light intensity when the user uses VR glasses, effectively preventing the user's eye fatigue and damage to the user's eyes.

[0069] It should be noted that the many details described in this specific embodiment are only helpful for understanding the technical concept of the present application and do not constitute a limitation of the present application. More simple transformations based on the technical concept of the present application should all be within the scope of protection of the present application.

[0070] Embodiment 3 The embodiment of the present invention also provides a brightness adjustment device, please refer to Figure 4 The brightness adjustment device is applied to a head mounted display device, and the brightness adjustment device comprises: An acquisition module 10 is used to acquire a first image brightness value corresponding to a current image frame to be played; A determination module 20 is used to determine the barrel current value mapped to the brightness value of the first image based on a preset brightness mapping relationship, wherein the brightness mapping relationship is calibrated by the matching relationship between the barrel current value and the image brightness value corresponding to the comfortable photosensitivity range of the human eye; The control module 30 is used to use the mapped lens barrel current value as a target lens barrel current value, and apply the target lens barrel current value when playing the image frame to be played, so as to control the current brightness of the lens barrel to be the brightness corresponding to the target lens barrel current value.

[0071] Optionally, the acquisition module 10 is further used for: Get the RGB value of each pixel in the current image frame to be played; According to the RGB values ​​of the pixels, a first image brightness value corresponding to the current image frame to be played is calculated.

[0072] Optionally, the acquisition module 10 is further used for: Accumulate the RGB values ​​of the pixels to obtain the total RGB value corresponding to the current image frame to be played; Based on a preset RGB value mapping relationship, determine the image brightness value of the RGB total value mapping; The mapped image brightness value is used as the first image brightness value corresponding to the current image frame to be played.

[0073] Optionally, the brightness adjustment device further includes a calibration module (not shown) for: Play a test video of a preset duration, wherein the test video includes a plurality of test image frames with different image brightness values; Obtaining the second image brightness value corresponding to the currently played test image frame and the current actual barrel current value; The eye image information corresponding to the test image frame currently played by the user is collected, and the mapping relationship between the barrel current value and the image brightness value is calibrated according to the second image brightness value, the actual barrel current value and the eye image information to obtain the brightness mapping relationship.

[0074] Optionally, the calibration module is further used for: Determining the pupil area and eye opening width of the user according to the eye image information; Collecting current light intensity information on the human eye side, wherein the light intensity information is affected by the brightness value of the second image and the actual lens barrel current value; Determining whether the light intensity information conforms to a comfortable light sensitivity range of human eyes according to the pupil area and the eye opening amplitude; The mapping relationship between the barrel current value and the image brightness value is calibrated according to the second image brightness value corresponding to the comfortable photosensitivity interval and the actual barrel current value.

[0075] Optionally, the calibration module is further used for: The second image brightness value corresponding to the comfortable photosensitivity interval is associated with the actual barrel current value to obtain a mapping relationship between the barrel current value and the image brightness value.

[0076] Optionally, the calibration module is further used for: If the pupil area is within the preset standard pupil area range, and the eye opening range is within the preset standard opening range, it is determined that the light intensity information meets the comfortable light sensitivity range of the human eye; If the pupil area does not fall within the preset standard pupil area range, and / or the eye opening amplitude does not fall within the preset standard opening amplitude range, it is determined that the light intensity information does not conform to the comfortable photosensitivity range of the human eye.

[0077] The brightness adjustment device provided in the embodiment of the present invention adopts the brightness adjustment method in the above embodiment 1 or embodiment 2, which can solve the technical problem that when a user uses a head-mounted display device, the screen brightness environment is too bright or too dark, causing excessive eye damage. Compared with the prior art, the beneficial effects of the brightness adjustment device provided in the embodiment of the present invention are the same as the beneficial effects of the brightness adjustment method provided in the above embodiment, and the other technical features of the brightness adjustment device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0078] Embodiment 4 An embodiment of the present invention provides a head-mounted display device, the head-mounted display device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the brightness adjustment method in the above-mentioned embodiment 1.

[0079] Reference below Figure 5 , which shows a schematic diagram of the structure of a head-mounted display device suitable for implementing the embodiment of the present disclosure. The head-mounted display device in the embodiment of the present disclosure includes but is not limited to a mixed reality (Mixed Reality)-MR device (such as MR glasses or MR helmet), an augmented reality (Augmented Reality)-AR device (such as AR glasses or AR helmet), a virtual reality (Virtual Reality)-VR device (such as VR glasses or VR helmet), an extended reality (Extended Reality)-XR device or some combination thereof, etc. head-mounted display devices. Figure 5 The head mounted display device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0080] like Figure 5 As shown, the head-mounted display device may include a processing device 1001 (such as a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM1002) or a program loaded from a storage device into a random access memory (RAM1004). In RAM1004, various programs and data required for the operation of the head-mounted display device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other via a bus 1005. An input / output (I / O) interface is also connected to the bus 1005.

[0081] Typically, the following systems may be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 1003 including, for example, a tape, a hard disk, etc.; and communication devices 1009. The communication device 1009 may allow the head-mounted display device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a head-mounted display device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be implemented or have instead.

[0082] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0083] The head-mounted display device provided by the present invention adopts the brightness adjustment method in the above embodiment, which can solve the technical problem that when a user uses the head-mounted display device, the screen brightness environment is too bright or too dark, causing excessive eye damage. Compared with the prior art, the beneficial effects of the head-mounted display device provided by the embodiment of the present invention are the same as the beneficial effects of the brightness adjustment method provided by the above embodiment, and the other technical features in the head-mounted display device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0084] It should be understood that the various parts of the present disclosure can be implemented with hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0085] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

[0086] Embodiment 5 An embodiment of the present invention provides a computer-readable storage medium having computer-readable program instructions stored thereon, and the computer-readable program instructions are used to execute the brightness adjustment method in the above embodiment.

[0087] The computer-readable storage medium provided in the embodiment of the present invention may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0088] The computer-readable storage medium may be included in the head-mounted display device; or may exist independently without being assembled into the head-mounted display device.

[0089] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by a head-mounted display device, the head-mounted display device: obtains a first image brightness value corresponding to a current image frame to be played; determines a lens barrel current value mapped to the first image brightness value based on a preset brightness mapping relationship, wherein the brightness mapping relationship is calibrated by a matching relationship between a lens barrel current value and an image brightness value corresponding to a comfortable photosensitivity range of a human eye; uses the mapped lens barrel current value as a target lens barrel current value, and when playing the image frame to be played, applies the target lens barrel current value to control the current brightness of the lens barrel to be the brightness corresponding to the target lens barrel current value.

[0090] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0091] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0092] The modules involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of the module does not limit the unit itself in some cases.

[0093] The computer-readable storage medium provided by the present invention stores computer-readable program instructions for executing the above-mentioned brightness adjustment method, which can solve the technical problem that when a user uses a head-mounted display device, the screen brightness environment is too bright or too dark, causing excessive eye damage. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in the embodiment of the present invention are the same as the beneficial effects of the brightness adjustment method provided in the above-mentioned embodiment 1 or embodiment 2, and will not be repeated here.

[0094] Embodiment 6 An embodiment of the present invention further provides a computer program product, including a computer program, and when the computer program is executed by a processor, the steps of the brightness adjustment method as described above are implemented.

[0095] The computer program product provided by the present application can solve the technical problem that when a user uses a head-mounted display device, the screen brightness is too bright or too dark, causing excessive eye damage. Compared with the prior art, the beneficial effects of the computer program product provided by the embodiment of the present invention are the same as the beneficial effects of the brightness adjustment method provided by the above-mentioned embodiment 1 or embodiment 2, and will not be repeated here.

[0096] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent processing scope of the present application.

Claims

1. A brightness adjustment method, characterized in that: The brightness adjustment method is applied to a head mounted display device, wherein the head mounted display device includes a lens barrel for displaying an image, and the method includes: Obtaining a first image brightness value corresponding to the current image frame to be played; Based on a preset brightness mapping relationship, determining a barrel current value mapped to the brightness value of the first image, wherein the brightness mapping relationship is calibrated by a matching relationship between the barrel current value and the image brightness value corresponding to a comfortable photosensitivity range of human eyes; The mapped lens barrel current value is used as a target lens barrel current value, and when playing the image frame to be played, the target lens barrel current value is applied to control the current brightness of the lens barrel to be the brightness corresponding to the target lens barrel current value.

2. The brightness adjustment method according to claim 1, characterized in that: Before the step of obtaining the first image brightness value corresponding to the current image frame to be played, the method further includes: Get the RGB value of each pixel in the current image frame to be played; According to the RGB values ​​of the pixels, a first image brightness value corresponding to the current image frame to be played is calculated.

3. The brightness adjustment method according to claim 2, characterized in that: The step of calculating the first image brightness value corresponding to the current image frame to be played according to the RGB value of each pixel comprises: Accumulate the RGB values ​​of the pixels to obtain the total RGB value corresponding to the current image frame to be played; Based on a preset RGB value mapping relationship, determine the image brightness value of the RGB total value mapping; The mapped image brightness value is used as the first image brightness value corresponding to the current image frame to be played.

4. The brightness adjustment method according to claim 1, characterized in that: The method further comprises: Play a test video of a preset duration, wherein the test video includes a plurality of test image frames with different image brightness values; Obtaining the second image brightness value corresponding to the currently played test image frame and the current actual barrel current value; The eye image information corresponding to the test image frame currently played by the user is collected, and the mapping relationship between the barrel current value and the image brightness value is calibrated according to the second image brightness value, the actual barrel current value and the eye image information to obtain the brightness mapping relationship.

5. The brightness adjustment method according to claim 4, characterized in that: The step of calibrating the mapping relationship between the lens barrel current value and the image brightness value according to the second image brightness value, the actual lens barrel current value and the eye image information comprises: Determining the pupil area and eye opening width of the user according to the eye image information; Collecting current light intensity information on the human eye side, wherein the light intensity information is affected by the brightness value of the second image and the actual lens barrel current value; Determining whether the light intensity information conforms to a comfortable light sensitivity range of human eyes according to the pupil area and the eye opening amplitude; The mapping relationship between the barrel current value and the image brightness value is calibrated according to the second image brightness value corresponding to the comfortable photosensitivity interval and the actual barrel current value.

6. The brightness adjustment method according to claim 5, characterized in that: The step of calibrating the mapping relationship between the barrel brightness value and the image brightness value according to the second image brightness value corresponding to the comfortable photosensitivity interval and the actual barrel current value comprises: The second image brightness value corresponding to the comfortable photosensitivity interval is associated with the actual barrel current value to obtain a mapping relationship between the barrel current value and the image brightness value.

7. The brightness adjustment method according to claim 5, characterized in that: The step of determining whether the light intensity information conforms to the comfortable light sensitivity range of the human eye according to the pupil area and the eye opening amplitude comprises: If the pupil area is within the preset standard pupil area range, and the eye opening range is within the preset standard opening range, it is determined that the light intensity information meets the comfortable light sensitivity range of the human eye; If the pupil area does not fall within the preset standard pupil area range, and / or the eye opening amplitude does not fall within the preset standard opening amplitude range, it is determined that the light intensity information does not conform to the comfortable photosensitivity range of the human eye.

8. A brightness adjustment device, characterized in that: The brightness adjustment device is applied to a head mounted display device, wherein the head mounted display device comprises a lens barrel for displaying an image, and the brightness adjustment device comprises: An acquisition module, used to acquire a first image brightness value corresponding to a current image frame to be played; A determination module, configured to determine the barrel current value mapped to the brightness value of the first image based on a preset brightness mapping relationship, wherein the brightness mapping relationship is calibrated by a matching relationship between the barrel current value and the image brightness value corresponding to a comfortable photosensitivity range of human eyes; The control module is used to use the mapped lens barrel current value as a target lens barrel current value, and apply the target lens barrel current value when playing the image frame to be played, so as to control the current brightness of the lens barrel to be the brightness corresponding to the target lens barrel current value.

9. A head mounted display device, characterized in that: The head mounted display device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the steps of the brightness adjustment method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program for implementing the brightness adjustment method, and the program for implementing the brightness adjustment method is executed by a processor to implement the steps of the brightness adjustment method according to any one of claims 1 to 7.

Images