Exposure adjustment method and apparatus for extended reality glasses

Abstract

The application discloses an exposure adjustment method and device of an extended reality glasses, and relates to the technical field of image processing. The method comprises the following steps: collecting an environment image, and detecting whether a hand region exists; when the hand region exists, determining a corresponding region of interest and obtaining a first pixel reference value based on a first gray value of pixels; when the hand region does not exist, determining a corresponding gray histogram and a weight coefficient corresponding to a second gray value, obtaining a weighted probability density corresponding to the second gray value based on the gray histogram and the weight coefficient, and obtaining a second pixel reference value; and dynamically adjusting an exposure parameter according to a current pixel reference value. The application distinguishes between scenes in which a hand exists and scenes in which a hand does not exist, focuses on the local brightness of the hand for the scene in which the hand exists, and obtains the second pixel reference value based on the weighted probability density for the scene in which the hand does not exist, so that the second pixel reference value can accurately reflect the global brightness, the accuracy of exposure adjustment is improved, and the brightness stability of the image is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to an exposure adjustment method and device for extended reality glasses. BACKGROUND

[0002] Extended reality (XR) glasses are a kind of intelligent device that integrates augmented reality (AR), virtual reality (VR) and mixed reality (MR) technologies, which realizes real-time interaction between virtual content and real environment through cameras, sensors and display modules. Generally, the core functions of XR glasses include gesture recognition, environment perception and virtual content rendering. Taking gesture recognition as an example, XR glasses need to capture images including user's hands through its camera in real time, and extract key points through algorithms to realize accurate interaction. However, the brightness stability of the images captured by XR glasses directly affects the accuracy of gesture recognition and user visual experience.

[0003] In the current technology, the exposure adjustment of extended reality glasses depends on the average value of the gray scale of the global image. For example, the brightness of the image is evaluated by calculating the average value of the gray scale of all pixels in the image, and the exposure adjustment is performed according to the average value of the gray scale.

[0004] However, in the current technology, the average value of the gray scale of the global image cannot accurately reflect the brightness of the image, resulting in inaccurate exposure adjustment, and ignoring the local brightness difference of the user's hands, resulting in loss of details of the user's hands and poor accuracy of gesture recognition of XR glasses. SUMMARY

[0005] Based on the above problems, the present application provides an exposure adjustment method and device for extended reality glasses, which distinguishes between scenes with and without user's hands. For scenes with user's hands, the exposure adjustment is directly focused on the local brightness of the user's hands, which prioritizes the clear visibility of the details of the user's hands and significantly improves the accuracy of gesture recognition of XR glasses. For scenes without user's hands, the pixel reference obtained by weighted probability density can accurately reflect the global brightness of the image, thereby improving the accuracy of exposure adjustment and the brightness stability of the images captured by XR glasses.

[0006] The embodiments of the present application disclose the following technical solutions:

[0007] In a first aspect, the embodiments of the present application provide an exposure adjustment method for extended reality glasses, comprising:

[0008] capturing an environment image and detecting whether a hand region exists in the environment image;

[0009] In response to the hand region existing in the environment image, a region of interest corresponding to the hand region is determined, and a first pixel reference value is obtained based on a first gray value of a pixel in the region of interest;

[0010] In response to the hand region not existing in the environment image, a gray histogram corresponding to the environment image and a weight coefficient corresponding to each second gray value in the environment image are determined, a weighted probability density corresponding to each second gray value is obtained based on the gray histogram and the weight coefficient corresponding to each second gray value, and a second pixel reference value is obtained based on the weighted probability density corresponding to each second gray value; wherein the weight coefficient corresponding to the second gray value is negatively correlated with a difference between the second gray value and a preset reference gray value.

[0011] The exposure parameter of the camera of the extended reality glasses is dynamically adjusted according to the current pixel reference value; wherein the current pixel reference value is the first pixel reference value or the second pixel reference value; and the exposure parameter includes at least one of an exposure time and a sensitivity.

[0012] In a possible implementation, the determination of the region of interest corresponding to the hand region in response to the hand region existing in the environment image includes:

[0013] In response to the hand region existing in the environment image, the hand region is intercepted to obtain the region of interest corresponding to the hand region; wherein the region of interest coincides with a center point of the hand region, and an area of the region of interest is smaller than an area of the hand region.

[0014] In a possible implementation, the determination of the region of interest corresponding to the hand region in response to the hand region existing in the environment image, and the obtaining of the first pixel reference value based on the first gray value of the pixel in the region of interest include:

[0015] In response to one hand region existing in the environment image, a region of interest corresponding to the one hand region is determined, and an average value of first gray values of all pixels in the region of interest of the environment image is calculated to obtain a first pixel reference value.

[0016] In response to multiple hand regions existing in the environment image, regions of interest corresponding to the multiple hand regions are determined, and an average value of first gray values of all pixels in each region of interest of the environment image is calculated to obtain a gray average value corresponding to each region of interest; and an average value of the gray average values corresponding to all the regions of interest is calculated to obtain a first pixel reference value.

[0017] In a possible implementation, the obtaining of the weighted probability density corresponding to each second gray value based on the gray histogram and the weight coefficient corresponding to each second gray value comprises:

[0018] The initial probability density corresponding to each second gray value is obtained based on the gray histogram.

[0019] The initial probability density corresponding to each second gray value is weighted based on the weight coefficient corresponding to each second gray value, to obtain the weighted probability density corresponding to each second gray value.

[0020] In a possible implementation, the obtaining of the second pixel reference value based on the weighted probability density corresponding to each second gray value comprises:

[0021] The sub-pixel reference value corresponding to each second gray value is obtained by multiplying the second gray value and the weighted probability density corresponding to the second gray value, and the second pixel reference value is obtained by summing the sub-pixel reference values corresponding to each second gray value.

[0022] In a possible implementation, the weight coefficient corresponding to the second gray value is obtained by the following manner:

[0023] The weight parameter corresponding to the second gray value is obtained based on the second gray value, the preset reference gray value and the preset standard deviation.

[0024] The weight coefficient corresponding to the second gray value is obtained by calculating the exponential function value of the weight parameter corresponding to the second gray value as the independent variable through an exponential function.

[0025] In a possible implementation, the dynamic adjustment of the exposure parameter of the camera of the extended reality glasses according to the current pixel reference value comprises:

[0026] The adjustment rate of the exposure parameter is determined based on the current pixel reference value and the preset brightness parameter.

[0027] It is judged whether the adjustment rate of the exposure parameter is between the preset minimum adjustment rate and the preset maximum adjustment rate.

[0028] When the adjustment rate of the exposure parameter is less than the preset minimum adjustment rate or the adjustment rate of the exposure parameter is greater than the preset maximum adjustment rate, the exposure parameter of the camera of the extended reality glasses is dynamically adjusted based on the adjustment rate of the exposure parameter.

[0029] In a possible implementation, when the adjustment rate of the exposure parameter is less than the preset minimum adjustment rate or the adjustment rate of the exposure parameter is greater than the preset maximum adjustment rate, the exposure parameter of the camera of the extended reality glasses is dynamically adjusted based on the adjustment rate of the exposure parameter, including:

[0030] When the adjustment rate of the exposure parameter is less than the preset minimum adjustment rate, it is determined whether the first exposure time corresponding to the environment image is a preset maximum exposure time.

[0031] When the first exposure time is not the preset maximum exposure time, the first exposure time corresponding to the environment image is adjusted according to the adjustment rate of the exposure parameter to obtain a second exposure time.

[0032] According to the second exposure time, the preset maximum exposure time and the preset minimum exposure time, a third exposure time is determined, and the exposure time of the camera of the extended reality glasses is adjusted to the third exposure time.

[0033] When the first exposure time is the preset maximum exposure time or the third exposure time is the preset maximum exposure time, the first light sensitivity corresponding to the environment image is adjusted according to the adjustment rate of the exposure parameter to obtain a second light sensitivity.

[0034] According to the second light sensitivity, the preset maximum light sensitivity and the preset minimum light sensitivity, a third light sensitivity is determined, and the light sensitivity of the camera of the extended reality glasses is adjusted to the third light sensitivity.

[0035] In a possible implementation, when the adjustment rate of the exposure parameter is less than the preset minimum adjustment rate or the adjustment rate of the exposure parameter is greater than the preset maximum adjustment rate, the exposure parameter of the camera of the extended reality glasses is dynamically adjusted based on the adjustment rate of the exposure parameter, including:

[0036] When the adjustment rate of the exposure parameter is greater than the preset maximum adjustment rate, it is determined whether the first light sensitivity corresponding to the environment image is a preset minimum light sensitivity.

[0037] When the first light sensitivity is not the preset minimum light sensitivity, the first light sensitivity corresponding to the environment image is adjusted according to the adjustment rate of the exposure parameter to obtain a second light sensitivity.

[0038] According to the second light sensitivity, the preset maximum light sensitivity and the preset minimum light sensitivity, a third light sensitivity is determined, and the light sensitivity of the camera of the extended reality glasses is adjusted to the third light sensitivity.

[0039] when the third ISO sensitivity is not the preset minimum ISO sensitivity, adjusting the exposure parameter of the camera of the extended reality glasses is completed;

[0040] when the first ISO sensitivity is the preset minimum ISO sensitivity or the third ISO sensitivity is the preset minimum ISO sensitivity, according to the adjustment rate of the exposure parameter, the first exposure time corresponding to the environment image is adjusted to obtain a second exposure time;

[0041] determining a third exposure time according to the second exposure time, a preset maximum exposure time and a preset minimum exposure time, and adjusting the exposure time of the camera of the extended reality glasses to the third exposure time.

[0042] In a second aspect, the embodiments of the present application provide an exposure adjustment device of extended reality glasses, comprising:

[0043] a hand region detection module, configured to collect an environment image and detect whether a hand region exists in the environment image;

[0044] a first reference determination module, configured to, in response to the hand region existing in the environment image, determine a region of interest corresponding to the hand region, and obtain a first pixel reference value based on a first gray value of a pixel in the region of interest;

[0045] a second reference determination module, configured to, in response to the hand region not existing in the environment image, determine a gray histogram corresponding to the environment image and a weight coefficient corresponding to each second gray value in the environment image, obtain a weighted probability density corresponding to each second gray value based on the gray histogram and the weight coefficient corresponding to each second gray value, and obtain a second pixel reference value based on the weighted probability density corresponding to each second gray value; wherein the weight coefficient corresponding to the second gray value is negatively correlated with the difference between the second gray value and a preset reference gray value;

[0046] an exposure parameter adjustment module, configured to dynamically adjust the exposure parameter of the camera of the extended reality glasses according to the current pixel reference value; wherein the current pixel reference value is the first pixel reference value or the second pixel reference value; and the exposure parameter comprises at least one of an exposure time and an ISO sensitivity.

[0047] Compared with the prior art, the present application has the following beneficial effects: the present application distinguishes between the presence of a user's hand and the absence of a user's hand. For the presence of a user's hand, the local brightness of the user's hand is directly focused on for exposure adjustment, giving priority to ensuring that the details of the user's hand are clear and visible, significantly improving the accuracy of XR glasses gesture recognition. For the absence of a user's hand, the second pixel reference obtained by weighting the probability density can accurately reflect the global brightness of the image, thereby improving the accuracy of exposure adjustment and improving the brightness stability of the image collected by the XR glasses. BRIEF DESCRIPTION OF DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0049] Figure 1 An application scenario diagram of an exposure adjustment method of an extended reality (XR) glasses provided by an embodiment of the present application;

[0050] Figure 2 A flowchart of an exposure adjustment method of an extended reality (XR) glasses provided by an embodiment of the present application;

[0051] Figure 3 A flowchart of an exposure parameter adjustment method provided by an embodiment of the present application;

[0052] Figure 4 A flowchart of another exposure parameter adjustment method provided by an embodiment of the present application;

[0053] Figure 5 A structural diagram of an exposure adjustment device of an extended reality (XR) glasses provided by an embodiment of the present application. DETAILED DESCRIPTION

[0054] As described above, the core functions of extended reality (XR) glasses include gesture recognition, environment modification, and virtual content rendering. Among them, the gesture recognition function is a key means to realize the interaction between the user and the XR glasses. The XR glasses recognize user gestures to realize direct interaction between user gestures and the virtual scene provided by the XR glasses. When the image collected by the XR glasses includes a user's hand, the key points of the user's hand need to be extracted by an algorithm for gesture recognition, which can realize real-time interaction between the user and the XR glasses.

[0055] However, the brightness stability of the image captured by the XR glasses directly affects the accuracy of gesture recognition and the user's visual experience. In one case, when the XR glasses need to perform gesture recognition, the key points of the user's hand need to be accurately recognized. However, when the user's hand region in the image is too bright or too dark, the key points of the user's hand cannot be recognized, and gesture recognition cannot be achieved. In another case, when the XR glasses do not need to perform gesture recognition, i.e., the user's hand is not in the field of view of the XR glasses, the image brightness captured by the XR glasses may frequently change with the change of the environment during the user's movement. This frequent change in brightness may cause eye discomfort and even disrupt the continuity of the display of virtual content. For example, when the user moves from indoors to a strong light environment or enters a shadow area, the brightness of the image captured by the XR glasses changes suddenly. This sudden change may cause the user to feel dizzy and tired, and even the virtual interface provided by the XR glasses may not be visible due to excessive brightness or darkness.

[0056] In the current technology, in order to ensure the brightness stability of the image captured by the XR glasses, the exposure adjustment is performed based on the average gray value of the image captured by the XR glasses. Specifically, the average gray value of all pixels in the global image is calculated to obtain the average gray value of the image. The brightness state of the image is represented by the average gray value, and the exposure adjustment is performed based on the average gray value.

[0057] However, in the current technology, the global average value may not accurately reflect the brightness state of the image due to reflections, shadows or noise interference in the background. For example, an image includes N pixels, of which N / 2 pixels have a gray value of 1 and the other N / 2 pixels have a gray value of 255. The average gray value of the image is 128, indicating that the brightness state of the global image is a median state, i.e., the brightness state of the global image is close to the median brightness. However, in fact, the image has a large number of pixels with extreme brightness (gray value 255) and extreme darkness (gray value 1), resulting in an actual brightness state of the pixels that is more biased towards the extreme rather than the median.

[0058] Moreover, in the current technology, there is no distinction between the presence of the user's hand and the absence of the user's hand, thereby ignoring the local brightness difference of the user's hand. For example, when the user's hand is in backlight or shadow area, the local brightness of the user's hand corresponds to a gray value that is much lower than the global average gray value, resulting in loss of details of the user's hand, poor accuracy of gesture recognition by the XR glasses, and even inability to achieve gesture recognition function.

[0059] Further, for the XR glasses, in the scene where the user's hand exists, the XR glasses need to preferentially ensure the brightness of the user's hand region and ensure that the key points of the user's hand can be accurately recognized to realize gesture recognition and realize the interaction between the user and the XR glasses; in the scene where the user's hand does not exist, the overall brightness stability of the image mainly needs to be maintained. In the current technology, the exposure adjustment mode based on a single gray average value cannot meet the needs of the above two scenes.

[0060] The present application provides an exposure adjustment method of extended reality glasses, comprising: collecting an environment image and detecting whether a hand region exists in the environment image; in response to the existence of the hand region in the environment image, determining a region of interest corresponding to the hand region, and obtaining a first pixel reference value based on a first gray value of a pixel in the region of interest; in response to the absence of the hand region in the environment image, determining a gray histogram corresponding to the environment image and a weight coefficient corresponding to each second gray value in the environment image, obtaining a weighted probability density corresponding to each second gray value based on the gray histogram and the weight coefficient corresponding to each second gray value, and obtaining a second pixel reference value based on the weighted probability density corresponding to each second gray value; and dynamically adjusting an exposure parameter of a camera of the extended reality glasses according to the current pixel reference value. The present application distinguishes between the scene where the user's hand exists and the scene where the user's hand does not exist. For the scene where the user's hand exists, the exposure adjustment is directly focused on the local brightness of the user's hand, which preferentially ensures that the details of the user's hand are clear and visible, and significantly improves the accuracy of gesture recognition of the XR glasses; for the scene where the user's hand does not exist, the second pixel reference obtained by the weighted probability density can accurately reflect the global brightness of the image, thereby improving the accuracy of exposure adjustment and the brightness stability of the image collected by the XR glasses.

[0061] In order for those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0062] Embodiment one:

[0063] The application environment of the exposure adjustment method of extended reality glasses provided by the embodiments of the present application will be described in detail below. Figures 1-4

[0064] First, the application environment of the exposure adjustment method of extended reality glasses provided by the embodiments of the present application will be described by way of example. As shown in Figure 1 Figure 1 ​​As shown, the application environment includes: a user 100, and an extended reality glasses 110 worn by the user 100. Illustratively, the user 100 can collect the current environment image through the extended reality glasses 110 worn by the user 100, and the extended reality glasses 110 can also superimpose virtual content on the collected environment image and display it to the user 100.

[0065] Extended reality (XR) glasses are a kind of intelligent device that integrates augmented reality (AR), virtual reality (VR), and mixed reality (MR) technologies. Illustratively, the XR glasses include: a camera for collecting real-time environment images; sensors such as inertial measurement units, ambient light sensors, etc.; a display module such as a Micro OLED / Micro LED display screen; and a processor such as an application processor, an image signal processor, an image processor, etc.

[0066] It should be noted that the exposure adjustment method of the extended reality glasses provided by the embodiments of the present application applied to the processor of the XR glasses can be a central processing unit (CPU), an image signal processor, etc.

[0067] As shown in Figure 2 The exposure adjustment method of the extended reality glasses provided by the embodiments of the present application includes the following steps:

[0068] S201, collect an environment image and detect whether a hand region exists in the environment image.

[0069] The environment image refers to the image collected by the camera of the extended reality glasses in real time. Specifically, the camera of the extended reality glasses is used to capture / collect real-time images within the user's field of view to provide visual input for gesture recognition, environment perception, and virtual content superimposition.

[0070] In one possible implementation, whether a hand region exists in the environment image is detected by a target detection model based on deep learning. Illustratively, the target detection model based on deep learning can be a YOLO series model.

[0071] Specifically, the environment image is input to the target detection model based on deep learning, so that the target detection model based on deep learning detects whether a hand region exists in the environment image. If a hand region exists, a hand bounding box of the hand region is recognized and an environment image including the hand bounding box of the hand region is output. If no hand region exists, the environment image is directly output.

[0072] When it is detected that the environment image has a hand region, S202 is performed.

[0073] When it is detected that the environment image has no hand region, S203 is performed.

[0074] S202, in response to the existence of the hand region in the environment image, determining a region of interest corresponding to the hand region, and obtaining a first pixel reference value based on first gray values of all pixels in the region of interest of the environment image.

[0075] The region of interest (ROI) is a local area in an image that needs special attention or processing, and usually contains key information (such as target objects, feature points, etc.). In this application, in response to the existence of the hand region in the environment image, the focus is on the region of interest corresponding to the hand region, so as to ensure that the brightness information of the core region of the hand is processed first, that is, the brightness of the hand region is prioritized, so as to ensure that the XR glasses realize gesture recognition.

[0076] The first gray value refers to the gray value of the pixel in the region of interest in the environment image where the hand region exists. The higher the gray value, the stronger the brightness perceived by the human eye, and the lower the gray value, the weaker the brightness perceived by the human eye.

[0077] In one possible implementation, when it is detected that the environment image has a hand region, the hand region is intercepted to obtain a region of interest corresponding to the hand region.

[0078] The region of interest coincides with the center point of the hand region, and the area of the region of interest is smaller than the area of the hand region.

[0079] For example, when the environment image has a hand region, a hand bounding box of the hand region (such as a hand bounding box output by a deep learning-based target detection model) is identified; 1 / 4 of the height of the hand bounding box is intercepted downward along the upper edge of the hand bounding box, 1 / 4 of the height of the hand bounding box is intercepted upward along the lower edge of the hand bounding box, 1 / 4 of the width of the hand bounding box is intercepted right along the left edge of the hand bounding box, and 1 / 4 of the width of the hand bounding box is intercepted left along the right edge of the hand bounding box to generate a bounding box of the region of interest corresponding to the hand region, thereby obtaining the region of interest corresponding to the hand region. The center point of the region of interest obtained by this way of interception coincides with the center point of the hand region, the area of the region of interest is smaller than the area of the hand region, the width of the region of interest is half of the width of the hand region, and the height of the region of interest is half of the height of the hand region. It should be noted that the hand region detected and identified in the embodiments of the present application is a rectangular region.

[0080] In the embodiments of the present application, when a hand region is detected in the environment image, a region of interest smaller than the hand region is obtained by reducing the hand region, so as to exclude the interference (such as shadow and reflection) possibly existing in the edge of the hand, and ensure that the brightness information of the core region of the hand (such as the palm and the fingertips) is preferentially processed. Moreover, the center point of the region of interest coincides with the center point of the hand region, so as to ensure that the region of interest can cover the key part of the hand region when the hand moves or the posture changes.

[0081] In a possible implementation, the number of hand regions existing in the environment image is one or more. For example, the environment image includes the right hand of the user of the XR glasses, and at this time, the environment image has one hand region; the environment image includes the left hand and the right hand of the user of the XR glasses, and at this time, the environment image has two hand regions.

[0082] In response to the fact that one hand region exists in the environment image, a region of interest corresponding to the one hand region is determined, and the average value of the first gray values of all the pixels in the region of interest of the environment image is calculated to obtain a corresponding first pixel reference value.

[0083] In response to the fact that multiple hand regions exist in the environment image, regions of interest corresponding to the multiple hand regions are determined respectively, and the average value of the first gray values of all the pixels in each region of interest of the environment image is calculated to obtain a corresponding gray average value of each region of interest; and the average value of the gray average values corresponding to all the regions of interest is calculated to obtain a corresponding first pixel reference value.

[0084] For example, when one hand region A1 is detected in the environment image, a region of interest a1 corresponding to the hand region A1 is determined; and the average value of the first gray values of all the pixels in the region of interest a1 (that is, the gray average value of the region of interest a1) is calculated to obtain a corresponding first pixel reference value. That is, the first pixel reference value = the gray average value of the region of interest a1.

[0085] When two hand regions, hand region A2 and hand region A3, are detected in the environment image, a region of interest a2 corresponding to the hand region A2 and a region of interest a3 corresponding to the hand region A3 are determined; the average value of the first gray values of all the pixels in the region of interest a2 is calculated to obtain the first gray average value of the region of interest a2, and the average value of the first gray values of all the pixels in the region of interest a3 is calculated to obtain the gray average value of the region of interest a3; and the average value of the gray average value of the region of interest a2 and the gray average value of the region of interest a3 is calculated to obtain a corresponding first pixel reference value. That is, the first pixel reference value = (the gray average value of the region of interest a2 + the gray average value of the region of interest a3) / 2.

[0086] In the embodiment of the present application, the overall brightness state of the hand region is quantified / characterized using the average gray value of the region of interest, which can eliminate local noise and background light interference, and more stably reflect the overall light and dark degree of the hand region. Compared with the average value of the global pixel gray value, when calculating the gray average value of the entire image, the brightness of the hand region may be covered due to the over-brightness (such as strong light reflection) or over-darkness (such as shadow) of the background. For example, when the hand is in the dark part, the high gray value of the bright area of the background will pull up the global average value, resulting in overestimation of the actual brightness of the hand, i.e., the actual brightness of the hand region is lower than the brightness characterized by the global gray average value. When the hand is in the bright part, the low gray value of the dark area of the background will pull down the global average value, resulting in underestimation of the actual brightness of the hand, i.e., the actual brightness of the hand region is higher than the brightness characterized by the global gray average value. Therefore, in the embodiment of the present application, the region of interest corresponding to the hand region is obtained based on the hand region, and the gray average value of the region of interest is taken as the first pixel reference value, so as to ensure that the first pixel reference value for characterizing the brightness of the hand region is calculated, and the interference of ambient light, background clutter area, etc. is excluded.

[0087] It should be noted that before calculating the corresponding pixel reference value, the environment image needs to be subjected to grayscale processing to obtain a grayscale image of the environment image, so as to obtain the first gray value of the environment image. Of course, the RGB value of the pixel of the environment image can also be directly obtained, and then the RGB value is converted into the first gray value.

[0088] S203, in response to the absence of the hand region in the environment image, determining a gray histogram corresponding to the environment image and a weight coefficient corresponding to each second gray value in the environment image, obtaining a weighted probability density corresponding to each second gray value based on the gray histogram and the weight coefficient corresponding to each second gray value, and obtaining a second pixel reference value based on the weighted probability density corresponding to each second gray value.

[0089] The gray histogram (Gray Histogram) is a statistical distribution of all pixel gray values in an image, the horizontal axis is the gray value (0-255), and the vertical axis is the number of pixels corresponding to the gray value. The gray histogram is used to reflect the overall brightness distribution of the image. In the embodiment of the present application, the gray histogram corresponding to the environment image is a statistical distribution of the second gray value of all pixels in the environment image, the horizontal axis is the second gray value, and the vertical axis is the number of pixels corresponding to the second gray value.

[0090] The second gray value refers to the gray value of the pixel in the environment image in which the hand region is absent.

[0091] It should be noted that the first gray value and the second gray value are actually the gray values of the pixels in essence, and in the present application, the first gray value and the second gray value are used to distinguish the gray values of the pixels in the environment image in which the hand region exists and the gray values of the pixels in the environment image in which the hand region does not exist.

[0092] wherein the weight coefficient refers to a dynamic adjustment factor of the second gray value, and the weight coefficient corresponding to the second gray value is negatively correlated with the difference between the second gray value and the preset reference gray value, that is, the greater the difference between the second gray value and the preset reference gray value, the smaller the weight coefficient corresponding to the second gray value, and vice versa.

[0093] The weight coefficient is used to suppress the interference of extreme second gray values (i.e. too bright or too dark) and highlight the contribution of the intermediate brightness region (such as the preset reference gray value).

[0094] wherein the weighted probability density refers to the adjusted probability density obtained by the weight coefficient and the initial probability density.

[0095] In one possible implementation, in response to the fact that the hand region does not exist in the environment image, the gray histogram corresponding to the environment image and the weight coefficient corresponding to each second gray value in the environment image are determined; the initial probability density corresponding to each second gray value is obtained based on the gray histogram corresponding to the environment image; and the probability density corresponding to each second gray value is weighted based on the weight coefficient corresponding to each second gray value, so as to obtain the weighted probability density corresponding to each second gray value.

[0096] wherein the initial probability density refers to the probability of the occurrence of each second gray value i in the image.

[0097] For the convenience of understanding, the weighted probability density corresponding to the second gray value will be introduced in detail below in combination with formula (1).

[0098] The weighted probability density corresponding to the second gray value is shown in formula (1):

[0099] (1)

[0100] wherein, is the second gray value i corresponding to the weighted probability density, is the second gray value i corresponding to the initial probability density, is the second gray value i corresponding to the weight coefficient.

[0101] The weight coefficient corresponding to the second gray value is obtained by the following manner: based on the second gray value, a preset reference gray value and a preset standard deviation, a weight parameter corresponding to the second gray value is obtained; and an exponential function is used to calculate an exponential function value of the weight coefficient corresponding to the second gray value as an argument, so as to obtain the weight coefficient corresponding to the second gray value.

[0102] For the convenience of understanding, the weight coefficient corresponding to the second gray value will be introduced in detail below in combination with formula (2).

[0103] The weight coefficient corresponding to the second gray value is shown in formula (2):

[0104] (2)

[0105] wherein, is the second gray value i corresponding to the weight coefficient; exp is an exponential function; i is the second gray value; u is a preset reference gray value, generally u =128 (median gray value); v is a preset standard deviation, used for controlling the width of the weight coefficient distribution, v the greater the weight coefficient decays slower (the wider the distribution), v the smaller, the weight coefficient decays faster (the narrower the distribution), generally v =10.

[0106] wherein, -( i - u ) 2 / 2 v 2 is the weight parameter corresponding to the second gray value i , exp[-( i - u ) 2 / 2 v 2 ] is an exponential function value of the weight coefficient corresponding to the second gray value as an argument.

[0107] As shown in formula (2), the closer the second gray value is to the preset reference gray value, the greater the weight coefficient corresponding to the second gray value is; the farther the second gray value is from the preset reference gray value, the smaller the weight coefficient corresponding to the second gray value is. And the exponential function makes the weight coefficient change continuously and naturally, avoiding sudden change.

[0108] In one possible implementation, each second gray value is multiplied by the weighted probability density corresponding to each second gray value, to obtain a sub-pixel reference value corresponding to each second gray value respectively, and the sub-pixel reference values corresponding to each second gray value respectively are summed to obtain a corresponding second pixel reference value.

[0109] For the convenience of understanding, the second pixel reference value is introduced in detail below in combination with formula (3) and formula (4).

[0110] The sub-pixel reference value is shown in formula (3):

[0111] (3)

[0112] wherein, is the second gray value i is the corresponding sub-pixel reference value, i is the second gray value, is the second gray value i is the corresponding weighted probability density.

[0113] The second pixel reference value is shown in formula (4):

[0114] (4)

[0115] wherein, Z is the corresponding second pixel reference value of the environment image, i is the second gray value, is the second gray value i is the corresponding weighted probability density.

[0116] In the embodiment of the present application, the weighted average gray value of all second gray values is obtained as the corresponding second pixel reference value of the environment image through the corresponding weighted probability density of each second gray value, the interference of extreme second gray values (too bright or too dark) is suppressed, the contribution of the intermediate brightness area is highlighted, the brightness represented by the second pixel reference value is closer to the brightness perceived by the human eye, and the situation that the global gray average value cannot represent the brightness state of the environment image due to local over-brightness or over-darkness is avoided.

[0117] In the current technology, the brightness state of the image is represented by the global gray average value, but the global gray average value is easily dominated by extreme brightness pixels (over-bright or over-dark pixels), for example: if there are a small number of over-bright pixels (such as gray value = 255), the global gray average value will be significantly high, and cannot accurately represent the brightness state of the image.

[0118] In the embodiments of the present application, by using the weight coefficient corresponding to the second gray value, and the weight coefficient of the second gray value being negatively correlated with the difference between the second gray value and the preset reference gray value (as shown in formula (2)), the interference of extreme brightness can be dynamically inhibited. If the weight coefficient of the over-bright / over-dark (i.e., extreme gray value) is small, the sharing of over-bright (i->255) or over-dark (i->0) pixels is reduced, and a high weight coefficient of the second gray value close to the preset reference gray value is retained, thereby highlighting the brightness area corresponding to the reference gray value, improving the accuracy of the second pixel reference value in reflecting / representing the brightness state of the environment image, and avoiding the case that the global gray average value cannot represent the brightness state of the environment image due to local over-brightness or over-darkness. Moreover, the human eye is more sensitive to intermediate brightness (i.e., the brightness corresponding to the preset reference gray value) and less sensitive to extreme brightness, so by enhancing the weight of the intermediate brightness area, the perception characteristics of the human eye are met, and the brightness state reflected by the second pixel reference value is closer to the brightness perceived by the human eye.

[0119] S204, dynamically adjusting the exposure parameter of the camera of the extended reality glasses according to the current pixel reference value.

[0120] The current pixel reference value is the first pixel reference value or the second pixel reference value. When it is detected that the hand region exists in the environment image, the current pixel reference value is the first pixel reference value; and when it is detected that the hand region does not exist in the environment image, the current pixel reference value is the second pixel reference value.

[0121] The exposure parameter includes at least one of an exposure time and a sensitivity.

[0122] The exposure parameter (Exposure Parameters) is a core parameter for controlling the brightness of an image.

[0123] The exposure time (Shutter Speed) refers to the length of time for which the camera sensor of the XR glasses receives light. For example, the longer the exposure time, the more light the camera sensor receives, and the brighter the image; conversely, the shorter the exposure time, the darker the image.

[0124] The sensitivity (ISO) refers to an index for measuring the sensitivity of the camera sensor of the XR glasses to light. For example, the larger the ISO value, the more sensitive the camera sensor is to light, and the brighter the image; conversely, the less sensitive it is to light, and the darker the image.

[0125] For example, only the exposure time of the camera of the XR glasses is dynamically adjusted according to the current pixel reference value; or only the sensitivity of the camera of the XR glasses is dynamically adjusted according to the current pixel reference value; or the exposure time and the sensitivity of the camera of the XR glasses are dynamically adjusted according to the current pixel reference value.

[0126] In a possible implementation, based on the current pixel reference value and the preset luminance parameter, a rate of adjustment of the exposure parameter is determined; it is judged whether the rate of adjustment of the exposure parameter is between a preset minimum adjustment rate and a preset maximum adjustment rate; when the rate of adjustment of the exposure parameter is less than the preset minimum adjustment rate, or the rate of adjustment of the exposure parameter is greater than the preset maximum adjustment rate, the exposure parameter of the camera of the extended reality glasses is dynamically adjusted based on the rate of adjustment of the exposure parameter; when the rate of adjustment of the exposure parameter is greater than or equal to the preset minimum adjustment rate, and the rate of adjustment of the exposure parameter is less than or equal to the preset maximum adjustment rate, the exposure parameter of the camera of the extended reality glasses does not need to be adjusted.

[0127] The preset luminance parameter is a preset reference value, which is a target luminance value (or a target gray value) set in advance. For example, a luminance value close to the comfortable range of human eye perception can be set in advance according to the adaptability of human eye to luminance.

[0128] Specifically, the rate of adjustment of the exposure parameter is shown in formula (5):

[0129] (5)

[0130] wherein, r the rate of adjustment of the exposure parameter, the current pixel reference value, b the preset luminance parameter.

[0131] The preset minimum adjustment rate is less than 0, and the preset maximum adjustment rate is greater than 0. When the rate of adjustment of the exposure parameter is less than the preset minimum adjustment rate, it indicates that the luminance represented by the current pixel reference value is darker than the luminance represented by the preset luminance parameter, and is too dark (out of the acceptable range). When the rate of adjustment of the exposure parameter is greater than the preset maximum adjustment rate, it indicates that the luminance represented by the current pixel reference value is brighter than the luminance represented by the preset luminance parameter, and is too bright (out of the acceptable range). Therefore, at this time, the exposure parameter of the camera of the XR glasses needs to be dynamically adjusted based on the rate of adjustment of the exposure parameter.

[0132] For the convenience of understanding, the following will introduce how to dynamically adjust the exposure parameter of the camera of the extended reality glasses when the rate of adjustment of the exposure parameter is less than the preset minimum adjustment rate and when the rate of adjustment of the exposure parameter is greater than the preset maximum adjustment rate, respectively. Figure 3 Figure 4

[0133] First, as shown in FIG. 1, when the rate of adjustment of the exposure parameter is less than the preset minimum adjustment rate in the embodiment of the application, the exposure parameter adjustment manner includes the following steps: Figure 3

[0134] ​​​S301, when the adjustment rate of the exposure parameter is less than the preset minimum adjustment rate, it is judged whether the first exposure time corresponding to the environment image is the preset maximum exposure time.

[0135] The preset maximum exposure time (Max Exposure Time) is determined by the hardware characteristics of the camera sensor of the XR glasses. For example, the preset maximum exposure time is determined according to the frame rate of the XR glasses. Assuming that the frame rate of the camera is 60fps, the frame period is 1 / 60 second (≈16.7ms), and the maximum exposure time is usually close to the frame period.

[0136] When the adjustment rate of the exposure parameter is less than the preset minimum adjustment rate, it indicates that the brightness of the environment image is too dark, and the brightness of the environment image needs to be improved. In the case of needing to improve the brightness of the environment image, the exposure time is first increased, and then the sensitivity is increased in the embodiment of the application. Because high sensitivity will introduce noise (graininess in the image) into the image, which reduces the quality of the image, the brightness of the image is preferentially improved by increasing the exposure time in the embodiment of the application, so as to avoid unnecessary noise caused by sensitivity.

[0137] When the first exposure time corresponding to the environment image is the preset maximum exposure time, it indicates that the brightness of the image cannot be improved by adjusting the exposure time, so S305 is performed.

[0138] When the first exposure time corresponding to the environment image is not the preset maximum exposure time, it indicates that the first exposure time is less than the preset maximum exposure time, that is, the brightness of the image can still be improved by adjusting the exposure time, so S302 is performed.

[0139] S302, according to the adjustment rate of the exposure parameter, the first exposure time corresponding to the environment image is adjusted to obtain the second exposure time.

[0140] Specifically, the second exposure time = the first exposure time x (1-adjustment rate).

[0141] S303, according to the second exposure time, the preset maximum exposure time and the preset minimum exposure time, the third exposure time is determined, and the exposure time of the camera of the extended reality glasses is adjusted to the third exposure time.

[0142] Specifically, when the second exposure time is less than the preset minimum exposure time, the third exposure time is the preset minimum exposure time; when the second exposure time is greater than the preset maximum exposure time, the third exposure time is the preset maximum exposure time; when the second exposure time is greater than or equal to the preset minimum exposure time and less than or equal to the preset maximum exposure time, the third exposure time is the second exposure time.

[0143] S304, determine whether the third exposure time is a preset maximum exposure time.

[0144] When the third exposure time is not the preset maximum exposure time, it indicates that the brightness of the collected image can be improved by adjusting the exposure time under the current brightness, and theoretically the user's expectation of image brightness can be met, so S307 is performed.

[0145] When the third exposure time is the preset maximum exposure time, it indicates that the brightness of the collected image can be improved by adjusting the exposure time under the current brightness, but theoretically it may not be able to meet the user's expectation of image brightness, because the second exposure time (i.e. the required exposure time) is greater than the preset maximum exposure time, and the third exposure time is the preset maximum exposure time. Therefore, when the third exposure time is the preset maximum exposure time, adjusting the exposure time of the camera to the third exposure time cannot theoretically meet the user's expectation of image brightness, so S305 is performed.

[0146] S305, adjust the first light sensitivity corresponding to the environmental image according to the adjustment rate of the exposure parameter to obtain a second light sensitivity.

[0147] Specifically, the second light sensitivity = the first light sensitivity x (1 - adjustment rate).

[0148] S306, determine a third light sensitivity according to the second light sensitivity, a preset maximum light sensitivity and a preset minimum light sensitivity, and adjust the light sensitivity of the camera of the extended reality glasses to the third light sensitivity.

[0149] Specifically, when the second light sensitivity is less than the preset minimum light sensitivity, the third light sensitivity is the preset minimum light sensitivity; when the second light sensitivity is greater than the preset maximum light sensitivity, the third light sensitivity is the preset maximum light sensitivity; when the second light sensitivity is greater than or equal to the preset minimum light sensitivity and less than or equal to the preset maximum light sensitivity, the third light sensitivity is the second light sensitivity.

[0150] S307, complete the adjustment of the exposure parameter of the camera of the extended reality glasses.

[0151] In the embodiments of the present application, for the case where the adjustment rate of the exposure parameter is less than the preset minimum adjustment rate, the exposure time is adjusted preferentially, and the light sensitivity adjustment is delayed, so as to reduce the accumulation of noise and avoid the loss of image details. If the exposure time has been adjusted to the maximum and still not enough brightness, then the light sensitivity is increased, which can limit the introduction amplitude of noise, rather than adjusting the light sensitivity from the beginning. Further improve the user's interactive experience.

[0152] First, as shown in Figure 4 When the adjustment rate of the exposure parameter is greater than the preset maximum adjustment rate in the embodiments of the present application, the exposure parameter adjustment method includes the following steps:

[0153] S401, when the adjustment rate of the exposure parameter is greater than the preset maximum adjustment rate, it is judged whether the first sensitivity corresponding to the environment image is the preset minimum sensitivity.

[0154] When the adjustment rate of the exposure parameter is greater than the preset maximum adjustment rate, it indicates that the brightness of the environment image is too bright, and the brightness of the environment image needs to be reduced. For the case where the brightness of the environment image needs to be reduced, the sensitivity is first reduced and then the exposure time is reduced in the embodiment of the present application. Because high sensitivity will introduce noise points (graininess in the image) in the image, reducing the quality of the image, the sensitivity is preferentially reduced to reduce the brightness of the image in the embodiment of the present application, so as to reduce the noise points introduced by the sensitivity while reducing the brightness of the image.

[0155] When the first sensitivity corresponding to the environment image is the preset minimum sensitivity, it indicates that the brightness of the image cannot be reduced by adjusting the sensitivity, so S305 is performed.

[0156] When the first sensitivity corresponding to the environment image is not the preset minimum sensitivity, it indicates that the first sensitivity is greater than the preset minimum sensitivity, that is, the brightness of the image can still be reduced by adjusting the sensitivity, so S402 is performed.

[0157] S402, adjusting the first sensitivity corresponding to the environment image according to the adjustment rate of the exposure parameter to obtain a second sensitivity.

[0158] Specifically, the second sensitivity = the first sensitivity x (1-adjustment rate).

[0159] S403, determining a third sensitivity according to the second sensitivity, the preset maximum sensitivity and the preset minimum sensitivity, and adjusting the sensitivity of the camera of the extended reality glasses to the third sensitivity.

[0160] Specifically, when the second sensitivity is less than the preset minimum sensitivity, the third sensitivity is the preset minimum sensitivity; when the second sensitivity is greater than the preset maximum sensitivity, the third sensitivity is the preset maximum sensitivity; when the second sensitivity is greater than or equal to the preset minimum sensitivity and less than or equal to the preset maximum sensitivity, the third sensitivity is the second sensitivity.

[0161] S404, judging whether the third sensitivity is the preset minimum sensitivity.

[0162] When the third sensitivity is not the preset minimum sensitivity, it indicates that the brightness of the image collected can be reduced by adjusting the sensitivity under the current brightness, and theoretically it can meet the user's expectation of the brightness of the image, so S407 is performed.

[0163] When the third ISO sensitivity is the preset minimum ISO sensitivity, it indicates that the brightness of the collected image can be reduced by adjusting the ISO sensitivity under the current brightness, but theoretically the user's expectation of the image brightness cannot be met because the second ISO sensitivity (i.e., the required ISO sensitivity) is less than the preset minimum ISO sensitivity. Therefore, when the third ISO sensitivity is the preset minimum ISO sensitivity, the user's expectation of the image brightness cannot be met theoretically by only adjusting the ISO sensitivity of the camera to the third ISO sensitivity, and thus S405 is performed.

[0164] S405, adjusting the first exposure time corresponding to the environmental image according to the adjustment rate of the exposure parameter to obtain a second exposure time.

[0165] Specifically, the second exposure time = the first exposure time x (1-adjustment rate).

[0166] S406, determining a third exposure time according to the second exposure time, the preset maximum exposure time, and the preset minimum exposure time, and adjusting the exposure time of the camera of the extended reality glasses to the third exposure time.

[0167] Specifically, when the second exposure time is less than the preset minimum exposure time, the third exposure time is the preset minimum exposure time; when the second exposure time is greater than the preset maximum exposure time, the third exposure time is the preset maximum exposure time; and when the second exposure time is greater than or equal to the preset minimum exposure time and less than or equal to the preset maximum exposure time, the third exposure time is the second exposure time.

[0168] S407, completing the adjustment of the exposure parameter of the camera of the extended reality glasses.

[0169] In the embodiments of the present application, for the case that the adjustment rate of the exposure parameter is greater than the preset maximum adjustment rate, the ISO sensitivity is adjusted first, and then the exposure time is adjusted, the image brightness is reduced first, and the image noise is reduced at the same time, so as to improve the image quality and avoid the loss of image details. If the ISO sensitivity has been adjusted to the minimum brightness and the image is still too bright, the exposure time is reduced at this time, so as to reduce the brightness of the image.

[0170] This application provides an exposure adjustment method for extended reality glasses, including: viewing an environmental image and detecting whether a hand region exists in the environmental image; in response to the presence of a hand region in the environmental image, determining a region of interest corresponding to the hand region, and obtaining a first pixel reference value based on the first grayscale value of a pixel in the region of interest; in response to the absence of a hand region in the environmental image, determining a grayscale histogram corresponding to the environmental image and a weighting coefficient corresponding to each second grayscale value in the environmental image, obtaining a weighted probability density corresponding to each second grayscale value based on the grayscale histogram and the weighting coefficient corresponding to each second grayscale value, and obtaining a second pixel reference value based on the weighted probability density corresponding to each second grayscale value; and dynamically adjusting the exposure parameters of the extended reality glasses' camera according to the current pixel reference value. This application's embodiments distinguish between scenarios with and without a user's hand. In scenarios with a user's hand, a first reference value is obtained based on the first grayscale value of the pixels in the region of interest corresponding to the hand area. Exposure adjustment is then directly focused on the local brightness of the user's hand, prioritizing the clear visibility of the user's hand details and significantly improving the accuracy of XR glasses gesture recognition. In scenarios without a user's hand, a second pixel reference value obtained through weighted probability density can accurately reflect the global brightness of the image, thereby improving the accuracy of exposure adjustment and thus enhancing the brightness stability of the image acquired by the XR glasses.

[0171] Furthermore, when the adjustment rate of the exposure parameter is less than the preset minimum adjustment rate, the exposure time is adjusted first, delaying the introduction of ISO adjustment to reduce noise accumulation and avoid loss of image detail. If the exposure time is already at its maximum and the brightness is still insufficient, then the ISO is increased, which limits the amount of noise introduced. When the adjustment rate of the exposure parameter is greater than the preset maximum adjustment rate, the ISO is adjusted first, then the exposure time, prioritizing the reduction of image brightness while simultaneously reducing image noise, thereby improving image quality and avoiding loss of image detail. This further enhances the user's interactive experience with XR glasses.

[0172] Example 2:

[0173] The following is combined with Figure 5 This application provides a detailed description of an exposure adjustment device for augmented reality glasses, as provided in the embodiments of this application.

[0174] like Figure 5 As shown in the embodiment of this application, an exposure adjustment device for augmented reality glasses includes the following modules:

[0175] The hand region detection module 501 is used to acquire environmental images and detect whether a hand region exists in the environmental images;

[0176] The first reference determining module 502 is configured to, in response to the hand region existing in the environment image, determine a region of interest corresponding to the hand region, and obtain a first pixel reference value based on a first gray value of a pixel in the region of interest.

[0177] The second reference determining module 503 is configured to, in response to the hand region not existing in the environment image, determine a gray histogram corresponding to the environment image and a weight coefficient corresponding to each second gray value in the environment image, obtain a weighted probability density corresponding to each second gray value based on the gray histogram and the weight coefficient corresponding to each second gray value, and obtain a second pixel reference value based on the weighted probability density corresponding to each second gray value; wherein the weight coefficient corresponding to the second gray value is negatively correlated with a difference between the second gray value and a preset reference gray value.

[0178] The exposure parameter adjusting module 504 is configured to dynamically adjust an exposure parameter of a camera of the extended reality glasses according to the current pixel reference value; wherein the current pixel reference value is the first pixel reference value or the second pixel reference value; and the exposure parameter includes at least one of an exposure time and a sensitivity.

[0179] In a possible implementation, the first reference determining module 502 is specifically configured to, in response to the hand region existing in the environment image, intercept the hand region to obtain a region of interest corresponding to the hand region; wherein the region of interest coincides with a center point of the hand region, and an area of the region of interest is smaller than an area of the hand region.

[0180] In a possible implementation, the number of hand regions is one or more; and the first reference determining module 502 is specifically configured to:

[0181] in response to one hand region existing in the environment image, determine a region of interest corresponding to the one hand region, and calculate an average value of gray values of all pixels in the region of interest of the environment image to obtain the first pixel reference value;

[0182] in response to multiple hand regions existing in the environment image, determine regions of interest corresponding to the multiple hand regions respectively, and calculate an average value of gray values of all pixels in each region of interest of the environment image to obtain a gray average value corresponding to each region of interest; and calculate an average value of the gray average values corresponding to all regions of interest to obtain the first pixel reference value.

[0183] In a possible implementation, the second reference determining module 503 is specifically configured to: obtain an initial probability density corresponding to each second gray value based on the gray histogram; and perform weighted processing on the initial probability density corresponding to each second gray value according to the weight coefficient corresponding to each second gray value to obtain the weighted probability density corresponding to each second gray value.

[0184] In a possible implementation, the second reference determining module 503 is specifically configured to: multiply each second gray value and the weighted probability density corresponding to the second gray value to obtain a sub-pixel reference value corresponding to each second gray value respectively, and perform summation processing on the sub-pixel reference values corresponding to each second gray value respectively to obtain a second pixel reference value.

[0185] In a possible implementation, the exposure parameter adjusting module 504 is specifically configured to: determine an adjustment rate of the exposure parameter based on the current pixel reference value and the preset luminance parameter; determine whether the adjustment rate of the exposure parameter is between a preset minimum adjustment rate and a preset maximum adjustment rate; and when the adjustment rate of the exposure parameter is less than the preset minimum adjustment rate or the adjustment rate of the exposure parameter is greater than the preset maximum adjustment rate, dynamically adjust the exposure parameter of the camera of the extended reality glasses based on the adjustment rate of the exposure parameter.

[0186] In a possible implementation, the exposure parameter adjusting module 504 includes a luminance increasing module and a luminance decreasing module. The luminance increasing module is configured to:

[0187] When the adjustment rate of the exposure parameter is less than the preset minimum adjustment rate, determine whether the first exposure time corresponding to the environment image is the preset maximum exposure time;

[0188] When the first exposure time is not the preset maximum exposure time, adjust the first exposure time corresponding to the environment image according to the adjustment rate of the exposure parameter to obtain a second exposure time;

[0189] determine a third exposure time according to the second exposure time, the preset maximum exposure time and the preset minimum exposure time, and adjust the exposure time of the camera of the extended reality glasses to the third exposure time;

[0190] When the first exposure time is the preset maximum exposure time or the third exposure time is the preset maximum exposure time, adjust the first light sensitivity corresponding to the environment image according to the adjustment rate of the exposure parameter to obtain a second light sensitivity;

[0191] determine a third light sensitivity according to the second light sensitivity, the preset maximum light sensitivity and the preset minimum light sensitivity, and adjust the light sensitivity of the camera of the extended reality glasses to the third light sensitivity.

[0192] In a possible implementation, the exposure parameter adjusting module 504 includes a luminance increasing module and a luminance decreasing module. The luminance decreasing module is configured to:

[0193] When the adjustment rate of the exposure parameter is greater than the preset maximum adjustment rate, determine whether the first light sensitivity corresponding to the environment image is the preset minimum light sensitivity;

[0194] when the first sensitivity is not the preset minimum sensitivity, adjusting the first sensitivity corresponding to the environment image according to the adjustment rate of the exposure parameter to obtain a second sensitivity;

[0195] determining a third sensitivity according to the second sensitivity, the preset maximum sensitivity and the preset minimum sensitivity, and adjusting the sensitivity of the camera of the extended reality glasses to the third sensitivity;

[0196] when the third sensitivity is not the preset minimum sensitivity, completing the adjustment of the exposure parameter of the camera of the extended reality glasses;

[0197] when the first sensitivity is the preset minimum sensitivity or the third sensitivity is the preset minimum sensitivity, adjusting the first exposure time corresponding to the environment image according to the adjustment rate of the exposure parameter to obtain a second exposure time;

[0198] determining a third exposure time according to the second exposure time, the preset maximum exposure time and the preset minimum exposure time, and adjusting the exposure time of the camera of the extended reality glasses to the third exposure time.

[0199] The embodiment of the application provides an exposure adjustment device of extended reality glasses, comprising: a hand region detection module 501, configured to collect an environment image and detect whether a hand region exists in the environment image; a first reference determination module 502, configured to determine a region of interest corresponding to the hand region in response to the existence of the hand region in the environment image, and obtain a first pixel reference value based on a first gray value of a pixel in the region of interest; a second reference determination module 503, configured to determine a gray histogram corresponding to the environment image and a weight coefficient corresponding to each second gray value in the environment image in response to the non-existence of the hand region in the environment image, obtain a weighted probability density corresponding to each second gray value based on the gray histogram and the weight coefficient corresponding to each second gray value, and obtain a second pixel reference value based on the weighted probability density corresponding to each second gray value; and an exposure parameter adjustment module 504, configured to dynamically adjust an exposure parameter of a camera of the extended reality glasses according to a current pixel reference value. The embodiment of the application distinguishes between a scene in which a user's hand exists and a scene in which the user's hand does not exist, obtains a first reference value based on a first gray value of a pixel in a region of interest corresponding to the hand region, directly focuses on local brightness of the user's hand for exposure adjustment for the scene in which the user's hand exists, preferentially ensures that the details of the user's hand are clear and visible, and significantly improves the accuracy of gesture recognition of the XR glasses; for the scene in which the user's hand does not exist, a second pixel reference value obtained through the weighted probability density can accurately reflect the global brightness of the image, thereby improving the accuracy of exposure adjustment, and thereby improving the brightness stability of the image collected by the XR glasses.

[0200] Further, for the case that the adjustment rate of the exposure parameter is less than the preset minimum adjustment rate, the exposure time is adjusted preferentially, and the introduction of the sensitivity is delayed, so as to reduce the noise accumulation and avoid the loss of image details. If the exposure time has been adjusted to the maximum and still not enough brightness, then the sensitivity is increased, which can limit the introduction amplitude of the noise. For the case that the adjustment rate of the exposure parameter is greater than the preset maximum adjustment rate, the sensitivity is adjusted preferentially, and then the exposure time is adjusted, so as to reduce the image noise while reducing the image brightness, thereby improving the image quality and avoiding the loss of image details. Further, the interaction experience of the user and the XR glasses is improved.

[0201] It should be noted that each of the embodiments in the specification is described in a progressive manner, and the same and similar parts between the embodiments can be referred to each other. Each embodiment focuses on the difference from other embodiments. Especially, the device embodiment is described more simply because it is basically similar to the method embodiment. The relevant parts can be referred to the part of the method embodiment. The device embodiment described above is only schematic, and the units described as separate components can or can not be physically separate, and the components indicated as units can or can not be physical units, that is, they can be located in one place, or distributed on multiple network units. According to the actual needs, some or all of the modules can be selected to achieve the purpose of the embodiment. Those skilled in the art can understand and implement it without creative labor.

[0202] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited to this. Any skilled person in the art can easily think of changes or replacements within the technical range disclosed in the present application, which should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method of exposure adjustment for an extended reality eyewear, the method comprising: The method comprises: capturing an environment image and detecting whether a hand region exists in the environment image; in response to the existence of the hand region in the environment image, determining a region of interest corresponding to the hand region, and obtaining a first pixel reference value based on a first gray value of a pixel in the region of interest; in response to the absence of the hand region in the environment image, determining a gray histogram corresponding to the environment image and a weight coefficient corresponding to each second gray value in the environment image, performing weighted processing on an initial probability density corresponding to the each second gray value and the weight coefficient corresponding to the each second gray value based on the gray histogram, to obtain a weighted probability density corresponding to the each second gray value, and obtaining a second pixel reference value based on the weighted probability density corresponding to the each second gray value; wherein the weight coefficient corresponding to the second gray value is negatively correlated with a difference between the second gray value and a preset reference gray value; and the initial probability density corresponding to the second gray value is a probability of occurrence of the second gray value in the environment image; dynamically adjusting an exposure parameter of a camera of an extended reality glasses according to a current pixel reference value; wherein the current pixel reference value is the first pixel reference value or the second pixel reference value; and the exposure parameter comprises at least one of an exposure time and a sensitivity. The weight coefficient corresponding to the second gray value is obtained by: obtaining a weight parameter corresponding to the second gray value based on the second gray value, the preset reference gray value and a preset standard deviation; calculating an exponential function value with the weight parameter corresponding to the second gray value as an independent variable by an exponential function, to obtain the weight coefficient corresponding to the second gray value.

2. The method of claim 1, wherein, The method comprises: in response to the existence of the hand region in the environment image, intercepting the hand region to obtain a region of interest corresponding to the hand region; wherein the region of interest coincides with a center point of the hand region, and an area of the region of interest is smaller than an area of the hand region.

3. The method of claim 1, wherein, The method comprises: in response to the existence of one hand region in the environment image, determining a region of interest corresponding to the one hand region, and calculating an average value of first gray values of all pixels in the region of interest of the environment image to obtain the first pixel reference value; in response to the existence of multiple hand regions in the environment image, determining regions of interest corresponding to the multiple hand regions respectively, calculating an average value of first gray values of all pixels in each of the regions of interest of the environment image to obtain a gray average value corresponding to each of the regions of interest, and calculating an average value of the gray average values corresponding to all the regions of interest to obtain the first pixel reference value.

4. The method of claim 1, wherein, The second pixel reference value is obtained based on the weighted probability density corresponding to each second gray value, including: The second pixel reference value is obtained by multiplying each second gray value and the weighted probability density corresponding to each second gray value, and then summing the sub-pixel reference values corresponding to each second gray value.

5. The method of claim 1, wherein, The exposure parameter of the camera of the extended reality glasses is dynamically adjusted according to the current pixel reference value, including: Based on the current pixel reference value and the preset brightness parameter, the adjustment rate of the exposure parameter is determined. It is judged whether the adjustment rate of the exposure parameter is between the preset minimum adjustment rate and the preset maximum adjustment rate. When the adjustment rate of the exposure parameter is less than the preset minimum adjustment rate, or the adjustment rate of the exposure parameter is greater than the preset maximum adjustment rate, the exposure parameter of the camera of the extended reality glasses is dynamically adjusted based on the adjustment rate of the exposure parameter.

6. The method of claim 5, wherein, When the adjustment rate of the exposure parameter is less than the preset minimum adjustment rate, or the adjustment rate of the exposure parameter is greater than the preset maximum adjustment rate, the exposure parameter of the camera of the extended reality glasses is dynamically adjusted based on the adjustment rate of the exposure parameter, including: When the adjustment rate of the exposure parameter is less than the preset minimum adjustment rate, it is judged whether the first exposure time corresponding to the environment image is the preset maximum exposure time. When the first exposure time is not the preset maximum exposure time, the first exposure time corresponding to the environment image is adjusted according to the adjustment rate of the exposure parameter to obtain a second exposure time. According to the second exposure time, the preset maximum exposure time and the preset minimum exposure time, a third exposure time is determined, and the exposure time of the camera of the extended reality glasses is adjusted to the third exposure time. When the first exposure time is the preset maximum exposure time or the third exposure time is the preset maximum exposure time, the first light sensitivity corresponding to the environment image is adjusted according to the adjustment rate of the exposure parameter to obtain a second light sensitivity. According to the second light sensitivity, the preset maximum light sensitivity and the preset minimum light sensitivity, a third light sensitivity is determined, and the light sensitivity of the camera of the extended reality glasses is adjusted to the third light sensitivity.

7. The method of claim 5, wherein, When the adjustment rate of the exposure parameter is greater than the preset maximum adjustment rate, it is judged whether the first light sensitivity corresponding to the environment image is the preset minimum light sensitivity. When the first light sensitivity is not the preset minimum light sensitivity, the first light sensitivity corresponding to the environment image is adjusted according to the adjustment rate of the exposure parameter to obtain a second light sensitivity. According to the second light sensitivity, the preset maximum light sensitivity and the preset minimum light sensitivity, a third light sensitivity is determined, and the light sensitivity of the camera of the extended reality glasses is adjusted to the third light sensitivity. ​ When the first ISO sensitivity is a preset minimum ISO sensitivity or the third ISO sensitivity is a preset minimum ISO sensitivity, a first exposure time corresponding to the environment image is adjusted according to an adjustment rate of the exposure parameter to obtain a second exposure time; A third exposure time is determined according to the second exposure time, a preset maximum exposure time and a preset minimum exposure time, and an exposure time of a camera of the extended reality glasses is adjusted to the third exposure time.

8. An exposure adjustment device of an extended reality eyeglass, characterized by, Comprise: A hand region detection module configured to collect an environment image and detect whether a hand region exists in the environment image; A first reference determination module configured to, in response to the hand region existing in the environment image, determine a region of interest corresponding to the hand region, and obtain a first pixel reference value based on a first gray value of a pixel in the region of interest; A second reference determination module configured to, in response to the hand region not existing in the environment image, determine a gray histogram corresponding to the environment image and a weight coefficient corresponding to each second gray value in the environment image, perform weighted processing on an initial probability density corresponding to the each second gray value represented by the gray histogram and the weight coefficient corresponding to the each second gray value to obtain a weighted probability density corresponding to the each second gray value, and obtain a second pixel reference value based on the weighted probability density corresponding to the each second gray value; wherein the weight coefficient corresponding to the second gray value is negatively correlated with a difference between the second gray value and a preset reference gray value; and the initial probability density corresponding to the second gray value is a probability of occurrence of the second gray value in the environment image; An exposure parameter adjustment module configured to dynamically adjust an exposure parameter of a camera of the extended reality glasses according to a current pixel reference value; wherein the current pixel reference value is the first pixel reference value or the second pixel reference value; and the exposure parameter comprises at least one of an exposure time and an ISO sensitivity. The weight coefficient corresponding to the second gray value is obtained in the following manner: A weight parameter corresponding to the second gray value is obtained based on the second gray value, the preset reference gray value and a preset standard deviation; An exponential function value with the weight parameter corresponding to the second gray value as an independent variable is calculated by an exponential function to obtain the weight coefficient corresponding to the second gray value.

Images