Electronic device with supplemental transmitter for tracking external objects

By using a supplementary transmitter and image sensor in a head-mounted device to optimize lighting conditions, the problem of insufficient sensor detection accuracy in low-light environments was solved, and stable image capture and hand tracking under different lighting conditions were achieved.

CN119027632APending Publication Date: 2024-11-26APPLE INC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410642668.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2024-03-10
Filing Date
2024-05-23
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

In low-light conditions, the sensors of head-mounted devices have difficulty accurately detecting the user's hand, leading to a decrease in tracking accuracy and performance.

Method used

The head-mounted device is equipped with a supplementary emitter to emit supplementary light to illuminate external objects in low-light conditions. Combined with an image sensor and a hand tracker, image capture and hand tracking are optimized by adjusting the output intensity of the supplementary emitter.

Benefits of technology

It improves the accuracy of external object tracking and sensor performance under low light conditions, and ensures stable image capture and hand motion detection under different lighting conditions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119027632A_ABST
    Figure CN119027632A_ABST
Patent Text Reader

Abstract

The invention relates to an electronic device with a supplemental transmitter for tracking an external object. A head-mounted device may include: one or more scene cameras configured to capture an image of a scene; one or more hand sensors configured to detect a hand of the user; one or more supplemental emitters, the one or more supplemental emitters configured to provide supplemental illumination to the user's hand; and a control circuit configured to determine whether the user's hand is sufficiently irradiated; determining whether the captured image is correctly exposed; and adjusting the one or more supplemental emitters based on whether the user's hand is sufficiently illuminated and based on whether the captured image is correctly exposed. The control circuit may activate the supplemental emitter when the captured image is too dark or when the detected one hand is too dark. The control circuit may deactivate the supplemental emitter when the detected one hand is too bright or when the captured image is overexposed.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] This application claims priority to U.S. Patent Application No. 18 / 600,730 filed on March 10, 2024, which claims the benefit of U.S. Provisional Patent Application No. 63 / 504,345 filed on May 25, 2023, which are hereby incorporated by reference in their entirety. Technical Field

[0002] The present disclosure relates generally to electronic devices, and more particularly to electronic devices such as head mounted devices. Background Art

[0003] An electronic device such as a head mounted device may have a sensor. For example, the head mounted device may have a camera for capturing images of the environment surrounding the head mounted device. The head mounted device may also include a sensor for tracking the user's hands.

[0004] Designing a sensor for detecting a user's hand can be challenging. In low light conditions, detecting a user's hand can become even more difficult. It is in this context that the embodiments of this document arise. Summary of the invention

[0005] A head mounted device is wearable on a user's head. The head mounted device is operable in a real world environment including illumination from one or more light sources. The head mounted device may include one or more sensors for capturing images from the real world environment and for tracking external objects and / or body parts of the user.

[0006] One aspect of the present disclosure provides a method of operating an electronic device, the method comprising: capturing an image of a scene using one or more image sensors; detecting an external object using one or more external object sensors; determining whether the external object is sufficiently illuminated based on information obtained from the one or more external object sensors; and activating one or more supplemental emitters to illuminate the external object in response to determining that the external object is not sufficiently illuminated. In response to determining that the external object is sufficiently illuminated, the one or more supplemental emitters may be deactivated. The method may also include activating the one or more supplemental emitters to illuminate the scene and the external object in response to determining that the captured image of the scene is not sufficiently bright. The method may include: calculating a scene average for the captured image of the scene; and comparing the calculated scene average with a low-light scene target. The method may include: calculating a first average associated with a first external object (e.g., a dark hand average); calculating a second average associated with a second external object (e.g., a bright hand average); and comparing the first average with an external object target.

[0007] The method may also include: determining whether the one or more external object sensors are currently detecting an external object; and in response to determining that the one or more external object sensors are not currently detecting an external object, gradually increasing the output intensity of the one or more supplemental emitters to converge the scene average toward the low-light scene target. The method may include calculating an external object average based on the first average and the second average. The method may include: adjusting the output intensity of the one or more supplemental emitters to converge the first average to the external object target; adjusting the output intensity of the one or more supplemental emitters to converge the scene average to the low-light scene target; and adjusting the output intensity of the one or more supplemental emitters to limit the second average to an external object clipping limit. The method may include deactivating the one or more supplemental emitters in response to determining that the second average is above the external object clipping limit when the output intensity of the one or more supplemental emitters is at a minimum level, or in response to determining that the scene average is above the scene clipping limit when the output intensity of the one or more supplemental emitters is at the minimum level.

[0008] One aspect of the present disclosure provides a method of operating a head-mounted device having one or more image sensors and a hand tracker. The method may include: capturing an image of a scene using the one or more image sensors; detecting a hand using the hand tracker; activating one or more supplemental emitters to illuminate the hand; detecting movement of the hand; and adjusting (reducing) the output intensity of the one or more supplemental emitters based on the detected movement of the hand. The method may include: calculating an average brightness value of the detected hand; and in response to determining that the calculated average brightness value is increasing at a rate greater than a threshold level, reducing the output intensity of the one or more supplemental emitters. The method may include: periodically deactivating the one or more supplemental emitters; and when the one or more supplemental emitters are deactivated, determining whether the scene average of the captured image is less than a low light scene target and determining whether the hand average of the detected hand is less than a dark hand target.

[0009] One aspect of the present disclosure provides a head-mounted device, the head-mounted device comprising: a support structure configured to mount the head-mounted device on a user's head; one or more scene cameras configured to capture an image of a scene; one or more hand cameras configured to detect a user's hand; one or more supplemental emitters configured to provide supplemental illumination for the user's hand; and a control circuit configured to: determine whether the user's hand is adequately illuminated; determine whether the captured image is properly exposed; and adjust the one or more supplemental emitters based on whether the user's hand is adequately illuminated and based on whether the captured image is properly exposed. The control circuit can be configured to determine whether the user's hand is adequately illuminated by the following steps: obtaining a dark hand average associated with a darker hand of the user's hand; obtaining a bright hand average associated with a brighter hand of the user's hand; and comparing the dark hand average with a predetermined dark hand target. The control circuit can be configured to determine whether the captured image is properly exposed by the following steps: obtaining a scene average of the captured image; and comparing the scene average with a predetermined low-light scene target. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 is a side view of an illustrative electronic device such as a head-mounted device according to some embodiments.

[0011] Figure 2 is a schematic diagram of an illustrative system with an electronic device according to some embodiments.

[0012] Figure 3 is a side view of an illustrative electronic device with multiple cameras having different fields of view according to some embodiments.

[0013] Figure 4 is a top view of an illustrative electronic device with multiple cameras having different fields of view according to some embodiments.

[0014] Figure 5 is a block diagram of an illustrative electronic device with a scene camera, a hand camera, and a supplemental transmitter for hand tracking according to some embodiments.

[0015] Figures 6 to 8 According to some embodiments, Figures 1 to 5 A flow chart of illustrative steps for an electronic device of the type shown. DETAILED DESCRIPTION

[0016] Figure 1is a side view of an illustrative head mounted electronic device of the type that may be provided with sensor circuitry for tracking one or more body parts of a user. Figure 1 As shown, head mounted device 10 may include head mounted support structure 26. Support structure 26 may have walls or other structures that separate an interior region of device 10, such as interior region 42, from an exterior region, such as exterior region 44, surrounding device 10. Electrical components 40 (e.g., integrated circuits, sensors, control circuits, light emitting diodes, lasers and other light emitting devices, other control circuits and input-output devices, etc.) may be mounted on printed circuits and / or other structures within device 10 (e.g., in interior region 42).

[0017] In order to present an image to the user for viewing from an eye zone such as eye zone 34, device 10 may include one or more displays such as display 14 and lenses such as lens 38. These components may be mounted in an optical module such as optical module 36 (e.g., a lens barrel) to form corresponding left and right optical systems. For example, there may be a left display 14 for presenting an image to the user's left eye through a left lens in the left eye zone and a right display 14 for presenting an image to the user's right eye in the right eye zone. When structure 26 is against the outer surface of the user's face (facial surface 30), the user's eye is located in eye zone 34 at the rear side R of device 10.

[0018] Support structure 26 may include a main support structure, such as main housing portion 26M (sometimes referred to as a main portion). Main housing portion 26M may form front side F of device 10. On rear side R, main housing portion 26M may have a cushioning structure to improve user comfort when portion 26M abuts facial surface 30. If desired, support structure 26 may include an optional headband, such as band 26B, and / or other structure that allows device 10 to be worn on a user's head.

[0019] exist Figure 2 A schematic diagram of an exemplary system that may include a head mounted device is shown in FIG. Figure 2 As shown, the system 8 may have one or more electronic devices 10. The device 10 may include a head mounted device (e.g., Figure 1 device 10), accessories such as headphones and watches, computing equipment (e.g., cellular phones, tablets, laptops, desktop computers, and / or remote computing equipment that supplies content to the head-mounted device), smart home devices, and / or other devices that communicate with each other.

[0020] Each electronic device 10 may have a control circuit 12. The control circuit 12 may include storage devices and processing circuits for controlling the operation of the device 10. The control circuit 12 may include storage devices, such as hard disk drive storage devices, non-volatile memory (e.g., electrically programmable read-only memory configured to form a solid-state drive), volatile memory (e.g., static or dynamic random access memory), etc. The processing circuits in the control circuit 12 may be based on one or more microprocessors, microcontrollers, digital signal processors, baseband processors, power management units, audio chips, graphics processing units, application-specific integrated circuits, and other integrated circuits. Software code may be stored on a storage device in the circuit 12 and run on a processing circuit in the circuit 12 to implement control operations for the device 10 (e.g., data acquisition operations, operations involving the use of control signals to regulate components of the device 10, etc.). The control circuit 12 may include wired and wireless communication circuits. For example, the control circuit 12 may include radio frequency transceiver circuits, such as cellular telephone transceiver circuits, wireless local area network transceiver circuits (e.g., circuits), millimeter wave transceiver circuits, and / or other wireless communication circuits.

[0021] During operation, the communication circuitry of devices in system 8 (e.g., the communication circuitry of control circuitry 12 of device 10) may be used to support communication between electronic devices. For example, one electronic device may transmit video data, audio data, and / or other data to another electronic device in system 8. Electronic devices in system 8 may use wired and / or wireless communication circuitry to communicate over one or more communication networks (e.g., the Internet, a local area network, etc.). The communication circuitry may be used to allow device 10 to receive data from and / or provide data to external equipment (e.g., a tethered computer, a portable device such as a handheld device or laptop computer, an online computing device such as a remote server or other remote computing equipment, or other electrical equipment).

[0022] Each device 10 in system 8 may include input-output devices 22. Input-output devices 22 may be used to allow a user to provide user input to device 10. Input-output devices 22 may also be used to gather information about the environment in which device 10 operates. Output components in circuitry 22 may allow device 10 to provide output to a user, and may be used to communicate with external electrical equipment.

[0023] like Figure 2As shown, the input-output device 22 may include one or more displays such as the display 14. In some configurations, the device 10 includes a left display device and a right display device. The device 10 may include, for example, left and right components such as a left scanning mirror display device and a right scanning mirror display device or other image projectors, a liquid crystal on silicon display device, a digital mirror device, or other reflective display devices; left and right display panels based on an array of light-emitting diode pixels (e.g., an organic light-emitting display with a polymer or semiconductor substrate or a display device based on an array of pixels formed by a crystalline semiconductor light-emitting diode die); a liquid crystal display panel; and / or other left and right display devices that provide images to the left and right eye zones for viewing by the user's left and right eyes, respectively.

[0024] During operation, display 14 may be used to display visual content for a user of device 10. The content presented on display 14 may include virtual objects and other content provided to display 14 by control circuitry 12. This virtual content may sometimes be referred to as computer-generated content. The computer-generated content may be displayed in the absence of real-world content, or may be combined with real-world content. In some configurations, a real-world image may be captured by a camera (e.g., a forward-facing camera, sometimes referred to as a front-facing camera) such that the computer-generated content may be electronically overlaid on a portion of the real-world image (e.g., when device 10 is a pair of virtual reality goggles).

[0025] The input-output circuit 22 may include a sensor 16. The sensor 16 may include, for example, a three-dimensional sensor (e.g., a three-dimensional image sensor such as a structured light sensor that emits a light beam and uses a two-dimensional digital image sensor to collect image data for a three-dimensional image from light points generated when a target is illuminated by the light beam, a binocular three-dimensional image sensor that uses two or more cameras in a binocular imaging arrangement to collect a three-dimensional image, a three-dimensional lidar (light detection and ranging) sensor, a three-dimensional radio frequency sensor, or other sensors that collect three-dimensional image data), a camera (e.g., an infrared and / or visible light digital image sensor), a gaze tracking sensor (e.g., a gaze tracking system based on an image sensor and, if necessary, a light source that emits one or more light beams that are directed at the user's eyes when the user is viewing the user's eyes). The device 10 may include a plurality of sensors that are capable of detecting the movement of a display 14 and / or a plurality of sensors that are capable of detecting the movement of a display 14. The device 10 may include a plurality of sensors that are capable of detecting the movement of a display 14 and / or a plurality of sensors that are capable of detecting the movement of a display 14. The device 10 may include a plurality of sensors that are capable of detecting the movement of a display 14 and / or a plurality of sensors that are capable of detecting the movement of a display 14. The device 10 may include a plurality of sensors that are capable of detecting the movement of a display 14 and / or a plurality of sensors that are capable of detecting the movement of a display 14.

[0026] User input and other information may be collected using sensors and other input devices in input-output devices 22. If desired, input-output devices 22 may include other devices 24 such as tactile output devices (e.g., vibrating components), light emitting diodes, lasers and other light sources, speakers such as ear speakers for producing audio output, circuits for receiving wireless power, circuits for wirelessly transmitting power to other devices, batteries and other energy storage devices (e.g., capacitors), joysticks, buttons, and / or other components.

[0027] Electronic device 10 may have a head-mounted support structure such as head-mounted support structure 26 (e.g., a head-mounted housing structure such as a housing wall, a band, etc.). The head-mounted support structure may be configured to be worn on a user's head (e.g., against the user's face, thereby covering the user's eyes) during operation of device 10, and may support display 14, sensor 16, other components 24, other input-output devices 22, and control circuitry 12 (see, e.g., FIG. 1 ). Figure 1 component 40 and optical module 36).

[0028] Figure 3is a cross-sectional side view of a head mounted device 10 in an illustrative configuration, wherein the device includes different types of sensors for monitoring the environment surrounding the device 10. Figure 3 The head mounted device 10 may have optical sensors. These sensors may include a camera. The camera of the device 10 may have a lens and an image sensor configured to capture images at ultraviolet wavelengths, visible wavelengths, and / or infrared wavelengths.

[0029] like Figure 3 As shown, the right hand side of the device 10 may include a first camera such as camera 50, which is used to capture images of the user's environment displayed in real time on the display 14 (e.g., capture real-time pass-through video). The left hand side of the device 10 may have a corresponding camera 50 oriented in the same manner. The viewing angles of the cameras 50 on the left and right sides may be configured to overlap in front of the device 10 so that there is no gap in the coverage in front of the user. If desired, the cameras 50 on the left and right sides of the device 10 may be replaced by a single camera or any number of cameras. The camera 50 used to capture real-time video displayed to the user as a pass-through video is sometimes referred to as a scene camera or a pass-through camera. The pass-through video obtained and presented to the user using the camera 50 is sometimes referred to as a pass-through video feed or a pass-through feed.

[0030] The device 10 may also include other cameras that can be used to track the position and movement of external objects. As an example, a tracking camera can track the user's hands or the user's torso or other body parts. As an example, gesture input can be used to control the operation of the device 10. Body part monitoring can be used to allow the user's body movements to be replicated by the content displayed in the virtual environment. If necessary, the device 10 may also include a camera for tracking the position of an external accessory (e.g., the position and movement of a controller moved by the user to control the device 10). In some scenarios, a visual inertial odometry (VIO) system or other systems that determine the position, movement and / or orientation of the device 10 relative to the surrounding environment of the device 10 can be formed by combining data from one or more cameras in the device 10 with additional sensor data (e.g., data from an inertial measurement unit). The camera can perform dedicated functions (tracking, visual inertial odometry functions, scene capture, ranging, three-dimensional image capture for facial recognition and environmental mapping, etc.), or two or more of these operations can be performed by a shared camera.

[0031] Still refer to Figure 3, the right-hand side of device 10 may also include a second camera 52 and a third camera 54. Camera 52 may be a forward-facing (front-facing) camera having a wider field of view than pass-through camera 50. For example, pass-through camera 50 may have a first field of view defined by angle A1 in the YZ plane, while camera 52 may have a second field of view defined by angle A2 in the YZ plane, which is greater than angle A1. Camera 54 may be a downward-facing camera, also having a relatively wide field of view. For example, camera 54 may have a third field of view defined by angle A3 in the YZ plane, which is also greater than angle A1. The left-hand side of device 10 may have corresponding cameras 52 and 54 oriented in the same manner. The viewing angles of cameras 52 on the left and right sides may be configured to overlap in front of device 10 so that there is no gap in the coverage in front of the user. The viewing angles of cameras 54 on the left and right sides may be configured to overlap below device 10 so that there is no gap in the coverage below device 10 where the user's hands and torso may be located when operating device 10. If desired, cameras 52 and 54 on the left and right sides of device 10 may be replaced by a single camera or any number of cameras.

[0032] Cameras 52 and 54 may be used to track the position and movement of external objects (e.g., to track a user's hands, torso, or other body parts, or to track external accessories), and thus may sometimes be referred to as external object tracking cameras or object tracking sensors. Cameras 52 and 54 may be used in conjunction with additional inertial measurement units (e.g., accelerometers, gyroscopes, gyrocompasses, magnetometers, etc.) as part of a visual inertial odometry (VIO) system for determining the position, movement, and / or orientation of device 10 relative to the surrounding environment, and thus may sometimes be referred to as orientation tracking cameras, orientation sensors, or motion sensors.

[0033] External object tracking sensors such as (e.g., cameras 52 and / or 54) can track external objects under normal to bright lighting conditions. However, in low light conditions, the accuracy and performance of such external object tracking sensors may be reduced. According to some embodiments, the device 10 is provided with supplemental emitters such as one or more supplemental emitters 60, which are configured to emit supplemental light to illuminate the external object to be tracked. The supplemental emitter 60 can emit signals at frequencies outside the visible spectrum so that the pass-through video captured from the scene camera and displayed to the user is substantially unchanged, but still within the detection capabilities of the object tracking sensor. The supplemental emitter 60 can be an infrared (IR) emitter component that is configured to emit infrared light (e.g., light with a wavelength in the range of 700 nanometers to approximately 1000 nanometers). In such an arrangement in which the supplemental emitter 60 is implemented as an IR emitter device, the external object tracking sensor can also be configured to detect infrared light.

[0034] This example in which the supplemental emitter 60 is configured to output infrared light is illustrative. In other embodiments, the supplemental emitter 60 may be configured to emit ultraviolet light or other invisible light. The left hand side of the device 10 may have a corresponding emitter 60 oriented in the same manner. The output coverage of the emitters 60 on the left and right sides may be configured to overlap in front of the device 10 so that there is no gap in the coverage in front of the user (for example, the coverage of the supplemental emitters 60 on the left and right sides may be configured to overlap in front of and below the device 10 so that there is no gap in the coverage below the device 10 where the user's hands and torso may be located when operating the device 10). If desired, the emitters 60 on the left and right sides of the device 10 may be replaced by a single emitter component or any number of emitters.

[0035] Figure 4 FIG. 1 is a top view of an exemplary electronic device 10, showing the left and right sides of the device. Figure 4 As shown, the right hand side of device 10 may include a pass-through (scene) camera 50, while the left hand side of device 10 may include another pass-through (scene) camera 50. Both pass-through cameras 50 on the left and right sides of device 10 are forward-facing cameras with the same or similar orientation. The viewing angles of the cameras 50 on the left and right sides (see, for example, the field of view marked by angle A1) can be configured to overlap in front of device 10 so that there is no gap in the coverage in front of the user. If desired, the cameras 50 on the left and right sides of device 10 can be replaced by a single camera or any number of outward-facing cameras.

[0036] The right hand side of device 10 may include an object / orientation tracking camera 52, while the left hand side of device 10 may include another object / orientation tracking camera 52. Both tracking cameras 52 on the left and right sides of device 10 are forward-facing cameras with the same or similar orientation. The viewing angles of cameras 52 on the left and right sides (see, for example, the field of view marked by angle A2) may be configured to overlap in front of device 10 so that there is no gap in coverage in front of the user. If desired, the cameras 52 on the left and right sides of device 10 may be replaced by a single camera or any number of outward-facing cameras.

[0037] In addition to cameras 50, 52 and 54 ( Figure 4In addition to the side-facing camera 56 (not shown in the figure to avoid complicating the drawing), device 10 may also include a side-facing camera 56. Side-facing camera 56 can be used to track the position and movement of external objects (e.g., to track a user's hands, torso, or other body parts, or to track external accessories), and thus may sometimes be referred to as an external object tracking camera or object tracking sensor. Side-facing camera 56 may be used in conjunction with an additional inertial measurement unit (e.g., accelerometer, gyroscope, etc.) as part of a visual inertial odometry (VIO) system for determining the position, movement, and / or orientation of device 10 relative to the surrounding environment, and thus may sometimes be referred to as an orientation tracking camera or orientation sensor.

[0038] like Figure 4 As shown, the right hand side of device 10 may include an object / orientation tracking camera 56, while the left hand side of device 10 may include another object / orientation tracking camera 56. Both tracking cameras 56 on the left and right sides of device 10 are side-facing cameras that point to the right hand side and left hand side of device 10, respectively. Side-facing cameras 56 may have a relatively wide field of view. For example, each of cameras 56 may have a field of view defined by an angle A4 in the XY plane that is greater than the field of view angle A1 associated with pass-through camera 50.

[0039] Still refer to Figure 4 , the right hand side of device 10 may include a first supplemental emitter 60, and the left hand side of device 10 may include a second supplemental emitter 60. Both emitters 60 on the left and right sides of device 10 are forward-facing emitters with the same or similar orientation. The output coverage of emitters 60 may be configured to overlap in front of and below device 10 so that there is no gap in the coverage in front of the user (e.g., the overlapping output coverage ensures that more than 90% of hand motions or hand positions can be correctly illuminated by the left and right emitters 60 together). If desired, the emitters 60 on the left and right sides of device 10 may be replaced by a single emitter or any number of outward-facing emitters.

[0040] Figure 5 is a block diagram of an exemplary electronic device 10 having different types of sensors and associated complementary transmitters. Figure 5As shown, the device may include one or more scene cameras 50 for capturing images of the external real-world environment around the device 10. The video feed output from the scene camera 50 may sometimes be referred to as a raw video feed or a real-time transparent video stream. The transparent feed output from the camera 50 may be processed by an image signal processor (ISP) 70 configured to perform image signal processing functions that rely solely on the input of the real-time camera feed itself. For example, the ISP block 70 may be configured to perform automatic exposure for controlling the exposure setting of the transparent video feed, automatic color correction for controlling white balance (sometimes referred to as automatic white balance), tone mapping, gamma correction, shading correction, noise reduction, black level adjustment, demosaicing, image sharpening, high dynamic range (HDR) correction, color space conversion and / or other image signal processing functions (to name a few) to output corresponding processed video frames.

[0041] Device 10 may also include a hand tracker 80 (sometimes referred to as a hand tracking subsystem, a hand tracking sensor, a gesture tracking subsystem, a gesture tracking sensor, etc.) configured to collect hand positions, gesture information, and / or other hand-related information. Hand tracker 80 may be one of sensors 16 (see Figure 2 ). Hand tracker 80 may include one or more cameras 82 and / or other gesture tracking system components (e.g., outward-facing components and / or light sources that emit light beams so that reflections of the light beams from the user's hands can be detected) to monitor the user's hands. Camera 82 within hand tracker 80 may be referred to and defined herein as hand camera 82, and may include cameras 52, 54, and / or 56 (see, e.g., Figure 3 and Figure 4 ). The hand camera 82 may be pointed at a user's hand and may track motion associated with the user's hand, may determine whether the user is performing a swiping motion with his / her hand, may determine whether the user is performing a contactless button press or object selection operation with his / her hand, may determine whether the user is performing a grasping or gripping motion with his / her hand, may determine whether the user is using his / her hand or fingers to point to a given object presented on the display 14, may determine whether the user is performing a swiping or striking motion with his / her hand, or may generally measure / monitor three-dimensional contactless gestures (“air gestures”) associated with the user's hand.

[0042] An image statistics collection subsystem such as an image statistics collection block 90 may be used to analyze the processed images or video frames output from the image signal processor 70. The image statistics collection block 90 may analyze the processed images and obtain information related to the following items: automatic exposure (AE) settings, automatic focus (AF) settings, automatic white balance (AWB) settings, brightness settings, black level compensation settings, brightness histograms (e.g., a graphical representation of the distribution of tones in a digital image), color histograms (e.g., a graphical representation of the distribution of colors in a digital image), sharpness maps, flicker detection information, object detection, spatial (depth) statistics, thumbnail information, illumination of the environment, image configuration, and / or other statistical information related to the captured image. The information collected by the image statistics collection block 90 may be collectively referred to as image statistics. According to one embodiment, the image statistics collection block 90 may obtain a scene average of the image and other image statistics. "Scene average" may be referred to and defined herein as the average brightness value of the captured scene image (e.g., the overall sensor average of the captured scene, possibly with a center-weighted spatial map as an example).

[0043] The image statistics collection block 90 may also be used to analyze the hand information output from the hand tracker 80. The image statistics collection block 90 may analyze the hand information and may obtain hand averages and other external object tracking statistics. In a typical scenario, the hand tracker 80 should be able to detect information associated with both hands of the user. It is rare for the left and right hands to be illuminated in the same way. Typically, one hand of the user will be brighter or darker than the other hand. The image statistics collection block 90 may identify a first hand average corresponding to the average brightness value associated with the darker hand, and may also identify a second hand average corresponding to the average brightness value associated with the brighter hand. Therefore, the first hand average is sometimes referred to and defined herein as the dark (darker) hand average, and may refer to the left hand or the right hand. Similarly, the second hand average is therefore sometimes referred to and defined herein as the bright (brighter) hand average, and may refer to the left hand or the right hand. The combined brightness average of the two hands may be referred to as a "hand average" (e.g., the hand average is equal to the average of the dark hand average and the bright hand average). These hand averages may be calculated first for each camera 82. The hand averages from each camera 82 may then be combined to obtain a dark hand average, a light hand average, and a hand average.

[0044] Image statistics collection block 90 may also output at least the scene average and the hand average (along with the dark hand average and the light hand average) and other statistics to a control circuit such as controller 92. Controller 92 may be configured to control emitter drive circuitry such as emitter driver 94 based on information received from image statistics collection block 90. ​​Thus, controller 92 is sometimes referred to as an emitter controller. Controller 92 may be considered Figure 295. The emitter driver 94 may be configured to output a series of pulses for driving one or more supplemental emitters 60. The controller 92 may adjust the output intensity of the emitter 60 by tuning the output pulses 95. For example, the controller 92 may direct the emitter driver 94 to increase the illumination provided by the emitter 60 by increasing the pulse width of the signal 95, by increasing the output drive current of the driver 94, by increasing the frequency of the signal 95, and / or by other means of increasing the output intensity of the emitter 60. Therefore, the signal 95 is sometimes referred to as a pulse width modulated signal or a gating signal. Conversely, the controller 92 may direct the emitter driver 94 to reduce the illumination provided by the emitter 60 by reducing the pulse width of the signal 95, by reducing the output drive current of the driver 94, by reducing the frequency of the signal 95, and / or by other means of reducing the output intensity of the emitter 60.

[0045] Upon receiving a pulse signal 95 from an emitter driver 94, the emitter 60 may output light 96 (e.g., infrared light) for illuminating one or more external objects such as external object 88. Object 88 may represent a user's hand, a user's torso, other user body parts, and / or other external objects associated or unassociated with the user. Light 96 emitted from the supplemental emitter 60 may bounce off the external object 88 as reflected light 98, which may then be received and sensed by the hand camera 82 and / or other external object tracking sensors within the device 10.

[0046] Care should be taken when controlling the supplemental emitter 60 to optimize sufficient hand detection capabilities while not consuming too much power. For example, the supplemental emitter 60 may be turned off (e.g., by deactivating the emitter driver 94) in bright (brighter) light conditions when the tracked external object is adequately illuminated. The emitter 60 may be turned on (e.g., by activating the emitter driver 94 to provide supplemental illumination for the tracked external object) in darker, dim, or low-light conditions where the tracked external object is not adequately illuminated by the existing ambient light within the environment.

[0047] Figure 6 Is used for operation Figures 1 to 5 1 is a flow chart of illustrative steps of an electronic device 10 of the type shown. During operation of block 100, a control circuit (e.g., Figure 5 Controller 92 or Figure 2 The control circuit 12 of the hand tracker 80 may be used to determine the current sensor gain of the hand tracker 80 (e.g., to determine the current sensor gain of the hand camera 82). During the operation of block 102, the control circuit may determine whether the current sensor gain of the hand camera 82 is less than the maximum sensor gain level. The current sensor gains of the various hand cameras 82 may be the same or may be different. Sometimes, a device configuration in which the sensor gains of all hand cameras 82 are the same is described herein as an example.

[0048] If the current sensor gain of the hand camera is indeed less than the maximum sensor gain level, the process may proceed to block 106 to deactivate the emitter driver 94 (e.g., to turn off the supplemental emitter 60), because the hand camera may still increase the sensor gain to improve the brightness level of the hand capture if necessary. However, if the current sensor gain of the hand camera is equal to the maximum sensor gain level (i.e., the sensor gain of the hand camera is at a maximum), the process may proceed to block 104.

[0049] During operation of block 104, logic in the control circuitry may check whether the captured scene is too dark or whether the darker hand is too dark. The control circuitry may receive a scene average and a dark hand average from the image statistics collection block 90. ​​The control circuitry may determine whether the captured scene is too dark by comparing the received scene average to a low light scene target. A "low light scene target" may be referred to and defined herein as a scene brightness threshold level, wherein a scene average below the threshold / target is considered too dark for proper external object sensing, and wherein a scene average above the threshold / target is considered sufficiently bright for proper external object sensing. The threshold level may be based on a minimum requirement of one or more downstream algorithms. The threshold may be pre-set or may be adjusted during operation of the device 10.

[0050] Logic in the control circuit can determine whether the darker hand is too dark by comparing the dark hand average to the dark hand target. The "dark hand target" may be referred to and defined herein as a predetermined dark hand brightness threshold level, wherein the dark hand average below the threshold / target is considered too dark for proper hand detection, and wherein the dark hand average above the threshold / target is considered sufficiently bright for proper hand detection. The dark hand target is sometimes referred to as an external object target or threshold. In an illustrative scenario in which the brightness range is represented using a 10-bit digital code ranging from 0 to 1023, both the low light scene target and the dark hand target can be set to 80 (as an example). This is merely illustrative. As another example, the low light scene target may be smaller than the dark hand target. As another example, the low light scene target may be larger than the dark hand target. If desired, any suitable number of bits may be used to encode the brightness range.

[0051] In response to determining that the scene average is less than the low-light scene target or in response to determining that the dark hand average is less than the dark hand target, processing may proceed to block 108. During operation at block 108, an emitter driver may be activated to turn on supplemental emitter 60. Once turned on, supplemental emitter 60 may provide additional illumination to help brighten the scene and / or hand being captured using various sensors in device 10.

[0052] During the operation of block 110, logic in the control circuit may check whether one or more hands are currently detected. For example, the control circuit may communicate with the hand tracker 80, may communicate with the image statistics collection block 90, may analyze statistics collected by block 90, or may analyze hand information output from the hand tracker 80 to determine whether the hand tracker is currently detecting any hands. If the control circuit determines that the hand tracker is not currently detecting any hands, the control circuit may direct the emitter driver to slowly increase the pulse intensity from a minimum level until the scene average converges toward a low light scene target (see the operation of block 112). The pulse intensity of the emitter driver may be increased by extending the pulse width, by increasing the drive current, by increasing the drive voltage, etc. Increasing the pulse intensity in this manner increases the overall brightness or intensity level of the light emitted from the supplemental emitter 60.

[0053] If the control circuit determines that the hand tracker has detected at least one hand, processing may then proceed to block 114. During operation of block 114, logic in the control circuit may monitor the hand average (e.g., a combined hand average based on the dark hand average and the bright hand average) and adjust (e.g., reduce) the emitter output intensity to converge the dark hand average to the dark hand target, converge the scene average to the low light scene target, and limit the bright hand average to the hand clipping limit. The control circuit may also receive the bright hand average from the image statistics collection block 90. ​​The "hand clipping limit" may be referred to and defined herein as a predetermined bright hand brightness threshold level, where bright hand averages above the threshold / target are considered too bright or overexposed using a clipped brightness value, and where bright hand averages below the threshold / target are considered not too bright for proper hand detection. In an illustrative scenario where the brightness range is represented using a 10-bit digital code ranging from 0 to 1023, the hand clipping limit may be set to 250 (as an example). The hand clipping limit is sometimes referred to as a bright hand clipping limit or an external object clipping limit.

[0054] The above example in which the transmitter output (pulse) intensity is reduced to help converge the dark hand average to the dark hand target is illustrative. Reducing the transmitter output intensity can reduce the brightness of the background, which may reduce the overall signal-to-noise ratio (SNR) of the captured scene in some scenes. To help maintain or improve the SNR of the background, the sensor gain (e.g., the analog gain of the hand tracking sensor) can optionally be reduced first before adjusting the transmitter output intensity to limit the bright hand average to the hand clipping limit. Reducing the analog sensor gain while maintaining the transmitter output intensity can help improve the overall SNR of the captured scene. This example in which the sensor gain is optionally reduced before reducing the transmitter pulse intensity is illustrative. If desired, the sensor gain can be optionally adjusted after first adjusting the transmitter output intensity.

[0055] When the control circuit adjusts the emitter output intensity so that the pulse intensity is at or below the minimum pulse intensity, processing may proceed to block 116. During operation of block 116, when the emitter driver pulse intensity is at the minimum pulse intensity, logic in the control circuit may check whether the bright hand average is above the hand clipping limit, or when the emitter driver pulse intensity is at the minimum pulse intensity, logic in the control circuit may check whether no hand is detected and whether the scene average is above the scene clipping limit. "Scene clipping limit" may be referred to and defined herein as a predetermined bright scene threshold level, wherein scene averages above this threshold / target are considered too bright or overexposed using clipped brightness values, and wherein scene averages below this threshold / target are considered not too bright or overexposed. In an illustrative scene in which the brightness range is represented using a 10-bit digital code ranging from 0 to 1023, the hand clipping limit may be set to 300 (as an example). In general, the scene clipping limit may be set to be greater than the hand clipping limit, equal to the hand clipping limit, or less than the hand clipping limit.

[0056] When the emitter driver pulse intensity is at the minimum pulse intensity, in response to determining that the bright hand average is above the hand clipping limit, processing may proceed to block 106 via path 118 to deactivate the emitter driver (e.g., to turn off supplemental emitter 60) because the brighter hand has been overexposed. When the emitter drive pulse intensity is at the minimum pulse intensity, in response to determining that a hand is not detected and the scene average is above the scene clipping limit, processing may also proceed from block 116 to block 106 via path 118 to deactivate the emitter driver because the scene has been overexposed.

[0057] Otherwise, if neither condition in box 116 is met (e.g., if the bright hand average is less than the hand clipping limit when the emitter driver pulse intensity is at a minimum and if the scene average is less than the scene clipping limit when the emitter driver pulse intensity is at a minimum and no hand is detected), the control circuit may adjust the emitter output intensity to a minimum level at box 120 and processing may loop back to box 114 as indicated by path 122.

[0058] Figure 7 Shown can be optionally with Figure 6150. During operation at block 150, logic in the control circuitry may determine whether any hands are moving toward device 10 or whether the hand averages are increasing rapidly (e.g., in response to determining that the hand averages are increasing at a rate greater than a threshold level). If either scene is detected, the exposure of the detected hands is likely to be overexposed in the near future, so processing may continue to block 152 to rapidly step down the pulse intensity of the emitter driver signal. Stepping down the emitter driver signal pulse intensity in this manner may proactively prevent the hands from being overexposed by the illumination provided from supplemental emitter 60.

[0059] Figure 8 Shown can be optionally with Figure 6 and Figure 7 During the operation of block 160, the emitter driver may be activated to help illuminate or brighten the scene and / or hand. For example, when the Figure 6 This may occur when at least one of the conditions of box 104 in FIG. 1 is met (eg, when the scene average is less than the low-light scene target or when the dark-hand average is less than the dark-hand target).

[0060] Subsequently, during operation at block 162, the emitter driver may be periodically deactivated to reduce power consumption. For example, after deactivating the supplemental emitter, if the scene average may remain above the low light scene target or if the dark hand average may remain above the dark hand target, the control circuit may determine that the supplemental emitter 60 is no longer needed and may remain in the off state to help save power. Operation may then loop back to block 160, as shown by path 164.

[0061] Combination of the above Figures 1 to 8 The described methods and operations may be performed by components of device 10, such as control circuitry 12, using software, firmware, logic components, and / or hardware (e.g., dedicated circuitry or hardware). Software code for performing these operations may be stored on a non-transitory computer-readable storage medium (e.g., a tangible computer-readable storage medium) stored on one or more of the components of device 10 (e.g., a computer-readable storage medium). Figure 1The software code may sometimes be referred to as software, data, instructions, program instructions, or code. Non-transitory computer-readable storage media may include a drive, non-volatile memory such as non-volatile random access memory (NVRAM), a removable flash drive or other removable media, other types of random access memory, etc. The software stored on the non-transitory computer-readable storage medium may be executed by processing circuitry on one or more of the components of device 10 (e.g., one or more processors in control circuitry 12). The processing circuitry may include a microprocessor, an application processor, a digital signal processor, a central processing unit (CPU), an application-specific integrated circuit having a processing circuit, or other processing circuitry.

[0062] The present disclosure recognizes that the use of such personal information in the disclosed technology can be used to benefit the user. For example, the personal information data can be used to deliver targeted content that the user is more interested in. Therefore, the use of such personal information data enables the user to have control over the delivered content. In addition, the present disclosure also anticipates other uses of personal information data that benefit the user. For example, health and fitness data can be used to provide insights into the user's overall health, the user's anatomy (e.g., whether the user has any tattoos on his hands), or can be used as positive feedback to individuals who use technology to pursue health goals.

[0063] A physical environment refers to the physical world that people can sense and / or interact with without the help of electronic devices. In contrast, an extended reality (XR) environment refers to a fully or partially simulated environment that people sense and / or interact with via electronic devices. For example, an XR environment may include augmented reality (AR) content, mixed reality (MR) content, virtual reality (VR) content, etc. In the case of an XR system, a subset of a person's physical movements, or a representation thereof, is tracked, and in response, one or more characteristics of one or more virtual objects simulated in the XR environment are adjusted in a manner that complies with at least one law of physics.

[0064] There are many different types of electronic systems that can enable a person to sense and / or interact with various XR environments. Examples include head-mounted systems, projection-based systems, heads-up displays (HUDs), vehicle windshields with integrated display capabilities, windows with integrated display capabilities, displays formed as lenses designed to be placed over a person's eyes (e.g., similar to contact lenses), headphones / earpieces, speaker arrays, input systems (e.g., wearable or handheld controllers with or without tactile feedback), smartphones, tablet computers, and desktop / laptop computers.

[0065] According to one embodiment, a method for operating an electronic device is provided, the method comprising: capturing an image of a scene using one or more image sensors; detecting an external object using one or more external object sensors; determining whether the external object is adequately illuminated based on information obtained from the one or more external object sensors; and in response to determining that the external object is not adequately illuminated, activating one or more supplemental emitters to illuminate the external object.

[0066] According to another embodiment, a method is provided that includes deactivating the one or more supplemental emitters in response to determining that the external object is sufficiently illuminated.

[0067] According to another embodiment, the method includes: determining whether the captured image of the scene is sufficiently bright; and in response to determining that the captured image of the scene is not sufficiently bright, activating the one or more supplemental emitters to illuminate the scene and the external object.

[0068] According to another embodiment, the method includes deactivating the one or more supplemental emitters in response to determining that the captured image of the scene is sufficiently bright.

[0069] According to another embodiment, the method includes calculating a scene average for the captured image of the scene, and determining whether the captured image of the scene is sufficiently bright includes comparing the calculated scene average to a low-light scene target.

[0070] According to another embodiment, the method includes: calculating a first average value associated with a first external object; and calculating a second average value associated with a second external object different from the first external object, the first average value being less than the second average value.

[0071] According to another embodiment, determining whether the external object is adequately illuminated includes comparing the calculated first average to an external object target.

[0072] According to another embodiment, the method includes: determining whether the one or more external object sensors currently detect an external object; and in response to determining that the one or more external object sensors currently do not detect an external object, gradually increasing the output intensity of the one or more supplemental emitters to converge the scene average toward the low-light scene target.

[0073] According to another embodiment, the method includes, in response to determining that the one or more external object sensors are currently detecting one or more external objects, calculating an external object average based on the first average and the second average.

[0074] According to another embodiment, the method includes adjusting the output intensity of the one or more supplemental emitters to converge the first average to the external object target.

[0075] According to another embodiment, the method includes adjusting the output intensity of the one or more supplemental emitters to converge the scene average to the low-light scene target.

[0076] According to another embodiment, the method includes adjusting the output intensity of the one or more supplemental transmitters to limit the second average value to an external object clipping limit.

[0077] According to another embodiment, the method includes: when the output intensity of the one or more supplementary transmitters is at a minimum level, determining whether the second average value is above the external object clipping limit; and when the output intensity of the one or more supplementary transmitters is at the minimum level, in response to determining that the second average value is above the external object clipping limit, deactivating the one or more supplementary transmitters.

[0078] According to another embodiment, the method includes limiting the second average value to the external object clipping limit by adjusting a gain of the one or more external object sensors prior to adjusting the output intensity of the one or more supplemental transmitters.

[0079] According to another embodiment, the method includes: when the output intensity of the one or more supplementary transmitters is at a minimum level, determining whether the scene average is above the scene clipping limit; and when the output intensity of the one or more supplementary transmitters is at the minimum level, in response to determining that the scene average is above the scene clipping limit, deactivating the one or more supplementary transmitters.

[0080] According to another embodiment, the method includes emitting infrared light using the one or more supplemental emitters to illuminate the external object.

[0081] According to one embodiment, a method of operating a head-mounted device having one or more image sensors and a hand tracker is provided, the method comprising: capturing an image of a scene using the one or more image sensors; detecting a hand using the hand tracker; activating one or more supplemental emitters to illuminate the hand; detecting movement of the hand; and adjusting the output intensity of the one or more supplemental emitters based on the detected movement of the hand.

[0082] According to another embodiment, adjusting the output intensity of the one or more supplemental transmitters based on the detected movement of the hand includes reducing the output intensity of the one or more supplemental transmitters in response to detecting that the hand is moving toward the hand tracker.

[0083] According to another embodiment, the method includes: calculating an average brightness value of the detected hand; and reducing the output intensity of the one or more supplemental emitters in response to determining that the calculated average brightness value is increasing at a rate greater than a threshold level.

[0084] According to another embodiment, the method includes: periodically deactivating the one or more supplemental transmitters; and when the one or more supplemental transmitters are deactivated, determining whether the scene average of the captured image is less than the low-light scene target and determining whether the hand average of the detected hand is less than the dark hand target.

[0085] According to one embodiment, a head-mounted device is provided, which includes: a support structure configured to mount the head-mounted device on a user's head; one or more scene cameras configured to capture images of a scene; one or more hand cameras configured to detect the user's hands; one or more supplemental emitters configured to provide supplemental illumination for the user's hands; and a control circuit configured to determine whether the user's hands are adequately illuminated, determine whether the captured image is correctly exposed, and adjust the one or more supplemental emitters based on whether the user's hands are adequately illuminated and based on whether the captured image is correctly exposed.

[0086] According to another embodiment, the control circuit is configured to determine whether the user's hand is adequately illuminated by: obtaining a dark hand average value associated with a darker one of the user's hands; obtaining a bright hand average value associated with a brighter one of the user's hands; and comparing the dark hand average value with a predetermined dark hand target.

[0087] According to another embodiment, the control circuit is configured to determine whether the captured image is correctly exposed by: obtaining a scene average for the captured image; and comparing the scene average to a predetermined low-light scene target.

[0088] The foregoing is merely illustrative and various modifications may be made to the embodiments described. The foregoing embodiments may be implemented independently or in any combination.

Claims

1. A method for operating an electronic device, the method comprising: capturing an image of the scene using one or more image sensors; detecting external objects using one or more external object sensors; determining whether the external object is adequately illuminated based on information obtained from the one or more external object sensors; as well as In response to determining that the external object is not adequately illuminated, one or more supplemental emitters are activated to illuminate the external object.

2. The method according to claim 1, further comprising: In response to determining that the external object is sufficiently illuminated, the one or more supplemental emitters are deactivated.

3. The method according to claim 1, further comprising: determining whether the captured image of the scene is sufficiently bright; as well as In response to determining that the captured image of the scene is not sufficiently bright, the one or more supplemental emitters are activated to illuminate the scene and the external object.

4. The method according to claim 3, further comprising: In response to determining that the captured image of the scene is sufficiently bright, the one or more supplemental emitters are deactivated.

5. The method according to claim 3, further comprising: A scene average is calculated for the captured image of the scene, wherein determining whether the captured image of the scene is sufficiently bright includes comparing the calculated scene average to a low-light scene target.

6. The method according to claim 5, further comprising: calculating a first average value associated with a first external object; as well as A second average value associated with a second external object different from the first external object is calculated, wherein the first average value is less than the second average value.

7. The method of claim 6, wherein determining whether the external object is sufficiently illuminated comprises comparing the calculated first average with an external object target, the method further comprising: determining whether the one or more external object sensors are currently detecting an external object; as well as In response to determining that the one or more external object sensors are not currently detecting an external object, the output intensity of the one or more supplemental emitters is gradually increased to converge the scene average toward the low-light scene target.

8. The method according to claim 7, further comprising: In response to determining that the one or more external object sensors are currently detecting one or more external objects, an external object average is calculated based on the first average and the second average.

9. The method according to claim 8, further comprising: The output intensities of the one or more supplemental emitters are adjusted to converge the first average to the external object target.

10. The method according to claim 8, further comprising: The output intensities of the one or more supplemental emitters are adjusted to converge the scene average to the low-light scene target.

11. The method according to claim 8, further comprising: The output intensities of the one or more supplemental transmitters are adjusted to limit the second average value to an external object clipping limit.

12. The method according to claim 11, further comprising: determining whether the second average is above the external object clipping limit when the output intensity of the one or more supplemental transmitters is at a minimum level; as well as In response to determining that the second average is above the external object clipping limit when the output intensity of the one or more supplemental transmitters is at the minimum level, the one or more supplemental transmitters are deactivated.

13. The method according to claim 11, further comprising: The second average is limited to the external object clipping limit by adjusting the gain of the one or more external object sensors before adjusting the output intensity of the one or more supplemental transmitters.

14. The method according to claim 8, further comprising: determining whether the scene average is above a scene clipping limit when the output intensity of the one or more supplemental transmitters is at a minimum level; as well as In response to determining that the scene average is above the scene clipping limit when the output intensity of the one or more supplemental transmitters is at the minimum level, the one or more supplemental transmitters are deactivated.

15. A method of operating a head mounted device having one or more image sensors and a hand tracker, the method comprising: capturing an image of a scene using the one or more image sensors; detecting a hand using the hand tracker; activating one or more supplemental emitters to illuminate the hand; detecting movement of the hand; as well as The output intensity of the one or more supplemental emitters is adjusted based on the detected movement of the hand.

16. The method according to claim 15, further comprising: Calculate the average brightness value of the detected hands; as well as In response to determining that the calculated average brightness value is increasing at a rate greater than a threshold level, the output intensity of the one or more supplemental emitters is reduced.

17. The method according to claim 15, further comprising: periodically deactivating the one or more supplemental transmitters; as well as While the one or more supplemental emitters are deactivated, determining whether a scene average of the captured image is less than a low light scene target and determining whether a hand average of the detected hand is less than a dark hand target.

18. A head mounted device, comprising: a support structure configured to mount the head mounted device on a user's head; one or more scene cameras configured to capture images of a scene; one or more hand cameras configured to detect a user's hands; one or more supplemental emitters configured to provide supplemental illumination to the user's hand; and A control circuit, the control circuit being configured to: determining whether the user's hand is adequately illuminated, Determine whether the captured image is correctly exposed, and The one or more supplemental emitters are adjusted based on whether the user's hand is adequately illuminated and based on whether the captured image is correctly exposed.

19. The head mounted device of claim 18, wherein the control circuit is configured to determine whether the user's hand is sufficiently illuminated by: obtaining a dark hand average value associated with a darker hand of the user; Obtaining a bright hand average associated with a brighter hand of the user's hands; and The dark hand average is compared to a predetermined dark hand target.

20. The head mounted device of claim 18, wherein the control circuit is configured to determine whether the captured image is correctly exposed by: obtaining a scene average of the captured images; and The scene average is compared to a predetermined low-light scene target.