Multi-mode camera frame allocation algorithm for computer vision (CV) features in XR head-mounted viewers

Abstract

The present disclosure provides a multi-mode camera frame allocation algorithm for computer vision (CV) features in an XR head-mounted viewer. Disclosed herein are systems and methods relating to an image capture device and one or more processors. An image capture device may capture one or more images. The one or more processors may determine that a parameter (e.g., gain or exposure) of the one or more images satisfies a threshold, and in response to the parameter satisfying the threshold, cause the image capture device to reduce a frame rate at which the one or more images are detected.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to and the benefit of U.S. non-provisional patent application No. 18 / 233,581, filed on August 14, 2023, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present disclosure generally relates to systems and methods for manipulating camera operations. Background Art

[0004] The development of computing devices and communication devices has promoted the development of wearable technology. Wearable devices can integrate various components in a compact form, so that wearable devices can be portable but perform complex processes. For example, a wearable device can be a smart watch that can access content through the network and can control or communicate with other computing devices, etc. Wearable devices can capture images. Summary of the invention

[0005] Various embodiments disclosed herein relate to a system. The system may include an image capture device. The image capture device may capture one or more images, which are presented by a display device. The system may include one or more processors. The one or more processors may determine that a parameter of the one or more images satisfies a threshold value. The one or more processors may cause the image capture device to reduce the frame rate at which the one or more images are captured in response to the parameter satisfying the threshold value. For example, the one or more processors may determine that a parameter satisfies a threshold value (e.g., exceeds a maximum threshold value; falls below a minimum threshold value), which indicates a standard for performing various image processing and / or computer vision operations using the captured image. The one or more processors may adjust the operation of the image capture device (e.g., to adjust the frame rate of the image capture device) based on an evaluation of the parameter and the threshold value, which may allow the system to maintain the standard even when various environmental conditions change or other factors (e.g., system resource usage) change.

[0006] In some embodiments, the parameters of the one or more images may include gain and exposure. The one or more processors may cause the image capture device to reduce the frame rate in response to the function of the gain and exposure exceeding a threshold. In some embodiments, the one or more processors may reduce the frame rate from a first frame rate to a second frame rate in response to the parameter exceeding a first threshold. The one or more processors may increase the frame rate from the second frame rate to the first frame rate in response to the parameter falling below a second threshold, the second threshold being less than the first threshold.

[0007] In some embodiments, the image capture device may be a first image capture device. The system may include a second image capture device. The one or more processors may keep the images captured by the first image capture device and the second image capture device aligned before and after causing the first image capture device to reduce the frame rate. In some embodiments, the system may include a wearable housing coupled to the image capture device, the display device, and the one or more processors. In some embodiments, the second frame rate may be 50 Hz (Hertz) or 60 Hz. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The accompanying drawings are not intended to be drawn to scale. The same reference numerals and names in different drawings represent the same elements. For clarity, not every component is labeled in every drawing.

[0009] Figure 1 is a schematic diagram of a wearable device according to an example embodiment of the present disclosure.

[0010] Figure 2 is a schematic diagram showing a perspective view of a head wearable display according to an example embodiment of the present disclosure.

[0011] Figure 3 is a schematic diagram of a system for manipulating camera operation based on environmental conditions according to an example embodiment of the present disclosure.

[0012] Figure 4 is a schematic diagram of example changes to camera operation based on environmental conditions according to an example implementation of the present disclosure.

[0013] Figure 5 is a flow chart illustrating a method of manipulating camera operation based on environmental conditions.

[0014] Figure 6 is a block diagram illustrating a computing environment of a wearable device according to an example implementation of the present disclosure.

[0015] Fig. 7A is a schematic diagram showing a perspective view of a wearable device including a computing device attached to a cradle according to an example embodiment of the present disclosure.

[0016] Figure 7B is a schematic diagram showing a perspective view of a wearable device including a computing device separated from a cradle according to an example embodiment of the present disclosure.

[0017] Figure 8 is a schematic diagram of a system environment including a wearable device according to an example embodiment of the present disclosure.

[0018] Fig. 9is a schematic diagram of an example wireless communication system according to an example implementation of the present disclosure. DETAILED DESCRIPTION

[0019] Before turning to the drawings that illustrate certain embodiments in detail, it should be understood that the present disclosure is not limited to the details or methods set forth in the specification or illustrated in the drawings. It should also be understood that the terminology used herein is for descriptive purposes only and is not intended to be limiting.

[0020] Disclosed herein are embodiments related to systems and methods for manipulating camera operations based on environmental conditions. An image capture device (e.g., a camera) may be used to perform various computing operations using images captured by the image capture device. For example, the captured images may be processed to perform a variety of operations, such as computer vision operations, head-mounted viewer tracking, hand tracking, controller tracking, object tracking, or various combinations thereof. The images may be processed to generate display data for presentation by a display device. Capturing images at a relatively high frame rate (e.g., a frame rate of 120 Hz) may be useful for various such operations (including but not limited to tracking operations). Typically, environmental conditions may affect the capture of images. For example, low brightness in an environment may result in reduced image quality, such as blurred or noisy images. Low brightness may be particularly problematic when capturing images at a high frame rate (e.g., 120 Hz). This may cause a negative experience for users interacting with the display device, and may also waste the computing resources of the camera and display device by attempting corrections, wasting frames, etc. These problems may be exacerbated if the camera operates at a higher frame rate.

[0021] Systems and methods according to the present disclosure can control camera operation to improve performance when environmental conditions affect image quality. The disclosed systems and methods can mitigate the impact of environmental conditions on image capture quality while performing at a high frame rate. When environmental conditions indicate that they will not affect image quality or user experience, the system can operate the image capture device at a higher (e.g., 120Hz) frame rate. Similarly, if environmental conditions are not suitable for a higher frame rate, the system can operate the image capture device at a lower (e.g., 50Hz or 60Hz) frame rate.

[0022] For displaying high-quality images, computer vision, and image tracking, it may be desirable to operate at a high frame rate. Operation at a higher frame rate may be suitable, for example, (including during certain environmental conditions) to provide sufficient data for a variety of device features or functions (e.g., hand tracking and / or operations that rely on hand tracking data). For example, the data from the captured image that can be used can be related to environmental conditions (e.g., ambient light in the environment); and can be related to how the operating parameters of the camera are implemented, such as gain, exposure (e.g., exposure time that can correspond to the amount of light received by the camera's sensor), or aperture, etc. For example, when the environment is bright enough to meet various criteria (e.g., to ensure that there is little flicker or image blur, etc.), the camera can operate at a high frame rate. Conversely, when the light in the environment is too low to meet the criteria for capturing an image - even when gain and / or exposure are controlled to try to meet the criteria - the device can reduce the frame rate, (e.g., by allowing a longer exposure time to be achieved) allowing more light to be received at the sensor. Thus, the device can control the frame rate to allow various operations, including but not limited to computer vision capabilities, to be performed more efficiently. Additionally, changing the operating parameters of the image capture device can reduce power consumption and wasted computing resources because higher frame rates will not be operational when doing so is impractical.

[0023] Now turning to the attached figure, Figure 1 1 is a schematic diagram of a wearable device (also referred to as a HWD device) 100 according to an example embodiment of the present disclosure. In some embodiments, the wearable device 100 may be the wearable device 810 or the wearable device 850 described herein. In some embodiments, the wearable device 100 includes a computing device 110 and a cradle ( Figure 1 (not shown), computing device 110 can be selectively attached to the cradle. In some embodiments, computing device 110 includes sensor 115, wireless communication interface 125 (also referred to herein as "wireless interface 125"), processor 130, non-transitory computer-readable medium 132, and display 145. These components can operate together to communicate with another device and generate or render content (e.g., artificial reality content). In other embodiments, wearable device 100 includes more than Figure 1 More or fewer parts than shown, or with Figure 1 The components shown in the figure are different components.

[0024] In some embodiments, sensor 115 includes an electronic component, or a combination of electronic and software components, that detects the proximity of a user wearing wearable device 100. For example, sensor 115 may include a Hall sensor that can detect whether a user is close to the sensor (e.g., less than 10 millimeters (mm) from the sensor), or whether a user is touching computing device 110 (e.g., touching and / or blocking a particular component of a device used / configured for wireless communication). Sensor 115 can detect the proximity of a user relative to computing device 110 and can generate sensor measurement data indicating the detected proximity.

[0025] In some embodiments, sensor 115 includes electronic components, or a combination of electronic components and software components, operable to sense / determine / measure the position and / or orientation of computing device 110. Examples of sensor 115 may include: one or more imaging sensors, one or more accelerometers, one or more gyroscopes, one or more magnetometers, one or more sensors of any other suitable type that detect motion and / or position. For example, one or more accelerometers may measure translational motion (e.g., forward / backward, up / down, left / right), and one or more gyroscopes may measure rotational motion (e.g., pitch, yaw, roll). In some embodiments, sensor 115 detects translational motion and rotational motion and determines the orientation and position of computing device 110. Sensor 115 may include one or more eye tracking sensors that detect movement of a user's eyes. Sensor 115 may generate sensor measurements indicating the detected position and orientation of computing device 110.

[0026] In some embodiments, the wireless communication interface 125 includes an electronic component, or a combination of electronic components and software components, that communicates with another device via a wireless communication link (e.g., communication links 814, 816, 818). In some embodiments, the wireless communication interface 125 includes a wireless interface for a cellular communication link (e.g., a 3G, 4G, LTE communication link). The wireless communication interface 125 may also include a wireless interface for a different communication link (e.g., a Wi-Fi or Bluetooth communication link). In some embodiments, the wireless communication interface 125 includes or is embodied as a transceiver or a communication modem coupled to these transceivers, which are used to send and receive data via a wireless medium. The wireless communication interface 125 can send or receive sensor measurement data indicating the position and orientation of the computing device 110. In addition, the wireless communication interface 125 can send or receive image data indicating or corresponding to an image to be rendered.

[0027] In some embodiments, the processor 130 includes electronic components, or a combination of electronic components and software components, that can execute instructions stored by the non-transitory computer-readable medium 132. The processor 130 may include one or more central processing units (CPUs), graphic processing units (GPUs), or a combination thereof. The non-transitory computer-readable medium 132 may store instructions for executing one or more applications that can be executed by the processor 130.

[0028] An example application, when executed by the processor 130, may cause the processor 130 to generate or process content for rendering. The processor 130 executing the application may cause the processor 130 to generate image data for rendering based on sensor measurement data from the sensor 115 or the wireless communication interface 125. For example, the processor 130 executing the application may determine a view of an artificial reality corresponding to the detected position and orientation in the sensor measurement data, and generate image data of the determined view of the artificial reality.

[0029] The example application, when executed by the processor 130, may cause the processor 130 to control or adjust the wireless communication interface 125. The example application includes a wireless interface controller.

[0030] Figure 2 850 according to an example embodiment. In some embodiments, the HWD 850 includes a front rigid body 205 and a strap 210. The front rigid body 205 includes a display 145 ( Figure 2 Not shown), lens ( Figure 2 ), sensor 115, wireless communication interface 125 and processor 130. Figure 2 In the illustrated embodiment, the wireless communication interface 125, processor 130, and sensor 115 are located within the front rigid body 205 and may not be visible to the user. Figure 2 For example, the wireless communication interface 125, the processor 130 and / or the sensor 115 may be located in a different configuration than that shown in FIG. Figure 2 Different positions shown in .

[0031] Figure 3is a schematic diagram of system 300. System 300 may include features of various systems and devices described herein, and / or may be used to implement various systems and devices described herein, including but not limited to wearable devices 100 and 850. System 300 may manipulate camera operation based on environmental conditions. For example, system 300 may include an image capture device 310 (e.g., coupled to HWD device 100) that may operate in two or more operating modes. In some embodiments, system 300 may select an operating mode based on one or more conditions, such as environmental conditions, power usage conditions, power availability conditions, processor and / or memory usage, or various combinations thereof.

[0032] For example, the image capture device 310 can operate in a first mode according to environmental conditions (e.g., sufficient brightness in the environment). The image capture device 310 can operate in a second mode in response to detecting that the brightness in the environment is insufficient, at least until the environmental conditions have met a threshold for returning to the first mode. The system 300 can also time the transition from one mode to another to reduce image flicker seen by the user. Compared to a system that cannot implement this automatic mode switching, the system and method according to the present disclosure can reduce overall image flicker and blur while providing a seamless user experience. This can enable the system 300 to reduce power consumption and waste of computing resources because it may not be necessary to remove blurred images and flicker during post-processing applications.

[0033] The system 300 may include one or more processors 305, one or more image capture devices 310, and / or one or more display devices 315. The processor 305, image capture device 310, and / or display device 315 may be included in, disposed within, or otherwise coupled to a wearable housing, such as the HWD device 100. In some embodiments, the image capture device 310 may capture images that the processor 305 may use to present the images (or use the images to render display data for presentation using the display device 315). The processor 305 may control the operation of the image capture device 310 based at least on the captured images.

[0034] like Figure 3, system 300 may include at least one image capture device 310. Image capture device 310 may be a camera or a video capture device. Image capture device 310 may include one or more camera sensors. The camera sensor may detect light to send a signal to image capture device 310 to capture one or more images 360. Image capture device 310 may capture images 360 using one or more methods (e.g., point and shoot or image tracking). Image capture device 310 may include operating parameters or settings that may affect images 360.

[0035] The operating parameters may include at least one of the following: gain, exposure, aperture, or frame rate, or various combinations thereof. The gain of the image capture device 310 may represent the amplification of the signal from the camera sensor and may be related to the apparent brightness of the image 360. The exposure time may represent the amount of time the sensor is exposed (e.g., exposed to light in the environment). The aperture may represent the size (e.g., diameter) of the opening that allows light to enter the camera sensor. The frame rate may refer to the speed at which the image capture device 310 captures the image 360 ​​or the speed at which the display device 315 displays the image 360. The frame rate may be related to the exposure (e.g., shutter speed). Each of these operating parameters may affect another operating parameter, may affect the image 360, and may be affected by the environment (including ambient light or light from the system 300). The operating parameters may be predetermined, or the operating parameters may change based on at least the image 360 ​​or the processor 305. The operating parameters may change periodically, or may change due to changes in the processor 305.

[0036] The image capture device 310 can operate in one or more operating modes. The operating mode can correspond to the value of one or more of a plurality of operating parameters of the image capture device 310, such as one or more values ​​of operating parameters of the image capture device 310 that can be controlled by the processor 305 (e.g., the processor 305 can provide at least one of a signal identifying the operating mode or a signal identifying one or more values ​​of one or more operating parameters to the image capture device 310, so that the image capture device 310 operates according to the identified mode and / or the identified one or more values). In some embodiments, the first mode can include a higher frame rate (e.g., 100 Hz or 120 Hz) than the second mode, which has a lower frame rate (e.g., 50 Hz or 60 Hz). In some embodiments, the image capture device 310 can change between the multiple modes based at least on environmental conditions or the image 360.

[0037] The image capture device 310 may capture an image 360 ​​(e.g., within an environment) and transmit the image 360 ​​to the processor 305. The processor 305 may be any combination of hardware and software for performing the operations described herein. The processor 305 may be similar or identical to the processor 130. The processor 305 may include an image parameter detector 325 and an application 330. The image capture device 310 may transmit the image 360 ​​to the image parameter detector 325 and the application 330 via a network (e.g., a local area network (LAN), a wide area network (WAN), Bluetooth) or via a hardwired connection (e.g., within the HWD device 100). The image parameter detector 325 may evaluate the image 360 ​​in response to receiving the image 360 ​​to identify one or more image parameters.

[0038] The image parameters may indicate the relative brightness or darkness of the image. The image parameters may include frame brightness or intensity, the signal to noise ratio (SNR) of the image, or the clarity of the image, etc. For example, brightness may indicate the perception or expected perception of light, such as the perception of the intensity and / or brightness of the image. The frame brightness may change at least according to environmental conditions. An example of an environmental condition may be a low light condition. A low light condition may refer to an environment in which the ambient light is insufficient to capture the image 360 ​​at a desired frame brightness and in a specified operating mode of the image capture device 310. The desired frame brightness may be part of a threshold value for determining the operating mode of the image capture device 310.

[0039] In some embodiments, the image parameter detector 325 evaluates the image 360 ​​to determine whether one or more of the image 360 ​​or its image data meets a threshold, and can output an indication of whether the image data meets the threshold. The threshold can be related to one or more of the following: image parameters (e.g., frame brightness, SNR, sharpness), operating parameters (e.g., gain, exposure, aperture), or a combination thereof. In some embodiments, the system 300 controls the operation of the image capture device 310 to meet the threshold frame brightness within environmental conditions. For example, the image parameter detector 325 can evaluate the image 360 ​​to meet the threshold frame brightness. The frame brightness and other image parameters can be related to the operating parameters of the image capture device 310. Meeting the threshold frame brightness can mean that the image is darker than the threshold, brighter than the threshold, or at the threshold.

[0040] The threshold value may be dynamic. The threshold value may be related to at least an image parameter, an operating parameter, or a combination thereof. The threshold value may change in response to an environmental condition or in response to an operating mode of the image capture device 310, etc. The threshold value may exhibit hysteresis. For example, the threshold value may change according to an operating mode, an environmental condition, or an image parameter of the image capture device 310. For example, a threshold value of a first mode of the image capture device 310 may be higher than a threshold value of a second mode of the image capture device 310. The threshold value may include a range. The threshold value may include an upper threshold value and a lower threshold value of the same operating parameter or image parameter. In this way, the upper threshold value and the lower threshold value of the same parameter may create a range. In some embodiments, the image parameter detector 325 may determine whether one or more image parameters of the image 360 ​​meet the corresponding threshold values ​​of the one or more image parameters. The image parameter detector 325 may determine whether the image 360 ​​meets a first threshold value associated with an operating parameter and a second threshold value associated with an image parameter. For example, if the frame brightness of image 360 ​​is below a first threshold and the gain and exposure of image capture device 310 are between or within a threshold range indicated by lower and upper thresholds for gain and exposure, image parameter detector 325 may determine that image 360 ​​satisfies the threshold.

[0041] The introduction of such hysteresis can reduce flickering caused by parameters of the system 300 hovering near a threshold. The threshold range can help eliminate excessive switching between two or more different camera operating modes when an operating parameter or an image parameter is toeing a threshold.

[0042] The image parameters may be related to the operating parameters of the image capture device 310. For example, the image parameter detector 325 may evaluate the image to obtain its frame brightness, which may be related to the gain and exposure of the image capture device. As an illustrative example, when the frame brightness decreases (e.g., due to low light conditions), the product of the gain and exposure of the image capture device 310 may increase to keep the frame brightness below a threshold. In this example, the gain and exposure of the image capture device 310 may increase to a saturation point of the image, or until the image meets a threshold for the frame brightness. In this way, exposure and gain may be used to keep the frame brightness below a threshold regardless of environmental conditions.

[0043] In some embodiments, the image parameter detector 325 can evaluate the image parameters to determine a model of the environmental conditions, for example, to determine a predicted value or expected value of one or more environmental conditions at a future time point, which can facilitate controlling the operation of the image capture device 310 in a predictive manner according to the value of the environmental conditions at the future time point. For example, the image parameter detector 325 can accumulate image parameter data to create an artificial intelligence prediction model. The model can be or include a neural network, a decision tree, a multivariate regression analysis, or other prediction model. The model can indicate how the image parameters will change according to environmental conditions, operating parameters, or time, etc. For example, the model can estimate that a second image transmitted after a first image may have a lower frame brightness than the first image. The model can be adjusted based on at least the image 360 ​​associated with the operating parameter. The model can predict whether an image in each image 360 ​​meets a threshold. The image parameter detector 325 can create an evaluation with or without a model, indicating whether one or more image parameters of each image 360 ​​meet a threshold.

[0044] The image parameter detector 325 may transmit the image data 350 to the camera controller 320. The camera controller 320 may receive an indication from the image parameter detector 325 whether the image data 350 (e.g., one or more parameters of the image 360) meets a threshold value in response to the image parameter detector 325 evaluating the image data 350. The processor 305 may operate the camera controller 320 to control the mode of the image capture device 310 based at least on the evaluation. The camera controller 320 may transmit a control signal 355 containing an instruction for the image capture device 310 to operate in a specified mode. The mode may be determined based on the image data 350 containing the evaluation. The evaluation may indicate in which mode the image capture device 310 is operated.

[0045] The mode in which the image capture device 310 is operated may depend on the mode in which the image capture device 310 is currently operating. For example, if the image capture device 310 is operating in a first mode corresponding to a high frame rate (e.g., 120 Hz), the camera controller 320 may transmit a control signal 355 based on an evaluation. The camera controller 320 may maintain the first mode or change to a second mode in response to receiving the evaluation. The evaluation may indicate that one or more image parameters meet a threshold. In some embodiments, the camera controller 320 may send a control signal 355 to cause the image capture device 310 to change to the second mode in response to an evaluation indicating that the image parameters meet the threshold. In other words, the image parameter detector 325 determines that the image parameters meet the threshold and may cause the image capture device 310 to change from a higher frame rate to a lower frame rate. In some embodiments, the evaluation may indicate that the image parameters meet a threshold frame brightness. The frame brightness of the image may be affected by low light conditions. In response to meeting the threshold frame brightness caused by the low light conditions, the camera controller 320 may transmit a control signal 355 to cause the image capture device 310 to switch from the first mode (high frame rate) to the second mode (low frame rate).

[0046] If the image capture device 310 is operating in the second mode (lower frame rate), the image data 350 may indicate that the image capture device 310 will continue in the second mode, or will change to the third mode. For example, if the gain and exposure meet the threshold gain and exposure for the second mode, and the frame brightness also meets the threshold, the camera controller 320 may cause the image capture device 310 to return to the first mode via the control signal 355. If the gain and exposure do not meet the threshold gain and exposure, and / or the frame brightness does not meet the threshold frame brightness, the camera controller 320 may transmit a control signal 355 with instructions to the image capture device 310 to maintain the current mode.

[0047] The camera controller 320 may generate a control signal 355 including instructions for changing the mode of the image capture device 310, for example to reduce or mitigate flicker. During mode transitions, flicker may result due to at least a change in exposure, image alignment, or timing. For example, during a mode transition, a sudden change in exposure may cause a user to notice a bright flash or dimness. To mitigate this, the camera controller 320 may indicate in its control signal 355 to maintain the same or similar exposure when switching modes. Additionally, in embodiments having more than one image capture device 310, the camera controller 320 may cause the images captured by each image capture device 310 to remain aligned before and after causing the image capture device 310 to change modes.

[0048] The camera controller 320 may include in the control signal 355 an instruction to not switch modes until a specified time. The specified switching time may be related to the frame rate and / or exposure of the image capture device 310. For example, the switch may occur at the end of the sequence of images 360. As an illustrative example, if the sequence of images 360 takes 66.666 milliseconds (ms) to complete and the switch is indicated at the beginning of the sequence, it may be 66.666ms before the mode switch. This may be a 4-frame delay at 60 Hz, or an 8-frame delay at 120 Hz. In some embodiments, the image capture device 310 may switch modes at the beginning of a sequence, at the end of a sequence, or during a sequence. The camera controller 320 may include in the control signal 355 an instruction to introduce gaps between multiple images 360 during the switch. For example, during a switch from a high frame rate mode to a low frame rate mode, it may be desirable to introduce gaps to facilitate the switch. As an illustrative example, when converting from 120 Hz to 60 Hz, system 300 may introduce a gap at the end of the sequence of images 360 equal to 50% (4.3 ms) of the 120 Hz frame before switching to the 60 Hz frame rate.

[0049] The system 300 can iteratively accept feedback from the image 360 ​​continuously transmitted from the image capture device 310. In this way, the image parameter detector 325 can continuously evaluate the image parameters for compliance with the threshold, and the camera controller 320 can continuously control the mode of the image capture device 310. In addition to transmitting the image 360 ​​to the image parameter detector 325 of the processor 305, the image capture device 310 can simultaneously, subsequently, or in advance transmit the image 360 ​​to the application 330 of the processor 305.

[0050] The application 330 may be a software application for executing features associated with the HWD device 100. The application 330 may be executed on the processor 305, the processor 130, or another computing device internal or external to the system. The application 330 may receive an image 360 ​​from the image capture device 310. The application 330 may process the image 360 ​​for transmission to the image renderer 335. Processing the image 360 ​​may include filtering the image, correcting the image, aligning the image, removing bad images, etc. The application 330 may generate an artificial image. The application 330 may encode the image 360 ​​into application data 345 for transmission to the image renderer 335.

[0051] The image renderer 335 may receive the application data 345 from the application 330 and may generate display data 365 for presenting the image 360 ​​via the display device 315. The image renderer 335 may be included as one of the operations of the processor 305. The image renderer 335 may render the display data 365 for presentation by the display device 315. The display data 365 may be encoded in a data packet for transmission (wired or wireless, e.g., via Bluetooth, Wi-Fi, etc.) to the display device 315 for presentation. The display data 365 may include data packets, bits, codes, etc., which include instructions for the display device to present at least an image, a hologram, a video, or other presentation to a user of the HWD device 100. The display data 365 may include the image 360, an artificial image generated by the application 330, or text, etc. The display data may be presented in two dimensions or three dimensions. The display data may be encoded in a video format (eg, QuickTime, MP4), a picture format (eg, JFIF, GIF), or other format suitable for displaying 3D graphics (eg, X3D, glTF, O3D).

[0052] Display device 315 may present display data 365. Display device 315 may be any hardware or software for displaying a presentation, such as display 145. The presentation may include image 360 ​​encoded as display data 365. Display device 315 may be or include a screen (e.g., a liquid crystal display (LCD) or a light-emitting diode (LED)), a hologram, a projection, or other display means for presenting display data 365.

[0053] By implementing the various control schemes described herein, the system 300 can improve the performance (e.g., accuracy) of computer vision operations, including but not limited to tracking operations, and can also reduce device power consumption and improve the efficiency of computing resource usage. Such operations can allow the system 300 to reduce flicker, noise, and / or blur in images. Automatic switching between modes of the image capture device 310 can achieve higher frame rates to enhance image presentation while allowing lower frame rates based on environmental conditions.

[0054] Figure 4 Schematic diagram 400 depicts an example change to camera operation based on environmental conditions. Schematic diagram 400 shows an example of threshold curves for image parameters and operating parameters that system 300 can use to trigger switching between a first higher frame rate mode and a second lower frame rate mode. Schematic diagram 400 shows multiple thresholds, including GE MAX,B405 (which may represent the maximum possible gain times exposure product when the system 300 is operating in the first mode), GE TH,L 410 (which may represent a threshold for controlling the transition from the second mode to the first mode), GE MAX,L 420 (which may represent the maximum gain multiplied by the exposure product when the system 300 is operating in the second mode), GE TH,H 425 (which may represent a threshold for controlling the transition from the first mode to the second mode), and FB MIN (Which may represent a minimum target frame brightness to be monitored in order to appropriately control operation in the first mode and / or the second mode.) Schematic 400 depicts regions 450, in which the gain and exposure of the image capture device are plotted, and 455, in which the ambient light 445 and frame brightness 435 are plotted, of the schematic.

[0055] Ambient light 445 may be ambient light in the environment in which the system operates. For example, the ambient light may be bright or low. Bright ambient light may refer to light having a higher light intensity (e.g., in lux or mW / cm2) than low ambient light. 2 ) in units). In some embodiments, the gain, exposure, or a combination thereof (e.g., the sum, product, or other operation of the gain and exposure) of the image capture device may increase in response to the reduced ambient light 445. As an illustrative example, when the ambient light 445 decreases in region 455, the gain and exposure product (GE) 415 may increase in region 450. In some embodiments, GE 415 may increase to a saturation point B. Saturation point B may be reached when the operating parameters of the image capture device are at a maximum. At or after exposure saturation, the image may appear dark, blurry, or otherwise of low quality, and may be difficult to discern with the human eye. If the ambient light further decreases while GE 415 has reached saturation point B, the frame brightness 435 of the image may decrease. In some embodiments, the frame brightness 435 may decrease to a frame brightness minimum (FB 435) at point D. MIN )440.

[0056] If the frame brightness 435 reaches point D, indicating that the image is underexposed, the system can determine to switch the mode of the image capture device. The system can make this determination by evaluating thresholds. For example, if the GE 415 is operating in the first mode (higher frame rate) when the frame brightness drops to point D, the system can evaluate whether other thresholds have been met. The system can evaluate whether the GE 415 has met the threshold GE for low light environments. TH,L 410. In some embodiments, if GE 415 has satisfied GE TH,L410 and the frame brightness 435 is reduced to FB MIN , the system may determine to change the mode of the image capture device. In some embodiments, if GE 415 has satisfied GE TH,L 410 and the frame brightness 435 decreases below the target frame brightness 430, the system can determine to change the mode of the image capture device from the first mode to the second mode. The transition from the first mode to the second mode can cause the frame brightness 435 to return to or approach the target frame brightness 430.

[0057] When the system is operating in the second mode, the mode transition or determination to transition may be made based on a threshold different from the threshold used in the first mode. For example, the second mode may include a gain and exposure product GE different from the threshold of the first mode. TH,H The minimum threshold of 425, and the gain and exposure product GE MAX,L In some embodiments, the system can operate in the second mode until GE 415 reaches GE TH,H 425. Region 450 may further depict that GE 415 decreases as ambient light 445 increases in region 455. System 300 may respond to GE 415 decreasing to GE TH,H 425 (eg, at point C), and determine to switch to the first mode.

[0058] Schematic diagram 400 depicts multiple thresholds for determining the mode in which the image capture device can operate. The multiple thresholds exhibit hysteresis to the system. The hysteresis can prevent rapid cycling between modes (which may be harmful to the components of the system or the user experience). The thresholds depicted herein can be dynamic. The two modes described herein are shown and can be extended to any number of modes. Similarly, GE 415 and frame brightness 435 can be examples of any operating parameters or image parameters, respectively.

[0059] Figure 5 A flowchart 500 is depicted showing a method of manipulating camera operation based on environmental conditions. The method 500 may include an action 510, in which the system may capture an image for presentation; an action 520, in which the system may determine that a parameter of the image satisfies a threshold; an action 530, in which the system may reduce a frame rate. The method 500 may include more or fewer actions than those described herein.

[0060] At act 510, the system may capture one or more images. The system may, for example, use image capture device 310 to capture the images. The images may be sent at least one after another as a sequence of images, or may be sent within a data packet, with or without processor confirmation. The images (e.g., image 360) may include a photograph or video of the system's surroundings. The images may have image parameters, such as brightness, sharpness, or blurriness of a frame (e.g., image), etc.

[0061] At action 520, the system determines that a parameter of the image satisfies a threshold. For example, the system may use the image parameter detector 325 to determine that the parameter satisfies the threshold. One or more parameters may satisfy one or more thresholds. For example, there may be a first parameter related to the frame brightness of the image and a second parameter related to the clarity of the image. The threshold may have an upper limit and a lower limit. For example, the first parameter may be below a lower threshold, between a lower threshold and an upper threshold, or above an upper threshold.

[0062] Determining that a parameter of an image satisfies a threshold may include determining, for each image, that the image satisfies one or more thresholds, or determining that a group of images satisfies one or more thresholds. Determining that a group of images satisfies one or more thresholds may mean that a majority of the images in the group satisfy the thresholds, or that a predetermined number of images (or a percentage of images) satisfy the thresholds. The system may determine that a group of images satisfies a threshold simultaneously, one by one, or in any other order or sequence.

[0063] The system may transmit images, data related to the images, or an evaluation of the images to a controller (e.g., camera controller 320) in response to determining that a parameter of the image satisfies a threshold. The evaluation may include information related to the images, the parameters of the images, and the threshold. The system may control the image capture device using at least the information contained in the evaluation.

[0064] In action 530, the system reduces the frame rate. The system can control the image capture device to reduce the frame rate. The system can reduce the frame rate in response to determining that a parameter of the image meets a threshold. For example, the system can reduce the frame rate of the image capture device in response to determining that the brightness of the image meets a threshold. The brightness of the image can correspond to an ambient condition (e.g., a low light condition). In some embodiments, the system can reduce the frame rate of the image capture device in response to determining that the brightness of the image meets a threshold and the product of the gain and exposure of the image capture device meets a threshold.

[0065] In addition to reducing the frame rate, the system may also take other control actions. The system may increase the frame rate of the image capture device. The system may increase the frame rate of the image capture device in response to determining that the image parameters meet a threshold value different from the threshold value used to reduce the frame rate. In some embodiments, the system may not change the frame rate of the display device. If, for example, the image parameters do not meet the threshold value, there is a manual override of the system, the displayed image needs to be presented at a specific frame rate, etc., the system may not change the frame rate of the display device.

[0066] The system may change other operating parameters of the image capture device. The system may control at least exposure, gain, aperture, shutter speed, or focus of the image capture device. The system may control more than one image capture device. The system may implement the same control for each image capture device, different controls for each image capture device, or the same control for a subset of image capture devices. The control may not include any actions. The system may keep the image aligned before, during, and after switching between multiple frame rates or changing to other operating parameters.

[0067] Various operations described herein can be implemented on a computer system. Figure 6 614 is a block diagram of a representative computing system that can be used to implement embodiments of the present disclosure. Figure 8 The wearable device 810 is implemented by a computing system 614. For example, the computing system 614 can be implemented as a consumer device, or implemented using a distributed computing device, such as a smartphone, other mobile phones, tablet computers, wearable computing devices (e.g., smart watches, glasses, head wearable displays), desktop computers, laptop computers. The computing system 614 can be implemented to provide VR experience, AR experience, MR experience. In some embodiments, the computing system 614 may include conventional computer components, such as a processor 616, a storage device 618, a network interface 620, a user input device 622, and a user output device 624.

[0068] The network interface 620 may provide a connection to a wide area network (e.g., the Internet) to which the WAN interface of the remote server system is also connected. The network interface 620 may include a wired interface (e.g., Ethernet) and / or a wireless interface implementing various radio frequency (RF) data communication standards, such as Wi-Fi, Bluetooth, or cellular data network standards (e.g., 3G, 4G, 5G, 60 GHz, LTE, etc.).

[0069] The user input device 622 may include any device (or devices) via which a user may provide a signal to the computing system 614, which the computing system 614 may interpret as indicating a particular user request or information. The user input device 622 may include any or all of the following: a keyboard, a touch pad, a touch screen, a mouse or other pointing device, a scroll wheel, a click wheel, a dial, a button, a switch, a keypad, a microphone, and a sensor (e.g., a motion sensor, an eye tracking sensor, etc.), etc.

[0070] User output device 624 may include any device through which computing system 614 can provide information to a user. For example, user output device 624 may include a display (not shown) for displaying images generated by or delivered to computing system 614. The display may be combined with various image generation technologies, such as a liquid crystal display (LCD), a light emitting diode (LED) including organic light-emitting diodes (OLED), a projection system, or a cathode ray tube (CRT), etc.; and supporting electronic devices (e.g., digital-to-analog or analog-to-digital converters, or signal processors, etc.). Devices used as both input devices and output devices (e.g., touch screens) may be used. User output device 624 may be provided in addition to or in place of a display. Examples include indicator lights, speakers, tactile "display" devices, and printers, etc.

[0071] Some embodiments include electronic components that store computer program instructions in a computer-readable storage medium (e.g., non-transient computer-readable medium), such as microprocessors, storage devices, and memories. Many of the features described in this disclosure can be implemented as the following process: these processes are specified as a set of program instructions encoded on a computer-readable storage medium. When these program instructions are executed by one or more processors, these instructions cause the processor to perform the various operations indicated in the program instructions. Examples of program instructions or computer codes include machine codes and files, which are generated, for example, by a compiler, and these files include high-level codes executed by a computer, an electronic component, or a microprocessor using an interpreter. Processor 616 can provide various functions for computing system 614 by appropriate programming, including any functions described herein as being performed by a server or client, or other functions associated with a message management service.

[0072] It will be understood that computing system 614 is illustrative, and various variations and modifications are possible. The computer system used in conjunction with the present disclosure may have other capabilities not specifically described herein. In addition, when describing computing system 614 with reference to specific blocks, it should be understood that these blocks are defined for ease of description and are not intended to imply a specific physical arrangement of component parts. For example, different blocks may be located in the same facility, in the same server rack, or on the same motherboard. In addition, these blocks do not need to correspond to physically different components. Each block can be configured to perform various operations, such as by programming a processor or providing appropriate control circuits, and various blocks may be reconfigurable or non-reconfigurable, depending on how the initial configuration is obtained. Embodiments of the present disclosure can be implemented in various devices, including electronic devices implemented using any combination of circuits and software.

[0073] Fig. 7A 7 is a perspective view showing a wearable device 810A (e.g., wearable device 810 in state 810A) including computing device 110 attached to cradle 720 according to an example embodiment of the present disclosure. Processor 130, wireless communication interface 725, and sensor 715 may be disposed within a housing of computing device 110 such that processor 130, wireless communication interface 725, and sensor 715 may not be visible to a user. Computing device 110 may also include display 145 located on front side 770 for presenting text or images. Figure 7B As shown, the computing device 110 can be detached from the bracket 720 (e.g., the wearable device 810 in state 810B). The computing device 110 can be detached from the bracket 720 to allow the user to charge the battery of the computing device 110, connect to another device via a cable, or capture images, etc. The bracket 720 can be a wearable structure or a component that is used to selectively hold, accommodate, connect, grasp and / or couple the computing device 110. The bracket 720 can include one or more couplers 750, and the back side 760 or other portion of the computing device 110 can be attached to the one or more couplers. The one or more couplers 750 can include a mechanical latch, a magnetic member, a hook and loop fastener, or any one or more components that allow the bracket 720 to selectively hold or couple the computing device 110. The bracket 720 can include or can be attached to the wristbands 710A, 720B.

[0074] Figure 8810 is a block diagram of an example system 800. In some embodiments, the system 800 includes a communication device 820, a wearable device 810, and a wearable device 850. The wearable device 810 can be a smart watch, and the wearable device 850 can be a head wearable device (HWD) 850. The communication device 820 can be an access point or any other communication device. The HWD 850 can be referred to as, include, or be part of a head mounted display (HMD), a head mounted device (HMD), a head worn display (HWD), or a head worn device (HWD). The wearable device 810 and the HWD 850 can communicate with each other via a communication link 814. The wearable device 810 and the communication device 820 can communicate with each other via a communication link 816, and the wearable device 850 and the communication device 820 can communicate with each other via a communication link 818. Wearable devices 810, 850 can access content (e.g., text, images, audio, video, etc.) from other devices via wireless links 816, 818. Communication links 814, 816, 818 can be wireless links (e.g., cellular links, Wi-Fi links, Bluetooth links, 60GHz links, ultra-wideband links, etc.). Communication links 814, 816, 818 can be based on the same protocol or different protocols. For example, communication links 816, 818 can comply with 3G, 4G, 5G, LTE, 60GHz protocols, where communication link 814 can comply with Wi-Fi links, Bluetooth, etc. In some embodiments, system 800 includes more than Figure 8 More or fewer parts than shown, or with Figure 8 The different components shown in .

[0075] In one aspect, the wearable device 810 and the wearable device 850 can operate together to provide / support artificial reality for the user. In one example, the wearable device 850 can detect the position and orientation of the wearable device 850 and generate sensor measurements indicating the detected position and orientation of the wearable device 850. The wearable device 850 can transmit the sensor measurements to the wearable device 810 via the communication link 814. The wearable device 810 can receive the sensor measurements and can generate or determine a view of the artificial reality corresponding to the detected position and orientation of the wearable device 850. The wearable device 810 can generate image data of the determined view of the artificial reality and transmit the image data to the wearable device 850 via the communication link 814. The HWD 850 can receive the image data and can present an image of the artificial reality to the user based on the image data. On the one hand, the process of detecting the position and orientation of the HWD 850 and rendering the image to the user should be performed within a frame time (e.g., 11ms or 16ms). Any delay between the movement of a user wearing the HWD and the images displayed corresponding to the user's movement may cause jitter, which may cause motion sickness and may degrade the user experience.

[0076] Fig. 9 An example wireless communication system 900 is shown. The wireless communication system 900 may include base stations 910A, 910B (also referred to as "wireless communication nodes 910" or "stations 910") and user equipment (UE) 920AA...920AN, 920BA...920BN (also referred to as "wireless communication devices 920" or "terminal devices 920"). The wireless communication link may be a cellular communication link that complies with 3G, 4G, 5G or other cellular communication protocols. In one example, the wireless communication link supports, adopts or is based on orthogonal frequency division multiple access (OFDMA). On the one hand, UE 920AA...920AN is located within a geographic boundary relative to base station 910A and can communicate with base station 910A or communicate through base station 910A. Similarly, UE 920BA...920BN is located within a geographic boundary relative to base station 910B and can communicate with base station 910B or communicate through base station 910B. The network between the UE 920 and the base station 910 may be referred to as a radio access network (RAN). Fig. 9 A greater, fewer, or different number of base stations 910 than shown in FIG.

[0077] In some embodiments, the UE 920 may be a user device, such as a mobile phone, a smart phone, a personal digital assistant (PDA), a tablet computer, a laptop computer, a wearable computing device (e.g., a head-mounted display, a smart watch), etc. Each UE 920 may communicate with the base station 910 via a corresponding communication link. For example, the UE 920 may transmit data to the base station 910 via a wireless communication link (e.g., a 3G, 4G, 5G or other cellular communication link), and / or receive data from the base station 910 via a wireless communication link (e.g., a 3G, 4G, 5G or other cellular communication link). Example data may include audio data, image data, text, etc. The communication or data transmission performed by the UE 920 to the base station 910 may be referred to as an uplink communication. The communication or data reception performed by the UE 920 from the base station 910 may be referred to as a downlink communication.

[0078] In some embodiments, the base station 910 may be an evolved node B (eNB), a serving eNB, a target eNB, a femto station, or a pico station. The base station 910 may be communicatively coupled to another base station 910 or other communication device via a wireless communication link and / or a wired communication link. The base station 910 may receive data (or RF signals) from a UE 920 in an uplink communication. Additionally or alternatively, the base station 910 may provide data to another UE 920, another base station, or another communication device. Thus, the base station 910 allows communication between multiple UEs 920 associated with the base station 910, or allows communication between other UEs associated with different base stations.

[0079] In some embodiments, the wireless communication system 900 includes a core network 970. The core network 970 may be a component or an aggregation of multiple components that ensures a reliable and secure connection of the UE 920 to the network. The core network 970 may be communicatively coupled to one or more base stations 910A, 910B via a communication link. The communication link between the core network 970 and the base station 910 may be a wireless communication link (e.g., 3G, 4G, 5G or other cellular communication link) or a wired communication link (e.g., Ethernet, optical communication link, etc.). In some embodiments, the core network 970 includes a user plane function (UPF), an access and mobility management function (AMF), a policy control function (PCF), etc. The UPF may perform packet routing and forwarding, packet inspection, quality of service (QoS) processing, and provide an external protocol data unit (PDU) session for interconnecting data networks (DN). The AMF may perform registration management, reachability management, connection management, etc. PCF can help operators (or operating equipment) to easily create and seamlessly deploy policies in wireless networks. The core network 970 may include additional components for managing or controlling the operation of the wireless network. In one aspect, the core network 970 may receive a message for performing network congestion control and perform the requested network congestion control. For example, the core network 970 may receive an explicit congestion notification (ECN) from the base station 910 and / or the UE 920 and perform network congestion control according to the ECN. For example, the core network 970 may adjust or control the amount of data generated in response to the ECN. Additionally or alternatively, the core network 970 may adjust or control the amount of data sent and / or received in response to the ECN.

[0080] In some embodiments, the wireless communication system 900 includes an application server 960. The application server 960 may be a component or device that generates, manages, or provides content data. The application server 960 may be communicatively coupled to one or more base stations 910A, 910B via a communication link. The communication link between the application server 960 and the base station 910 may be a wireless communication link (e.g., a 3G, 4G, 5G, or other cellular communication link) or a wired communication link (e.g., an Ethernet, an optical communication link, etc.). In one aspect, the application server 960 may receive a request for data from the UE 920 via the base station 910, and provide the requested data to the UE 920 via the base station 910. In one aspect, the application server 960 may receive a message for performing network congestion control, and perform the requested network congestion control. For example, the application server 960 may receive an explicit congestion notification (ECN) from the base station 910, the UE 920, or the core network 970, and perform network congestion control according to the ECN. For example, the application server 960 may adjust or control the amount of data generated in response to the ECN. Additionally or alternatively, application server 960 may adjust or control the amount of data sent and / or received in response to the ECN.

[0081] In some embodiments, the communication between the base station 910, the UE 920, the application server 960, and the core network 970 is based on one or more layers of the Open Systems Interconnection (OSI) model. The OSI model may include multiple layers, including: a physical layer, a medium access control (MAC) layer, a radio link control (RLC) layer, a packet data convergence protocol (PDCP) layer, a radio resource control (RRC) layer, a non-access stratum (NAS) layer or an Internet Protocol (IP) layer, and other layers.

[0082] In some embodiments, the base station 910 includes a wireless interface, a processor, a storage device, and one or more antennas. These components can be embodied as hardware, software, firmware, or a combination thereof. For example, the base station 910 can include an electronic display and / or an input device. For example, the base station 910 can include additional antennas and wireless interfaces.

[0083] In some embodiments, system 900 may include an antenna (not shown). The antenna may be a component that receives a radio frequency (RF) signal and / or sends an RF signal through a wireless medium. The RF signal may be at a frequency between 200 megahertz (MHz) and 100 GHz. The RF signal may have data packets, symbols, or frames that correspond to data for communication. The antenna may be a dipole antenna, a patch antenna, a loop antenna, or any suitable antenna for wireless communication. In one aspect, a single antenna is used to send and receive RF signals. In one aspect, different antennas are used to send and receive RF signals. In one aspect, multiple antennas are used to support multiple-input, multiple-out, MIMO communications.

[0084] An antenna may be a component that receives a radio frequency (RF) signal and / or transmits an RF signal through a wireless medium. The antenna may be a dipole antenna, a patch antenna, a loop antenna, or any suitable antenna for wireless communication. In one aspect, a single antenna is used to transmit and receive RF signals. In one aspect, different antennas are used to transmit and receive RF signals. In one aspect, multiple antennas are used to support multiple-input multiple-output (MIMO) communications.

[0085] The wireless interface includes or is embodied as a transceiver, which is used to send and receive RF signals through one or more antennas. The wireless interface can communicate with the wireless interface of UE 920 through a wireless communication link. In one configuration, the wireless interface is coupled to one or more lines. In one aspect, the wireless interface can receive an RF signal at an RF frequency received by an antenna, and downconvert the RF signal to a baseband frequency (e.g., 0 GHz to 1 GHz). The wireless interface can provide a downconverted signal to the processor. In one aspect, the wireless interface can receive a baseband signal for sending at a baseband frequency from a processor, and upconvert the baseband signal to generate an RF signal. The wireless interface can send an RF signal through an antenna.

[0086] The processor is a component that processes data. The processor can be embodied as a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a logic circuit, etc. The processor can obtain instructions from a storage device and execute these instructions. In one aspect, the processor can receive down-converted data at a baseband frequency from a wireless interface and decode or process these down-converted data. For example, the processor can generate audio data or image data based on the down-converted data. In one aspect, the processor can generate or obtain data transmitted at a baseband frequency and encode or process these data. For example, the processor can encode or process image data or audio data at a baseband frequency and provide the encoded or processed data to the wireless interface for transmission. In one aspect, the processor can set, dispatch, schedule or allocate communication resources for different UEs 120. For example, the processor can set different modulation schemes, time slots, channels, frequency bands, etc. for UE 920 to avoid interference. The processor may generate data (or an uplink configuration grant (UL CG)) indicating the configuration of communication resources and provide the data (or UL CG) to the wireless interface for transmission to the UE 920.

[0087] A storage device is a component that stores data. The storage device may be embodied as a random access memory (RAM), a flash memory, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), a register, a hard disk, a removable disk, a compact disk read-only memory (CD-ROM), or any device capable of storing data. The storage device may be embodied as a non-transitory computer-readable medium that stores instructions executable by a processor that are used to perform various functions of the base station 910 disclosed herein. In some embodiments, the storage device and the processor are integrated into a single component.

[0088] Now that some illustrative embodiments have been described, it is apparent that the foregoing, which has been presented by way of example, is illustrative rather than restrictive. Specifically, although many of the examples presented herein relate to specific combinations of multiple method actions or multiple system elements, these actions and these elements may be combined in other ways to achieve the same goals. The actions, elements, and features discussed in conjunction with one embodiment are not intended to be excluded from similar roles in other one or more embodiments.

[0089] The hardware and data processing components for implementing the various processes, operations, exemplary logic, logic blocks, modules and circuits described in conjunction with the embodiments disclosed herein may be implemented or executed using the following items: a general purpose single-chip or multi-chip processor; a digital signal processor (DSP); an application specific integrated circuit (ASIC); a field programmable gate array (FPGA); or other programmable logic devices, discrete gate or transistor logic, discrete hardware components or any combination thereof designed to perform the functions described herein. A general purpose processor may be a microprocessor, or any conventional processor, controller, microcontroller or state machine. The processor may also be implemented as a combination of multiple computing devices, such as a combination of a DSP and a microprocessor, a combination of multiple microprocessors, one or more microprocessors in combination with a DSP core, or any other such configuration. In some embodiments, specific processes and methods may be performed by circuits specific to a given function. A memory (e.g., memory, storage unit, storage device, etc.) may include one or more devices (e.g., RAM, ROM, flash memory, hard disk storage, etc.) for storing data and / or computer code, which are used to complete or facilitate the various processes, layers and modules described in this disclosure. The memory may be or include volatile memory or non-volatile memory, and may include a database component, an object code component, a script component, or any other type of information structure for supporting the various activities and information structures described in the present disclosure. According to an exemplary embodiment, the memory is communicatively connected to the processor via the processing circuit, and the memory includes computer code for executing (e.g., by the processing circuit and / or the processor) one or more processes described herein.

[0090] The present disclosure contemplates the method, system and program product on any machine-readable medium for completing various operations. The embodiments of the present disclosure can be implemented using an existing computer processor, or implemented by a special-purpose computer processor of an appropriate system incorporated for this or another purpose, or implemented by a hard-wired system. The embodiments within the scope of the present disclosure include program products, which include machine-readable media, which carry or have machine-executable instructions or data structures stored thereon. This machine-readable medium can be any available medium that can be accessed by a general-purpose computer or a special-purpose computer or other machine with a processor. As an example, this machine-readable medium can include RAM, ROM, EPROM, EEPROM or other optical disk storage, disk storage or other magnetic storage device, or can be used to carry or store any other medium of the desired program code in the form of machine-executable instructions or data structures, which can be accessed by a general-purpose computer or a special-purpose computer or other machine with a processor. The above-mentioned combinations are also included in the scope of machine-readable media. Machine-executable instructions include instructions and data that, for example, enable a general-purpose computer, a special-purpose computer or a special-purpose processing machine to perform a specific function or a group of functions.

[0091] The words and terms used herein are for illustrative purposes and should not be considered limiting. As used herein, "includes," "comprising," "having," "covering," "involving," "characterized by," "characterized by," and variations thereof are intended to include the items listed thereafter, their equivalents, and additional items, as well as alternative embodiments consisting of the items specifically listed thereafter. In one embodiment, the systems and methods described herein include one, each combination of more than one, or all of the described elements, actions, or components.

[0092] Any reference to the embodiments or elements or actions of the systems and methods mentioned herein in the singular may also encompass embodiments including a plurality of these elements, and any plural reference to any embodiments or elements or actions herein may also encompass embodiments including only a single element. References in singular or plural form are not intended to limit the currently disclosed systems or methods, their parts, actions or elements to a single or multiple configurations. References to any actions or elements based on any information, action or element may include embodiments in which the action or element is based at least in part on any information, action or element.

[0093] Any embodiment disclosed herein may be combined with any other embodiment or implementation, and references to "an embodiment," "some embodiments," or "an embodiment," etc. are not necessarily mutually exclusive and are intended to indicate that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment or implementation. These terms as used herein do not necessarily all refer to the same embodiment. Any embodiment may be combined, inclusively or exclusively, with any other embodiment in any manner consistent with the various aspects and embodiments disclosed herein.

[0094] Where a technical feature in the drawings, detailed description or any claims is followed by a reference numeral, the reference numeral has been included to increase the intelligibility of the drawings, detailed description and claims. Therefore, neither the reference numeral nor the absence of a reference numeral has any limiting effect on the scope of any claim element.

[0095] The systems and methods described herein may be implemented in other specific forms without departing from their characteristics. Unless otherwise expressly stated, reference to "about," "approximately," "substantially," or other degree terms includes variations of + / -10% of a given measurement, unit, or range. The coupled elements may be electrically, mechanically, or physically coupled to each other directly or to intermediate elements. Accordingly, the scope of the systems and methods described herein is indicated by the appended claims rather than the foregoing description, and changes that fall within the meaning and range of equivalents of the claims are encompassed therein.

[0096] The term "coupled" and variations thereof include two components being directly or indirectly connected to one another. Such connection may be constant (e.g., permanent or fixed) or removable (e.g., removable or releasable). Such connection may be achieved by two members being directly coupled to one another or coupled to one another, wherein the two members are coupled to one another using a separate intermediate member and any additional intermediate members that are coupled to one another, or the two members are coupled to one another using an intermediate member that is integrally formed as a single unity with one of the two members. If "coupled" or variations thereof are modified by additional terms (e.g., directly coupled), the general definition of "coupled" provided above is modified by the plain language meaning of the additional terms (e.g., "directly coupled" refers to the connection of two members without any separate intermediate members), resulting in a narrower definition than the general definition of "coupled" provided above. Such coupling may be mechanical, electrical, or fluid.

[0097] References to "or" may be interpreted as inclusive, such that any term described using "or" may indicate a single, more than one, and all of the terms described. References to "at least one of 'A' and 'B'" may include only 'A', only 'B', and both 'A' and 'B'. Such references used in conjunction with "including" or other open-ended terms may include additional items.

[0098] Without departing substantially from the teachings and advantages of the subject matter disclosed herein, modifications may be made to the elements and actions described, such as variations in size, dimensions, structure, shape, and proportion of the various elements, variations in parameter values, mounting arrangements, use of materials, colors, orientations. For example, an element shown as being integrally formed may be composed of multiple parts or elements, the positions of which may be reversed or otherwise changed, and the nature or number or position of discrete elements may be changed or varied. Other substitutions, modifications, changes, and omissions may also be made in the design, operating conditions, and arrangement of the disclosed elements and operations without departing from the scope of the present disclosure.

[0099] References herein to the positions of elements (e.g., "top," "bottom," "above," "below") are only used to describe the orientation of the various elements in the drawings. According to other exemplary embodiments, the orientation of the various elements may be different, and these variations are intended to be included in the present disclosure.

Claims

1. A system comprising: an image capture device configured to capture one or more images; as well as One or more processors configured to: determining that a parameter of the one or more images satisfies a threshold; and In response to the parameter satisfying the threshold, the image capture device is caused to reduce a frame rate at which the one or more images are captured.

2. The system of claim 1, wherein: The parameters of the one or more images correspond to gain and exposure; and The one or more processors are configured to cause the image capture device to reduce the frame rate in response to a function of the gain and the exposure exceeding a threshold.

3. The system according to claim 1, wherein: The one or more processors are configured to cause the image capture device to reduce the frame rate in response to the parameter satisfying a low light condition.

4. The system according to claim 1, wherein: The one or more processors are configured to: In response to the parameter exceeding a first threshold, reducing the frame rate from a first frame rate to a second frame rate; and In response to the parameter falling below a second threshold, the frame rate is increased from the second frame rate to the first frame rate, the second threshold being less than the first threshold.

5. The system of claim 1, wherein: The image capture device is a first image capture device; The system includes a second image capture device; and The one or more processors are configured to maintain alignment of images captured by the first image capture device and the second image capture device before and after causing the first image capture device to reduce the frame rate.

6. The system of claim 1, further comprising a wearable housing coupled to the image capture device, display device, and the one or more processors.

7. The system according to claim 1, wherein: The one or more processors are configured to reduce the frame rate from a first frame rate of 120 Hz to a second frame rate of 50 Hz or 60 Hz.

8. The system according to claim 1, wherein: The image parameters correspond to one or more of: gain, exposure, frame brightness, aperture, a signal-to-noise ratio SNR of the one or more images, or a sharpness of the one or more images.

9. The system according to claim 1, wherein: The one or more processors are configured to perform a tracking operation using the one or more images.

10. A method comprising: capturing one or more images by an image capture device; determining, by one or more processors coupled to the memory, that a parameter of the one or more images satisfies a threshold; as well as In response to the parameter satisfying the threshold, the image capture device is caused to reduce a frame rate at which the one or more images are captured.

11. The method according to claim 10, comprising: in, The parameters of the one or more images correspond to gain and exposure; and The one or more processors cause the image capture device to reduce the frame rate in response to the function of the gain and the exposure exceeding a threshold.

12. The method according to claim 10, comprising: The one or more processors cause the image capture device to reduce the frame rate in response to the parameter satisfying a low light condition.

13. The method according to claim 10, comprising: reducing, by the one or more processors, the frame rate from a first frame rate to a second frame rate in response to the parameter exceeding a first threshold; as well as The frame rate is increased, by the one or more processors, from a second frame rate to the first frame rate in response to the parameter falling below a second threshold, the second threshold being less than the first threshold.

14. The method according to claim 10, comprising: in, The image capture device is a first image capture device; as well as The one or more processors maintain alignment of images captured by the first and second image capture devices before and after causing the first image capture device to reduce the frame rate.

15. The method of claim 10, comprising a wearable housing coupled to the image capture device, the display device, and the one or more processors.

16. The method according to claim 13, wherein: The second frame rate is 50 Hz or 60 Hz.

17. The method according to claim 10, wherein: The image parameters correspond to one or more of: gain, exposure, frame brightness, shutter speed, aperture, a signal-to-noise ratio (SNR) of the image, or sharpness of the image.

18. A non-transitory computer-readable medium having instructions embodied thereon, the instructions, when executed by one or more processors, causing the one or more processors to perform operations comprising: capturing one or more images by an image capture device; determining, by one or more processors coupled to the memory, that a parameter of the one or more images satisfies a threshold; as well as In response to the parameter satisfying the threshold, the image capture device is caused to reduce a frame rate at which the one or more images are captured.

19. The non-transitory computer readable medium of claim 18, wherein: The operation also include: wherein said parameters of said one or more images correspond to gain and exposure; as well as The one or more processors cause the image capture device to reduce the frame rate in response to the function of the gain and the exposure exceeding a threshold.

20. The non-transitory computer readable medium of claim 18, wherein: The operations also include: reducing, by the one or more processors, the frame rate from a first frame rate to a second frame rate in response to the parameter exceeding a first threshold; and The frame rate is increased, by the one or more processors, from a second frame rate to the first frame rate in response to the parameter falling below a second threshold, the second threshold being less than the first threshold.

21. The non-transitory computer readable medium of claim 18, wherein: The operations also include: wherein the image capturing device is a first image capturing device; and The one or more processors maintain alignment of images captured by the first and second image capture devices before and after causing the first image capture device to reduce the frame rate.

Images