Gate on array (GOA) circuits for gaze tracking foven

Abstract

A gate-on-array (GOA) circuit for gaze tracking fovea centrally displays is provided. A gate circuit is described that includes a plurality of flip-flops each driven by one of two clock signals, and the plurality of flip-flops are arranged such that adjusting the frequency of one of the clock signals changes the number of subsets of display elements driven simultaneously. In a foveated display, an image can be produced with different effective resolutions at different portions of the image by regarding a group of display elements as a single pixel. For example, in the case where each display element has an independent color, a more detailed portion of an image may be generated, while in the case where multiple consecutive display elements have the same color, a less detailed portion of the image may be generated. The gate circuits described herein may allow multiple rows to be activated to generate outputs simultaneously with each other from at least some of the display elements of the rows.

Description

[0001] Cross-reference to related applications

[0002] This application claims the benefit of U.S. Provisional Patent Application No. 63 / 665,027, filed June 27, 2024, entitled “GOA Circuit for Gaze Tracked Foveated Displays,” and U.S. Non-Provisional Patent Application No. 19 / 242,187, filed June 18, 2025, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This disclosure generally relates to techniques for driving foveated displays. Background Technology

[0004] Foveated imaging is a display technique in which the image resolution on a single image can be varied. For example, the portion of the image corresponding to the center of the retina (fovea) can be rendered at a higher resolution than the portion of the image farther from the retina. Because the peripheral areas of the eye have lower contrast sensitivity compared to their center, the image area rendered at a lower resolution may not be noticeable. Summary of the Invention

[0005] Embodiments of this disclosure may provide a gate circuit comprising a plurality of flip-flops, each driven by one of two clock signals, and arranged such that adjusting the frequency of one of these clock signals changes the number of simultaneously driven subsets of display elements. For example, if the display elements are arranged in rows (e.g., in a grid), multiple rows can be activated simultaneously. In a recessed display, by treating a group of display elements as a single pixel, an image can be produced in which different portions have different effective resolutions. For example, a more detailed portion of the image can be produced when each display element is controlled independently, while a less detailed portion can be produced when multiple consecutive display elements are controlled together to produce the same brightness and / or color. The gate circuit described herein can allow the simultaneous activation of multiple rows of display elements to produce output from at least some of the display elements in these rows simultaneously. Furthermore, the process can be performed dynamically so that the combination of rows activated to produce a given image frame can vary from frame to frame. This approach differs from typical displays, which address rows (or certain other subsets of display elements) one by one in the same order within each image frame. Attached Figure Description

[0006] The accompanying drawings illustrate several exemplary embodiments and are part of this specification. These drawings, together with the following description, illustrate and explain various principles of this disclosure.

[0007] Figure 1 An illustrative concave display according to some embodiments of the present disclosure is depicted.

[0008] Figure 2 Addressing of display elements in a display according to some embodiments of the present disclosure is described.

[0009] Figure 3 An illustrative gate circuit according to some embodiments of the present disclosure is depicted.

[0010] Figure 4 and Figure 5 Some embodiments according to this disclosure are depicted. Figure 3 Examples of signals at various points within the illustrative gate circuit.

[0011] Figure 6 An illustrative system is described, which can be operated according to the techniques described herein, according to some embodiments of the present disclosure.

[0012] Figure 7 This is a flowchart of an exemplary method for rendering image frames in a central concave display according to some embodiments of the present disclosure.

[0013] Figure 8 This is an illustration of an example artificial reality system according to some embodiments of the present disclosure.

[0014] Figure 9 This is an illustration of an example artificial reality system with a handheld device according to some embodiments of the present disclosure.

[0015] Figure 10A These are illustrations of example user interactions within an artificial reality system according to some embodiments of the present disclosure.

[0016] Figure 10B These are illustrations of example user interactions within an artificial reality system according to some embodiments of the present disclosure.

[0017] Figure 11A These are illustrations of example user interactions within an artificial reality system according to some embodiments of the present disclosure.

[0018] Figure 11B These are illustrations of example user interactions within an artificial reality system according to some embodiments of the present disclosure.

[0019] Figure 12This is an illustration of an example wrist-worn wearable device of an artificial reality system according to some embodiments of the present disclosure.

[0020] Figure 13 This is an illustration of an example wearable artificial reality system according to some embodiments of the present disclosure.

[0021] Figure 14 This is an illustration of an example augmented reality system according to some embodiments of the present disclosure.

[0022] Figure 15A This is an illustration of an example virtual reality system according to some embodiments of the present disclosure.

[0023] Figure 15B yes Figure 15A The illustration shows another perspective of the virtual reality system.

[0024] Figure 16 It is a block diagram showing the system components of an example artificial reality system and an example virtual reality system.

[0025] Figure 17 This is a diagram of an example system that includes an eye-tracking subsystem capable of tracking one or both of a user's eyes.

[0026] Figure 18 yes Figure 17 A more detailed illustration of the various aspects of the eye-tracking subsystem shown.

[0027] Throughout the accompanying drawings, the same reference numerals and descriptions denote similar but not necessarily identical elements. While the exemplary embodiments described herein are readily adaptable to various modifications and alternatives, several specific embodiments are illustrated by way of example in these drawings, and these specific embodiments will be described in detail herein. However, the exemplary embodiments described herein are not intended to be limited to the specific forms disclosed. Rather, this disclosure covers all modifications, equivalents, and alternatives falling within the scope of the appended claims. Detailed Implementation

[0028] This disclosure generally relates to techniques for driving foveated displays. As will be explained in more detail below, embodiments of this disclosure may include a gate circuit that generates drive signals to address display elements in the foveated display. Typically, a foveated display can be driven to activate a consecutive subset of display elements in the display. For example, a foveated display can drive each row of a pixel array, where each pixel row is activated independently. However, this approach may require numerous control signals to drive the individual subsets of display elements, resulting in complex and potentially unreliable circuitry for displays with a large number of display elements. For example, some displays are driven by clock modules, each of which can be operated to drive rows of the display, where each module is driven by a clock signal that activates the module, and each module also utilizes multiple clock signals to drive each of the rows connected to that module. For large displays, this approach may require hundreds of clock signals.

[0029] Embodiments of this disclosure may provide a gate circuit comprising a plurality of flip-flops, each driven by one of two clock signals, and arranged such that adjusting the frequency of one of these clock signals changes the number of simultaneously driven subsets of display elements. For example, if the display elements are arranged in rows (e.g., in a grid), multiple rows can be activated simultaneously. In a recessed display, by treating a group of display elements as a single pixel, an image can be produced in which different portions have different effective resolutions. For example, a more detailed portion of the image can be produced when each display element is controlled independently, while a less detailed portion can be produced when multiple consecutive display elements are controlled together to produce the same brightness and / or color. The gate circuit described herein can allow the simultaneous activation of multiple rows of display elements to produce output from at least some of the display elements in these rows simultaneously. Furthermore, the process can be performed dynamically so that the combination of rows activated to produce a given image frame can vary from frame to frame. This approach differs from typical displays, which address rows (or certain other subsets of display elements) one by one in the same order within each image frame.

[0030] As a further explanation, Figure 1 An illustrative concave display according to some embodiments of the present disclosure is depicted, the concave display comprising a grid of 576 (24 × 24) display elements (e.g., pixels and / or subpixels). Figure 1In the example, an image was generated using a concave pattern that includes a central high-detail area, where each of the 64 display elements in that area independently produces light of the desired color and / or brightness. In this concave pattern, a medium-detail area surrounds the high-detail area, and a low-detail area surrounds the medium-detail area. In the medium-detail area, groups of four display elements (2×2 blocks) are operated to produce the same output. That is, when the display generates an image using the depicted concave pattern, in principle, any display element in the high-detail area might look different from each other, while in the medium-detail area, each 2×2 block will look consistent, effectively acting as a larger display element. Similarly, in Figure 1 In the low detail area, a 4×4 block of display elements is manipulated to produce the same output, thus effectively acting as a single display element that is larger than the display elements in the medium detail area.

[0031] It will be recognized that, generally speaking, high-detail areas can be geographically distributed across different locations on the display depending on the image frame, and Figure 1 This is merely an illustrative example of a centrally recessed pattern, where the high-detail area is precisely located in the center. Furthermore, any number of consecutive display elements arranged in any suitable size and shape can be manipulated to produce the same output in any given area. For example, a 3×3 block or a 5×5 block of display elements can be manipulated in this way in a given area of ​​an image. Similarly, any number of areas with different levels of detail can be generated in a given image, and the techniques described herein are not limited to this. Figure 1 The example shows three regions.

[0032] exist Figure 1 In the example, the display element is addressed when both gate signal 102 and display signal 101 are provided to it. As shown, the gate signal can be provided along the rows of the display, and the display signal along the columns. Thus, a single display element can be addressed by providing a gate signal to the row where the display element is located and a display signal to the column where the display element is located. According to some embodiments, each gate signal can be a digital value (e.g., 0 or 1) indicating whether the row is active, and each display signal can be an analog signal indicating how to operate the display element (e.g., indicating a brightness level). In a typical display, an image is generated by sequentially activating rows of the display via successive gate signals, and as each gate signal activates a given row, a display signal is provided along successive columns to indicate how to operate each display element. In this way, each display element is operated to generate light (e.g., row by row from top left to bottom right).

[0033] Figure 2This type of addressing according to some embodiments of this disclosure is described in more detail. Figure 2 The example depicts six display elements arranged in three distinct rows and two distinct columns, which could represent a portion of a larger array of display elements with many rows and columns. Figure 2 As shown, each of the display elements 211, 212, 221, 222, 231, and 232 can be activated by providing both display signal inputs and gate signal inputs along both columns and rows, respectively. For example, display element 211 is activated when both gate signal 1 and display signal 1 are active; otherwise, display element 211 is not activated. As described above, typically, display elements in a display are activated sequentially, one by one, to generate a single image frame. For example, in... Figure 2 In the example, gate signal 1 can be set to active and display signal 1 is provided, causing display element 211 to operate according to display signal 1; then, display signal 1 can be stopped and display signal 2 can be provided, while gate signal 1 remains active, causing display element 212 to operate according to display signal 2; and so on.

[0034] A typical display driver coordinates this row-by-row addressing process by generating a continuous gate signal for each row. In some arrangements, the gate signal is generated by circuitry that sends a valid signal to the desired row by receiving a control input (e.g., a clock signal) associated with that row. In other words, for each different gate signal (e.g., each row of the display), a signal path from a controller to the gate signal generator is required. However, given the complexity of controlling a large number of rows, large displays may experience unsatisfactory errors or other unexpected results.

[0035] As described above, embodiments of this disclosure can provide a gate circuit comprising a plurality of flip-flops, each driven by one of two clock signals, and arranged such that adjusting the frequency of one of the clock signals changes the number of a subset of display elements simultaneously driven. This method for generating gate signals is simpler and more robust than the methods described above that require control signal paths to turn each different gate signal on and off.

[0036] Figure 3 An illustrative gate circuit according to some embodiments of the present disclosure is depicted. Figure 3In the example, gate circuit 300 includes a plurality of first flip-flops 311, 312, 313, 314, 315, and 316, which operate as a shift register. An input pattern 301 (e.g., a digital signal comprising a sequence of high and low voltages) is provided as an input to flip-flops 311, and this input pattern is shifted down through the flip-flops in the shift register according to a clock signal 330. At the clock edge (e.g., a rising or falling edge) of the clock signal 330, each of flip-flops 311, 312, 313, 314, 315, and 316 samples its input and holds it as an output until the next clock edge. Because the output of each of flip-flops 311, 312, 313, 314, 315, and 316 is coupled to the input of the next flip-flop in the shift register, this causes the input pattern to be shifted through the flip-flops in the shift register according to the clock signal 330.

[0037] Additionally, the gate circuit 300 includes a second plurality of flip-flops 321, 322, 323, 324, 325, and 326, each having an input coupled to a corresponding output of a first plurality of flip-flops 311, 312, 313, 314, 315, and 316 (e.g., the output of flip-flop 311 is coupled to the input of flip-flop 321, etc.). The second plurality of flip-flops 321, 322, 323, 324, 325, and 326 operate according to a 1x clock signal 340; therefore, the gate signal outputs (also referred to as gate signals) 331, 332, 333, 334, 335, and 336 are set to active or inactive based on the output of each of the second plurality of flip-flops 321, 322, 323, 324, 325, and 326.

[0038] As will be further described below, the number of simultaneously active gate signals can be controlled by changing the frequency of the Nx clock signal 330 midway through an image frame. For example, in Figure 1 For an illustrative display, it may be necessary to provide a valid gate signal to each of the first four rows of display elements simultaneously. Simultaneously, display signals (or multiple instances of the same display signal) can be directed to four columns (e.g., the first four columns), allowing sixteen display elements to be addressed simultaneously to produce the same output from a 4×4 block of display elements. Therefore, image frames can be generated more efficiently compared to other methods where each display element is addressed individually.

[0039] According to some embodiments, the frequency of Nx clock signal 330 is an integer multiple of the frequency of 1x clock signal 340. For example, the frequency of Nx clock signal 330 can be equal to the frequency of 1x clock signal 340 (the frequency of 1x clock signal 340 × 1), or it can be equal to 2, 3, 4, 5, 6, 7, etc., times the frequency of 1x clock signal 340. A suitable controller can be configured to generate one or both of clock signals 330 and 340, and control the frequency of clock signal 330 based on data describing the image to be generated by the display. The following describes... Figure 6 An example of this controller is described.

[0040] exist Figure 3 In the example, these flip-flops are configured to be triggered by a negative edge (falling edge), as indicated by the "bubble" (circle) at the input of each of flip-flops 311, 312, 313, 314, 315, and 316. While the depicted circuitry does not need to be configured in this manner in all embodiments, arranging the circuitry in this way helps prevent run-on issues between clock signals 330 and 340. This configuration allows flip-flops 311, 312, 313, 314, 315, and 316 to stabilize to new values ​​before flip-flops 321, 322, 323, 324, 325, and 326 latch new inputs.

[0041] Figure 4 Examples of signal variations over time at various points within an illustrative gate circuit 300 according to some embodiments of the present disclosure are depicted. Figure 4 In the example, the digital values ​​of each signal are depicted from left to right over time. As shown in the figure, in... Figure 4 During the time period depicted, the frequency of Nx clock 330 is twice the frequency of 1x clock 340.

[0042] exist Figure 4 In the example, digital input mode 301 is provided as an input to gate circuit 300, causing the input mode to be output from flip-flop 311 on the falling edge of Nx clock 330. Flip-flop 311 also sends the input mode to the input of flip-flop 312. Figure 4 In the example, the Nx clock 330 is triggered on the falling edge of the Nx clock signal. Subsequently, on the falling edge of the second pulse of the Nx clock 330, the output of flip-flop 312 is triggered, and since input mode 301 is still input to flip-flop 311, flip-flop 311 also outputs a high-level signal.

[0043] exist Figure 4In the example, the clock pulses of 1x clock 340 are highlighted with vertical dashed lines; these clock pulses coincide with the rising edge of the 1x clock signal 340. It should be noted that, unlike the clock pulses of Nx clock, which are triggered on the falling edge of the Nx clock signal, the clock pulses of 1x clock 340 are triggered on the rising edge of the 1x clock signal.

[0044] like Figure 4 As shown, at the first pulse of the 1x clock 340, the outputs of flip-flops 311 and 312 are both high, which causes each of the flip-flops 321 and 322 to output a high-level signal, thereby simultaneously generating valid gate signals 331 and 332.

[0045] At the third pulse of Nx clock 330, since the input mode signal is now low, the output of flip-flop 311 goes low, while flip-flop 312 receives a high-level signal from flip-flop 311, and flip-flop 313 receives a high-level signal from flip-flop 312. At the fourth pulse of Nx clock 330, since the output of flip-flop 311 is now low, the output of flip-flop 312 goes low, while flip-flop 313 receives a high-level signal from flip-flop 312, and flip-flop 314 receives a high-level signal from flip-flop 313. Subsequently, at the second pulse of 1x clock 340, the outputs of flip-flops 313 and 314 are both high, causing each of flip-flops 323 and 324 to output a high-level signal, thereby simultaneously generating active gate signals 333 and 334.

[0046] Such as Figure 4 The processes shown can be repeated to simultaneously generate gate signal pairs, thereby activating display row pairs simultaneously. However, as mentioned above, this behavior can be adjusted at any time, including midway through the generation of an image frame, by changing the frequency of Nx clock 330 (e.g., changing it to some other integer multiple of the frequency of 1x clock 340).

[0047] Figure 5 Depicting and Figure 4 The examples depicted are similar to those in other examples, only in Figure 5 In the example, after the initial generation of gate signal 331, the frequency of Nx clock 330 is adjusted from twice the frequency of 1x clock 340 to equal the frequency of 1x clock. Specifically, by combining with the above... Figure 4Operating in the same manner described, at the first pulse of the 1x clock, the outputs of flip-flops 311 and 312 are both high, causing each of flip-flops 321 and 322 to output a high-level signal, thereby simultaneously generating valid gate signals 331 and 332. Subsequently, the clock frequency of the Nx clock 330 decreases to the frequency of the 1x clock 340, thus requiring a longer time for the input mode signal to pass through the shift register including flip-flops 311, 312, 313, 314, etc. Therefore, at the second pulse of the 1x clock, only the valid gate signal 333 is triggered. Similarly, at the third pulse of the 1x clock, only the valid gate signal 334 is triggered.

[0048] This allows for multiple control over the number of gates activated simultaneously during a single image frame. For example, for Figure 1 The display shown initially allows the Nx clock frequency to be four times the 1x clock frequency to simultaneously activate four rows and address a 4×4 block of display elements. For rows containing at least a portion of the medium detail area, the Nx clock frequency can be reduced to twice the 1x clock frequency to simultaneously activate two rows and address a 2×2 block of display elements. For the high detail area, the Nx clock frequency can be reduced to 1x clock frequency to activate a single row and address a single display element. Subsequently, the Nx clock frequency can be increased to twice the 1x clock frequency to simultaneously activate two rows and address a 2×2 block of display elements, thus creating the lower portion of the medium detail area; then, the Nx clock frequency can be increased to four times the 1x clock frequency to simultaneously activate four rows and address a 4×4 block of display elements, thus creating the lower portion of the low detail area.

[0049] However, the above is an example; typically, the frequency of the Nx clock can be modified multiple times when generating a single image frame, including any one or more increases and / or decreases in any suitable order. Furthermore, although... Figure 1 The sizes of the medium and low detail regions in the image are multiples of two, but the rendering performed using the techniques described herein is not limited to this method. For example, the Nx clock frequency can be adjusted to three times the 1x clock frequency to activate three rows simultaneously and address a 3x3 block of display elements.

[0050] Figure 6 An illustrative system 600, operable according to some embodiments of the present disclosure and capable of operating according to the techniques described herein, is depicted. System 600 may, for example, be part of a wearable device such as an artificial reality system, examples of which are described below.

[0051] exist Figure 6In the example, system 600 is partially controlled by system-on-a-chip (SoC) 610, which generates image data for display on display 640. Control of display 640 is provided by display driver 630, which receives image data from SoC 610 and generates gate signals and display signals to address display elements in the display as described above. Figure 6 In the example, the gate signal is generated by the gate circuit 631, which can be implemented, for example, as follows: Figure 3 The gate circuit 300 shown.

[0052] According to some embodiments, the image data provided by the SoC 610 includes digital values ​​indicating the colors of multiple display elements of the display 640. The number of such digital values ​​for a single image frame may be less than the number of display elements in the display because, in a recessed display implemented as described herein, display signals can be provided simultaneously to multiple columns to address multiple display elements concurrently with multiple gate signals. Therefore, the size of the data stream representing the image frame can be smaller than that required to address each display element individually, potentially leading to improved display rendering efficiency.

[0053] In some embodiments, the image data provided by SoC 610 may further include information relating to the recessed layout of the image frame to be rendered, which informs controller 632 how to operate the gate circuitry (e.g., how to control the frequency of the Nx clock of the gate circuitry) to simultaneously activate multiple rows of display 640, at least in part, by simultaneously providing multiple gate signals to the display elements of the display, thereby rendering the image frame. In some embodiments, the information relating to the recessed layout of the image frame to be rendered may include the coordinate positions of various locations in the recessed layout within the image frame, such as the center of a region (e.g., a high-detail region, a medium-detail region, or a low-detail region), a corner of a region, etc. Additionally or alternatively, the information relating to the recessed layout of the image frame to be rendered may include the display element density of a particular region, the size of the region (e.g., horizontal and / or vertical dimensions), and / or any other information indicating which display elements should be addressed simultaneously to operate in the same manner.

[0054] exist Figure 6 In the example, eye-tracking system 620 can detect and measure the position of one or both eyes of the wearer of the device including system 600. The following is combined with... Figure 17 and Figure 18An illustrative eye-tracking system is described. Eye-tracking system 620 can provide eye-tracking data, including indications of the position of one or both of a user's eyes, to SoC 610. The SoC uses this eye-tracking data to generate image data for transmission to display driver 630. For example, SoC 610 can determine the location of high-detail regions of an image frame based on the position of one or both eyes and generate image data accordingly, as described above.

[0055] exist Figure 6 In the example, controller 632 is configured to generate the aforementioned Nx clock signal and 1x clock signal, and provide these clock signals to gate circuitry 631. Controller 632 is also configured to control a display signal generator to generate display signals that, together with gate signals generated by gate circuitry 631, address display elements in display 640. In some embodiments, controller 632 may (e.g., by operating the display signal generator according to the 1x clock signal) generate the display signal and gate signal synchronously. In some embodiments, display signal generator 633 may include a digital-to-analog converter (DAC) configured to convert (e.g., received from SoC 610) digital values ​​indicating color or brightness into one or more analog display signals. For example, RGB digital values ​​may be converted into one or more (e.g., three) analog display signals addressing display elements (e.g., pixels or subpixels) of display 640. Although Figure 6 Not shown, but controller 632 can also be configured to control which column of the columns the display signals are routed to.

[0056] although Figure 6 The various elements are described as separate subsystems, but it will be appreciated that these elements do not need to be implemented in this way. For example, the display driver 630 can be implemented as an integrated circuit that at least implements the gate circuit 631, the controller 632, and the display signal generator 633.

[0057] Based on the general principles described herein, features from any of the embodiments described herein can be combined with each other. These and other embodiments, features, and advantages will be more fully understood upon reading the following detailed description in conjunction with the accompanying drawings and claims.

[0058] Figure 7 This is a flowchart of an exemplary computer implementation of a method 700 for rendering image frames in a concave display according to the techniques described herein. Figure 7 The steps shown can be performed by any suitable computer-executable code and / or computing system (including...). Figure 6 (One or more systems shown) are used to execute this. In one example, Figure 7 Each step shown can represent an algorithm as follows: the structure of the algorithm includes multiple sub-steps and / or is represented by multiple sub-steps.

[0059] Method 700 begins with action 710, in which an input mode is provided to a circuit including a shift register operating at a first clock frequency. For example, the input mode may be provided as an input to gate circuit 300, which includes a shift register comprising flip-flops 311, 312, 313, etc., and which operates at a first clock frequency of Nx clock 330. In action 720, one or more first gate signals are generated from the circuit. For example, one or more first gate signals may be generated from one or more flip-flops of a plurality of flip-flops (e.g., one or more flip-flops 321, 322, 323, etc. in gate circuit 300), each of which is coupled to a corresponding flip-flop in the shift register.

[0060] In action 730, the first gate signal generated in action 720 is directed to one or more rows of display elements of the display. More generally, the first gate signal may be directed to one or more subsets of the display elements of the display. In action 740, the first clock frequency is increased or decreased. For example, the frequency of the Nx clock signal in gate circuit 300 may be increased or decreased. According to some embodiments, the circuit may include components that operate according to another clock signal (e.g., clock 1x), and the first clock frequency may be adjusted to an integer multiple of the frequency of the other clock signal.

[0061] In action 750, one or more second gate signals are generated from the circuit. For example, one or more second gate signals may be generated from one or more flip-flops of a plurality of flip-flops (e.g., one or more flip-flops of flip-flops 321, 322, 323, etc. in gate circuit 300), each of which is coupled to a corresponding flip-flop in a shift register.

[0062] It will be understood that the terms "same time" or "simultaneously" used herein refer to driving multiple rows or columns of display elements, meaning that each row and / or each column is addressed simultaneously. The signals driving the rows and / or columns do not necessarily need to start or end simultaneously to be considered "same time" or "simultaneously" generated. For example, two signals that both trigger on their rising edges can end at different times without affecting their simultaneous addressing. Similarly, two signals that both trigger on their falling edges can start at different times without affecting their simultaneous addressing.

[0063] Example Implementation

[0064] Example 1. A system comprising: a plurality of addressable display elements, the plurality of addressable display elements including a plurality of subsets; a shift register including a first plurality of flip-flops, each of the first plurality of flip-flops coupled to a first clock input; a second plurality of flip-flops, each of the second plurality of flip-flops configured to receive an output from a corresponding flip-flop among the first plurality of flip-flops, each of the second plurality of flip-flops coupled to a second clock input, and each of the second plurality of flip-flops configured to output a gate signal to a corresponding subset of the plurality of subsets of the plurality of addressable display elements; and a controller configured to control one or more subsets of the plurality of addressable display elements to simultaneously receive gate signals from flip-flops among the second plurality of flip-flops by controlling the frequency of the first clock input.

[0065] Example 2. The system according to Example 1, wherein the plurality of addressable display elements are arranged in a grid; and wherein the plurality of subsets of the plurality of addressable display elements are plurality of addressable display element rows in the grid.

[0066] Example 3. A system according to any of Examples 1 to 2, wherein the plurality of subsets of the plurality of addressable display elements are plurality of first subsets of the plurality of addressable display elements; wherein the controller is further configured to send display signals to a selected subset of the plurality of second subsets of the plurality of addressable display elements; and wherein each subset of the plurality of second subsets intersects with each subset of the plurality of first subsets.

[0067] Example 4. A system according to any of Examples 1 to 3, wherein the plurality of addressable display elements are arranged in a grid; and wherein the plurality of first subsets are plurality of addressable display element rows in the grid, and wherein the plurality of second subsets are plurality of addressable display element columns in the grid.

[0068] Example 5. A system according to any of Examples 1 to 4, wherein a display signal controls the brightness and / or color of light to be emitted by display elements in a plurality of second subsets of the plurality of addressable display elements, the display elements in the plurality of second subsets also receiving gate signals from one or more of a plurality of second flip-flops.

[0069] Example 6. A system based on any of Examples 1 to 5, wherein the frequency of the first clock input is an integer multiple of the frequency of the second clock input.

[0070] Example 7. A system according to any of Examples 1 to 6, wherein the plurality of addressable display elements comprises a plurality of pixels and / or a plurality of subpixels.

[0071] Example 8. A system according to any of Examples 1 to 7, wherein at least some of the first plurality of flip-flops in the shift register are configured to receive input from the output of another flip-flop in the first plurality of flip-flops.

[0072] Example 9. A system according to any of Examples 1 to 8, wherein the first flip-flop of a first plurality of flip-flops in the shift register is configured to receive an input mode signal; and wherein each of the first plurality of flip-flops other than the first flip-flop is configured to receive an input from the output of another flip-flop of the first plurality of flip-flops.

[0073] Example 10. A system according to any of Examples 1 to 9, wherein the controller is configured to adjust the frequency of a first clock input while the plurality of addressable display elements are operated to generate an image frame by continuously providing gate signals to each subset of the plurality of subsets of the addressable display elements.

[0074] Example 11. A system based on any of Examples 1 to 10, wherein the controller, shift register, and a plurality of flip-flops are implemented as integrated circuits.

[0075] Example 12. A method comprising: providing an input pattern to a shift register, the shift register including a first plurality of flip-flops, each of the first plurality of flip-flops operating according to a first clock frequency; generating a first gate signal from a first subset of a second plurality of flip-flops, each of the second plurality of flip-flops being configured to receive an output from a corresponding flip-flop in the first plurality of flip-flops; directing each first gate signal to a corresponding row of display elements in a display element grid; increasing or decreasing the first clock frequency; and generating a second gate signal from a second subset of the second plurality of flip-flops, wherein the number of the second plurality of flip-flops in the first subset is different from the number of the second plurality of flip-flops in the second subset.

[0076] Example 13. The method according to Example 12 further includes: directing each second gate signal to a corresponding display element row in the display element grid.

[0077] Example 14. A method according to any of Examples 12 to 13, comprising: generating a corresponding first gate signal from every two flip-flops in a second plurality of flip-flops; reducing the first clock frequency by a factor of 2; and generating a single second gate signal from one of the flip-flops in the second plurality of flip-flops.

[0078] Example 15. The method according to any of Examples 12 to 14 further includes: directing a display signal to one or more columns of display elements in a display element grid, thereby simultaneously directing both the display signal and one or more first gate signals to one or more display elements.

[0079] Example 16. A method according to any of Examples 12 to 15, comprising: directing a display signal to two columns of display elements in a display element grid, and simultaneously directing a corresponding gate signal of a first gate signal to two rows of display elements, thereby simultaneously directing both the display signal and one of the first gate signals to four display elements.

[0080] Example 17. A method according to any of Examples 12 to 16, wherein increasing or decreasing the first clock frequency includes decreasing the first clock frequency by a factor of 2; wherein the display signal is a first display signal; and wherein the method includes: after simultaneously directing both the first display signal and one of a plurality of first gate signals to four display elements, directing a second display signal to a single display element column in a display element grid, and simultaneously directing a single second gate signal to a single display element row.

[0081] Example 18. The method of any of Examples 12 to 17, wherein the display element row comprises a pixel row and / or a subpixel row.

[0082] Example 19. A method according to any of Examples 12 to 18, including: increasing the first clock frequency by an integer multiple.

[0083] Example 20. A method according to any of Examples 12 to 19, comprising: increasing or decreasing a first clock frequency while generating an image frame from a display element grid by continuously providing a gate signal in a first gate signal to each of a plurality of display element rows.

[0084] Example 21. The method of any of Examples 12 to 20, wherein each of the second plurality of flip-flops operates according to a second clock frequency.

[0085] Example 22. The method of any of Examples 12 to 21, wherein the first clock frequency is an integer multiple of the second clock frequency.

[0086] Example 23. A system comprising: a plurality of addressable display elements arranged in a plurality of rows and a plurality of columns; a shift register including a first plurality of flip-flops, each of the first plurality of flip-flops coupled to a first clock input; a second plurality of flip-flops, each of the second plurality of flip-flops configured to receive an output from a corresponding flip-flop among the first plurality of flip-flops, each of the second plurality of flip-flops coupled to a second clock input, and each of the second plurality of flip-flops configured to output a gate signal to a corresponding row among the plurality of rows of the plurality of addressable display elements; and a controller configured to control one or more rows of the plurality of addressable display elements to simultaneously receive gate signals from flip-flops among the second plurality of flip-flops by controlling the frequency of the first clock input.

[0087] The embodiments of this disclosure may include various types of Artificial-Reality (AR) systems, or may be implemented by combining various types of AR systems. AR can be any overlay functionality and / or sensorially detectable content presented by an AR system within a user's physical environment. In other words, AR is a form of reality that has been adjusted in some way before being presented to the user. AR may include and / or represent virtual reality (VR), augmented reality, mixed AR (MAR), or some combination and / or variation of these types of reality. Similarly, AR environments may include VR environments (including non-immersive VR environments, semi-immersive VR environments, and fully immersive VR environments), augmented reality environments (including marker-based augmented reality environments, markerless augmented reality environments, location-based augmented reality environments, and projection-based augmented reality environments), mixed reality environments, and / or any other type or form of mixed reality environment or alternative reality environment.

[0088] AR content can consist entirely of computer-generated content or include computer-generated content combined with acquired (e.g., real-world) content. AR content can include video, audio, haptic feedback, or some combination thereof, any of which can be presented in a single channel or multiple channels (e.g., stereoscopic video providing a three-dimensional (3D) effect to the viewer). Additionally, in some embodiments, AR can be associated with applications, products, accessories, services, or some combination thereof, which are used, for example, to create content in artificial reality and / or otherwise for use in artificial reality (e.g., to perform activities in artificial reality).

[0089] AR systems can be implemented in a variety of different shapes and configurations. Some AR systems can be designed to operate without a near-eye display (NED). Other AR systems may include NEDs, which also provide visibility into the real world (e.g., Figure 14 Augmented reality systems (also known as head-mounted wearable devices or AR glasses)

[1400] or systems that allow users to visually immerse themselves in an artificial reality (e.g., Figure 15A and Figure 15B Virtual reality systems (also known as head-mounted wearable devices or VR headsets)

[1510] . While some AR devices may be standalone systems, others may communicate with and / or cooperate with external devices to provide an AR experience to the user. Examples of such external devices include handheld controllers, mobile devices, desktop computers, devices worn by the user, devices worn by one or more other users, and / or any other suitable external system.

[0090] Figures 8 to 11B An example artificial reality (AR) system according to some embodiments is shown. Figure 8 A first AR system 800 and a first example user interaction are shown, which uses a wrist wearable device 802, a head wearable device (e.g., AR glasses 804) and / or a handheld intermediary processing device (HIPD) 806. Figure 9 A second AR system 900 and a second example user interaction are shown, which uses a wrist wearable device 902, AR glasses 904 and / or HIPD 906. Figure 10A and Figure 10B The interaction between the third AR system 1000 and the third example user 1008 is shown, which uses a wrist wearable device 1002, a head wearable device (e.g., a VR headset 1050) and / or a HIPD 1006. Figure 11A and Figure 11B The interaction between the fourth AR system 1100 and the fourth example user 1108 is shown, which uses a wrist wearable device 1130, a VR headset 1120 and / or a haptic device 1160 (e.g., wearable gloves).

[0091] The following is for reference Figure 12 and Figure 13 This describes a wrist-worn wearable device 1200 and one or more of its components. The wrist-worn wearable device 1200 can be used in wrist-worn wearable devices 802, 902, 1002, and 1130; the following references... Figures 14 to 16The description includes head-worn devices 1400 and 1510 and one or more components thereof, which can be used in AR glasses 804 and 904 or VR headsets 1050 and 1120, respectively.

[0092] refer to Figure 8 The wrist-worn wearable device 802, AR glasses 804, and / or HIPD 806 can be communicatively coupled via a network 825 (e.g., cellular, near-field, Wi-Fi, personal area network, wireless local area network (LAN), etc.). Additionally, the wrist-worn wearable device 802, AR glasses 804, and / or HIPD 806 can also be communicatively coupled to one or more servers 830, computers 840 (e.g., laptops, computers, etc.), mobile devices 850 (e.g., smartphones, tablets, etc.) and / or other electronic devices via the network 825 (e.g., cellular, near-field, Wi-Fi, personal area network, wireless LAN, etc.).

[0093] exist Figure 8 The image shows a user 808 wearing a wrist-worn wearable device 802 and AR glasses 804, with a HIPD 806 placed on their table. The wrist-worn wearable device 802, AR glasses 804, and HIPD 806 facilitate the user's interaction with the AR environment. Specifically, as shown in the first AR system 800, the wrist-worn wearable device 802, AR glasses 804, and / or HIPD 806 enable the presentation of one or more avatars 810, digital representations of contacts 812, and virtual objects 814. As discussed below, the user 808 can interact with one or more avatars 810, digital representations of contacts 812, and virtual objects 814 via the wrist-worn wearable device 802, AR glasses 804, and / or HIPD 806.

[0094] User 808 may use any of the wrist-worn wearable device 802, AR glasses 804, and / or HIPD 806 to provide user input. For example, user 808 may perform one or more gestures to provide user input, which are provided by the wrist-worn wearable device 802 (e.g., using the following reference). Figure 12 and Figure 13 The description includes one or more electromyography (EMG) sensors and / or IMUs that detect and / or transmit signals via AR glasses 804 (e.g., using the following references). Figures 14 to 16The detection is performed by one or more image sensors or cameras. Alternatively or additionally, user 808 may provide user input via one or more touch surfaces of the wrist wearable device 802, AR glasses 804, and / or HIPD 806, and / or voice commands acquired by the microphone of the wrist wearable device 802, AR glasses 804, and / or HIPD 806. In some embodiments, the wrist wearable device 802, AR glasses 804, and / or HIPD 806 includes a digital assistant for assisting user 808 in providing user input (e.g., completing a series of actions, recommending different actions or commands, providing reminders, confirming commands, etc.). In some embodiments, user 808 may provide user input via one or more facial gestures and / or facial expressions. For example, the camera of the wrist wearable device 802, AR glasses 804, and / or HIPD 806 may track user 808's eyes for navigating the user interface.

[0095] The wrist-worn wearable device 802, AR glasses 804, and / or HIPD 806 can operate individually or in combination to allow user 808 to interact with the AR environment. In some embodiments, HIPD 806 is configured to operate as a central hub or control center for the wrist-worn wearable device 802, AR glasses 804, and / or another communication-coupled device. For example, user 808 can provide input for interacting with the AR environment at any of the wrist-worn wearable device 802, AR glasses 804, and / or HIPD 806, and HIPD 806 can identify one or more backend and frontend tasks to enable the execution of the requested interaction and distribute instructions to enable the execution of the one or more backend and frontend tasks at the wrist-worn wearable device 802, AR glasses 804, and / or HIPD 806. In some embodiments, backend tasks are user-insensible background processing tasks (e.g., rendering content, decompressing, compressing, etc.), while frontend tasks are user-insensible user-facing tasks (e.g., presenting information to the user, providing feedback to the user, etc.). HIPD 806 can perform backend tasks and provide operational data corresponding to the backend tasks to the wrist wearable device 802 and / or AR glasses 804, enabling the wrist wearable device 802 and / or AR glasses 804 to perform frontend tasks. In this way, HIPD 806 (which has more computing resources and greater thermal headroom than the wrist wearable device 802 and / or AR glasses 804) performs computationally intensive tasks and reduces the computing resource utilization and / or power consumption of the wrist wearable device 802 and / or AR glasses 804.

[0096] In the example shown in the first AR system 800, HIPD 806 identifies one or more backend and frontend tasks associated with a user request initiating an AR video call with one or more other users (represented by avatar 810 and digital representation 812 of the contact), and issues instructions to execute the one or more backend and frontend tasks. Specifically, HIPD 806 performs backend tasks for processing and / or rendering image data (and other data) associated with the AR video call, and provides AR glasses 804 with operational data associated with the performed backend tasks, causing AR glasses 804 to perform frontend tasks for presenting the AR video call (e.g., presenting avatar 810 and digital representation 812 of the contact).

[0097] In some embodiments, HIPD 806 can function as a focal point or anchor point for information presentation. This allows user 808 to know where the information is presented. For example, as shown in the first AR system 800, an avatar 810 and a digital representation 812 of a contact are presented above HIPD 806. Specifically, HIPD 806 and AR glasses 804 operate in conjunction to determine the location for presenting the avatar 810 and the digital representation 812 of the contact. In some embodiments, information can be presented within a predetermined distance from HIPD 806 (e.g., within 5 meters). For example, as shown in the first AR system 800, a virtual object 814 is presented on a table at a distance from HIPD 806. Similar to the examples above, HIPD 806 and AR glasses 804 can operate in conjunction to determine the location for presenting the virtual object 814. Alternatively, in some embodiments, the presentation of information is not constrained by HIPD 806. More specifically, the avatar 810, the digital representation 812 of the contact, and the virtual object 814 need not be presented within a predetermined distance of HIPD 806.

[0098] The user input provided at the wrist wearable device 802, AR glasses 804, and / or HIPD 806 is coordinated to enable the user to initiate, continue, and / or complete an operation using any device. For example, user 808 may provide user input to AR glasses 804 to cause AR glasses 804 to present virtual object 814, and while AR glasses 804 presents virtual object 814, user 808 may provide one or more gestures via wrist wearable device 802 to interact with and / or manipulate virtual object 814.

[0099] Figure 9The image shows user 908 wearing wrist-worn wearable device 902 and AR glasses 904, and holding HIPD 906. In the second AR system 900, wrist-worn wearable device 902, AR glasses 904, and / or HIPD 906 are used to receive one or more messages and / or provide one or more messages to user 908's contacts. Specifically, wrist-worn wearable device 902, AR glasses 904, and / or HIPD 906 detect and coordinate one or more user inputs to launch a messaging application, and prepare a response to the received message via the messaging application.

[0100] In some embodiments, user 908 launches an application on wrist wearable device 902, AR glasses 904, and / or HIPD 906 via user input, causing the application to launch on at least one device. For example, in a second AR system 900, user 908 performs a gesture associated with a command to launch a messaging application (represented by messaging user interface 916); wrist wearable device 902 detects the gesture and, based on determining that user 908 is wearing AR glasses 904, causes AR glasses 904 to present the messaging user interface 916 of the messaging application. AR glasses 904 may present the messaging user interface 916 to user 908 via its display (e.g., as shown in user 908's field of view 918). In some embodiments, the application is launched and executed on a device (e.g., wrist wearable device 902, AR glasses 904, and / or HIPD 906) that detects user input for launching the application, and that device provides operational data to another device to cause the messaging application to be presented. For example, the wrist-worn wearable device 902 can detect user input to launch a messaging application, launch and run the messaging application, and provide operational data to the AR glasses 904 and / or HIPD 906 to render the messaging application. Alternatively, the application can be launched and executed on a different device than the one that detected the user input. For example, the wrist-worn wearable device 902 can detect gestures associated with launching the messaging application and enable the HIPD 906 to run the messaging application and coordinate its rendering.

[0101] Furthermore, user 908 can provide user input at the wrist wearable device 902, AR glasses 904, and / or HIPD 906 to continue and / or complete an operation initiated at another device. For example, after launching a messaging application via the wrist wearable device 902 and when the messaging user interface 916 is presented on the AR glasses 904, user 908 can provide input at the HIPD 906 to prepare a response (e.g., as shown by a swipe gesture performed on the HIPD 906). The gesture performed by user 908 on the HIPD 906 can be provided and / or displayed on another device. For example, a swipe gesture performed on the HIPD 906 is displayed on the virtual keyboard of the messaging user interface 916 displayed by the AR glasses 904.

[0102] In some embodiments, the wrist wearable device 902, AR glasses 904, HIPD 906, and / or other communication-coupled devices may present one or more notifications to the user 908. The notification may be an indication of a new message, incoming call, application update, status update, etc. The user 908 may select a notification via the wrist wearable device 902, AR glasses 904, or HIPD 906, causing an application or action associated with the notification to be presented on at least one device. For example, the user 908 may receive a notification of a received message at the wrist wearable device 902, AR glasses 904, HIPD 906, and / or other communication-coupled devices, and provide user input at the wrist wearable device 902, AR glasses 904, and / or HIPD 906 to view the notification. The device that detects the user input may cause the application associated with the notification to be launched and / or the application associated with the notification to be presented on the wrist wearable device 902, AR glasses 904, and / or HIPD 906.

[0103] While the examples above describe coordinated input for interacting with messaging applications, user input can be coordinated to interact with any number of applications, including but not limited to gaming applications, social media applications, camera applications, web-based applications, financial applications, etc. For example, AR glasses 904 can present game application data to user 908, while HIPD 906 can use a controller to provide input to the game. Similarly, user 908 can use wrist wearable device 902 to activate the camera of AR glasses 904, and user 908 can use wrist wearable device 902, AR glasses 904, and / or HIPD 906 to manipulate image acquisition (e.g., zoom in or out, apply filters, etc.) and acquire image data.

[0104] Users can interact with the device described in this article in a variety of ways. For example, such as Figure 10A and Figure 10BAs shown, user 1008 can interact with AR system 1000 by wearing VR headset 1050, holding HIPD 1006, and wearing wrist wearable device 1002. In this example, AR system 1000 allows the user to interact with game 1010 by waving their arm. One or more of VR headset 1050, HIPD 1006, and wrist wearable device 1002 can detect the gesture and, in response, display a sword strike in game 1010. Similarly, in Figure 11A and Figure 11B In this example, user 1108 can interact with AR system 1100 by wearing VR headset 1120, haptic device 1160, and wrist wearable device 1130. In this example, AR system 1100 allows the user to interact with game 1110 by waving their arm. One or more of the VR headset 1120, haptic device 1160, and wrist wearable device 1130 can detect the gesture and, in response, display the casting of a spell in game 1110.

[0105] Having discussed the example AR system, this paper will now discuss in more detail the devices used to interact with such AR systems and other computing systems in a wider range. For ease of reference, this paper provides some explanations of the devices and components that may be included in some or all of the example devices discussed below. Certain types of parts described below may be more suitable for a particular set of devices and less suitable for different sets of devices. However, subsequent references to parts defined herein should be considered as included in the description provided.

[0106] In some of the embodiments discussed below, several example devices and systems, including electronic devices and systems, will be discussed. These example devices and systems are not intended to be limiting, and those skilled in the art will understand that alternative devices and systems to the example devices and systems described herein can be used to perform the operations described herein and to construct the systems and devices described herein.

[0107] An electronic device can be a device that uses electrical energy to perform a specific function. An electronic device can be any physical object that contains electronic components such as transistors, resistors, capacitors, diodes, and integrated circuits. Examples of electronic devices include smartphones, laptops, digital cameras, televisions, game consoles, and music players, as well as the various example electronic devices discussed herein. As described herein, an intermediate electronic device can be a device located between two other electronic devices and / or between subsets of multiple components of one or more electronic devices, facilitating communication, data processing, and / or data transmission between the respective electronic devices and / or electronic components.

[0108] An integrated circuit (IC) can be an electronic device composed of multiple interconnected electronic components, such as transistors, resistors, and capacitors. These components can be etched onto a small piece of semiconductor material, such as silicon. ICs can include analog ICs, digital ICs, mixed-signal ICs, and / or any other suitable type or form of IC. Examples of ICs include application-specific integrated circuits (ASICs), processing units, central processing units (CPUs), coprocessors, and accelerators.

[0109] Analog integrated circuits (such as sensors, power management circuits, and operational amplifiers) can process continuous signals and perform analog functions such as amplification, active filtering, demodulation, and mixing. Examples of analog integrated circuits include linear integrated circuits and radio frequency circuits.

[0110] Digital integrated circuits (which may be referred to as logic integrated circuits) may include microprocessors, microcontrollers, memory chips, interfaces, power management circuits, programmable devices, and / or any other suitable type or form of integrated circuit. In some embodiments, examples of integrated circuits include central processing units (CPUs).

[0111] Processing units, such as CPUs, can be electronic components responsible for executing instructions and controlling the operation of electronic devices (e.g., computers). Various types of processors exist, which can be used interchangeably or may be specifically required by the embodiments described herein. For example, a processor can be: (i) a general-purpose processor designed to perform a wide range of tasks, such as running software applications, managing operating systems, and performing arithmetic and logical operations; (ii) a microcontroller designed for specific tasks, such as controlling electronic devices, sensors, and motors; (iii) an accelerator, such as a graphics processing unit (GPU) designed to accelerate the creation and rendering of images, videos, and animations (e.g., virtual reality animation, such as 3D modeling); (iv) a field-programmable gate array (FPGA) that can be programmed and reconfigured post-manufacturing and / or customized to perform specific tasks, such as signal processing, encryption, and machine learning; and / or (v) a digital signal processor (DSP) designed to perform mathematical operations on signals (e.g., audio, video, and radio waves). One or more processors of one or more electronic devices may be used in the various embodiments described herein.

[0112] Memory broadly refers to electronic components in a computer or electronic device that store data and instructions for access and manipulation by a processor. Examples of memory may include: (i) random access memory (RAM) configured to temporarily store data and instructions; (ii) read-only memory (ROM) configured to permanently store data and instructions (e.g., one or more portions of system firmware and / or a bootloader) and / or semi-permanently store data and instructions; (iii) flash memory configured to store data in an electronic device (e.g., a universal serial bus (USB) drive, a memory card, and / or a solid-state drive (SSD)); and / or (iv) cache memory configured to temporarily store frequently accessed data and instructions. As described herein, memory may store structured data (e.g., SQL databases, MongoDB databases, GraphQL data, JSON data, etc.). Other examples of data stored in memory may include: (i) data data, including user account data, user settings, and / or other user data stored by the user; (ii) sensor data detected by one or more sensors and / or otherwise acquired; (iii) media content data, including stored image data, audio data, and documents; (iv) application data, which may include data collected and / or otherwise acquired and stored during the use of the application; and / or any other types of data described herein.

[0113] A controller can be an electronic component that manages and coordinates the operation of other components within an electronic device (e.g., controlling inputs, processing data, and / or generating outputs). Examples of controllers may include: (i) microcontrollers, which include small, low-power controllers commonly used in embedded systems and Internet of Things (IoT) devices; (ii) programmable logic controllers (PLCs), which can be configured for use in industrial automation systems to control and monitor manufacturing processes; (iii) system-on-a-chip (SoC) controllers, which integrate multiple components such as processors, memory, input / output (I / O) interfaces, and other peripherals into a single chip; and / or (iv) digital signal processors (DSPs).

[0114] The power system of an electronic device can be configured to convert input power into a form usable for operating the device. The power system may include various components, such as: (i) a power source, which may be an alternating current (AC) adapter power source or a direct current (DC) adapter power source; (ii) a charger input, which may be configured to use wired and / or wireless connections (this charger input may be part of a peripheral interface, such as USB, microUSB, near-field magnetic coupling, magnetic induction and magnetic resonance charging, and / or radio frequency (RF) charging); (iii) a power management integrated circuit configured to distribute power to the various components of the device and ensure that the device operates within safe limits (e.g., regulating voltage, controlling current, and / or managing heat dissipation); and / or (iv) a battery configured to store power to provide usable power to the components of one or more electronic devices.

[0115] Peripheral interfaces can be electronic components (e.g., electronic components of an electronic device) that allow the electronic device to communicate with other devices or peripheral devices and can provide the ability to input and output data and signals. Examples of peripheral interfaces may include: (i) a Universal Serial Bus (USB) interface and / or a microUSB interface configured to connect a device to an electronic device; (ii) a Bluetooth interface configured to allow multiple devices to communicate with each other, including Bluetooth Low Energy (BLE); (iii) a near-field communication (NFC) interface configured as a short-range wireless interface for operations such as access control; (iv) a POGO pin, which may be a small, spring-loaded pin configured to provide a charging interface; (v) a wireless charging interface; (vi) a global positioning system (GPS) interface; (vii) a Wi-Fi interface for providing connectivity between the device and a wireless network; and / or (viii) a sensor interface.

[0116] Sensors can be electronic components configured to detect physical and environmental changes and generate electrical signals (e.g., electronic components in electronic devices (e.g., wearable devices) and / or electronic components that otherwise communicate electronically with electronic devices). Examples of sensors may include: (i) imaging sensors for collecting imaging data (e.g., including one or more cameras disposed on a corresponding electronic device); (ii) biopotential signal sensors; (iii) inertial measurement units (e.g., IMUs) for detecting, for example, changes in angular velocity, force, magnetic field, and / or acceleration; (iv) heart rate sensors for measuring a user's heart rate; (v) blood oxygen saturation (SpO2) sensors for measuring a user's blood oxygen saturation and / or other biometric data; (vi) capacitive sensors for detecting potential changes at a site on a user's body (e.g., a sensor-skin interface); and / or (vii) light sensors (e.g., time-of-flight sensors, infrared sensors, visible light sensors, etc.).

[0117] Biopotential signal sensing components can be devices used to measure electrical activity within the body (e.g., biopotential signal sensors). Some types of biopotential signal sensors include: (i) electroencephalography (EEG) sensors configured to measure electrical activity in the brain to diagnose neurological disorders; (ii) electrocardiography (ECG or EKG) sensors configured to measure electrical activity in the heart to diagnose heart problems; (iii) electromyography (EMG) sensors configured to measure electrical activity in muscles and diagnose neuromuscular disorders; and (iv) electrooculography (EOG) sensors configured to measure electrical activity in eye muscles to detect eye movements and diagnose eye disorders.

[0118] Applications (e.g., software) stored in the memory of an electronic device may include instructions stored in the memory. Examples of such applications include: (i) games; (ii) word processors; (iii) messaging applications; (iv) media streaming applications; (v) financial applications; (vi) calendars; (vii) clocks; and (viii) communication interface modules for enabling wired and / or wireless connections between different corresponding electronic devices (e.g., IEEE 802.15.4, Wi-Fi, ZigBee, 6LoWPAN, Thread, Z-Wave, Bluetooth Smart, ISA100.11a, WirelessHART, or MiWi; custom wired protocols or standard wired protocols (e.g., Ethernet or HomePlug); and / or any other suitable communication protocol).

[0119] A communication interface can be a mechanism that enables different systems or devices to exchange information and data with each other, including hardware, software, or a combination of both. For example, a communication interface can refer to a physical connector and / or port on a device that enables communication with other devices (e.g., USB, Ethernet, High Definition Multimedia Interface (HDMI), Bluetooth). In some embodiments, a communication interface can refer to a software layer that enables different software programs to communicate with each other (e.g., application programming interface (API), protocols such as Hypertext Transfer Protocol (HTTP) and Transmission Control Protocol / Internet Protocol (TCP / IP)).

[0120] A graphics module can be a component or software module designed to handle graphics operations and / or graphical processes, and the graphics module may include hardware modules and / or software modules.

[0121] Non-transitory computer-readable storage media can be physical devices or storage media that can be used to store electronic data in a non-transitory form (e.g., such that the data is permanently stored until it is intentionally deleted or modified).

[0122] Figure 12 and Figure 13 An example wrist-worn wearable device 1200 and an example computing system 1300 are illustrated according to some embodiments. The wrist-worn wearable device 1200 is described herein... Figure 8 The wearable device 802 described herein is such that the wearable device 802 should be understood as having the characteristics of the wrist wearable device 1200, and vice versa. Figure 13Multiple components of a wrist-worn wearable device 1200 are shown, which can be used individually or in combination, including combinations that include other electronic devices and / or electronic components.

[0123] Figure 12 The image shows a wearable strap 1210 and a watch body 1220 (or capsule) coupled together to form a wrist-worn wearable device 1200. The wrist-worn wearable device 1200 can perform various functions and / or operations associated with navigating in a user interface and selectively activating applications, as well as the above references. Figures 8 to 11B The described functions and / or operations.

[0124] As will be described in more detail below, the operations performed by the wrist-worn wearable device 1200 may include: (i) presenting content to a user (e.g., displaying visual content via display 1205); (ii) detecting (e.g., sensing) user input (e.g., sensing touches on peripheral buttons 1223 and / or touches on the touchscreen of display 1205, sensing gestures detected by sensors (e.g., biopotential sensors); (iii) sensing biometric data (e.g., neuromuscular signals, heart rate, temperature, sleep, etc.) via one or more sensors 1213; messaging (e.g., text, voice, video, etc.); image acquisition via one or more imaging devices or cameras 1225; wireless communication (e.g., cellular, near-field, Wi-Fi, personal area network, etc.); location determination; financial transactions; providing haptic feedback; providing alarms; providing notifications; providing biometric authentication; providing health monitoring; providing sleep monitoring, etc.

[0125] The example functions described above can be performed independently in the watch body 1220, independently in the wearable band 1210, and / or via electronic communication between the watch body 1220 and the wearable band 1210. In some embodiments, the functions can be performed on the wrist wearable device 1200 when an AR environment is presented (e.g., via one of the AR systems 800 to 1100). The wearable device described herein can also be used with other types of AR environments.

[0126] The wearable band 1210 can be configured to be worn by a user such that the inner surface of the wearable structure 1211 of the wearable band 1210 contacts the user's skin. In this example, the sensor 1213 can contact the user's skin when worn by the user. In some examples, one or more of the sensors 1213 can sense biometric data, such as the user's heart rate, saturated oxygen level, temperature, sweat level, neuromuscular signals, or combinations thereof. One or more of the sensors 1213 can also sense data about the user's environment, including the user's motion, altitude, location, orientation, gait, acceleration, position, or combinations thereof. In some embodiments, one or more of the sensors 1213 can be configured to track the position and / or motion of the wearable band 1210. One or more of the sensors 1213 may include the features defined above and / or the following regarding... Figure 12 Any of the multiple sensors discussed.

[0127] One or more sensors of each sensor 1213 may be distributed on the inner and / or outer surface of the wearable band 1210. In some embodiments, one or more sensors of each sensor 1213 are evenly spaced along the wearable band 1210. Alternatively, in some embodiments, one or more sensors of each sensor 1213 are located at different points along the wearable band 1210. Figure 12 As shown, one or more of the sensors 1213 may be the same or different. For example, in some embodiments, one or more of the sensors 1213 may be shaped as a pill (e.g., sensor 1213a), oval, circular, square, elliptical (e.g., sensor 1213c), and / or any other shape that maintains contact with the user's skin (e.g., so that neuromuscular signals and / or other biometric data can be accurately measured at the user's skin). In some embodiments, one or more of the sensors 1213 are aligned to form sensor pairs (e.g., for sensing neuromuscular signals based on differential sensing within each respective sensor). For example, sensor 1213b may be aligned with an adjacent sensor to form sensor pair 1214a, and sensor 1213d may be aligned with an adjacent sensor to form sensor pair 1214b. In some embodiments, the wearable band 1210 does not have sensor pairs. Alternatively, in some embodiments, the wearable band 1210 has a predetermined number of sensor pairs (one sensor pair, three sensor pairs, four sensor pairs, six sensor pairs, sixteen sensor pairs, etc.).

[0128] The wearable band 1210 may include any suitable number of sensors 1213. In some embodiments, the number and arrangement of the sensors 1213 depend on the specific application using the wearable band 1210. For example, the wearable band 1210 may be configured as an armband, wristband, or chest band including multiple sensors 1213, with each use case (e.g., a medical use case) having a different number of sensors 1213, individual sensors of various types among the multiple sensors 1213, and different arrangements compared to gaming use cases or general everyday use cases.

[0129] According to some embodiments, the wearable band 1210 also includes an electrically grounding electrode and a shielding electrode. Similar to sensor 1213, the electrically grounding electrode and shielding electrode may be distributed on the inner surface of the wearable band 1210 such that they contact a portion of the user's skin. For example, the electrically grounding electrode and shielding electrode may be located on the inner surface of coupling mechanism 1216 or on the inner surface of wearable structure 1211. The electrically grounding electrode and shielding electrode may be formed and / or use the same components as sensor 1213. In some embodiments, the wearable band 1210 includes more than one electrically grounding electrode and more than one shielding electrode.

[0130] Sensor 1213 may be formed as part of the wearable structure 1211 of the wearable band 1210. In some embodiments, sensor 1213 is flush or substantially flush with the wearable structure 1211 such that the sensors do not extend beyond the surface of the wearable structure 1211. Although flush with the wearable structure 1211, sensor 1213 is still configured to contact the user's skin (e.g., via a skin contact surface). Alternatively, in some embodiments, sensor 1213 extends beyond the wearable structure 1211 by a predetermined distance (e.g., 0.1 mm to 2 mm) to contact and press into the user's skin. In some embodiments, sensor 1213 is coupled to an actuator (not shown) configured to adjust the extension height of sensor 1213 (e.g., distance from the surface of the wearable structure 1211) such that sensor 1213 contacts and presses into the user's skin. In some embodiments, the actuator adjusts the extension height between 0.01 mm and 1.2 mm. This allows users to customize the position of the sensor 1213 to improve the overall comfort of the wearable band 1210 when worn, while still allowing the sensor 1213 to contact the user's skin. In some embodiments, the sensor 1213 is not distinguishable from the wearable structure 1211 when worn by the user.

[0131] The wearable structure 1211 may be formed of an elastic material, elastomer, or the like, which is configured to be stretched and adapted for wear by a user. In some embodiments, the wearable structure 1211 is a textile or woven fabric. As described above, the sensor 1213 may be formed as part of the wearable structure 1211. For example, the sensor 1213 may be molded into the wearable structure 1211, integrated into the woven fabric (e.g., the sensor 1213 may be sewn into the fabric and simulate the flexibility of the fabric and / or may be composed of a series of woven fabric threads).

[0132] Wearable structure 1211 may include sensors 1213, electronic circuits and / or other electronic components (hereinafter referred to as such) to be included in wearable band 1210. Figure 13 (As described above) Flexible electronic connectors for interconnection. In some embodiments, the flexible electronic connectors are configured to interconnect sensors 1213, electronic circuitry, and / or other electronic components of the wearable band 1210 with corresponding sensors and / or other electronic components of another electronic device (e.g., the watch body 1220). The flexible electronic connectors are configured to move together with the wearable structure 1211 such that adjustments made to the wearable structure 1211 by the user (e.g., resizing, pulling, folding, etc.) do not cause stress or strain on the electrical coupling of the components of the wearable band 1210.

[0133] As described above, the wearable band 1210 is configured to be worn by a user. Specifically, the wearable band 1210 may be shaped or otherwise manipulated for wear by a user. For example, the wearable band 1210 may be shaped to have a generally circular shape, such that the wearable band may be configured to be worn on the user's forearm or wrist. Alternatively, the wearable band 1210 may be shaped to be worn on another part of the user's body (e.g., the user's upper arm (e.g., around the biceps), forearm, chest, leg, etc.). The wearable band 1210 may include a retaining mechanism 1212 (e.g., a hook and loop fastener, etc.) for securing the wearable band 1210 to the user's wrist or other body part. When the wearable band 1210 is worn by the user, the sensor 1213 senses data from the user's skin (referred to as sensor data). In some examples, the sensor 1213 of the wearable band 1210 acquires (e.g., senses and records) neuromuscular signals.

[0134] Sensed data (e.g., sensed neuromuscular signals) can be used to detect and / or determine a user's intention to perform certain motor actions. In some examples, sensor 1213 can sense and record neuromuscular signals from the user when the user performs muscle activation (e.g., movement, gesture, etc.). The detected and / or determined motor actions (e.g., phalanges (or fingers) movement, wrist movement, hand movement, and / or other muscle intentions) can be used to determine control commands or control information (instructions to execute certain commands after the data is sensed) for causing the computing device to execute one or more input commands. For example, sensed neuromuscular signals can be used to control certain user interfaces displayed on display 1205 of the wrist-worn wearable device 1200, and / or can be sent to a device responsible for rendering an artificial reality environment (e.g., a head-mounted display) to perform actions in the associated artificial reality environment (e.g., to control the movement of a virtual device displayed to the user). Muscle activation performed by a user can include: static gestures, such as placing the user's palm down on a table; dynamic gestures, such as grasping a physical or virtual object; and covert gestures that are imperceptible to another person, such as slightly tensing a joint by coordinating the contraction of opposing muscles or using submuscular activation. Muscle activation performed by a user can also include symbolic gestures (e.g., gestures mapped to other gestures, interactions, or commands based on a gesture vocabulary that specifies a mapping from gestures to commands).

[0135] Sensor data sensed by sensor 1213 can be used to provide users with enhanced interaction with physical objects (e.g., devices communicatively coupled to wearable band 1210) and / or virtual objects in artificial reality applications generated by artificial reality systems (e.g., user interface objects presented on display 1205 or another computing device (e.g., smartphones)).

[0136] In some embodiments, the wearable band 1210 includes one or more tactile devices 1346 (e.g., vibratory tactile actuators) configured to provide tactile feedback (e.g., skin sensation and / or kinesthetic sensation) to the user's skin. Sensors 1213 and / or tactile devices 1346 (such as...) Figure 13 (As shown) can be configured to run in combination with multiple applications, including but not limited to health monitoring, social media, games and artificial reality (e.g., applications associated with artificial reality).

[0137] The wearable band 1210 may also include a coupling mechanism 1216 for detachably coupling the capsule (e.g., computing unit) or the watch body 1220 to the wearable band 1210 (via a coupling surface of the watch body 1220). For example, the bracket or shape of the coupling mechanism 1216 may correspond to the shape of the watch body 1220 of the wrist wearable device 1200. In particular, the coupling mechanism 1216 may be configured to receive a coupling surface of the watch body 1220 near the bottom side (e.g., the side opposite the front side where the display 1205 of the watch body 1220 is located) so that a user can push the watch body 1220 down into the coupling mechanism 1216 to attach the watch body 1220 to the coupling mechanism 1216. In some embodiments, the coupling mechanism 1216 may be configured to receive the top side of the watch body 1220 (e.g., the side near the front of the display 1205 of the watch body 1220) which is pushed upward into the bracket rather than downward into the coupling mechanism 1216. In some embodiments, the coupling mechanism 1216 is an integrated component of the wearable strap 1210, such that the wearable strap 1210 and the coupling mechanism 1216 are a single unified structure. In some embodiments, the coupling mechanism 1216 is a frame or housing that allows the coupling surface of the watch body 1220 to be held within or on the coupling mechanism 1216 of the wearable strap 1210 (e.g., bracket, tracking strap, support base, buckle, etc.).

[0138] The coupling mechanism 1216 allows the watch body 1220 to be detachably coupled to the wearable strap 1210 via friction engagement, magnetic coupling, a rotation-based connector, a scissor pin coupling, a retaining spring, one or more magnets, clips, pins, hook-and-loop fasteners, or combinations thereof. A user can perform any type of action to couple the watch body 1220 to the wearable strap 1210 and to detach the watch body 1220 from the wearable strap 1210. For example, a user can twist, slide, rotate, push, pull, or rotate (or combinations thereof) the watch body 1220 relative to the wearable strap 1210 to attach the watch body 1220 to and detach the watch body 1220 from the wearable strap 1210. Alternatively, as discussed below, in some embodiments, the watch body 1220 can be detached from the wearable strap 1210 by actuation of the release mechanism 1229.

[0139] The wearable band 1210 can be coupled to the watch body 1220 to enhance the functionality of the wearable band 1210 (e.g., converting the wearable band 1210 into a wrist-worn wearable device 1200, adding additional computing units and / or batteries to increase the computing resources and / or battery life of the wearable band 1210, adding additional sensors to improve sensed data, etc.). As described above, the wearable band 1210 and coupling mechanism 1216 are configured to operate independently of the watch body 1220 (e.g., perform functions independently of the watch body). For example, the coupling mechanism 1216 may include one or more sensors 1213 that, with or without the watch body 1220, contact the user's skin when the user wears the wearable band 1210 and can provide sensor data for determining control commands.

[0140] Users can detach the watch body 1220 from the wearable strap 1210 to reduce the burden of the wrist wearable device 1200 on the user. In embodiments where the watch body 1220 is detachable, the watch body 1220 may be referred to as a detachable structure, such that in these embodiments, the wrist wearable device 1200 includes a wearable portion (e.g., the wearable strap 1210) and a detachable structure (e.g., the watch body 1220).

[0141] Turning to the watch body 1220, in some examples, the watch body 1220 may have a generally rectangular or circular shape. The watch body 1220 is configured to be worn by a user on their wrist or another body part. More specifically, the watch body 1220 is sized for easy carrying by a user, easy attachment to a part of a user's clothing, and / or easy coupling to a wearable strap 1210 (thus forming a wrist wearable device 1200). As described above, the watch body 1220 may have a shape corresponding to the coupling mechanism 1216 of the wearable strap 1210. In some embodiments, the watch body 1220 includes a single release mechanism 1229 or multiple release mechanisms (e.g., two release mechanisms 1229 positioned on opposite sides of the watch body 1220, such as spring-supported buttons) to detach the watch body 1220 from the wearable strap 1210. The release mechanism 1229 may include, but is not limited to, buttons, knobs, plugs, handles, levers, fasteners, buckles, dials, latches, or combinations thereof.

[0142] A user can actuate the release mechanism 1229 by pushing, rotating, lifting, pressing, moving, or performing other actions on it. Actuation of the release mechanism 1229 can release (e.g., detach) the watch body 1220 from the coupling mechanism 1216 of the wearable band 1210, allowing the user to use the watch body 1220 independently of the wearable band 1210, and vice versa. For example, detaching the watch body 1220 from the wearable band 1210 allows the user to use the rear camera 1225b to capture images. Although the release mechanism 1229 is shown positioned at a corner of the watch body 1220, it can be positioned anywhere on the watch body 1220 that is easily actuated by the user. Additionally, in some embodiments, the wearable band 1210 may also include a corresponding release mechanism for detaching the watch body 1220 from the coupling mechanism 1216. In some embodiments, the release mechanism 1229 is optional, and as described above, the body 1220 can be separated from the coupling mechanism 1216 (e.g., by twisting, rotating, etc.).

[0143] The watch body 1220 may include one or more peripheral buttons 1223 and 1227 for performing various operations at the watch body 1220. For example, peripheral buttons 1223 and 1227 may be used to turn on or wake up the display 1205 (e.g., to bring the display 1205 from sleep to active state), unlock the watch body 1220, increase or decrease the volume, increase or decrease the brightness, interact with one or more applications, interact with one or more user interfaces, etc. Additionally or alternatively, in some embodiments, the display 1205 acts as a touchscreen and allows the user to provide one or more inputs for interacting with the watch body 1220.

[0144] In some embodiments, the watch body 1220 includes one or more sensors 1221. The sensors 1221 of the watch body 1220 may be the same as or different from the sensors 1213 of the wearable band 1210. The sensors 1221 of the watch body 1220 may be distributed on the inner and / or outer surfaces of the watch body 1220. In some embodiments, the sensors 1221 are configured to contact the user's skin when the user wears the watch body 1220. For example, the sensors 1221 may be placed on the underside of the watch body 1220, and the coupling mechanism 1216 may be a bracket with an opening that allows the underside of the watch body 1220 to directly contact the user's skin. Alternatively, in some embodiments, the watch body 1220 does not include sensors configured to contact the user's skin (e.g., sensors including those inside and / or outside the watch body 1220, configured to sense data from the watch body 1220 and data from the surrounding environment). In some embodiments, the sensors 1221 are configured to track the position and / or movement of the watch body 1220.

[0145] The watch body 1220 and the wearable band 1210 can share data using wired communication methods (e.g., Universal Asynchronous Receiver / Transmitter (UART), USB transceiver, etc.) and / or wireless communication methods (e.g., Near Field Communication, Bluetooth, etc.). For example, the watch body 1220 and the wearable band 1210 can share data sensed by sensors 1213 and 1221, as well as application-specific and device-specific information (e.g., active and / or available applications, output devices (e.g., displays, speakers, etc.), input devices (e.g., touchscreens, microphones, imaging sensors, etc.)).

[0146] In some embodiments, the watch body 1220 may include, but is not limited to, a front-facing camera 1225a and / or a rear-facing camera 1225b, sensors 1221 (e.g., biometric sensors, IMUs, heart rate sensors, oxygen saturation sensors, neuromuscular signal sensors, altimeter sensors, temperature sensors, bioimpedance sensors, pedometer sensors, optical sensors (e.g., imaging sensors 1363), touch sensors, sweat sensors, etc.). In some embodiments, the watch body 1220 may include one or more haptic devices 1376 (e.g., vibratory haptic actuators) configured to provide haptic feedback to the user (e.g., skin sensation and / or kinesthetic sensation, etc.). Sensors 1321 and / or haptic devices 1376 may also be configured to operate in conjunction with multiple applications, including but not limited to health monitoring applications, social media applications, gaming applications, and artificial reality applications (e.g., applications associated with artificial reality).

[0147] As described above, the watch body 1220 and the wearable strap 1210, when coupled, can form a wrist wearable device 1200. The watch body 1220 and the wearable strap 1210, when coupled, can function as a single device to perform the functions (operation, detection, communication, etc.) described herein. In some embodiments, each device may be provided with specific instructions for performing one or more operations of the wrist wearable device 1200. For example, if it is determined that the watch body 1220 does not include a neuromuscular signal sensor, the wearable strap 1210 may include alternative instructions for performing the associated instructions (e.g., providing sensed neuromuscular signal data to the watch body 1220 via different electronic devices). The operations of the wrist wearable device 1200 may be performed by the watch body 1220 alone or by the watch body in conjunction with the wearable strap 1210 (e.g., via a corresponding processor and / or hardware component), or vice versa. In some embodiments, the operations of the wrist wearable device 1200, the watch body 1220, and / or the wearable strap 1210 may be performed in conjunction with one or more processors and / or hardware components.

[0148] For reference below Figure 13 As described in the block diagram, the wearable band 1210 and / or the watch body 1220 may each include independent resources required to perform functions independently. For example, the wearable band 1210 and / or the watch body 1220 may each include a power source (e.g., a battery), memory, data storage, a processor (e.g., a central processing unit (CPU)), communication, a light source, and / or input / output devices.

[0149] Figure 13 Block diagrams are shown of a computing system 1330 corresponding to a wearable strap 1210 and a computing system 1360 corresponding to a watch body 1220, according to some embodiments. According to some embodiments, the computing system 1300 of the wrist wearable device 1200 includes a combination of components of the wearable strap computing system 1330 and components of the watch body computing system 1360.

[0150] The watch body 1220 and / or wearable strap 1210 may include one or more components shown in the watch body computing system 1360. In some embodiments, a single integrated circuit may include all or most of the components of the watch body computing system 1360, which are included in a single integrated circuit. Alternatively, in some embodiments, the components of the watch body computing system 1360 may be included in multiple communication-coupled integrated circuits. In some embodiments, the watch body computing system 1360 may be configured (e.g., via a wired or wireless connection) to couple with the wearable strap computing system 1330, which may allow the two computing systems to share components, distribute tasks, and / or (individually or as a single device) perform other operations described herein.

[0151] The computing system 1360 of the meter may include one or more processors 1379, controllers 1377, peripheral interfaces 1361, power systems 1395, and memory (e.g., memory 1380).

[0152] The power system 1395 may include a charger input 1396, a power-management integrated circuit (PMIC) 1397, and a battery 1398. In some embodiments, the watch body 1220 and the wearable band 1210 may have their own batteries (e.g., batteries 1398 and 1359) and may share power with each other. The watch body 1220 and the wearable band 1210 may use various technologies to receive charge. In some embodiments, the watch body 1220 and the wearable band 1210 may use wired charging components (e.g., a power cord) to receive charge. Alternatively or additionally, the watch body 1220 and / or the wearable band 1210 may be configured for wireless charging. For example, a portable charging device may be designed to mate with a portion of the watch body 1220 and / or a portion of the wearable band 1210 and wirelessly deliver available power to the battery 1398 of the watch body 1220 and / or the battery 1359 of the wearable band 1210. The watch body 1220 and the wearable band 1210 may have independent power systems (e.g., power systems 1395 and 1356, respectively) to enable each to operate independently. The watch body 1220 and the wearable band 1210 may also share power (e.g., one can charge the other) via their respective PMICs (e.g., PMICs 1397 and 1358) and charger inputs (e.g., charger inputs 1357 and 1396), which can share power via power conductors and ground conductors and / or via wireless charging antennas.

[0153] In some embodiments, the peripheral interface 1361 may include one or more sensors 1321. Sensor 1321 may include one or more coupling sensors 1362 for detecting when the watch body 1220 is coupled to another electronic device (e.g., the wearable band 1210). Sensor 1321 may include one or more imaging sensors 1363 (e.g., one or more of a camera 1325 and / or a separate imaging sensor 1363 (e.g., a thermal imaging sensor)). In some embodiments, sensor 1321 may include one or more SpO2 sensors 1364. In some embodiments, sensor 1321 may include one or more bioelectric potential signal sensors (e.g., an EMG sensor 1365, which may be disposed on the user-facing internal portion of the watch body 1220 and / or the wearable band 1210). In some embodiments, sensor 1321 may include one or more capacitive sensors 1366. In some embodiments, sensor 1321 may include one or more heart rate sensors 1367. In some embodiments, sensor 1321 may include one or more IMU sensors 1368. In some embodiments, one or more IMU sensors 1368 may be configured to detect the movement of a user’s hand, or the movement of the watch body 1220 in other positions where it is placed or held.

[0154] In some embodiments, one or more of the sensors 1321 may provide an example human-machine interface. For example, a set of neuromuscular sensors (e.g., EMG sensor 1365) may be arranged circumferentially along the wearable band 1210, wherein the inner surface of the EMG sensor 1365 is configured to contact the user's skin. Any suitable number of neuromuscular sensors may be used (e.g., 2 to 20 sensors). The number and arrangement of the neuromuscular sensors may depend on the specific application for which the wearable device is used. For example, the wearable band 1210 may be used to generate control information for controlling an augmented reality system, controlling a robot, controlling a vehicle, scrolling text, controlling a virtual avatar, or controlling any other suitable control task.

[0155] In some embodiments, the neuromuscular sensors may be coupled together using flexible electronics incorporated into a wireless device, and the outputs of one or more sensing elements may optionally be processed using hardware signal processing circuitry (e.g., to perform amplification, filtering, and / or rectification). In other embodiments, at least some of the signal processing of the outputs of the sensing elements may be performed in software (e.g., processor 1379). Therefore, signal processing of the signals sampled by the sensors may be performed in hardware, in software, or by any suitable combination of hardware and software, as the aspects of the techniques described herein are not limited in this respect.

[0156] Neuromuscular signals can be processed in various ways. For example, the output of the EMG sensor 1365 can be provided to an analog front-end that can be configured to perform analog processing (e.g., amplification, noise reduction, filtering, etc.) on the recorded signal. The processed analog signal can then be provided to an analog-to-digital converter (ADC) that can convert the analog signal into a digital signal that can be processed by one or more computer processors. Furthermore, although this example is discussed in the context of interfacing with an EMG sensor, the embodiments described herein can also be implemented in wearable interfaces with other types of sensors, including but not limited to mechanomyography (MMG) sensors, sonomyography (SMG) sensors, and electrical impedance tomography (EIT) sensors.

[0157] In some embodiments, the peripheral interface 1361 includes a Near Field Communication (NFC) component 1369, a Global Positioning System (GPS) component 1370, a Long-Term Evolution (LTE) component 1371, and / or a Wi-Fi and / or Bluetooth communication component 1372. In some embodiments, the peripheral interface 1361 includes one or more buttons 1373 (e.g., ...). Figure 12 The peripheral interface 1361 includes peripheral buttons 1223 and 1227, which, when selected by the user, cause an operation to be performed at the body 1220. In some embodiments, the peripheral interface 1361 includes one or more indicators (e.g., light-emitting diodes, LEDs) to provide the user with visual indicators (e.g., received message, low battery, active microphone and / or camera, etc.).

[0158] The watch body 1220 may include at least one display 1205 for displaying a visual representation of information or data to a user, including user interface elements and / or three-dimensional virtual objects. The display may also include a touchscreen for inputting user input (e.g., touch gestures and swipe gestures). The watch body 1220 may include at least one speaker 1374 and at least one microphone 1375 for providing audio signals to the user and receiving audio input from the user. The user may provide user input through the microphone 1375 and may also receive audio output from the speaker 1374 as part of a haptic event provided by a haptic controller 1378. The watch body 1220 may include at least one camera 1325, including a front-facing camera 1325a and a rear-facing camera 1325b. The camera 1325 may include an ultra-wide-angle camera, a wide-angle camera, a fisheye camera, a spherical camera, a telephoto camera, a depth-sensing camera, or other types of cameras.

[0159] The watch body computing system 1360 may include one or more haptic controllers 1378 and associated components (e.g., haptic devices 1376) for providing haptic events at the watch body 1220 (e.g., a vibrational sensation or audio output responding to an event at the watch body 1220). The haptic controllers 1378 may communicate with one or more haptic devices 1376 (e.g., electroacoustic devices), which may include speakers in one or more loudspeakers 1374 and / or other audio components and / or electromechanical devices that convert energy into linear motion (e.g., motors, electromagnetic coils, electroactive polymers, piezoelectric actuators, electrostatic actuators, or other haptic output generating components (e.g., components that convert electrical signals into haptic outputs on the device)). The haptic controllers 1378 may provide haptic events that a user of the watch body 1220 can perceive. In some embodiments, the one or more haptic controllers 1378 may receive input signals from an application in application 1382.

[0160] In some embodiments, the table body computing system 1360 may include a memory 1380, which may be controlled by one or more memory controllers of the controller 1377. In some embodiments, software components stored in the memory 1380 include one or more applications 1382 configured to perform operations at the table body 1220. In some embodiments, the one or more applications 1382 may include games, word processors, messaging applications, calling applications, web browsers, social media applications, media streaming applications, financial applications, calendars, clocks, etc. In some embodiments, software components stored in the memory 1380 include one or more communication interface modules 1383 as defined above. In some embodiments, software components stored in the memory 1380 include: one or more graphics modules 1384 for rendering, encoding, and / or decoding audio data and / or video data; and one or more data management modules 1385 for collecting, organizing, and / or providing access to data 1387 stored in the memory 1380. In some embodiments, one or more applications and / or one or more modules in the applications 1382 may work together to perform various tasks at the table body 1220.

[0161] In some embodiments, the software components stored in the memory 1380 may include one or more operating systems 1381 (e.g., a Linux-based operating system, an Android operating system, etc.). The memory 1380 may also include data 1387. The data 1387 may include data 1388A, sensor data 1389A, media content data 1390, and application data 1391.

[0162] It should be understood that the table body computing system 1360 is an example of a computing system within the table body 1220, and the table body 1220 may have more or fewer components than those shown in the table body computing system 1360, may combine two or more components, and / or may have different configurations and / or arrangements of these components. The various components shown in the table body computing system 1360 are implemented in hardware, software, firmware, or combinations thereof (including one or more signal processing circuits and / or application-specific integrated circuits).

[0163] Turning to wearable band computing system 1330, one or more components that may be included in wearable band 1210 are shown. Wearable band computing system 1330 may include more or fewer components than those shown in body computing system 1360, may combine two or more components, and / or may have different configurations and / or arrangements of some or all of these components. In some embodiments, all or most of the components of wearable band computing system 1330 are included in a single integrated circuit. Alternatively, in some embodiments, the components of wearable band computing system 1330 are included in multiple communication-coupled integrated circuits. As described above, in some embodiments, wearable band computing system 1330 is configured to couple with body computing system 1360 (e.g., via a wired or wireless connection), which allows the two computing systems to share components, assign tasks, and / or (individually or as a single device) perform other operations described herein.

[0164] Similar to the wearable computing system 1360, the wearable computing system 1330 may include: one or more processors 1349; one or more controllers 1347 (including one or more haptic controllers 3148); a peripheral interface 1331, which may include one or more sensors 1313 and other peripheral devices; a power supply (e.g., a power system 1356); and a memory (e.g., a memory 1350), which includes an operating system (e.g., an operating system 1351), data (e.g., data 1354, which includes data 1388B, sensor data 1389B, etc.) and one or more modules (e.g., a communication interface module 1352, a data management module 1353, etc.).

[0165] One or more of the sensors 1313 may be similar to sensor 1321 of the watch computing system 1360. For example, sensor 1313 may include one or more coupling sensors 1332, one or more SpO2 sensors 1334, one or more EMG sensors 1335, one or more capacitive sensors 1336, one or more heart rate sensors 1337, and one or more IMU sensors 1338.

[0166] Peripheral interface 1331 may also include other components similar to those included in peripheral interface 1361 of watch computing system 1360, as described above with reference to peripheral interface 1361. These other components include NFC component 1339, GPS component 1340, LTE component 1341, Wi-Fi and / or Bluetooth communication component 1342 and / or one or more haptic devices 1346. In some embodiments, peripheral interface 1331 includes one or more buttons 1343, a display 1333, a speaker 1344, a microphone (MIC) 1345, and a camera 1355. In some embodiments, peripheral interface 1331 includes one or more indicators, such as LEDs.

[0167] It should be understood that the wearable band computing system 1330 is an example of a computing system within the wearable band 1210, and the wearable band 1210 may have more or fewer components than those shown in the wearable band computing system 1330, may combine two or more components, and / or may have different configurations and / or arrangements of these components. The various components shown in the wearable band computing system 1330 may be implemented as one or more of a combination of hardware, software, or firmware (including one or more signal processing circuits and / or application-specific integrated circuits).

[0168] refer to Figure 12 The wrist wearable device 1200 is an example of a wearable strap 1210 and a watch body 1220 coupled together, and therefore the wrist wearable device 1200 will be understood to include the components shown and described for the wearable strap computing system 1330 and the watch body computing system 1360. In some embodiments, the wrist wearable device 1200 has a split architecture (e.g., a split mechanical architecture, a split electronic architecture, etc.) between the watch body 1220 and the wearable strap 1210. In other words, all the components shown in the wearable strap computing system 1330 and the watch body computing system 1360 can be accommodated or otherwise arranged in the combined wrist wearable device 1200, or accommodated or otherwise arranged within a single component in the watch body 1220, the wearable strap 1210 and / or portions thereof (e.g., the coupling mechanism 1216 of the wearable strap 1210).

[0169] The above technology can be used with any device for sensing neuromuscular signals, but it can also be used with other types of wearable devices for sensing neuromuscular signals, such as body wearables or head wearables that may have neuromuscular sensors closer to the brain or spine.

[0170] In some embodiments, the wrist wearable device 1200 may be used in conjunction with a head wearable device (e.g., AR glasses 1400 and VR system 1510) and / or HIPD, and the wrist wearable device 1200 may also be configured to allow a user to control any aspect of the artificial reality (e.g., by controlling user interface objects in the artificial reality using EMG-based gestures, and / or by allowing a user to interact with a touchscreen on the wrist wearable device to also control aspects of the artificial reality). Having described example wrist wearable devices in this manner, attention now turns to example head wearable devices, such as AR glasses 1400 and VR headset 1510.

[0171] Figures 14 to 16 An example artificial reality system is shown, which can be used as or in conjunction with a wrist-worn wearable device 1200. In some embodiments, such as Figure 14 As shown, the AR system 1400 includes glasses device 1402. In some embodiments, such as Figure 15A and Figure 15B As shown, VR system 1510 includes a head-mounted display (HMD) 1512. In some embodiments, AR system 1400 and VR system 1510 may include one or more similar components (e.g., components for presenting an interactive artificial reality environment, such as a processor, memory, and / or presentation device, including one or more displays and / or one or more waveguides), some of which will be referenced to Figure 16 A more detailed description is provided below. As described herein, a head-mounted wearable device may include components of glasses device 1402 and / or head-mounted display 1512. Some embodiments of the head-mounted wearable device do not include any display, including any of the displays described with reference to AR system 1400 and / or VR system 1510. Although the example artificial reality systems are described herein as AR system 1400 and VR system 1510, respectively, one or both of the example AR systems described herein may be configured to present a fully immersive virtual reality scene presented in substantially the entire field of view of the user, or a more subtle augmented reality scene presented in a portion (not the entirety) of the user's field of view.

[0172] Figure 14 An example visual depiction of an AR system 1400 including glasses device 1402 (e.g., herein, the AR system 1400 may also be described as augmented reality glasses and / or smart glasses) is shown. The AR system 1400 may include… Figure 14Additional electronic components (e.g., wearable accessory devices and / or intermediate processing devices, not shown) that communicate electronically with or are otherwise configured to be used in conjunction with the eyewear device 1402. In some embodiments, the wearable accessory device and / or intermediate processing device may be configured to be connected via a coupling sensor 1624 ( Figure 16 The electronic communication coupling mechanism is coupled to the eyewear device 1402, wherein the coupling sensor 1624 can detect when the electronic device is physically or electronically coupled to the eyewear device 1402. In some embodiments, the eyewear device 1402 may be configured to couple to the housing 1690. Figure 16 The housing may include one or more additional coupling mechanisms configured to couple with additional accessory devices. Figure 14 The components shown can be implemented in hardware, software, firmware, or a combination thereof (including one or more signal processing components and / or application-specific integrated circuits (ASICs)).

[0173] The eyewear device 1402 includes mechanical eyewear components, which include a frame 1404 configured to hold one or more lenses (e.g., one or both of lenses 1406-1 and 1406-2). Those skilled in the art will recognize that the eyewear device 1402 may include additional mechanical components, such as hinges configured to allow partial folding and unfolding of the frame 1404 of the eyewear device 1402, a bridge configured to span the gap between lenses 1406-1 and 1406-2 and rest on the user's nose, nose pads configured to rest on the bridge of the nose and provide support for the eyewear device 1402, earpieces configured to rest on the user's ears and provide additional support for the eyewear device 1402, and temples configured to extend from the hinges to the earpieces of the eyewear device 1402, etc. Those skilled in the art will also recognize that some examples of the AR system 1400 may not include the mechanical components described herein. For example, a smart contact lens configured to present artificial reality to a user may not include any component of the eyewear device 1402.

[0174] The eyeglasses device 1402 includes multiple electronic components, many of which will be described in the following reference. Figure 16 To provide a more detailed description. Figure 14The diagram illustrates some example electronic components, including acoustic sensors 1425-1, 1425-2, 1425-3, 1425-4, 1425-5, and 1425-6, which may be distributed along a large portion of the frame 1404 of the eyeglasses device 1402. The eyeglasses device 1402 also includes a left camera 1439A and a right camera 1439B located on different sides of the frame 1404. The eyeglasses device 1402 also includes a processor 1448 (or any other suitable type or form of integrated circuit) embedded in a portion of the frame 1404.

[0175] Figure 15A and Figure 15B A VR system 1510 according to some embodiments is shown, which includes a head-mounted display (HMD) 1512 (e.g., also referred to herein as an artificial reality head-mounted device, a head-worn device, a VR head-mounted device, etc.). As mentioned, some artificial reality systems (e.g., AR system 1400) may substantially use virtual experiences to replace one or more of a user's visual and / or other sensory perceptions of the real world, rather than blending artificial reality with real reality.

[0176] The HMD 1512 includes a front body 1514 and a frame 1516 (e.g., a strip or strap) shaped to fit the user's head. In some embodiments, the front body 1514 and / or frame 1516 include one or more electronic components (e.g., a display, IMU, tracking transmitter, or detector) for facilitating the presentation and / or interaction with AR and / or VR systems. In some embodiments, such as Figure 15B As shown, the HMD 1512 includes an output audio transducer (e.g., audio transducer 1518). In some embodiments, such as Figure 15B As shown, one or more components (e.g., one or more output audio transducers 1518 and frame 1516) (e.g., a portion or all of frame 1516 and / or audio transducers 1518) may be configured to be attached to and detached from HMD 1512 (e.g., detachably attached to HMD 1512). In some embodiments, coupling a detachable component to HMD 1512 enables the detachable component to enter into electronic communication with HMD 1512.

[0177] Figure 15A and Figure 15BThe VR system 1510 is also shown to include one or more cameras, such as a left camera 1539A and a right camera 1539B, which may resemble the left and right cameras 1439A and 1439B on the frame 1404 of the glasses device 1402. In some embodiments, the VR system 1510 includes one or more additional cameras (e.g., cameras 1539C and 1539D) that can be configured to enhance the image data acquired by the left camera 1539A and right camera 1539B by providing more information. For example, camera 1539C may be used to provide color information not identified by cameras 1539A and 1539B. In some embodiments, one or more of the cameras 1539A through 1539D may include an optional infrared (IR) cutoff filter configured to remove IR light received at the respective camera sensor.

[0178] Figure 16 A computing system 1620 and an optional housing 1690 are shown, both of which display components that may be included in the AR system 1400 and / or the VR system 1510. In some embodiments, the optional housing 1690 may include more or fewer components, depending on the practical limitations of the respective AR system described.

[0179] In some embodiments, the computing system 1620 may include one or more peripheral interfaces 1622A, and / or an optional housing 1690 may include one or more peripheral interfaces 1622B. Each of the computing system 1620 and the optional housing 1690 may also include one or more power systems 1642A and 1642B, one or more controllers 1646 (including one or more haptic controllers 1647), one or more processors 1648A and 1648B (as defined above, including any examples provided), and memories 1650A and 1650B, all of which may communicate electronically with each other. For example, the one or more processors 1648A and 1648B may be configured to execute instructions stored in memories 1650A and 1650B, which may cause a controller in one or more controllers 1646 to cause operations to be performed at one or more peripheral devices connected to peripheral interfaces 1622A and / or 1622B. In some embodiments, each of the described operations may be powered by power provided by power systems 1642A and / or 1642B.

[0180] In some embodiments, peripheral interface 1622A may include one or more devices configured as part of computing system 1620, some of which have been defined and / or referenced above. Figure 12 and Figure 13 The wrist-worn wearable device shown is described. For example, peripheral interface 1622A may include one or more sensors 1623A. Some example sensors 1623A include one or more coupling sensors 1624, one or more acoustic sensors 1625, one or more imaging sensors 1626, one or more EMG sensors 1627, one or more capacitive sensors 1628, one or more IMU sensors 1629, and / or any other type of sensor explained above or described with reference to any other embodiments discussed herein.

[0181] In some embodiments, peripheral interfaces 1622A and 1622B may include one or more additional peripheral devices, including one or more NFC devices 1630, one or more GPS devices 1631, one or more LTE devices 1632, one or more Wi-Fi and / or Bluetooth devices 1633, one or more buttons 1634 (e.g., including slide-able or otherwise adjustable buttons), one or more displays 1635A and 1635B, one or more speakers 1636A and 1636B, one or more microphones 1637, one or more cameras 1638A and 1638B (e.g., including a left camera 1639A and / or a right camera 1639B), one or more haptic devices 1640, and / or any other type of peripheral device as defined above or described with reference to any other embodiments discussed herein.

[0182] AR systems can include various types of visual feedback mechanisms (e.g., presentation devices). For example, the display devices in AR system 1400 and / or VR system 1510 can include one or more liquid-crystal displays (LCDs), light-emitting diode (LED) displays, organic LED (OLED) displays, and / or any other suitable type of display. An AR system can include a single display (e.g., configured for binocular viewing), or can provide a separate display for each eye, which can provide additional flexibility for zoom adjustment and / or for correcting refractive errors associated with the user's vision. Some embodiments of AR systems also include an optical subsystem with one or more lenses (e.g., conventional concave or convex lenses, Fresnellens, or adjustable liquid lenses) through which the user views the display.

[0183] For example, corresponding displays 1635A and 1635B may be coupled to each of lenses 1406-1 and 1406-2 of the AR system 1400. Displays 1635A and 1635B may be coupled to each of lenses 1406-1 and 1406-2, and they may present an image or a series of images to the user together or independently. In some embodiments, the AR system 1400 includes a single display 1635A or 1635B (e.g., a near-eye display) or two or more displays 1635A and 1635B. In some embodiments, a first group of one or more displays 1635A and 1635B may be used to present an augmented reality environment, and a second group of one or more display devices 1635A and 1635B may be used to present a virtual reality environment. In some embodiments, one or more waveguides are used in conjunction with presenting artificial reality content to the user of the AR system 1400 (e.g., as a means of delivering light from one or more displays 1635A and 1635B to the user's eyes). In some embodiments, one or more waveguides are wholly or partially integrated into the eyewear device 1402. In addition to or as an alternative to a display screen, some artificial reality systems include one or more projection systems. For example, each display device in AR system 1400 and / or VR system 1510 may include (e.g., using waveguides) a micro-LED projector that projects light onto the display device, such as a transparent combination lens that allows ambient light to pass through. The display device can refract the projected light toward the user's pupil, allowing the user to simultaneously view both artificial reality content and the real world. Artificial reality systems can also be configured with any other suitable type or form of image projection system. In some embodiments, one or more waveguides are provided in addition to or as an alternative to one or more displays 1635A and 1635B.

[0184] The computing system 1620 of the AR system 1400 or the optional housing 1690 may include some or all of the components of the power systems 1642A and 1642B. The power systems 1642A and 1642B may include one or more charger inputs 1643, one or more PMICs 1644, and / or one or more batteries 1645A and 1644B.

[0185] Memory 1650A and 1650B may include instructions and data, some or all of which may be stored within memory 1650A and 1650B as a non-transitory computer-readable storage medium. For example, memory 1650A and 1650B may include one or more operating systems 1651, one or more applications 1652, one or more communication interface applications 1653A and 1653B, one or more graphics applications 1654A and 1654B, one or more AR processing applications 1655A and 1655B, and / or any other type of data as defined above or described with reference to any other embodiments discussed herein.

[0186] Memory 1650A and 1650B also include data 1660A and 1660B, which can be used in conjunction with one or more of the applications discussed above. Data 1660A and 1660B may include data 1661, sensor data 1662A and 1662B, media content data 1663A, AR application data 1664A and 1664B, and / or any other type of data as defined above or described with reference to any other embodiments discussed herein.

[0187] In some embodiments, the controller 1646 of the glasses device 1402 can process information generated by sensors 1623A and / or 1623B on the glasses device 1402, and / or by another electronic device within the AR system 1400. For example, the controller 1646 can process information from acoustic sensors 1425-1 and 1425-2. For each detected sound, the controller 1646 can perform direction of arrival (DOA) estimation to estimate the direction from which the detected sound arrives at the glasses device 1402 of the AR system 1400. When one or more of the acoustic sensors 1625 (e.g., acoustic sensors 1425-1, 1425-2) detect sound, the controller 1646 can use this information to populate an audio dataset (e.g., in...). Figure 16 (represented as sensor data 1662A and 1662B).

[0188] In some embodiments, physical electronic connectors can transmit information between the eyewear device 1402 and another electronic device, and / or between one or more processors 1448, 1648A, 1648B and controller 1646 in the AR system 1400 or VR system 1510. This information can be in the form of optical data, electronic data, wireless data, or any other transmissible data format. Offloading the processing of information generated by the eyewear device 1402 to an intermediate processing device can reduce the weight and heat of the eyewear device, making it more comfortable and safer for the user. In some embodiments, optional wearable accessory devices (e.g., electronic neckbands) are coupled to the eyewear device 1402 via one or more connectors. The connectors can be wired or wireless connectors and can include electronic components and / or non-electronic (e.g., structural) components. In some embodiments, the eyewear device 1402 and the wearable accessory device can operate independently without any wired or wireless connection between them.

[0189] In some cases, pairing an external device (e.g., a mid-processing device (e.g., HIPD 806, 906, 1006)) with eyeglasses device 1402 (e.g., as part of AR system 1400) enables eyeglasses device 1402 to achieve a form factor similar to that of a pair of glasses while still providing sufficient battery power and computing power for the extended capabilities. Some or all of the battery power, computing resources, and / or additional features of AR system 1400 may be provided by the paired device or shared between the paired device and eyeglasses device 1402, thereby reducing the overall weight, heat profile, and form factor of eyeglasses device 1402 while allowing eyeglasses device 1402 to maintain its desired functionality. For example, wearable accessory devices may allow components otherwise included on eyeglasses device 1402 to be included in the wearable accessory device and / or mid-processing device, thereby transferring the weight load from the user's head and neck to one or more other parts of the user's body. In some embodiments, the mid-processing device has a larger surface area thereon that diffuses and disperses heat to the surrounding environment. Therefore, the intermediate processing device allows for greater battery capacity and computing power compared to what might otherwise be possible on a standalone glasses device 1402. Since the weight carried by the wearable accessory device is less intrusive to the user than the weight carried by the glasses device 1402, users can tolerate wearing a lighter glasses device and carrying or wearing paired devices for longer periods compared to enduring the weight of a standalone glasses device, thus allowing the artificial reality environment to be more fully integrated into the user's daily activities.

[0190] AR systems can include various types of computer vision components and computer vision subsystems. For example, AR system 1400 and / or VR system 1510 can include one or more optical sensors, such as two-dimensional (2D) cameras or three-dimensional (3D) cameras, time-of-flight depth sensors, structured light emitters and detectors, single-beam rangefinders or scanning laser rangefinders, 3D LiDAR sensors, and / or any other suitable type or form of optical sensor. AR systems can process data from one or more of these sensors to identify the user's location and / or aspects of the user's real-world physical environment, including the location of real-world objects within the real-world physical environment. In some embodiments, the methods described herein are used to map the real world to provide the user with context about their surroundings, and / or to generate digital twins (e.g., interactive virtual objects), and various other functionalities. For example, Figure 15A and Figure 15B A VR system 1510 with cameras 1539A to 1539D is shown. This VR system can be used to provide depth information to create voxel fields and two-dimensional meshes, thereby providing the user with object information to avoid collisions.

[0191] In some embodiments, the AR system 1400 and / or VR system 1510 may include a haptic feedback system that can be integrated into a headpiece, gloves, bodysuit, handheld controller, environmental device (e.g., a chair or mat), and / or any other type of device or system (e.g., wearable devices discussed herein). The haptic feedback system can provide various types of skin feedback, including vibration, force, tension, shear stress, texture, and / or temperature. The haptic feedback system can also provide various types of kinematic feedback, such as motion and compliance. Haptic feedback can be implemented using motors, piezoelectric actuators, fluid systems, and / or various other types of feedback mechanisms. The haptic feedback system can be implemented independently of other artificial reality devices, within other artificial reality devices, and / or in conjunction with other artificial reality devices.

[0192] In some embodiments of artificial reality systems (e.g., AR system 1400 and / or VR system 1510), ambient light (e.g., a real-time feed of the user's surroundings as normally perceived) may penetrate the display elements of the corresponding head-mounted wearable device that is presenting aspects of the AR system. In some embodiments, ambient light may penetrate a portion (less than the entire AR environment) of the AR environment presented within the user's field of view (e.g., a portion of the AR environment where physical objects are located within a designated boundary (e.g., a guard boundary) in the user's real-world environment, which is configured to be used by the user while interacting with the AR environment). For example, visual user interface elements (e.g., notification user interface elements) may be presented on the head-mounted wearable device, and a certain amount of ambient light (e.g., 15% to 50% of ambient light) may penetrate the user interface element, allowing the user to distinguish at least a portion of the physical environment on which the user interface element is displayed.

[0193] In some embodiments, the system described herein may also include an eye-tracking subsystem designed to identify and track various characteristics of one or both eyes of a user (e.g., the user's gaze direction). In some examples, the phrase "eye tracking" may refer to the process of measuring, detecting, sensing, determining, and / or monitoring the position, orientation, and / or movement of the eyes. The disclosed system may measure the position, orientation, and / or movement of the eyes in a variety of different ways, including by using various optical-based eye-tracking techniques, ultrasonic-based eye-tracking techniques, etc. The eye-tracking subsystem may be configured in a variety of different ways and may include a variety of different eye-tracking hardware components or other computer vision components. For example, the eye-tracking subsystem may include a variety of different optical sensors, such as two-dimensional (2D) cameras or 3D cameras, time-of-flight depth sensors, single-beam rangefinders or scanning laser rangefinders, 3D LiDAR sensors, and / or any other suitable type or form of optical sensor. In this example, the processing subsystem can process data from one or more of these sensors to measure, detect, determine, and / or otherwise monitor the position, orientation, and / or movement of a user's single or binocular eye.

[0194] Figure 17 This is an illustration of an example system 1700 that incorporates an eye-tracking subsystem capable of tracking a user's single or binocular eyes. (See example 1700.) Figure 17As depicted, system 1700 may include a light source 1702, an optical subsystem 1704, an eye-tracking subsystem 1706, and / or a control subsystem 1708. In some examples, light source 1702 may generate light for an image (e.g., to be presented to the viewer's eye 1701). Light source 1702 may represent any of a variety of suitable devices. For example, light source 1702 may include a two-dimensional projector (e.g., an LCoS display), a scanning source (e.g., a scanning laser), or other devices (e.g., an LCD, LED display, OLED display, active-matrix OLED (AMOLED) display, transparent OLED (TOLED) display, waveguide, or some other display capable of generating light for presenting an image to a viewer). In some examples, the image may represent a virtual image, which may refer to an optical image formed according to the apparent divergence of light rays from a point in space, rather than an image formed according to the actual divergence of light rays.

[0195] In some embodiments, the optical subsystem 1704 may receive light generated by the light source 1702 and generate a converging beam 1720 including an image based on the received light. In some examples, the optical subsystem 1704 may include any number of lenses (e.g., Fresnel lenses, convex lenses, concave lenses), apertures, filters, mirrors, prisms, and / or other optical components that may be combined with actuators and / or other devices. In particular, the actuators and / or other devices may translate and / or rotate one or more of the plurality of optical components to change one or more aspects of the converging beam 1720. Furthermore, various mechanical couplings may be used to maintain the relative spacing and / or orientation of the respective optical components in any suitable combination.

[0196] In one embodiment, the eye-tracking subsystem 1706 can generate tracking information indicating the gaze angle of a viewer's eye 1701. In this embodiment, the control subsystem 1708 can control aspects of the optical subsystem 1704 (e.g., the angle of incidence of the converging beam 1720) based at least in part on this tracking information. Additionally, in some examples, the control subsystem 1708 can store and utilize historical tracking information (e.g., a history of tracking information over a given duration (e.g., a fraction of a second)) to predict the gaze angle of the eye 1701 (e.g., the angle between the visual axis and the anatomical axis of the eye 1701). In some embodiments, the eye-tracking subsystem 1706 can detect radiation emitted from a part of the eye 1701 (e.g., the cornea, iris, or pupil, etc.) to determine the current gaze angle of the eye 1701. In other examples, the eye-tracking subsystem 1706 can employ a wavefront sensor to track the current position of the pupil.

[0197] Any number of techniques can be used to track the eye 1701. Some techniques may involve illuminating the eye 1701 with infrared light and measuring the reflection using at least one optical sensor tuned to be sensitive to infrared light. Information about how infrared light is reflected from the eye 1701 can be analyzed to determine one or more locations, one or more orientations, and / or one or more movements of one or more eye features (e.g., cornea, pupil, iris, and / or retinal vessels).

[0198] In some examples, the radiation acquired by the sensors of the eye-tracking subsystem 1706 can be digitized (i.e., converted into electronic signals). Furthermore, the sensors can send a digital representation of this electronic signal to one or more processors (e.g., a processor associated with a device including the eye-tracking subsystem 1706). The eye-tracking subsystem 1706 can include any of a variety of sensors in various different configurations. For example, the eye-tracking subsystem 1706 can include an infrared detector that responds to infrared radiation. The infrared detector can be a thermal detector, a photon detector, and / or any other suitable type of detector. The thermal detector can include a detector that responds to the thermal effects of incident infrared radiation.

[0199] In some examples, one or more processors may process digital representations generated by one or more sensors of the eye-tracking subsystem 1706 to track the movement of the eye 1701. In another example, these processors may track the movement of the eye 1701 by executing an algorithm represented by computer-executable instructions stored in non-transitory memory. In some examples, on-chip logic (e.g., an application-specific integrated circuit or ASIC) may be used to execute at least a portion of such an algorithm. As noted, the eye-tracking subsystem 1706 may be programmed to track the movement of the eye 1701 using the output of one or more sensors. In some embodiments, the eye-tracking subsystem 1706 may analyze the digital representations generated by the sensors to extract eye rotation information from changes in reflection. In one embodiment, the eye-tracking subsystem 1706 may use corneal reflections or flashes (also known as Purkinje images) and / or the center of the pupil 1722 of the eye as features to be tracked over time.

[0200] In some embodiments, the eye-tracking subsystem 1706 can generate a corneal reflection using the center of the pupil 1722 of the eye and uncollimated infrared or near-infrared light. In these embodiments, the eye-tracking subsystem 1706 can calculate the gaze direction of the eye 1701 using the vector between the center of the pupil 1722 and the corneal reflection. In some embodiments, the disclosed system can perform a calibration process on an individual (using, for example, supervised or unsupervised techniques) before tracking the user's eyes. For example, the calibration process may include guiding the user to look at one or more points displayed on a display while the eye-tracking system records values ​​corresponding to each gaze position associated with each point.

[0201] In some embodiments, the eye-tracking subsystem 1706 may use two types of infrared and / or near-infrared (also known as active light) eye-tracking techniques: bright-pupil eye-tracking and dark-pupil eye-tracking, which can be distinguished based on the position of the illumination source relative to the optical elements used. If the illumination is coaxial with the light path, the eye 1701 can act as a retroreflector when light is reflected back from the retina, producing a bright-pupil effect similar to the red-eye effect in photography. If the illumination source is offset from the light path, the pupil 1722 of the eye may appear darker because the retroreflection from the retina is deviated from the sensor orientation. In some embodiments, bright-pupil tracking can produce greater iris / pupil contrast to allow robust eye-tracking in the presence of iris pigmentation and can be characterized by reduced interference (e.g., interference from eyelashes and other blurry features). Bright-pupil tracking can also allow tracking in illumination conditions ranging from complete darkness to very bright environments.

[0202] In some embodiments, control subsystem 1708 may control light source 1702 and / or optical subsystem 1704 to reduce optical aberrations (e.g., chromatic aberration and / or monochromatic aberration) in the image that may be caused by or affected by eye 1701. In some examples, as mentioned above, control subsystem 1708 may use tracking information from eye-tracking subsystem 1706 to perform this control. For example, when controlling light source 1702, control subsystem 1708 may (e.g., through image rendering) modify the light generated by light source 1702 to modify (e.g., pre-distort) the image, thereby reducing aberrations in the image caused by eye 1701.

[0203] The disclosed system can track both the position and relative size of the pupil (e.g., due to pupil dilation and / or constriction). In some examples, the eye-tracking devices and components (e.g., sensors and / or sources) used to detect and / or track the pupil may be different (or differently calibrated) for different types of eyes. For example, the frequency range of the sensors may be different (or individually calibrated) for eyes of different colors and / or different pupil types, and / or sizes, etc. Therefore, it may be necessary to calibrate the various eye-tracking components (e.g., infrared sources and / or sensors) described herein for each individual user and / or eye.

[0204] The disclosed system can track eyes with and without ophthalmic correction (e.g., correction provided by contact lenses worn by the user). In some embodiments, ophthalmic corrective elements (e.g., adjustable lenses) can be directly integrated into the artificial reality system described herein. In some examples, the color of the user's eyes may require modification of the corresponding eye-tracking algorithm. For example, the eye-tracking algorithm may need to be modified at least in part based on, for example, the different color contrast between brown and blue eyes.

[0205] Figure 18 yes Figure 17 A more detailed illustration of various aspects of the eye-tracking subsystem shown is provided below. As illustrated, the eye-tracking subsystem 1800 may include at least one source 1804 and at least one sensor 1806. Source 1804 generally represents an element capable of emitting radiation of any type or form. In one example, source 1804 may generate visible radiation, infrared radiation, and / or near-infrared radiation. In some examples, source 1804 may radiate the non-collimated infrared and / or near-infrared portions of the electromagnetic spectrum toward the user's eye 1802. Source 1804 may utilize various sampling rates and velocities. For example, the disclosed system may use a source with a high sampling rate to acquire fixational eye movements of the user's eye 1802 and / or accurately measure the saccade dynamics of the user's eye 1802. As mentioned above, any type or form of eye-tracking technology (including optical eye-tracking technology, ultrasound-based eye-tracking technology, etc.) can be used to track the user's eye 1802.

[0206] Sensor 1806 broadly represents any type or form of element capable of detecting radiation (e.g., radiation reflected from a user's eye 1802). Examples of sensor 1806 include, but are not limited to, charge-coupled devices (CCDs), photodiode arrays, and / or sensor devices based on complementary metal-oxide-semiconductor (CMOS). In one example, sensor 1806 may represent a sensor with predetermined parameters, including but not limited to dynamic resolution range, linearity, and / or other characteristics specifically selected and / or designed for eye tracking.

[0207] As detailed above, the eye-tracking subsystem 1800 can generate one or more blinks. As detailed above, blink 1803 can represent a reflection of radiation from the user's eye structure (e.g., infrared radiation from an infrared source (e.g., source 1804)). In various embodiments, eye-tracking algorithms executed by a processor (located within or outside the AIVR) can be used to track blink 1803 and / or the user's pupils. For example, the AIVR may include a processor and / or storage devices to perform eye tracking locally, and / or include transceivers to send and receive data required for performing eye tracking on external devices (e.g., mobile phones, cloud servers, or other computing devices).

[0208] Figure 18 An example image 1805 acquired by an eye-tracking subsystem (e.g., eye-tracking subsystem 1800) is shown. In this example, image 1805 may include both the user's pupil 1808 and a flicker 1810 near the user's pupil. In some examples, an artificial intelligence-based algorithm (e.g., a computer vision-based algorithm) may be used to identify the pupil 1808 and / or the flicker 1810. In one embodiment, image 1805 may represent a single frame in a series of frames that can be continuously analyzed to track the user's eye 1802. Furthermore, the pupil 1808 and / or the flicker 1810 may be tracked over a period of time to determine the user's gaze.

[0209] In one example, the eye-tracking subsystem 1800 can be configured to identify and measure the user's interpupillary distance (IPD). In some embodiments, the eye-tracking subsystem 1800 can measure and / or calculate the user's IPD while the user is wearing the artificial reality system. In these embodiments, the eye-tracking subsystem 1800 can detect the position of the user's eyes and can use this information to calculate the user's IPD.

[0210] As mentioned, the eye-tracking system or subsystem disclosed herein can track a user's eye position and / or eye movements in various ways. In one example, one or more light sources and / or optical sensors can acquire images of the user's eyes. The eye-tracking subsystem can then use the acquired information to determine the interpupillary distance, interocular distance, and / or 3D position (e.g., for distortion correction purposes) of each eye, including the magnitude of torsion and rotation (i.e., roll, pitch, and yaw) of each eye and / or the gaze direction of each eye. In one example, infrared light can be emitted by the eye-tracking subsystem and reflected from each eye. The reflected light can be received or detected by optical sensors and analyzed to extract eye rotation data from changes in the infrared light reflected by each eye.

[0211] An eye-tracking subsystem can use any of a variety of different methods to track a user's eyes. For example, a light source (e.g., an infrared LED) can emit a dot pattern onto each of the user's eyes. The eye-tracking subsystem can then detect and analyze the reflections of the dot pattern from each of the user's eyes (e.g., via optical sensors coupled to the artificial reality system) to identify the position of each of the user's pupils. Thus, the eye-tracking subsystem can track up to six degrees of freedom for each eye (i.e., 3D position, roll, pitch, and yaw) and can combine at least a subset of the tracking amounts from the user's eyes to estimate the gaze point (i.e., the 3D position or location in the virtual scene the user is looking at) and / or IPD.

[0212] In some cases, the distance between a user's pupil and the display can change when the user's eyes move to view in different directions. This change in distance between the pupil and the display when the viewing direction changes is known as "pupil migration," and because light focuses at different locations when the pupil-display distance changes, it can cause perceived distortion for the user. Therefore, measuring the distortion at different eye positions and pupillary distances relative to the display and generating distortion corrections for these different positions and distances allows for mitigation of distortion caused by pupil migration by tracking the 3D position of the user's eyes and applying distortion corrections corresponding to the 3D position of each eye at a given point in time. Thus, knowing the 3D position of each eye in the user's eyes allows for mitigation of distortion caused by changes in the distance between the pupil and the display by applying distortion corrections to each 3D eye position. Furthermore, as mentioned above, knowing the position of each eye in the user's eyes also enables the eye-tracking subsystem to automatically adjust the user's IPD (Interpupillary Distance).

[0213] In some embodiments, the display subsystem may include various additional subsystems that can operate in conjunction with the eye-tracking subsystem described herein. For example, the display subsystem may include a zoom subsystem, a scene rendering module, and / or a vergence-processing module. The zoom subsystem can cause the left and right display elements to change the focal length of the display device. In one embodiment, the zoom subsystem can physically change the distance between the display and the optics, through which the display can be viewed, by moving the display, the optics, or both. Alternatively, moving or translating two lenses relative to each other can also be used to change the focal length of the display. Therefore, the zoom subsystem may include actuators or motors that move the display and / or the optics to change the distance between them. The zoom subsystem may be separate from or integrated into the display subsystem. The zoom subsystem may also be integrated into or separate from the actuation subsystem and / or eye-tracking subsystem described herein.

[0214] In one example, the display subsystem may include a convergence processing module configured to determine the convergence depth of a user's gaze based on the gaze point and / or the estimated intersection of the gaze lines determined by the eye-tracking subsystem. Convergence can refer to the simultaneous movement or rotation of both eyes in opposite directions to maintain a single binocular vision, which can be performed naturally and automatically by the human eye. Therefore, the location where the user's eyes approach is the location the user is looking at, and is often also the location where the user's eyes are focused. For example, the convergence processing module may perform triangulation on the gaze line to estimate the distance or depth from the user, associated with the intersection of the gaze lines. The depth associated with the intersection of the gaze lines can then be used as an approximation of the adjustment distance, which can identify the distance between the location where the user's eyes are pointing and the user. Thus, the convergence distance can allow determination of the location where the user's eyes should focus, and the depth of the user's eyes from the point of focus, thereby providing information for rendering adjustments to the virtual scene (e.g., objects or focal planes).

[0215] The convergence processing module can collaborate with the eye-tracking subsystem described herein to adjust the display subsystem to take into account the user's convergence depth. When a user focuses on a distant object, their pupils may be slightly further apart than when focusing on a near object. The eye-tracking subsystem can acquire information about the user's convergence or depth of focus and can adjust the display subsystem to be closer when the user's eyes focus on or approach a near object, and to be further away when the user's eyes focus on or approach a distant object.

[0216] For example, the eye-tracking information generated by the aforementioned eye-tracking subsystem can also be used to modify various aspects of how different computer-generated images are presented. For instance, the display subsystem can be configured to modify at least one aspect of how computer-generated images are presented based on the information generated by the eye-tracking subsystem. For example, the computer-generated image can be modified based on the user's eye movements so that if the user looks up, the computer-generated image can move upwards on the screen. Similarly, if the user looks to the side or down, the computer-generated image can move to the side or down on the screen. If the user's eyes are closed, the computer-generated image can be paused or removed from the display and resumed once the user's eyes are open again.

[0217] The aforementioned eye-tracking subsystem can be incorporated in various ways into one or more of the various artificial reality systems described herein. For example, one or more components of the various components of system 1700 and / or eye-tracking subsystem 1800 can be incorporated into any of the augmented reality and / or virtual reality systems described herein to enable these systems to perform various eye-tracking tasks (including one or more of the multiple eye-tracking operations described herein).

[0218] As detailed above, the computing devices and systems described and / or illustrated herein broadly refer to any type or form of computing device or system capable of executing computer-readable instructions (e.g., those contained in the modules described herein). In the most basic configuration of these one or more computing devices, each of these one or more computing devices may include at least one storage device and at least one physical processor.

[0219] In some examples, the term "storage device" broadly refers to any type or form of volatile or non-volatile storage device or storage medium capable of storing data and / or computer-readable instructions. In one example, a storage device may store, load, and / or maintain one or more of the modules described herein. Examples of storage devices include, but are not limited to, random access memory (RAM), read-only memory (ROM), flash memory, hard disk drive (HDD), solid-state drive (SSD), optical disk drive, cache memory, variations or combinations of one or more of the above, or any other suitable storage memory.

[0220] In some examples, the term "physical processor" broadly refers to any type or form of hardware-implemented processing unit capable of interpreting and / or executing computer-readable instructions. In one example, a physical processor can access and / or modify one or more modules stored in the aforementioned storage device. Examples of physical processors include, but are not limited to, microprocessors, microcontrollers, central processing units (CPUs), field-programmable gate arrays (FPGAs) implementing soft-core processors, application-specific integrated circuits (ASICs), some of the above, variations or combinations of the above, or any other suitable physical processor.

[0221] Although shown as separate elements, the multiple modules described and / or illustrated herein may represent portions of a single module or application. Additionally, in some embodiments, one or more of these modules may represent one or more software applications or programs that, when executed by a computing device, enable the computing device to perform one or more tasks. For example, one or more of the multiple modules described and / or illustrated herein may represent a module stored on and configured to run on one or more of the multiple computing devices or systems described and / or illustrated herein. One or more of these modules may also represent all or part of one or more dedicated computers configured to perform one or more tasks.

[0222] Additionally, one or more of the modules described herein can convert data, physical devices, and / or representations of physical devices from one form to another. Alternatively or additionally, one or more of the modules described herein can convert the processor, volatile memory, non-volatile memory, and / or any other parts of a computing device from one form to another by executing on, storing data on, and / or otherwise interacting with the computing device.

[0223] In some embodiments, the term "computer-readable medium" broadly refers to any form of device, carrier, or medium capable of storing or carrying computer-readable instructions. Examples of computer-readable media include, but are not limited to, transport media and non-transitory media, such as carrier waves, magnetic storage media (e.g., hard disk drives, magnetic tape drives, and floppy disks), optical storage media (e.g., compact disks (CDs), digital video disks (DVDs), and Blu-ray discs), electronic storage media (e.g., solid-state drives and flash memory media), and other distribution systems.

[0224] The process parameters and order of steps described and / or illustrated herein are given by way of example only and may be changed as needed. For example, while multiple steps shown and / or described herein may be shown or discussed in a particular order, these steps do not necessarily need to be performed in the shown or discussed order. The various exemplary methods described and / or illustrated herein may also omit one or more of the multiple steps described or illustrated herein, or may include additional steps in addition to those disclosed.

[0225] The foregoing description has been provided to enable others skilled in the art to optimally utilize various aspects of the exemplary embodiments disclosed herein. This exemplary description is not intended to be exhaustive or limited to any precise form disclosed. Many modifications and variations are possible without departing from the spirit and scope of this disclosure. The embodiments disclosed herein should be considered illustrative in all respects and not restrictive. Reference should be made to the appended claims and their equivalents in determining the scope of this disclosure.

[0226] Unless otherwise stated, the terms “connected to” and “coupled to” (and their derivatives) as used in this specification and claims shall be construed as allowing both direct and indirect connections (i.e., via other elements or components). Additionally, the terms “a” or “an” as used in this specification and claims shall be construed as meaning “at least one of…”. Finally, for ease of use, the terms “comprising” and “having” (and their derivatives) as used in this specification and claims are interchangeable with the word “including” and have the same meaning as the word “comprising”.

Claims

1. A system comprising: a plurality of addressable display elements, the plurality of addressable display elements comprising a plurality of subsets; a shift register comprising a first plurality of flip-flops, each coupled to a first clock input; a second plurality of flip-flops, each configured to receive an output from a respective flip-flop of the first plurality of flip-flops, each flip-flop of the second plurality of flip-flops coupled to a second clock input, and each flip-flop of the second plurality of flip-flops configured to output a gate signal to a respective subset of the plurality of subsets of the plurality of addressable display elements; and a controller configured to control which subset or subsets of the plurality of addressable display elements simultaneously receive a gate signal from a flip-flop of the second plurality of flip-flops by controlling a frequency of the first clock input.

2. The system of claim 1, wherein, the plurality of addressable display elements are arranged in a grid; and wherein the plurality of subsets of the plurality of addressable display elements are rows of the plurality of addressable display elements in the grid.

3. The system of claim 1, wherein, the plurality of subsets of the plurality of addressable display elements are a first plurality of subsets of the plurality of addressable display elements; wherein the controller is further configured to send a display signal to a selected subset of a second plurality of subsets of the plurality of addressable display elements; and wherein each subset of the second plurality of subsets intersects each subset of the first plurality of subsets.

4. The system of claim 3, wherein, the plurality of addressable display elements are arranged in a grid; and wherein the first plurality of subsets are rows of the plurality of addressable display elements in the grid, and wherein the second plurality of subsets are columns of the plurality of addressable display elements in the grid.

5. The system of claim 3, wherein, the display signal controls a luminance and / or a color of light to be emitted by a display element of the second plurality of subsets of the plurality of addressable display elements, the display element of the second plurality of subsets also receiving a gate signal from one or more flip-flops of the second plurality of flip-flops.

6. The system of claim 1, wherein, the frequency of the first clock input is an integer multiple of the frequency of the second clock input.

7. The system of claim 1, wherein, at least some of the first plurality of flip-flops in the shift register are configured to receive an input from an output of another flip-flop of the first plurality of flip-flops.

8. The system of claim 1, wherein, a first flip-flop of the first plurality of flip-flops in the shift register is configured to receive an input mode signal; and wherein each flip-flop of the first plurality of flip-flops other than the first flip-flop is configured to receive an input from an output of another flip-flop of the first plurality of flip-flops.

9. The system of claim 1, wherein, the controller is configured to adjust the frequency of the first clock input while the plurality of addressable display elements are operated to produce an image frame by successively providing a gate signal to each subset of the plurality of subsets of the plurality of addressable display elements.

10. The system of claim 1, wherein, the controller, the shift register, and the second plurality of flip-flops are implemented as an integrated circuit.

11. A method comprising: An input pattern is provided to a shift register, the shift register comprising a first plurality of flip-flops each operating according to a first clock frequency; A first gate signal is generated from a first subset of a second plurality of flip-flops each configured to receive an output from a respective flip-flop of the first plurality of flip-flops; Each of the first gate signals is directed to a respective row of display elements in a grid of display elements; The first clock frequency is increased or decreased; And A second gate signal is generated from a second subset of the second plurality of flip-flops, wherein the number of the second plurality of flip-flops in the first subset is different from the number of the second plurality of flip-flops in the second subset.

12. The method of claim 11, further comprising: Each of the second gate signals is directed to a respective row of display elements in the grid of display elements.

13. The method of claim 11, comprising: A respective first gate signal is generated from each two of the second plurality of flip-flops; The first clock frequency is decreased by a factor of two; And A single second gate signal is generated from one of the second plurality of flip-flops.

14. The method of claim 11, further comprising: A display signal is directed to one or more columns of display elements in the grid of display elements, thereby directing both the display signal and one or more of the first gate signals to one or more display elements.

15. The method of claim 14, comprising: The display signal is directed to two columns of display elements in the grid of display elements and simultaneously a respective one of the first gate signals is directed to two rows of display elements, thereby simultaneously directing both the display signal and one of the first gate signals to four display elements.

16. The method of claim 15, wherein, Increasing or decreasing the first clock frequency comprises decreasing the first clock frequency by a factor of two, wherein the display signal is a first display signal, and wherein the method comprises, after simultaneously directing both the first display signal and one of the first gate signals to four display elements, directing a second display signal to a single column of display elements in the grid of display elements and simultaneously directing a single second gate signal to a single row of display elements.

17. The method of claim 11, comprising: The first clock frequency is increased or decreased while generating an image frame from the grid of display elements by successively providing a gate signal of the first gate signals to each of a plurality of rows of display elements.

18. The method of claim 11, wherein, The second plurality of flip-flops each operate according to a second clock frequency.

19. The method of claim 11, wherein, The first clock frequency is an integer multiple of the second clock frequency.

20. A system comprising: a plurality of addressable display elements arranged in a plurality of rows and a plurality of columns; a shift register comprising a first plurality of flip-flops each coupled to a first clock input; a second plurality of flip-flops each coupled to a second clock input; and a plurality of gate signals each generated from a respective subset of the second plurality of flip-flops. a second plurality of flip-flops, each configured to receive an output from a respective flip-flop in the first plurality of flip-flops, each flip-flop in the second plurality of flip-flops coupled to a second clock input, and each flip-flop in the second plurality of flip-flops configured to output a gate signal to a respective row of the plurality of rows of the plurality of addressable display elements; and a controller configured to control which row or rows of the plurality of addressable display elements simultaneously receive a gate signal from a flip-flop in the second plurality of flip-flops by controlling a frequency of the first clock input.

Images